Data processing method, device, electronic device and storage medium

Abstract

An embodiment of the present application proposes a data processing method, device, electronic device and computer storage medium, the method comprising: obtaining a data table in a database; the data table is used to store business data; when the data of at least one record in the data table changes, the primary key ID of the changed data is stored in a pre-established intermediate storage medium; based on at least one primary key ID in the intermediate storage medium, the data of the record corresponding to the at least one primary key ID is filtered out from the data table.

Description

Technical Field

[0001] The present application relates to the field of big data technology, and in particular to a data processing method, device, electronic device, and computer storage medium. Background Art

[0002] In daily system applications or business scenarios, users may encounter situations where they need to filter out changed data in the database. In this case, the conventional method is to add a field to the data table in the database that stores such data to identify whether the data has changed. If the data has changed, the field is set to a value. Then, during the data query process, the changed data is filtered out based on the value of this field. After processing the changed data, the field is reset to the default state value.

[0003] However, when the data volume in the data table is large, the above method only uses this field identifier to filter the changed data, which makes the Structured Query Language (SQL) execution efficiency very low. In addition, due to the low discreteness of this field, even adding an index cannot achieve a significant optimization effect and also takes up storage space. Therefore, a method is needed that can accurately and efficiently filter out changed data in the case of large data volumes. Summary of the Invention

[0004] The present application provides a data processing method, device, electronic device and computer storage medium.

[0005] The technical solution of this application is achieved as follows:

[0006] The present invention provides a data processing method, which includes:

[0007] Obtain a data table in a database; the data table is used to store business data;

[0008] When data of at least one record in the data table is changed, the primary key identity document (ID) of the changed data is stored in a pre-established intermediate storage medium;

[0009] Based on the at least one primary key ID in the intermediate storage medium, data of the record corresponding to the at least one primary key ID is filtered out from the data table.

[0010] In some embodiments, the method further comprises:

[0011] When the use of the data recorded corresponding to the at least one primary key ID is completed, the primary key ID corresponding to the changed data whose use has been completed is deleted from the intermediate storage medium.

[0012] In some embodiments, the filtering out, from the data table, data corresponding to the at least one primary key ID based on the at least one primary key ID in the intermediate storage medium includes:

[0013] Based on the primary key ID in the intermediate storage medium, data of the record corresponding to the primary key ID in the intermediate storage medium is filtered out from the data table.

[0014] In some embodiments, the filtering out, from the data table, data corresponding to the at least one primary key ID based on the at least one primary key ID in the intermediate storage medium includes:

[0015] Filtering at least one primary key ID from the primary key IDs in the intermediate storage medium according to a preset condition;

[0016] Filter out data of records corresponding to the at least one primary key ID from the data table.

[0017] In some embodiments, the intermediate storage medium includes: an intermediate table or a distributed cache middleware.

[0018] The embodiment of the present application further proposes a data processing device, which includes an acquisition module, a storage module and a screening module, wherein:

[0019] An acquisition module is used to acquire a data table in a database; the data table is used to store business data;

[0020] A storage module, configured to store the primary key ID of the changed data in a pre-established intermediate storage medium when the data of at least one record in the data table is changed;

[0021] The screening module is used to screen out data corresponding to the at least one primary key ID from the data table based on the at least one primary key ID in the intermediate storage medium.

[0022] An embodiment of the present application provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the data processing method provided by one or more of the aforementioned technical solutions is implemented.

[0023] An embodiment of the present application provides a computer storage medium storing a computer program; after the computer program is executed, it can implement the data processing method provided by one or more of the aforementioned technical solutions.

[0024] An embodiment of the present application proposes a data processing method, device, electronic device and computer storage medium, the method comprising: obtaining a data table in a database; the data table is used to store business data; when the data of at least one record in the data table changes, storing the primary key ID of the changed data in a pre-established intermediate storage medium; based on at least one primary key ID in the intermediate storage medium, filtering out the data of the record corresponding to the at least one primary key ID from the data table; in this way, when the amount of data in the data table is large, it is only necessary to query the primary key ID of the changed data from the intermediate storage medium with a smaller amount of data, and then directly filter out the data of the corresponding record in the data table through the primary key ID; that is, there is no need to directly search for the changed data in the data table with massive business data, which greatly improves the efficiency of filtering the changed data; in addition, due to the uniqueness of the primary key ID, querying the data of the corresponding record in the data table through the primary key ID can ensure the accuracy of the query result. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 is a flow chart of a data processing method in an embodiment of the present application;

[0026] Figure 2a This is a schematic diagram of the process of storing and changing data in an embodiment of the present application;

[0027] Figure 2b This is a flowchart of querying and changing data in an embodiment of the present application;

[0028] Figure 3 A schematic diagram of the structure of a data processing device according to an embodiment of the present application;

[0029] Figure 4 This is a schematic structural diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0030] The present application will be further described in detail below in conjunction with the accompanying drawings and examples. It should be understood that the embodiments provided herein are merely intended to explain the present application and are not intended to limit the present application. In addition, the embodiments provided below are partial embodiments for implementing the present application, rather than providing all embodiments for implementing the present application. In the absence of conflict, the technical solutions described in the embodiments of the present application may be implemented in any combination.

[0031] It should be noted that, in the embodiments of the present application, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a method or apparatus comprising a series of elements includes not only the elements explicitly stated, but also other elements not explicitly listed, or also includes elements inherent to the implementation of the method or apparatus. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other related elements (such as steps in the method or units in the apparatus, for example, a unit may be a portion of a circuit, a portion of a processor, a portion of a program or software, etc.) in the method or apparatus comprising the element.

[0032] The term "and / or" herein is merely a description of an association relationship between associated objects, indicating that three relationships may exist. For example, "I and / or J" may represent the existence of I alone, the existence of both I and J, and the existence of J alone. Furthermore, the term "at least one" herein refers to any combination of at least two of any one or more of a plurality of items. For example, "at least one of I, J, and R" may represent any one or more elements selected from the set consisting of I, J, and R.

[0033] For example, the data processing method provided in the embodiment of the present application includes a series of steps, but the data processing method provided in the embodiment of the present application is not limited to the recorded steps. Similarly, the data processing device provided in the embodiment of the present application includes a series of modules, but the data processing device provided in the embodiment of the present application is not limited to including the modules explicitly recorded, and may also include modules required to obtain relevant time series data or perform processing based on time series data.

[0034] The embodiments of the present application can be applied to a computer system consisting of a terminal device and a server, and can operate with many other general-purpose or special-purpose computing system environments or configurations. Here, the terminal device can be a thin client, a thick client, a handheld or laptop device, a microprocessor-based system, a set-top box, a programmable consumer electronic product, a network personal computer, a minicomputer system, etc., and the server can be a server computer system, a minicomputer system, a mainframe computer system, and a distributed cloud computing technology environment including any of the above systems, etc.

[0035] Electronic devices such as terminals and servers can be described in the general context of computer system executable instructions (such as program modules) executed by a computer system. Generally, program modules can include routines, programs, object programs, components, logic, data structures, etc., which perform specific tasks or implement specific abstract data types. Computer systems / servers can be implemented in a distributed cloud computing environment, where tasks are performed by remote processing devices linked via a communication network. In a distributed cloud computing environment, program modules can be located on local or remote computing system storage media, including storage devices.

[0036] In related technologies, the conventional method for filtering changed data in database tables is to add a field to the existing database table. If a piece of data has changed, the field identifier of the data is modified to a specific value. Since the identifier data has been modified, SQL conditions are spliced ​​through the field when querying the changed data. After the query is completed, the field is reset to the default state value to achieve this function. However, using this method to filter changed data has the following technical problems:

[0037] 1) The business data table structure needs to be modified. Since the data table is usually created for a specific business scenario, adding a new field to identify the modification status of the data may make it difficult to understand the meaning of the field over time and difficult to maintain.

[0038] 2) If the database table contains a large amount of data, perhaps hundreds of thousands or even millions, filtering the changed data using only this newly added field identifier—that is, filtering out a small amount of qualifying data from the massive amount of data—will result in low SQL query execution efficiency. Furthermore, because the field has a low discreteness, adding an index will not significantly optimize the query. This will not only increase disk space overhead, but changes in the field's value will also lead to frequent index updates, reducing update efficiency.

[0039] In order to solve the above technical problems, the following embodiments are proposed.

[0040] In some embodiments of the present application, the data processing method can be implemented using a processor in a data processing device, and the above-mentioned processor can be at least one of an application-specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a central processing unit (CPU), a controller, a microcontroller, and a microprocessor.

[0041] Figure 1 This is a flow chart of a data processing method in an embodiment of the present application. Figure 1 As shown, the method includes the following steps:

[0042] Step 100: Obtain a data table in the database; the data table is used to store business data.

[0043] Here, the database is a warehouse for storing data. Its essence is a file system that stores data in a specific format. Users can add, modify, delete and query data in the database.

[0044] Databases store data in tables. A table, or table for short, consists of three parts: the table name, the fields within it, and the records within it. Each row in a table is called a record, which consists of several fields. Fields, also called domains, are columns in a table. Each field has corresponding descriptive information, such as the field name, field type, and field width.

[0045] In one embodiment, assume that a data table stores the test scores of all students in a class, where each row in the table corresponds to a student and includes the student's student ID, name, and grades for each course. For this data table, the table name may be Class X Test Scores. Each student's student ID, name, and grades for each course in the table constitute a record. The columns corresponding to the student ID, name, and grades for each course in the table represent different fields.

[0046] In an embodiment of the present application, a data table is used to store business data in different business scenarios; here, the business data can be data related to the basic information of the user, such as name, gender or age, etc.; it can be data related to user behavior, such as the type of behavior performed by the user, and the time or place where the behavior occurs, etc.; it can also be data related to goods, such as product name, product category or product inventory, etc.; the embodiment of the present application does not limit the type of business data in the data table.

[0047] In one embodiment, the storage of business data in the data table can be implemented through SQL statements; SQL is a database query and programming language used to access data and query, update, and manage data tables in the database.

[0048] In one embodiment, data tables in a database can be obtained through a database management system (DBMS). A database management system (DBMS) is a large-scale software program used to operate and manage databases. It is used to establish, use, and maintain databases, and to provide unified management and control of databases to ensure their security and integrity. Common database management systems include MySQL, Oracle, and SQL Server.

[0049] Step 101: When data of at least one record in a data table is changed, the primary key ID of the changed data is stored in a pre-established intermediate storage medium.

[0050] In one embodiment, data in a data table may change during data storage. In some daily system applications or business scenarios, it is necessary to filter out the changed data in the data table to facilitate subsequent operations.

[0051] Here, the embodiments of the present application do not limit the reasons for the changes in the data in the data table; for example, it may be caused by data redundancy; it may also be caused by improper concurrency control; it may also be caused by software or hardware failure or operational errors.

[0052] Usually, a primary key is set during the design of a data table. The purpose of the primary key is to effectively manage the data in the data table. The primary key exists as a unique identification column, which can distinguish each row of records in the table. The primary key of each row of records cannot be empty, that is, when the data table generates a record, the primary key value must exist.

[0053] In the embodiment of the present application, the value of the primary key is the primary key ID, and there is no limitation on the method for generating the primary key ID; for example, it can be generated by an auto-increment method, a snowflake algorithm, or a universally unique identifier (UUID).

[0054] In one embodiment, when a data table generates a primary key ID by auto-incrementing, there is no limit on the value of each auto-increment; assuming that the primary key ID of data table 1 is 0 when generating the first record, the primary key ID when generating the second record can be 1 or 2.

[0055] In an embodiment of the present application, the intermediate storage medium is used to store intermediate processing results for the data table; in some embodiments, the intermediate storage medium may include: an intermediate table or a distributed cache middleware; here, the distributed cache middleware may include: a remote dictionary service (Remote Dictionary Server, redis) or a cache system (Memcached) based on memory storage.

[0056] Here, since the distributed cache middleware is memory storage, its query efficiency is much higher than that of the database disk storage. Therefore, when the amount of data in the data table is large, the use of distributed cache middleware can improve data query efficiency.

[0057] In the embodiment of the present application, each record in the data table includes several fields, wherein the data of each record is changed, which may be the data of any field included in each record being changed.

[0058] For example, assume that the basic information of users is stored in a data table, each row in the table corresponds to a user, and each row of records includes three fields, namely primary key ID, name, and age; when the user's name changes, the primary key ID of the changed data "name" is stored in a pre-established intermediate storage medium; when the user's age changes, the primary key ID of the changed data "age" is stored in a pre-established intermediate storage medium.

[0059] It can be seen that when the data of at least two fields in a record in a data table changes, there is a situation where the primary key ID is stored repeatedly; exemplarily, in order to avoid the situation where the primary key ID is stored repeatedly, the method may also include: when determining the primary key ID of the changed data stored in the pre-established intermediate storage medium, ignoring the primary key ID of the changed data, thereby avoiding the situation where the same primary key ID is stored repeatedly.

[0060] In one implementation, assume that the first record in a data table includes fields 1 and 2 in addition to the primary key ID. If the data in field 1 changes from A to B, the primary key ID is stored in the intermediate storage medium. At this point, the primary key ID of the first record has already been stored. Subsequently, if the data in field 2 changes from C to D, the primary key ID of the changed data in field 2 is ignored; that is, the primary key ID corresponding to field 2 is no longer stored. This reduces the amount of data in the intermediate storage medium, thereby improving processing efficiency when performing subsequent data queries on the intermediate storage medium.

[0061] Step 102: Based on at least one primary key ID in the intermediate storage medium, filter out data of a record corresponding to at least one primary key ID from the data table.

[0062] In the embodiment of the present application, after the above steps, the intermediate storage medium has stored the primary key IDs of all changed data in the data table.

[0063] Here, the intermediate storage medium also stores data based on the data table. Each row in the data table corresponds to a change data, and each row consists of two fields; one is the primary key ID field of the data table itself in the intermediate storage medium, and the other is the primary key ID field of the change data in the original data table.

[0064] In one embodiment, assuming that the primary key ID of the first changed data stored in the intermediate storage medium is 100, and the primary key ID of the data table itself in the intermediate storage medium is 1; then the status record of the first changed data in the intermediate storage medium is in the form of "primary key id = 1, primary key ID = 100".

[0065] In some embodiments, based on at least one primary key ID in the intermediate storage medium, filtering out data corresponding to at least one primary key ID from the data table may include: based on the primary key ID in the intermediate storage medium, filtering out data corresponding to the primary key ID in the intermediate storage medium from the data table.

[0066] In one embodiment, the primary key IDs of all changed data in the intermediate storage medium may be queried, and then, data of records corresponding to the primary key IDs of all changed data may be screened out in a data table of the database.

[0067] For example, assuming that ten records of changed data are stored in the intermediate storage medium, the primary key IDs of the ten changed data in the intermediate storage medium can be queried to determine the data of the records corresponding to the primary key IDs of the ten changed data.

[0068] It can be seen that the above method can filter out all the changed data in the original data table, that is, it can ensure the comprehensiveness of the data screening results.

[0069] In some embodiments, based on at least one primary key ID in the intermediate storage medium, filtering out data corresponding to at least one primary key ID from the data table may include: filtering out at least one primary key ID from the primary key ID in the intermediate storage medium according to preset conditions; filtering out data corresponding to at least one primary key ID from the data table.

[0070] In actual use, it may not be necessary to filter all the changed data in the data table. That is, when only part of the changed data in the data table needs to be used, the primary key ID of these changed data can be queried in the intermediate storage medium according to the preset conditions; then, the data corresponding to the records of these primary key IDs can be filtered out from the data table.

[0071] Here, the preset condition is determined according to actual usage requirements; the preset condition can be used to determine the primary key ID of the changed data to be queried in the intermediate storage medium.

[0072] In one embodiment, assuming that there are ten records of changed data stored in the intermediate storage medium, the preset condition can be to query the primary key IDs of the first three changed data in the intermediate storage medium, or to query the primary key IDs of the first five changed data in the intermediate storage medium.

[0073] Here, selectively filtering out some of the changed data from the original data table according to preset conditions can better meet the actual use needs of the changed data.

[0074] In some embodiments, the above method further includes: when the use of the data recorded corresponding to at least one primary key ID is completed, deleting the primary key ID corresponding to the changed data in the intermediate storage medium.

[0075] In an embodiment of the present application, after the use of the data corresponding to the filtered primary key ID is completed, the primary key ID corresponding to the changed data whose use has been completed is deleted in the intermediate storage medium; for example, assuming that the primary key IDs of ten changed data are stored in the intermediate storage medium, if the use of the data corresponding to the primary key IDs of the first three changed data is completed, the primary key IDs of the first three changed data in the intermediate storage medium are deleted.

[0076] Here, by deleting the primary key ID of the changed data after use, not only can the amount of data in the intermediate storage medium be reduced and memory be saved, but also the query efficiency of the primary key ID of other changed data can be improved.

[0077] The embodiment of the present application proposes a data processing method, device, electronic device and computer storage medium, which includes: obtaining a data table in a database; the data table is used to store business data; when the data of at least one record in the data table changes, the primary key ID of the changed data is stored in a pre-established intermediate storage medium; based on at least one primary key ID in the intermediate storage medium, the data of the record corresponding to at least one primary key ID is filtered out from the data table; in this way, when the amount of data in the data table is large, it is only necessary to query the primary key ID of the changed data from the intermediate storage medium with a smaller amount of data, and then directly filter out the data of the corresponding record in the data table through the primary key ID; that is, there is no need to directly search for the changed data in the data table with massive business data, which greatly improves the efficiency of filtering the changed data; in addition, due to the uniqueness of the primary key ID, the data of the corresponding record in the data table can be queried through the primary key ID to ensure the accuracy of the query result; further, the method does not need to modify the data table structure and is easy to maintain.

[0078] In order to better reflect the purpose of this application, further examples are given based on the above embodiments of this application.

[0079] In the embodiment of the present application, the primary key ID of the changed data is mainly recorded with the help of an intermediate table or a distributed cache middleware with high data query efficiency; first, after a certain data in the data table is changed, the primary key ID of the changed data is recorded in the intermediate table or distributed cache middleware, and when the changed data needs to be obtained, the primary key ID of all changed data is obtained from the intermediate table or distributed cache middleware; then, other data of the record is queried in the data table through the primary key ID. The efficiency of querying data through the primary key ID is extremely high. After the data queried by the primary key ID is used, the primary key ID of the changed data that has been used can be deleted from the intermediate table or distributed cache middleware.

[0080] In one implementation, if the amount of changed data is small, an intermediate table of a database can be used, and the query efficiency of the intermediate table is also relatively high; if the amount of changed data is large, the changed data can be recorded through a distributed cache such as redis; since redis data is stored in memory, the query efficiency is much higher than that of the database disk storage. This storage method can solve the problem of excessive business data in the data table and slow query efficiency of changed data.

[0081] Figure 2a This is a flow chart of storing and changing data in an embodiment of the present application. Figure 2a As shown, the process includes the following steps:

[0082] Step A1: Use the change data operation to modify the data in the business table M.

[0083] In one embodiment, business table M is a data table in the database of the above embodiment. Assume that the table structure of business table M includes three fields: user ID, user name, and age. The user ID field is the primary key ID of business table M. For example, the modified data of business table M obtained by the data change operation is: user ID = 1, user name = zhangsan, age = 20.

[0084] Step A2: Store the change data record in the change data record table N.

[0085] In one embodiment, change data record table N is a representation of the intermediate storage medium in the above-described embodiment. The table structure of change data record table N includes two fields: id and user ID. Here, id is the key ID corresponding to the record of the user ID in change data record table N. When it is determined that the user name "zhangsan" and age "20" in business table M have changed, the user ID "1" corresponding to this user name and age is stored in change data record table N. That is, "id = 1, user ID = 1" can be used as the status record of the change data, and the process ends.

[0086] Figure 2b This is a flow chart of querying and changing data in an embodiment of the present application. Figure 2b As shown, the process includes the following steps:

[0087] Step A3: Use the change data operation to query the change data record.

[0088] In one embodiment, the record of the changed data queried from the changed data record table N is "id=1, user ID=1".

[0089] Step A4: Use the user ID as the query condition to query the data of the corresponding record in the business table M.

[0090] In one embodiment, the user ID in the change data record table N is used as a query condition to query the data corresponding to the record of "user ID=1" in the business table M and use it.

[0091] Step A5: After the data is used, the data record corresponding to the change data record table N is deleted.

[0092] In one embodiment, after the data is used, the primary key ID "user ID=1" corresponding to the changed data is deleted from the change data record table N, the modification status is reset, and the process ends.

[0093] It can be seen that the embodiment of the present application decouples and separates the changed data from the original data table by adopting an intermediate storage medium, thereby removing the data table's dependence on the fields in the modified state, and reducing the time and resource consumption of data modification and rebuilding the table index. Especially when the data table has a large amount of business data and a relatively small amount of changed data, it is only necessary to query the changed data from the change data record table with a smaller amount of data, without the need to search in the data table of massive business data, which greatly improves the efficiency of data query.

[0094] Figure 3 FIG. 1 is a schematic diagram of the structure of a data processing device according to an embodiment of the present application. Figure 3 As shown, the device includes: an acquisition module 300, a storage module 301 and a screening module 302, wherein:

[0095] The acquisition module 300 is used to obtain a data table in the database; the data table is used to store business data;

[0096] The storage module 301 is used to store the primary key ID of the changed data in a pre-established intermediate storage medium when the data of at least one record in the data table is changed;

[0097] The screening module 302 is configured to screen out data corresponding to at least one primary key ID from the data table based on at least one primary key ID in the intermediate storage medium.

[0098] In some embodiments, the screening module 302 is further configured to:

[0099] When the use of the data recorded corresponding to at least one primary key ID is completed, the primary key ID corresponding to the changed data whose use has been completed is deleted from the intermediate storage medium.

[0100] In some embodiments, the filtering module 302 is configured to filter out data corresponding to at least one primary key ID from the data table based on at least one primary key ID in the intermediate storage medium, including:

[0101] Based on the primary key ID in the intermediate storage medium, data of the record corresponding to the primary key ID in the intermediate storage medium is filtered out from the data table.

[0102] In some embodiments, the filtering module 302 is configured to filter out data corresponding to at least one primary key ID from the data table based on at least one primary key ID in the intermediate storage medium, including:

[0103] Filter out at least one primary key ID from the primary key IDs in the intermediate storage medium according to preset conditions;

[0104] Filter out data of at least one record corresponding to the primary key ID from the data table.

[0105] In some embodiments, the intermediate storage medium includes: an intermediate table or a distributed cache middleware.

[0106] In actual applications, the above-mentioned acquisition module 300, storage module 301 and screening module 302 can all be implemented by a processor located in an electronic device, and the processor can be at least one of ASIC, DSP, DSPD, PLD, FPGA, CPU, controller, microcontroller, and microprocessor.

[0107] In addition, the functional modules in this embodiment may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated units may be implemented in the form of hardware or software functional modules.

[0108] If the integrated unit is implemented in the form of a software functional module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment, or the part that contributes to the relevant technology, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of the method of this embodiment. The aforementioned storage medium includes: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc., various media that can store program code.

[0109] Specifically, the computer program instructions corresponding to a data processing method in this embodiment can be stored on a storage medium such as a CD, a hard disk, or a USB flash drive. When the computer program instructions corresponding to a data processing method in the storage medium are read or executed by an electronic device, any data processing method in the aforementioned embodiments is implemented.

[0110] Based on the same technical concept as the above embodiment, see Figure 4 , which shows the electronic device 400 provided by this application, which may include: a memory 401 and a processor 402; wherein,

[0111] Memory 401, used to store computer programs and data;

[0112] The processor 402 is configured to execute the computer program stored in the memory to implement any one of the data processing methods in the aforementioned embodiments.

[0113] In actual applications, the above-mentioned memory 401 can be a volatile memory (volatile memory), such as RAM; or a non-volatile memory (non-volatile memory), such as ROM, flash memory, hard disk drive (HDD) or solid-state drive (SSD); or a combination of the above types of memory, and provide instructions and data to the processor 402.

[0114] The processor 402 may be at least one of an ASIC, a DSP, a DSPD, a PLD, an FPGA, a CPU, a controller, a microcontroller, and a microprocessor. It is understood that for different data processing devices, the electronic device used to implement the processor function may also be other, and this embodiment of the application does not specifically limit this.

[0115] In some embodiments, the functions or modules provided by the apparatus of the present application can be used to execute the method described in the above method embodiment. The specific implementation thereof can refer to the description of the above method embodiment. For the sake of brevity, it will not be repeated here.

[0116] The above description of the various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other. For the sake of brevity, this article will not repeat them.

[0117] The methods disclosed in the various method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0118] The features disclosed in the various product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0119] The features disclosed in the various method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0120] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of hardware embodiments, software embodiments, or embodiments combining software and hardware. Furthermore, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage and optical storage, etc.) containing computer-usable program code.

[0121] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0122] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0123] The above are merely preferred embodiments of the present application and are not intended to limit the scope of protection of the present application.

Claims

1. A data processing method, characterized in that: The method comprises: Obtain a data table in a database; the data table is used to store business data; When data of at least one record in the data table is changed, the primary key identity ID of the changed data is stored in a pre-established intermediate storage medium; Based on at least one primary key ID in the intermediate storage medium, filter out data of records corresponding to the at least one primary key ID from the data table; The method of filtering out data corresponding to the record of at least one primary key ID in the intermediate storage medium from the data table based on the at least one primary key ID in the intermediate storage medium includes: filtering out data corresponding to the record of the primary key ID in the intermediate storage medium from the data table based on the primary key ID in the intermediate storage medium.

2. The method according to claim 1, characterized in that The method further comprises: When the use of the data recorded corresponding to the at least one primary key ID is completed, the primary key ID corresponding to the changed data whose use has been completed is deleted from the intermediate storage medium.

3. The method according to claim 1, characterized in that The step of filtering out data corresponding to the at least one primary key ID from the data table based on the at least one primary key ID in the intermediate storage medium includes: Filtering at least one primary key ID from the primary key IDs in the intermediate storage medium according to a preset condition; Filter out data of records corresponding to the at least one primary key ID from the data table.

4. The method according to claim 1, wherein The intermediate storage medium includes: an intermediate table or a distributed cache middleware.

5. A data processing device, characterized in that: The device comprises: An acquisition module is used to acquire a data table in a database; the data table is used to store business data; A storage module, configured to store the primary key ID of the changed data in a pre-established intermediate storage medium when the data of at least one record in the data table is changed; A screening module, configured to screen out data corresponding to the at least one primary key ID from the data table based on the at least one primary key ID in the intermediate storage medium; The screening module is further configured to screen out data corresponding to the primary key ID in the intermediate storage medium from the data table based on the primary key ID in the intermediate storage medium.

6. The device according to claim 5, characterized in that The screening module is configured to screen out data corresponding to the at least one primary key ID from the data table based on the at least one primary key ID in the intermediate storage medium, including: Filtering at least one primary key ID from the primary key IDs in the intermediate storage medium according to a preset condition; Filter out data of records corresponding to the at least one primary key ID from the data table.

7. An electronic device, characterized in that: The device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the method according to any one of claims 1 to 4 when executing the program.

8. A computer storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.

Images