Data processing method and system and non-real-time data query device
By integrating data from different business systems in the data processing system and generating index tables, the problem of inefficient data query is solved, efficient and accurate cross-system data query is achieved, and user experience is improved.
Patent Information
- Application Number
- CN202411174427.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-26
- Publication Date
- 2024-12-06
AI Technical Summary
In the prior art, data is stored in different business systems in a dispersed manner, resulting in low data query efficiency, complex data integration across systems, and easy to make mistakes in manual operations. Especially when facing massive data, traditional real-time query methods are inefficient and resource consumption is high. .
In the data processing system, new data is obtained based on preset basic data keywords and time intervals, and the business system keyword mapping table is used to integrate the data of the same order number, and generate comprehensive data tables and index tables to improve query efficiency and accuracy , realize unified query of cross-system data.
It improves the efficiency and accuracy of data query, simplifies user operations, improves user experience, and reduces system resource consumption.
Smart Images

Figure CN119088837A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of data processing, and in particular to a data processing method, system and non-real-time data query device. Background Art
[0002] In the prior art, data is stored in different business systems, which makes data query and analysis difficult. When faced with massive amounts of data, traditional real-time query methods are inefficient, take a long time to query, and consume a lot of system resources. In addition, cross-system data integration is complex, and manual operations are prone to errors and inefficient. Summary of the invention
[0003] In view of this, the purpose of the present invention is to provide a data processing method, system and non-real-time data query device, which can improve data query efficiency, improve data query accuracy, and thus enhance user experience.
[0004] In a first aspect, an embodiment of the present invention provides a data processing method, which is applied to a data processing system, wherein the data processing system is connected to a business system of at least one peripheral device; the data processing method comprises: based on a preset basic data keyword, obtaining the newly added basic data in each business system within the current time interval according to a preset time interval; the basic data keyword includes the order number; based on a business system keyword mapping table, integrating the newly added basic data corresponding to the same order number into a data set to be updated, and updating a comprehensive data table based on each data set to be updated; the comprehensive data table stores the association relationship between the basic data keyword and the basic data; for each data set to be updated, based on the updated comprehensive data table, determining the newly added basic data and index information corresponding to the preset query field in the data set to be updated as the newly added index table data; the index information includes a pointer to the comprehensive data table; and updating the index table based on the newly added index table data.
[0005] Furthermore, the data processing system is also connected to the query system of the peripheral device; the method also includes: obtaining a data query request sent by the query system; the data query request includes at least one query field; according to the data query request, determining the target index information corresponding to the query field in the index table; based on the target index information, querying from the comprehensive data table to obtain the basic data to be queried; and returning the basic data to be queried to the query system.
[0006] Furthermore, the comprehensive data table is set up in the following manner: obtaining a request for creating a comprehensive data table; the request for creating the comprehensive data table includes interface information of the business system and basic data keywords; establishing a connection with the business system based on the interface information; obtaining basic data in each business system based on the basic data keywords; integrating basic data corresponding to the same order number into an initial data set based on the business system keyword mapping table; and generating a comprehensive data table based on the initial data set.
[0007] Furthermore, the index table is set up in the following manner: obtaining an index table creation request; the index table creation request includes a query field; obtaining basic data corresponding to the query field in the comprehensive data table and index information corresponding to the basic data; generating an index table based on the query field, basic data and index information; the index table stores the association relationship between the query field, basic data and index information.
[0008] Furthermore, after the step of obtaining the basic data in each business system based on the basic data keywords, the method also includes: determining the business system keywords corresponding to each business system based on the basic data keywords; the business system keywords include the order number; determining that the business system whose business system keywords include the order number is the main business system, and the business systems other than the main business system are the slave business systems; judging whether the business system keywords corresponding to the slave business system are the same as the business system keywords corresponding to the main business system; if the business system keywords corresponding to a slave business system are the same as the business system keywords corresponding to a master business system, determining that the same business system keywords are the target business system keywords; establishing a business system keyword mapping table based on the target business system keywords; the business system keyword mapping table is used to characterize the association relationship between the main business system and each slave business system.
[0009] Furthermore, the step of returning the basic data to be queried to the query system includes: encrypting the basic data to be queried based on a preset encryption method to obtain the encrypted basic data to be queried; and sending the encrypted basic data to be queried to the query system.
[0010] In a second aspect, an embodiment of the present invention provides a data processing system, which is connected to a business system of at least one peripheral device; the data processing system includes: a basic data acquisition module, which is used to acquire the newly added basic data in each business system within the current time interval according to a preset time interval based on a preset basic data keyword; the basic data keyword includes the order number; a comprehensive data table update module, which is used to integrate the newly added basic data corresponding to the same order number into a data set to be updated based on the business system keyword mapping table, and update the comprehensive data table based on each data set to be updated; the comprehensive data table stores the association relationship between the basic data keyword and the basic data; an index table update module, which is used to determine, for each data set to be updated, based on the updated comprehensive data table, that the newly added basic data and index information corresponding to the preset query field in the data set to be updated are the newly added index table data; the index information includes a pointer to the comprehensive data table; the index table update module is also used to update the index table based on the newly added index table data.
[0011] In a third aspect, an embodiment of the present invention provides a non-real-time data query device, comprising: at least one business system and a query system; and also comprising the above-mentioned data processing system; the data processing system is connected to the business system and the query system respectively.
[0012] In a fourth aspect, an embodiment of the present invention provides an electronic device, including a memory and a processor, wherein the memory stores a computer program executable on the processor, and the processor implements the method described above when executing the computer program.
[0013] In a fifth aspect, an embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored, and the program code enables the processor to execute the method as described above.
[0014] The embodiment of the present invention provides a data processing method, system and non-real-time data query device, which are applied to a data processing system, and the data processing system is connected to at least one peripheral business system; the data processing method includes: based on a preset basic data keyword, obtaining the newly added basic data in each business system within the current time interval according to a preset time interval; the basic data keyword includes an order number; based on a business system keyword mapping table, the newly added basic data corresponding to the same order number is integrated into a data set to be updated, and a comprehensive data table is updated based on each data set to be updated; the comprehensive data table stores the association relationship between the basic data keyword and the basic data; for each data set to be updated, based on the updated comprehensive data table, the newly added basic data and index information corresponding to the preset query field in the data set to be updated are determined as newly added index table data; the index information includes a pointer to the comprehensive data table; and the index table is updated based on the newly added index table data. In this method, by integrating the data stored in different systems into a comprehensive data table and generating an index table according to the comprehensive data table, the query efficiency of the data can be improved, the accuracy of the data query can be improved, and the user experience can be improved.
[0015] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention are realized and obtained by the structures particularly pointed out in the description, claims and drawings.
[0016] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0018] Figure 1 A flow chart of a data processing method provided in Embodiment 1 of the present invention;
[0019] Figure 2 A flowchart of a method for setting a comprehensive data table provided in Embodiment 1 of the present invention;
[0020] Figure 3 A flowchart of a method for setting a keyword mapping table for a business system provided in Embodiment 1 of the present invention;
[0021] Figure 4 A flowchart of a method for setting an index table provided in Embodiment 1 of the present invention;
[0022] Figure 5 A flowchart of a method for returning basic data to be queried to a query system provided in the first embodiment of the present invention;
[0023] Figure 6 A schematic diagram of a data processing system provided in Embodiment 2 of the present invention;
[0024] Figure 7 A schematic diagram of another data processing system provided in Embodiment 2 of the present invention;
[0025] Figure 8 This is a schematic diagram of a non-real-time data query device provided in Embodiment 3 of the present invention.
[0026] Icons: 1- basic data acquisition module; 2- comprehensive data table update module; 3- index table update module; 4- data query module; 5- business system; 6- query system; 7- data processing system. DETAILED DESCRIPTION
[0027] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0028] As time goes by, the company's order volume surges, the number of cooperating merchants also increases rapidly, and the amount of data is getting larger and larger. Anti-fraud risk control personnel also have an increased demand for data statistics. They need to obtain various latitude information of cooperating merchants, analyze various data, control risks or reduce risk costs. Order information, product information, and merchant information are all stored in different places. There is no unified place to query, and the data caliber in each place may be different. After querying the data, it needs to be checked in different places, which is cumbersome and error-prone.
[0029] The present application provides a data processing method, system and non-real-time data query device, which can improve data query efficiency and improve data query accuracy, thereby enhancing user experience.
[0030] To facilitate understanding of this embodiment, the embodiment of the present invention is described in detail below.
[0031] Embodiment 1:
[0032] Figure 1 This is a flow chart of a data processing method provided in Embodiment 1 of the present invention.
[0033] The data processing method is applied to a data processing system, and the data processing system is connected to at least one peripheral business system.
[0034] Specifically, the business system may be an order system, a merchant information system, a product information system, and the like.
[0035] Reference Figure 1 , data processing methods, including:
[0036] Step S101, based on preset basic data keywords, new basic data in each business system within the current time interval is obtained according to preset time intervals; the basic data keywords include order numbers.
[0037] Here, the preset time interval can be preset according to actual conditions, and can be set to 24 hours. Among them, the embodiment of the present invention is aimed at non-real-time query scenarios. Given that the validity of all data is the previous day, the time for obtaining new basic data is set to early morning. The new basic data of the previous day is synchronized at early morning every day to keep the data up-to-date and the query efficient.
[0038] Basic data keywords are pre-set according to actual conditions. Basic data keywords are the core fields of the business system and are used to uniquely identify basic data in the business system. Basic data keywords in different business systems may be different. For example, the order system includes basic data keywords such as order code, product code, merchant code, and order time. The merchant information system includes basic data keywords such as merchant code, merchant name, and merchant address. The product information system includes basic data keywords such as product code, product name, and product category.
[0039] Step S102, based on the business system keyword mapping table, the newly added basic data corresponding to the same order number are integrated into a data set to be updated, and the comprehensive data table is updated based on each data set to be updated; the comprehensive data table stores the association relationship between the basic data keywords and the basic data.
[0040] Here, the comprehensive data table is used to store all detailed data records, including order, merchant and product information. The comprehensive data table is mainly used to accommodate and integrate all data obtained from different business systems to ensure data integrity and consistency.
[0041] The comprehensive data table stores data from different places and latitudes.
[0042] Different places means that data is stored in different systems, databases or tables. For example, order data is stored in the order system, merchant information is stored in the merchant information system, and product information is stored in the product information system.
[0043] Different dimensions refer to different aspects or attributes of data. For example, order data may include order number, product number, order time, etc.; merchant information may include merchant number, merchant name, address, etc.; product information may include product number, product name, category, etc.
[0044] In one embodiment, referring to Figure 2 , the comprehensive data table is set up in the following way:
[0045] Step S201, obtaining a comprehensive data table creation request; the comprehensive data table creation request includes interface information of the business system and basic data keywords.
[0046] Here, determine from which business system the data needs to be obtained based on the interface information.
[0047] Step S202: Establish a connection with the business system based on the interface information.
[0048] Here, the data processing system establishes a data transmission channel with the business system of each peripheral device through the interface set in the interface information.
[0049] Step S203, based on the basic data keyword, obtain the basic data in each business system.
[0050] Specifically, based on the basic data keywords, all basic data are extracted from the order system, merchant information system and product information system.
[0051] Extract basic data from different business systems based on basic data keywords. For example, extract order data from the order system, extract merchant information from the merchant information system, and extract product data from the product information system.
[0052] In one embodiment, referring to Figure 3 After step S203, the method further includes:
[0053] Step S301, based on the basic data keywords, determine the business system keywords corresponding to each business system; the business system keywords include the order number.
[0054] Step S302, determining that the business system whose business system keyword includes the order number is the primary business system, and the business systems other than the primary business system are secondary business systems.
[0055] Specifically, the order system including the order number is determined as the main business system, and the merchant information system and the product information system are determined as the slave information systems.
[0056] Step S303, determining whether the business system keyword corresponding to the slave business system is the same as the business system keyword corresponding to the master business system.
[0057] Step S304: if a business system keyword corresponding to a slave business system is the same as a business system keyword corresponding to a master business system, the same business system keyword is determined as a target business system keyword.
[0058] Specifically, the order system includes basic data keywords such as order code, product code, merchant code and order time. The merchant information system includes basic data keywords such as merchant code, merchant name and merchant address. The product information system includes basic data keywords such as product code, product name and product category.
[0059] The merchant code in the merchant information system is the same as the merchant code in the order system, and the product code in the product information system is the same as the product code in the order system.
[0060] Determine the merchant code and product code as the target business system keywords.
[0061] Step S305 , establishing a business system keyword mapping table based on the target business system keyword; the business system keyword mapping table is used to characterize the association relationship between the master business system and each slave business system.
[0062] Specifically, a business system keyword mapping table is established based on the target business system keyword as shown in Table 1 below.
[0063] Table 1 Business system keyword mapping table
[0064]
[0065]
[0066] Step S204: Based on the business system keyword mapping table, basic data corresponding to the same order number are integrated into an initial data set.
[0067] Here, based on the target business system keywords, the basic data corresponding to the same order number are associated and integrated into a unified initial data set.
[0068] Specifically, the newly added basic data obtained from the order system is shown in Table 2:
[0069] Table 2 New basic data of order system
[0070] Order code Product Code Merchant Code Order time 1001 5001 200 2024-07-01 1002 5002 201 2024-07-02
[0071] The newly added basic data obtained from the merchant information system is shown in Table 3:
[0072] Table 3 Newly added basic data of merchant information system
[0073] Merchant Code Business Name Business address 200 Merchant A Address A 201 Merchant B Address B
[0074] The newly added basic data obtained from the product information system is shown in Table 4:
[0075] Table 4 Newly added basic data of product information system
[0076] Product Code Product Name Product Categories 5001 Product A Category A 5002 Product B Category B
[0077] The newly added basic data is integrated into a complete data set to be updated through order codes, product codes and merchant codes.
[0078] Step S205, generating a comprehensive data table according to the initial data set.
[0079] Here, the comprehensive data table is used to store all the integrated initial data sets.
[0080] Specifically, the comprehensive data table is shown in Table 5:
[0081] Table 5 Comprehensive data table
[0082]
[0083] Step S103, for each data set to be updated, based on the updated comprehensive data table, determine that the newly added basic data and index information corresponding to the preset query field in the data set to be updated are newly added index table data; the index information includes a pointer to the comprehensive data table.
[0084] Here, the index table is a composite index table including more than one query field, which is used to quickly locate and query basic data in the comprehensive data table, and includes commonly used query fields and index information thereof.
[0085] The query fields are pre-set based on actual conditions, such as order time, order number, merchant number, and product category.
[0086] Specifically, the data set to be updated includes the newly added basic data in each business system within the current time interval, and the data set to be updated includes: order code: 1003, product code: 5003, merchant code: 200, order time: 2024-07-03, product name: Product C, merchant name: Merchant A, merchant address: Address A, product category: Category C.
[0087] The comprehensive data table is updated based on the data set to be updated. The updated comprehensive data table is shown in Table 6:
[0088] Table 6 Updated comprehensive data table
[0089]
[0090]
[0091] Step S104: update the index table based on the newly added index table data.
[0092] Specifically, based on the updated comprehensive data table, the index table is updated, and the updated index table is shown in Table 7:
[0093] Table 7 Updated index table
[0094] Order code Merchant Code Order time Product Categories Index information 1001 200 2024-07-01 Category A ->1001 1002 201 2024-07-02 Category B ->1002 1003 200 2024-07-03 Category C ->1003
[0095] In one embodiment, referring to Figure 4 , the index table is set up in the following way:
[0096] Step S401, obtaining an index table creation request; the index table creation request includes a query field.
[0097] Step S402, obtaining basic data corresponding to the query field in the comprehensive data table and index information corresponding to the basic data.
[0098] Here, the pointer in the index information is used to point to the data location of the basic data corresponding to the query field in the comprehensive data table. The pointer is a logical pointer, and the pointer can be set to an order code to point to the order code in the comprehensive data table.
[0099] Step S403, generating an index table according to the query fields, basic data and index information; the index table stores the association relationship between the query fields, basic data and index information.
[0100] Here, an index table is generated based on the query field, basic data and index information.
[0101] In one embodiment, the data processing system is also connected to a query system of the peripheral device.
[0102] Here, the user queries the basic data stored in the data processing system through the query system.
[0103] Reference Figure 5 , the method further comprises:
[0104] Step S501: obtaining a data query request sent by a query system; the data query request includes at least one query field.
[0105] Here, the user enters query conditions, such as merchant code and time range, through the query system.
[0106] Specifically, the user inputs: merchant code: 200; time range: 2024-07-01 to 2024-07-02.
[0107] Step S502: Determine target index information corresponding to the query field in the index table according to the data query request.
[0108] Specifically, in the index table, search for records with a merchant code of 200 and an order time between 2024-07-01 and 2024-07-02, and locate the order code 1001 corresponding to the order that meets the conditions.
[0109] Step S503: query the comprehensive data table based on the target index information to obtain the basic data to be queried.
[0110] Specifically, the basic data corresponding to the order code 1001 in the comprehensive data table is obtained, and the basic data corresponding to the order code 1001 is determined to be the basic data to be queried.
[0111] Step S504, returning the basic data to be queried to the query system.
[0112] In one embodiment, step S504 includes:
[0113] The basic data to be queried is encrypted based on a preset encryption method to obtain the encrypted basic data to be queried.
[0114] Send the encrypted basic data to be queried to the query system.
[0115] Here, the preset encryption method is pre-set according to actual conditions and may be an Advanced Encryption Standard (AES) encryption algorithm.
[0116] A unique key is generated for each query or a preset key is used, and the data to be queried is encrypted using the AES algorithm and the generated key to obtain the encrypted data to be queried.
[0117] An SSL / TLS (Secure Sockets Layer / Transport Layer Security) encrypted connection is established between the data processing system and the query system, and the encrypted data to be queried is sent to the query system through a secure connection to ensure the security of the data during transmission.
[0118] The embodiment of the present invention provides a data processing method, which is applied to a data processing system, and the data processing system is connected to a business system of at least one peripheral device; the data processing method includes: based on a preset basic data keyword, obtaining the newly added basic data in each business system within the current time interval according to a preset time interval; the basic data keyword includes an order number; based on a business system keyword mapping table, the newly added basic data corresponding to the same order number is integrated into a data set to be updated, and a comprehensive data table is updated based on each data set to be updated; the comprehensive data table stores the association relationship between the basic data keyword and the basic data; for each data set to be updated, based on the updated comprehensive data table, the newly added basic data and index information corresponding to the preset query field in the data set to be updated are determined as newly added index table data; the index information includes a pointer to the comprehensive data table; and the index table is updated based on the newly added index table data. In this method, by integrating the data stored in different systems into a comprehensive data table and generating an index table based on the comprehensive data table, the query efficiency of the data can be improved, the accuracy of the data query can be improved, and the user experience can be improved.
[0119] Embodiment 2:
[0120] Figure 6 A schematic diagram of a data processing system provided in Embodiment 2 of the present invention.
[0121] Reference Figure 6 , the data processing system includes:
[0122] The basic data acquisition module 1 is used to acquire the newly added basic data in each business system within the current time interval according to the preset time interval based on the preset basic data keywords; the basic data keywords include the order number.
[0123] The comprehensive data table update module 2 is used to integrate the newly added basic data corresponding to the same order number into a data set to be updated based on the business system keyword mapping table, and update the comprehensive data table based on each data set to be updated; the comprehensive data table stores the association relationship between the basic data keywords and the basic data.
[0124] The index table updating module 3 is used to determine, for each data set to be updated, based on the updated comprehensive data table, the newly added basic data and index information corresponding to the preset query field in the data set to be updated as the newly added index table data; the index information includes a pointer to the comprehensive data table.
[0125] The index table updating module 3 is also used to update the index table based on the newly added index table data.
[0126] In one embodiment, referring to Figure 7 The data processing system further includes: a data query module 4, which is connected to the index table updating module 3. The data query module 4 is used to:
[0127] A data query request sent by a query system is obtained; the data query request includes at least one query field.
[0128] According to the data query request, the target index information corresponding to the query field in the index table is determined.
[0129] Based on the target index information, the basic data to be queried is obtained from the comprehensive data table.
[0130] Return the basic data to be queried to the query system.
[0131] In one embodiment, referring to Figure 6 and Figure 7 , the comprehensive data table update module 2 is also used for:
[0132] A request for creating a comprehensive data table is obtained; the request for creating a comprehensive data table includes interface information of the business system and basic data keywords.
[0133] Establish a connection with the business system based on interface information.
[0134] Based on the basic data keywords, obtain the basic data in each business system.
[0135] Based on the business system keyword mapping table, the basic data corresponding to the same order number is integrated into an initial data set.
[0136] Generate a comprehensive data table from the initial data set.
[0137] In one embodiment, referring to Figure 6 and Figure 7 , the index table updating module 3 is also used for:
[0138] Get an index table creation request; the index table creation request includes a query field.
[0139] Get the basic data corresponding to the query field in the comprehensive data table and the index information corresponding to the basic data.
[0140] An index table is generated according to the query fields, basic data and index information; the index table stores the association relationship between the query fields, basic data and index information.
[0141] In one embodiment, referring to Figure 6 and Figure 7 , the comprehensive data table update module 2 is also used for:
[0142] Based on the basic data keywords, the business system keywords corresponding to each business system are determined; the business system keywords include the order number.
[0143] The business system whose business system keyword includes the order number is determined as the primary business system, and the business systems other than the primary business system are determined as secondary business systems.
[0144] Determine whether the business system keyword corresponding to the slave business system is the same as the business system keyword corresponding to the master business system.
[0145] If a business system keyword corresponding to a slave business system is the same as a business system keyword corresponding to a master business system, the same business system keyword is determined as the target business system keyword.
[0146] A business system keyword mapping table is established based on the target business system keyword; the business system keyword mapping table is used to characterize the association relationship between the main business system and each slave business system.
[0147] In one embodiment, referring to Figure 7 , the data query module 4 is also used for:
[0148] The basic data to be queried is encrypted based on a preset encryption method to obtain the encrypted basic data to be queried.
[0149] Send the encrypted basic data to be queried to the query system.
[0150] An embodiment of the present invention provides a data processing system, in which the data stored in different systems is integrated into a comprehensive data table, and an index table is generated based on the comprehensive data table, so as to improve the efficiency of data query and the accuracy of data query, thereby improving the user experience.
[0151] Embodiment three:
[0152] Figure 8 This is a schematic diagram of a non-real-time data query device provided in Embodiment 3 of the present invention.
[0153] Reference Figure 8 The non-real-time data query device includes: at least one business system 5 and a query system 6; it also includes the above-mentioned data processing system 7; the data processing system 7 is connected to the business system 5 and the query system 6 respectively.
[0154] The embodiment of the present invention provides a non-real-time data query device, which can query data at any historical time and latitude, improve data query efficiency, improve data query accuracy, and thus enhance user experience.
[0155] An embodiment of the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the data processing method provided in the above embodiment are implemented.
[0156] An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. The computer-readable medium stores a computer program, and when the computer program is executed by a processor, the steps of the data processing method of the above embodiment are executed.
[0157] The computer program product provided in the embodiment of the present invention includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the method described in the previous method embodiment. The specific implementation can be found in the method embodiment and will not be repeated here.
[0158] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system and device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.
[0159] In addition, in the description of the embodiments of the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.
[0160] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.
[0161] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.
[0162] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.
Claims
1. A data processing method, It is characterized in that Applied to a data processing system, the data processing system is connected to at least one peripheral business system; the data processing method comprises: Based on a preset basic data keyword, acquiring new basic data in each of the business systems within a current time interval according to a preset time interval; the basic data keyword includes an order number; Based on the business system keyword mapping table, the newly added basic data corresponding to the same order number are integrated into a data set to be updated, and a comprehensive data table is updated based on each data set to be updated; the comprehensive data table stores the association relationship between the basic data keywords and the basic data; For each of the data sets to be updated, based on the updated comprehensive data table, determining that the newly added basic data and index information corresponding to the preset query field in the data set to be updated are newly added index table data; the index information includes a pointer to the comprehensive data table; The index table is updated based on the newly added index table data.
2. The data processing method according to claim 1, It is characterized in that The data processing system is also connected to a query system of an external device; the method further comprises: Acquire a data query request sent by the query system; the data query request includes at least one of the query fields; Determine, according to the data query request, target index information corresponding to the query field in the index table; Based on the target index information, query the comprehensive data table to obtain the basic data to be queried; Return the basic data to be queried to the query system.
3. The data processing method according to claim 1, It is characterized in that The comprehensive data table is set up in the following way: Obtaining a comprehensive data table creation request; the comprehensive data table creation request includes the interface information of the business system and the basic data keyword; Establishing a connection with the business system based on the interface information; Based on the basic data keyword, acquiring the basic data in each of the business systems; Based on the business system keyword mapping table, the basic data corresponding to the same order number are integrated into an initial data set; The comprehensive data table is generated according to the initial data set.
4. The data processing method according to claim 3, It is characterized in that The index table is set up in the following way: Obtaining an index table creation request; the index table creation request includes the query field; Acquire the basic data corresponding to the query field in the comprehensive data table and the index information corresponding to the basic data; Generate the index table according to the query field, the basic data and the index information; The index table stores the association relationship between the query field, the basic data and the index information.
5. The data processing method according to claim 3, It is characterized in that After the step of acquiring the basic data in each of the business systems based on the basic data keyword, the method further includes: Based on the basic data keyword, determining a business system keyword corresponding to each of the business systems; the business system keyword includes the order number; Determine that the business system whose business system keyword includes the order number is the primary business system, and the business systems other than the primary business system are secondary business systems; Determine whether the business system keyword corresponding to the slave business system is the same as the business system keyword corresponding to the master business system; If a business system keyword corresponding to one of the slave business systems is the same as a business system keyword corresponding to one of the master business systems, determining the same business system keyword as a target business system keyword; The business system keyword mapping table is established based on the target business system keyword; the business system keyword mapping table is used to characterize the association relationship between the master business system and each of the slave business systems.
6. The data processing method according to claim 2, It is characterized in that The step of returning the basic data to be queried to the query system includes: Encrypting the basic data to be queried based on a preset encryption method to obtain encrypted basic data to be queried; The encrypted basic data to be queried is sent to the query system.
7. A data processing system, It is characterized in that The data processing system is connected to at least one peripheral business system; the data processing system comprises: A basic data acquisition module, used to acquire new basic data in each of the business systems within a current time interval according to a preset time interval based on a preset basic data keyword; the basic data keyword includes an order number; A comprehensive data table updating module is used to integrate the newly added basic data corresponding to the same order number into a data set to be updated based on the business system keyword mapping table, and update the comprehensive data table based on each data set to be updated; the comprehensive data table stores the association relationship between the basic data keywords and the basic data; An index table updating module, for each of the data sets to be updated, based on the updated comprehensive data table, determines that the newly added basic data and index information corresponding to the preset query field in the data set to be updated are newly added index table data; the index information includes a pointer to the comprehensive data table; The index table updating module is further used to update the index table based on the newly added index table data.
8. A non-real-time data query device, It is characterized in that include: At least one business system and query system; also includes the data processing system described in claim 7; the data processing system is connected to the business system and the query system respectively.
9. An electronic device, It is characterized in that The method comprises a memory and a processor, wherein the memory stores a computer program executable on the processor, and is characterized in that the processor implements the data processing method described in any one of claims 1 to 6 when executing the computer program.
10. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, the data processing method according to any one of claims 1 to 6 is implemented.
