A service data query method and device, electronic equipment and storage medium

Abstract

The application relates to the technical field of data processing, in particular to a business data query method and device, electronic equipment and a storage medium, to improve the query efficiency of business data. The method comprises the following steps: receiving a business data query request, the business data query request carrying a target business identifier of a target business to be queried and a column identifier set; acquiring a distributed storage table, the distributed storage table containing the business identifiers of various resource transfer businesses and various business attribute data, each business identifier being a unique identifier of each row of data in the distributed storage table, each row of data in the distributed storage table containing various business attribute data of a corresponding resource transfer business, and each column of data in the distributed storage table containing one kind of business attribute data; and acquiring at least one business attribute data corresponding to the target business identifier and the column identifier set from the distributed storage table as target data. In this way, the distributed storage table is used to query business data, and the query efficiency is improved.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a business data query method, apparatus, electronic device, and storage medium. Background Technology

[0002] Currently, with the rapid development of banking services, mobile online platforms provide users with resource transfer services such as fund services, wealth management services, insurance services, and deposit services. The business data of mobile online platforms is also becoming increasingly massive, with an average daily transaction volume of nearly 2 million, a peak online transaction volume per minute (Transactions Per Minute, TPM) of nearly 3,000, and a central processing unit (CPU) utilization rate of 50%, which is approaching the utilization rate threshold. The average transaction response time reaches 60ms.

[0003] In related technologies, when querying business data, it is common to query fields such as basic fund information, basic investment information, and yield indicators in relational databases (Oracle). These fields are stored in different Oracle tables.

[0004] However, when querying business data using this method in related technologies, the application server needs to access each related Oracle table because the Oracle database needs to store the data in tables with relationships between them. This increases the application server's access overhead to the Oracle database, resulting in longer response times, slower queries, and higher CPU usage.

[0005] Therefore, the efficiency of querying business data under the relevant technologies needs to be further improved. Summary of the Invention

[0006] This application provides a business data query method, apparatus, electronic device, and storage medium to improve the efficiency of business data query.

[0007] The specific technical solutions provided in this application are as follows:

[0008] Firstly, a business data query method is provided, including:

[0009] Receive a business data query request, which carries the target business identifier and column identifier set of the target business to be queried.

[0010] Obtain the distributed storage table, which contains: the business identifier and business attribute data of each resource transfer service. Each business identifier is a unique identifier for each row of data in the distributed storage table. Each row of data in the distributed storage table contains the business attribute data of the corresponding resource transfer service. Each column of data in the distributed storage table contains one type of business attribute data.

[0011] From the distributed storage table, obtain at least one business attribute data corresponding to the target business identifier and column identifier set as the target data.

[0012] Optionally, before receiving a business data query request, the method further includes:

[0013] Obtain the source data file set corresponding to each information type. Each source data file set contains: the business identifier and business attribute data of at least one resource transfer business under the corresponding information type.

[0014] For each obtained source data file set, perform the following operations respectively: Batch process the business identifier and business attribute data of at least one resource transfer service contained in a source data file set, and map the obtained business identifier and business attribute data to the target data table associated with the information type of the source data file set respectively;

[0015] For each target data table obtained, perform the following operations: According to the preset table entries, map the business attribute data of each resource transfer business contained in a target data table to a column of data in a pre-set distributed storage table, and use the business identifier of each resource transfer business contained in a target data table as the unique identifier of the corresponding row of data in the distributed storage table.

[0016] Optionally, each information type shall include at least: basic business types, business number types, and business indicator types;

[0017] Then obtain the source data file set corresponding to each information type, including:

[0018] Obtain the first source data file set corresponding to each basic business type. Each first source data file set contains: basic attribute data and business identifier of at least one resource transfer business under the corresponding basic business type.

[0019] Obtain the second source data file set corresponding to the business number type, wherein the second source data file set contains: number attribute data and business identifier of at least one resource transfer business under the business number type;

[0020] Obtain the third source data file set corresponding to the business indicator type. The third source data file set contains: indicator attribute data and business identifiers for at least one resource transfer business under the business indicator type.

[0021] Optionally, batch processing is performed on the service identifiers and service attribute data of at least one resource transfer service contained in a source data file set. The obtained service identifiers and service attribute data are then mapped to target data tables associated with the information types corresponding to the source data file set, including:

[0022] For each basic business type, the following operations are performed: take the business identifiers and basic attribute data of the first source data file set corresponding to a basic business type as the first batch of data, and map the first batch of data to the target data table associated with the corresponding basic business type based on the first number of processing times and the first processing time.

[0023] The business identifiers and number attribute data of the second source data file set are used as the second batch data. Based on the second number of processing times and the second processing time, the second batch data is mapped to the target data table associated with the business number type.

[0024] The business identifiers and indicator attribute data of the third source data file set are used as the third batch data. Based on the third processing number and the third processing time, the third batch data is mapped to the target data table associated with the business indicator type.

[0025] Optionally, according to preset table entries, one type of business attribute data from each resource transfer service contained in a target data table is mapped to a column of data in a pre-set distributed storage table, including:

[0026] Convert the data format of the business identifiers and business attribute data of each resource transfer business contained in a target data table;

[0027] Based on the preset table entries, the transformed business attribute data is stored in the corresponding data columns of the pre-set distributed storage table.

[0028] Optionally, from the distributed storage table, at least one business attribute data corresponding to the target business identifier and column identifier set is obtained as the target data, including:

[0029] From the distributed storage table, filter out all data columns corresponding to the target business identifier;

[0030] Extract at least one business attribute data corresponding to the column identifier set from all data columns as the target data.

[0031] Secondly, a business data query device is provided, comprising:

[0032] The receiving module is used to receive business data query requests, which carry the target business identifier and column identifier set of the target business to be queried.

[0033] The acquisition module is used to acquire the distributed storage table. The distributed storage table contains: the business identifier and business attribute data of each resource transfer service. Each business identifier is a unique identifier for each row of data in the distributed storage table. Each row of data in the distributed storage table contains the business attribute data of the corresponding resource transfer service. Each column of data in the distributed storage table contains one type of business attribute data.

[0034] The determination module is used to obtain at least one business attribute data corresponding to the target business identifier and column identifier set from the distributed storage table as the target data.

[0035] Optionally, before receiving a business data query request, the device further includes a storage module, the storage module being used for:

[0036] Obtain the source data file set corresponding to each information type. Each source data file set contains: the business identifier and business attribute data of at least one resource transfer business under the corresponding information type.

[0037] For each obtained source data file set, perform the following operations respectively: Batch process the business identifier and business attribute data of at least one resource transfer service contained in a source data file set, and map the obtained business identifier and business attribute data to the target data table associated with the information type of the source data file set respectively;

[0038] For each target data table obtained, perform the following operations: According to the preset table entries, map the business attribute data of each resource transfer business contained in a target data table to a column of data in a pre-set distributed storage table, and use the business identifier of each resource transfer business contained in a target data table as the unique identifier of the corresponding row of data in the distributed storage table.

[0039] Optionally, each information type shall include at least: basic business types, business number types, and business indicator types;

[0040] When retrieving the source data file sets corresponding to each information type, the storage module is also used for:

[0041] Obtain the first source data file set corresponding to each basic business type. Each first source data file set contains: basic attribute data and business identifier of at least one resource transfer business under the corresponding basic business type.

[0042] Obtain the second source data file set corresponding to the business number type, wherein the second source data file set contains: number attribute data and business identifier of at least one resource transfer business under the business number type;

[0043] Obtain the third source data file set corresponding to the business indicator type. The third source data file set contains: indicator attribute data and business identifiers for at least one resource transfer business under the business indicator type.

[0044] Optionally, when batch processing the service identifiers and service attribute data of at least one resource transfer service contained in a source data file set, and mapping the obtained service identifiers and service attribute data to the target data table associated with the information type corresponding to the source data file set, the storage module is further configured to:

[0045] For each basic business type, the following operations are performed: take the business identifiers and basic attribute data of the first source data file set corresponding to a basic business type as the first batch of data, and map the first batch of data to the target data table associated with the corresponding basic business type based on the first number of processing times and the first processing time.

[0046] The business identifiers and number attribute data of the second source data file set are used as the second batch data. Based on the second number of processing times and the second processing time, the second batch data is mapped to the target data table associated with the business number type.

[0047] The business identifiers and indicator attribute data of the third source data file set are used as the third batch data. Based on the third processing number and the third processing time, the third batch data is mapped to the target data table associated with the business indicator type.

[0048] Optionally, when mapping business attribute data of various resource transfer services contained in a target data table to a column of data in a pre-set distributed storage table according to preset table entries, the storage module is further used for:

[0049] Convert the data format of the business identifiers and business attribute data of each resource transfer business contained in a target data table;

[0050] Based on the preset table entries, the transformed business attribute data is stored in the corresponding data columns of the pre-set distributed storage table.

[0051] Optionally, when retrieving at least one business attribute data corresponding to the target business identifier and column identifier set from the distributed storage table as the target data, the determining module is further configured to:

[0052] From the distributed storage table, filter out all data columns corresponding to the target business identifier;

[0053] Extract at least one business attribute data corresponding to the column identifier set from all data columns as the target data.

[0054] Thirdly, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the method described in any of the first aspects above.

[0055] Fourthly, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in any of the first aspects above.

[0056] Fifthly, a computer program product is provided, the computer program product comprising a computer program stored in a computer-readable storage medium; when a processor of an electronic device reads the computer program from the computer-readable storage medium, the processor executes the computer program, causing the electronic device to perform the steps of the method described in any of the first aspects above.

[0057] In this embodiment, the service device receives a service data query request sent by the terminal device. The service data query request carries the target service identifier and column identifier set of the target service to be queried. A distributed storage table is obtained. The distributed storage table contains: the service identifier and service attribute data of each resource transfer service. Each service identifier is a unique identifier for each row of data in the distributed storage table. Each row of data in the distributed storage table contains service attribute data of the corresponding resource transfer service. Each column of data in the distributed storage table contains one type of service attribute data. From the distributed storage table, at least one service attribute data corresponding to the target service identifier and column identifier set is obtained as the target data. Thus, each service identifier is a unique identifier for each row of data in the distributed storage table, and each row of data in the distributed storage table contains service attribute data of the corresponding resource transfer service. By querying the target service data from the distributed storage table using the target service identifier and column identifier set, the service attribute data of the target service to be queried can be quickly retrieved, improving the query efficiency of service data and enabling the output of one or more columns of data specified by the target service identifier. Attached Figure Description

[0058] Figure 1 This is a schematic diagram of the application scenario in the embodiments of this application;

[0059] Figure 2 This is a schematic diagram of the first process of a business data query method in an embodiment of this application;

[0060] Figure 3 This is a schematic diagram of the process for obtaining each source data file set in an embodiment of this application;

[0061] Figure 4 This is a schematic diagram illustrating the acquisition of various source data file sets in an embodiment of this application;

[0062] Figure 5 This is a schematic diagram of the process for obtaining each target data table in an embodiment of this application;

[0063] Figure 6 This is a schematic diagram illustrating the acquisition of each target data table in an embodiment of this application;

[0064] Figure 7 This is a schematic diagram illustrating the design of a distributed storage table in an embodiment of this application;

[0065] Figure 8 This is a schematic diagram of the process for storing business attribute data in an embodiment of this application;

[0066] Figure 9 This is a schematic diagram illustrating data format conversion in an embodiment of this application;

[0067] Figure 10 This is a schematic diagram illustrating the storage of business attribute data in an embodiment of this application;

[0068] Figure 11 This is a schematic diagram of the second process of business data query in an embodiment of this application;

[0069] Figure 12 This is a schematic diagram of a distributed storage table in an embodiment of this application;

[0070] Figure 13 This is a flowchart illustrating the process of determining target data in an embodiment of this application;

[0071] Figure 14 This is a schematic diagram illustrating the determination of target data in an embodiment of this application;

[0072] Figure 15 This is a schematic diagram of the structure of the business data query device in the embodiments of this application;

[0073] Figure 16 This is a schematic diagram of the structure of the electronic device in the embodiments of this application. Detailed Implementation

[0074] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0075] The following explanations of some terms used in the embodiments of this application are provided to facilitate understanding by those skilled in the art.

[0076] (1) Distributed Database (Hadoop Database, HBase): HBase is a column-oriented distributed database. Its underlying storage is based on the Hadoop Distributed File System (HDFS), and cluster management is based on ZooKeeper. HBase's robust distributed architecture enables rapid storage and random access to massive amounts of data. Based on data replication and partitioning mechanisms, online scaling up, scaling down, and data disaster recovery can be easily achieved. HBase features easy scalability, massive storage, column-oriented storage, high reliability, and sparsity.

[0077] (2) Hive: Hive is a data warehouse tool based on Haplo Hadoop, used for data extraction, transformation, and loading. It is a mechanism for storing, querying, and analyzing large-scale data stored in Hadoop. Hive can map structured data files to a database table and provides Structured Query Language (SQL) query functionality, which can transform SQL statements into simplified MapReduce tasks for execution. Hive is suitable for data warehouse applications that perform related static data analysis, do not require fast response to provide results, and where the data itself does not change frequently.

[0078] (3) Spark: Spark is a fast and general-purpose big data processing framework. Spark is based on in-memory computing and introduces the Database Availability Group (DAG) execution engine, which has advantages over MapReduce such as faster computing speed, data locality, scheduling optimization and transmission optimization.

[0079] (4) DAT file: Data file. DAT is a text file extension. Most database software has the function of converting its own formatted data files into text format files, one of which has the extension .dat. This type of text file is plain text and does not contain information about data attributes or structure.

[0080] The preferred embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0081] like Figure 1The diagram illustrates an application scenario of an embodiment of this application. The application scenario diagram includes a service device 110 and terminal devices 120 (including terminal devices 1201, 1202, ..., 120n). The service device 110 receives a service data query request sent by the terminal devices 120. Upon receiving the service data query request, the service device 110 obtains a distributed storage table. The distributed storage table contains: the service identifier and service attribute data for each resource transfer service. Each service identifier is a unique identifier for each row of data in the distributed storage table. Each row of data in the distributed storage table contains service attribute data for the corresponding resource transfer service. Each column of data in the distributed storage table contains one type of service attribute data. Finally, from the distributed storage table, at least one service attribute data corresponding to the target service identifier and column identifier set is obtained as the target data.

[0082] Service device 110 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. Terminal device 120 and service device 110 can be directly or indirectly connected via wired or wireless communication, which is not limited herein.

[0083] Terminal device 120 can be a smartphone, tablet, laptop, desktop computer, smart TV, etc., but is not limited to these.

[0084] Based on the above embodiments, see Figure 2 The diagram shown is a first flowchart of a business data query method according to an embodiment of this application, specifically including:

[0085] Step 20: Obtain the source data file set corresponding to each information type.

[0086] Each source data file set contains: a business identifier and various business attribute data for at least one resource transfer service under the corresponding information type.

[0087] In this embodiment of the application, the service device obtains the source data file set corresponding to each information type from the corresponding upstream system.

[0088] The information types include at least: basic business types, business number types, business indicator types, etc. The source data file can be a DAT file, and this application embodiment does not impose any restrictions on this.

[0089] Specifically, during step 20, the service device performs the following operations. (See also...) Figure 3 As shown, this is a schematic diagram of the process for obtaining each source data file set in an embodiment of this application. The following is a detailed explanation in conjunction with the attached diagram. Figure 3 The specific operations to be performed will be explained in detail:

[0090] Step 200: Obtain the first source data file set corresponding to each basic business type.

[0091] Each first source data file set contains: basic attribute data and business identifiers for at least one resource transfer business under the corresponding basic business type.

[0092] In addition, the basic types of each business include at least: basic types of fund business, basic types of wealth management business, basic types of deposit business, basic types of insurance business, etc., and the basic attribute data includes at least: business name, business term, business status, business risk level, business currency, etc., but this application embodiment does not impose any restrictions on this.

[0093] In this embodiment of the application, the following operations are performed for each basic business type: obtain the first source data file set corresponding to the basic business type from the upstream system corresponding to the basic business type.

[0094] For details, please refer to Figure 4 The diagram illustrates the acquisition of various source data file sets in this embodiment. From the upstream fund system corresponding to the basic fund business type, the first source data file set corresponding to the basic fund business type is accessed to obtain the business name, business term, business status, business risk level, and business currency of each fund resource transfer business. From the upstream wealth management system corresponding to the basic wealth management business type, the first source data file set corresponding to the basic wealth management business type is accessed to obtain the business name, business term, business status, business risk level, and business currency of each wealth management resource transfer business. From the upstream deposit system corresponding to the basic deposit business type, the first source data file set corresponding to the basic deposit resource transfer business type is accessed to obtain the business name, business term, business status, business risk level, and business currency of each deposit business. From the upstream insurance system corresponding to the basic insurance business type, the first source data file set corresponding to the basic insurance business type is accessed to obtain the business name, business term, business status, business risk level, and business currency of each insurance resource transfer business.

[0095] For example, the first source data file set obtained from the upstream fund system corresponding to the basic types of fund business includes: Business ID of Fund Resource Transfer Business 1: 00001, Business Name: Fund Product 1, Business Term: 1127 days, Business Status: On Sale, Business Risk Level: R2 (Low to Medium Risk), Business Currency: RMB; Business ID of Fund Resource Transfer Business 2: 00001, Business Name: Fund Product 2, Business Term: 351 days, Business Status: On Sale, Business Risk Level: R1 (Low to Medium Risk), Business Currency: HKD; Business ID of Fund Resource Transfer Business 3: 00001, Business Name: Fund Product 3, Business Term: None, Business Status: On Sale, Business Risk Level: R2 (Low to Medium Risk), Business Currency: USD.

[0096] Step 201: Obtain the second source data file set corresponding to the business number type.

[0097] The second source data file set contains: number attribute data and business identifier for at least one resource transfer business under the business number type.

[0098] In addition, the number attribute data includes at least the business code and business number, but this application embodiment does not impose any restrictions on this.

[0099] In this embodiment of the application, the second source data file set corresponding to the business number type is obtained from the upstream system corresponding to the business number type.

[0100] Specifically, such as Figure 4 As shown, the system accesses the second source data file set corresponding to the business number type from the upstream product platform system to obtain the business code and business number of each resource transfer business.

[0101] For example, the second source data file set obtained from the upstream product platform system corresponding to the business number type includes: Business identifier: 00001, Business code: JJJC***21, Business number: 1 for Fund Resource Transfer Business 1; Business identifier: 00002, Business code: QQZZ***1A, Business number: 2 for Fund Resource Transfer Business 2; Business identifier: 00003, Business code: LXTT***1B, Business number: 3 for Fund Resource Transfer Business 3; Business identifier: 00004, Business code: LXTT***1B, Business number: 3 for Wealth Management Resource Transfer Business 1; and Business ID: 00004. Business Code: WFZQ***10, Business Number: 4, Business Identifier for Wealth Management Resource Transfer Business 2: 00005, Business Code: HMYX***10, Business Number: 5, Business Identifier for Deposit Resource Transfer Business 1: 00006, Business Code: AMRS***10, Business Number: 6, Business Identifier for Deposit Resource Transfer Business 2: 00007, Business Code: GRSD***75, Business Number: 7, Business Identifier for Insurance Resource Transfer Business 1: 00008, Business Code: AGSB***0, Business Number: 8.

[0102] Step 202: Obtain the third source data file set corresponding to the business indicator type.

[0103] The third source data file set contains: under the business indicator type, the indicator attribute data and business identifier of at least one resource transfer business.

[0104] In addition, the indicator attribute data includes at least: business return rate and business net value, etc., but this application embodiment does not impose any restrictions on this.

[0105] In this embodiment of the application, a third source data file set corresponding to the business indicator type is obtained from the upstream system corresponding to the business indicator type.

[0106] Specifically, such as Figure 4 As shown, the system accesses the third source data file set corresponding to the business indicator type from the upstream fund system and upstream wealth management system to obtain the business rate of return and net asset value of each fund resource transfer business and each wealth management resource transfer business.

[0107] For example, the third-source data file set obtained from the upstream fund system and upstream wealth management system corresponding to the business indicator types includes: Business ID of Fund Resource Transfer Business 1: 00001, Business Yield: 3.41%, Business Net Value: 1.0000; Business ID of Fund Resource Transfer Business 2: 00002, Business Yield: 3.10%, Business Net Value: 1.0101; Business ID of Fund Resource Transfer Business 3: 00003, Business Yield: 3.22%, Business Net Value: 1.0104; Business ID of Wealth Management Resource Transfer Business 1: 00004, Business Yield: 2.94%, Business Net Value: 1.0532; Business ID of Wealth Management Resource Transfer Business 2: 00005, Business Yield: 3.08%, Business Net Value: 1.0161.

[0108] Step 21: For each obtained source data file set, perform the following operations respectively: Batch process the business identifier and business attribute data of at least one resource transfer service contained in a source data file set, and map the obtained business identifier and business attribute data to the target data table associated with the information type of the source data file.

[0109] In this embodiment of the application, after obtaining the source data file sets corresponding to each information type, the following operations are performed on each obtained source data file set: according to the preset processing time and number of processing times, the service identifiers and service attribute data of at least one resource transfer service contained in a source data file set are processed in batches, and the obtained service identifiers and service attribute data are mapped to the target data table associated with the information type corresponding to the source data file set.

[0110] The target data table can be a Hive table, and this embodiment does not impose any restrictions on it.

[0111] Specifically, during step 21, the service device performs the following operations. (See also...) Figure 5 As shown, this is a flowchart illustrating the process of obtaining each target data table in an embodiment of this application. The following is a detailed explanation in conjunction with the attached diagram. Figure 5 The specific operations to be performed will be explained in detail:

[0112] Step 210: For each basic business type, perform the following operations: take the business identifiers and basic attribute data of the first source data set corresponding to a basic business type as the first batch of data, and map the first batch of data to the target data table associated with the corresponding basic business type based on the first number of processing times and the first processing time.

[0113] In this embodiment of the application, for each of the first source data file sets corresponding to each basic business type, the following operations are performed respectively: taking each business identifier and each basic attribute data of the first source data file set corresponding to a basic business type as the first batch of data, and processing the first batch of data into the target data table associated with the basic business type through Spark SQL based on the first number of processing times and the first processing time.

[0114] The first number of processing times and the first processing time can be set according to the actual situation, and there is no limitation on them in this embodiment.

[0115] For example, see Figure 6 The diagram shown illustrates the acquisition of target data tables in this embodiment of the application. Assuming the first processing time is 1 time and the first processing time is 1 AM every day, then at 1 AM every day, Spark SQL processes the first batch of data corresponding to the basic type of fund business into the target data table associated with the basic type of fund business; Spark SQL processes the first batch of data corresponding to the basic type of wealth management business into the target data table associated with the basic type of wealth management business; Spark SQL processes the first batch of data corresponding to the basic type of deposit business into the target data table associated with the basic type of deposit business; and Spark SQL processes the first batch of data corresponding to the basic type of insurance business into the target data table associated with the basic type of insurance business.

[0116] Step 211: Take the business identifiers and number attribute data of the second source data file set as the second batch data, and map the second batch data to the target data table associated with the business number type based on the second processing number and the second processing time.

[0117] In this embodiment of the application, after obtaining the second source data file set, the business identifiers and number attribute data of the second source data file set are used as the second batch data. Based on the second number of processing times and the second processing time, the second batch data is processed into the target data table associated with the business number type using SparkSQL.

[0118] The second number of processing steps and the second processing time can be set according to the actual situation. The second number of processing steps and the first number of processing steps can be the same, and the second processing time and the first processing time can be the same.

[0119] For example, such as Figure 6 As shown, assuming the second processing time is 1 time and the second processing time is 1 AM every day, then at 1 AM every day, Spark SQL processes the second batch of data corresponding to the business number type into the target data table associated with the business number type.

[0120] Step 212: Take the business identifiers and indicator attribute data of the third source data file set as the third batch data, and map the third batch data to the target data table associated with the business indicator type based on the third processing number and the third processing time.

[0121] In this embodiment of the application, after obtaining the third source data file set, the business identifiers and number attribute data of the third source data file set are used as the second batch of data. Based on the third processing number and the third processing time, the third batch of data is processed into the target data table associated with the business indicator type by SparkSQL.

[0122] The number of third processing steps and the third processing time can be set according to the actual situation. It is worth noting that the number of third processing steps is greater than the number of first and second processing steps.

[0123] For example, such as Figure 6 As shown, assuming the third processing time is 4 times, and the third processing time is 2:00, 6:00, 8:00 and 14:00 every day, then at 2:00, 6:00, 8:00 and 14:00 every day, the third batch of data corresponding to the business indicator type is processed by Spark SQL and sent to the target data table associated with the business indicator type.

[0124] In this way, each source data file set is processed in batches based on preset processing times and processing times, ensuring the real-time synchronization of business attribute data. Furthermore, the number of processing times for business indicator types is greater than that for other information types, ensuring that the business indicator data for fund resource transfer business and wealth management resource transfer business are consistent with the upstream system, making the business indicator data queried by users more accurate.

[0125] Step 22: For each of the obtained target data tables, perform the following operations: According to the preset table entries, map the business attribute data of each resource transfer business contained in a target data table to a column of data in a pre-set distributed storage table, and use the business identifier of each resource transfer business contained in a target data table as the unique identifier of the corresponding row of data in the distributed storage table.

[0126] In this embodiment, after obtaining each target data table, according to preset table entries, the basic attribute data of each resource transfer service contained in the target data table corresponding to each basic business type is mapped to a column of data in a preset distributed storage table. The business identifier of each resource transfer service contained in the target data table corresponding to each basic business type is used as a unique identifier for the corresponding row of data in the distributed storage table. According to preset table entries, the number attribute data of each resource transfer service contained in the target data table corresponding to the business number type is mapped to a column of data in the preset distributed storage table. Similarly, according to preset table entries, the indicator attribute data of each resource transfer service contained in the target data table corresponding to the business indicator type is mapped to a column of data in the preset distributed storage table.

[0127] The pre-configured distributed storage table can be an HBase table. The business identifier of each resource transfer business is used as the row key of the distributed storage table, and column families are designed to distribute business attribute data on the column families.

[0128] For example, see Figure 7 The diagram shown is a schematic of the distributed storage table design in this embodiment of the application. The column families include at least: basic, number, and target. Basic attribute data is distributed on the column family basic, number attribute data is distributed on the column family number, and indicator attribute data is distributed on the column family target. The preset table entries include at least: business name, business term, business status, business risk level, business currency, business code, business number, business yield, and business net value. The business identifier of each resource transfer business is used as the unique identifier of the corresponding row of data in the distributed storage table.

[0129] In this way, designing a distributed storage table can improve the retrieval efficiency of row keys and reduce the overhead of accessing wide rows, thereby improving the query efficiency of business data. It also facilitates the horizontal expansion of business attribute data of resource transfer businesses of different business types, making the design more flexible. If a new resource transfer business comes in, the distributed storage table design can be reused.

[0130] Specifically, when mapping business attribute data of various resource transfer services contained in a target data table to a column of data in a pre-set distributed storage table according to preset table entries, the service device performs the following operations. (See also...) Figure 8 As shown, it is a flowchart illustrating the process of storing business attribute data in an embodiment of this application. The following is a detailed explanation in conjunction with the attached diagram. Figure 8 The specific operations to be performed will be explained in detail:

[0131] Step 220: Convert the data format of the business identifiers and business attribute data of each resource transfer business contained in a target data table.

[0132] In this embodiment of the application, the business identifiers and business attribute data of each resource transfer service contained in a target data table are converted according to a preset data format to obtain the converted business attribute data.

[0133] The preset data format can be key-value format, and this application embodiment does not impose any restrictions on it.

[0134] For example, see Figure 9 The diagram illustrates the data format conversion in this embodiment. Following the key-value format, the business identifiers and attribute data of each resource transfer business contained in the target data table corresponding to the business indicator type are converted. Assuming the target data table contains the business identifier of Fund Resource Transfer Business 1: 00001, business return rate: 3.10%, business net value: 1.0000, and the business identifier of Fund Resource Transfer Business 2: 00002, business return rate: 3.41%, business net value: 1.0101, then the converted business indicator data are: {rowkey: 00001, columnfamily: target, column qualifier: business return rate, value: 3.10%}, {rowkey: 00002, columnfamily: target, column qualifier: business return rate, value: 3.41%}, {rowkey: 00001 ...10%}, {rowkey: 00002, columnfamily: target, column qualifier: business return rate, value: 3.41%}, {rowkey: 00001, columnfamily: target, column qualifier: business return rate, value: 3.41%}, {rowkey: 00001, columnfamily: target, column qualifier: business return rate, value: 3.41%}, {rowkey: 00001, columnfamily: target, column { qualifier: net business value, value: 1.0000}, { rowkey: 00002, columnfamily: target, column qualifier: net business value, value: 1.0101}.

[0135] Step 221: Based on the preset table entries, store the transformed business attribute data into the corresponding data columns of the pre-set distributed storage table.

[0136] In this embodiment of the application, after obtaining the converted business attribute data, the converted business attribute data is stored in the corresponding data column of a pre-set distributed storage table based on preset table entries.

[0137] For example, see Figure 10The diagram shown is a schematic of storing business attribute data in an embodiment of this application. It is assumed that the preset table entries include at least: business name, business term, business status, business risk level, business currency, business code, business number, business yield, and business net value. The converted business attribute data are: {rowkey: 00001, column family: target, column qualifier: business yield, value: 3.10%}, {rowkey: 00002, column family: target, column qualifier: business yield, value: 3.41%}, {rowkey: 00001, column family: target, column qualifier: business net value, value: 1.0000}, {rowkey: 00002, column family: target, column qualifier: business net value, value: 1.0000}, {rowkey: 00002, column family: target, column qualifier: business yield, value: 1.0000}, {rowkey: 00002, column family: target, column qualifier: business risk level ... If qualifier: net business value, value: 1.0101}, then 3.10% will be stored in the data column corresponding to the business yield rate and the data row corresponding to row key 00001 in the distributed storage table; 3.41% will be stored in the data column corresponding to the business yield rate and the data row corresponding to row key 00002 in the pre-set distributed storage table; 1.0000 will be stored in the data column corresponding to the net business value and the data row corresponding to row key 00001 in the distributed storage table; and 1.0101 will be stored in the data column corresponding to the net business value and the data row corresponding to row key 00002 in the distributed storage table.

[0138] Furthermore, in this embodiment of the application, after establishing the distributed storage table, business data can be queried from the distributed storage table, see [reference]. Figure 11 As shown, this is a schematic diagram of the second process of business data query in an embodiment of this application. The following is a detailed explanation in conjunction with the attached diagram. Figure 11 The specific operations to be performed will be explained in detail:

[0139] Step 23: Receive business data query requests.

[0140] The business data query request carries the target business identifier and column identifier set of the target business to be queried.

[0141] In this embodiment of the application, the service device receives a service data query request sent by the terminal device.

[0142] For example, suppose the target business to be queried by the terminal device is fund business 3, and the business attribute data to be queried is yield and net value. The column identifier corresponding to the yield is 8, the column identifier corresponding to the net value is 9, and the target business identifier of fund business 3 is 00003. Then the business data query request carries the target business identifier 00003 and the column identifier set 8 and 9 of the target business.

[0143] Step 24: Obtain the distributed storage table.

[0144] The distributed storage table contains: the business identifier and business attribute data for each resource transfer service. Each business identifier is a unique identifier for each row of data in the distributed storage table. Each row of data in the distributed storage table contains the business attribute data for the corresponding resource transfer service. Each column of data in the distributed storage table contains one type of business attribute data.

[0145] In this embodiment of the application, a distributed storage table is obtained from a repository.

[0146] For example, see Figure 12 The diagram shown is a schematic of the distributed storage table in this embodiment of the application. The data types of each business attribute of each resource transfer business include at least: business name, business term, business status, business risk level, business currency, business code, business code, business yield, business net value, etc. This embodiment of the application does not impose any restrictions on these. The business identifier of each resource transfer business is a unique identifier of the corresponding row in the distributed storage table. Each column of data in the distributed storage table contains one type of business attribute data.

[0147] Step 25: From the distributed storage table, obtain at least one business attribute data corresponding to the target business identifier and column identifier set as the target data.

[0148] In this embodiment of the application, after obtaining the distributed storage table, based on the target business identifier and column identifier set, at least one business attribute data corresponding to the target business identifier and column identifier set is determined from the distributed storage table as the target data.

[0149] Specifically, during step 25, the service device performs the following operations. (See also...) Figure 13 As shown, this is a flowchart illustrating the process of determining target data in an embodiment of this application. The following is a detailed explanation in conjunction with the attached diagram. Figure 13 The specific operations to be performed will be explained in detail:

[0150] Step 250: From the distributed storage table, filter out all data columns corresponding to the target business identifier.

[0151] In this embodiment of the application, based on the target service identifier, all data columns corresponding to the target service identifier are filtered out from the distributed storage table.

[0152] For example, see Figure 14 The diagram shown is a schematic diagram of determining target data in an embodiment of this application. Assuming the target service identifier is 00003, all data columns corresponding to the target service identifier 00003 are filtered out from the distributed storage table.

[0153] Step 251: Extract at least one business attribute data corresponding to the column identifier set from all data columns as the target data.

[0154] In this embodiment of the application, after filtering out all data columns corresponding to the target business identifier, at least one business attribute data corresponding to the column identifier set is determined from all data columns as the target data based on the column identifier set.

[0155] For example, such as Figure 14 As shown, assuming the column identifier set is column identifier 8 and column identifier 9, the business attribute data corresponding to column identifier 8 is determined to be 3.22% and the business attribute data corresponding to column identifier 9 is 1.0104 from all data columns. 3.22% and 1.0104 are taken as target data.

[0156] In this way, when querying business data, the distributed storage table is retrieved, and the target business data is retrieved from the distributed storage table through the target business identifier and column identifier set. This realizes the output of one or more columns of data specified by the target business identifier, which improves the query efficiency of business data and makes it easier for users to quickly obtain the business attribute data they need.

[0157] Optionally, after retrieving at least one business attribute data corresponding to the target business identifier and column identifier set from the distributed storage table as the target data, the service device returns the target data to the terminal device and displays the query results to the user through the display interface.

[0158] Based on the same inventive concept, this application also provides a business data query device, see reference. Figure 15 The diagram shown is a structural schematic of the business data query device in an embodiment of this application, specifically including:

[0159] The receiving module 1501 is used to receive a business data query request, which carries the target business identifier and column identifier set of the target business to be queried.

[0160] The acquisition module 1502 is used to acquire the distributed storage table. The distributed storage table contains: the business identifier and business attribute data of each resource transfer service. Each business identifier is a unique identifier for each row of data in the distributed storage table. Each row of data in the distributed storage table contains the business attribute data of the corresponding resource transfer service. Each column of data in the distributed storage table contains one type of business attribute data.

[0161] The determination module 1503 is used to obtain at least one business attribute data corresponding to the target business identifier and column identifier set from the distributed storage table as the target data.

[0162] Optionally, before receiving a business data query request, the device further includes a storage module 1504, which is used for:

[0163] Obtain the source data file set corresponding to each information type. Each source data file set contains: the business identifier and business attribute data of at least one resource transfer business under the corresponding information type.

[0164] For each obtained source data file set, perform the following operations respectively: Batch process the business identifier and business attribute data of at least one resource transfer service contained in a source data file set, and map the obtained business identifier and business attribute data to the target data table associated with the information type of the source data file set respectively;

[0165] For each target data table obtained, perform the following operations: According to the preset table entries, map the business attribute data of each resource transfer business contained in a target data table to a column of data in a pre-set distributed storage table, and use the business identifier of each resource transfer business contained in a target data table as the unique identifier of the corresponding row of data in the distributed storage table.

[0166] Optionally, each information type shall include at least: basic business types, business number types, and business indicator types;

[0167] When retrieving the source data file sets corresponding to each information type, storage module 1504 is also used for:

[0168] Obtain the first source data file set corresponding to each basic business type. Each first source data file set contains: basic attribute data and business identifier of at least one resource transfer business under the corresponding basic business type.

[0169] Obtain the second source data file set corresponding to the business number type, wherein the second source data file set contains: number attribute data and business identifier of at least one resource transfer business under the business number type;

[0170] Obtain the third source data file set corresponding to the business indicator type. The third source data file set contains: indicator attribute data and business identifiers for at least one resource transfer business under the business indicator type.

[0171] Optionally, when batch processing the service identifiers and service attribute data of at least one resource transfer service contained in a source data file set, and mapping the obtained service identifiers and service attribute data to the target data table associated with the information type corresponding to the source data file set, the storage module 1504 is further configured to:

[0172] For each basic business type, the following operations are performed: take the business identifiers and basic attribute data of the first source data file set corresponding to a basic business type as the first batch of data, and map the first batch of data to the target data table associated with the corresponding basic business type based on the first number of processing times and the first processing time.

[0173] The business identifiers and number attribute data of the second source data file set are used as the second batch data. Based on the second number of processing times and the second processing time, the second batch data is mapped to the target data table associated with the business number type.

[0174] The business identifiers and indicator attribute data of the third source data file set are used as the third batch data. Based on the third processing number and the third processing time, the third batch data is mapped to the target data table associated with the business indicator type.

[0175] Optionally, when mapping business attribute data of various resource transfer services contained in a target data table to a column of data in a pre-set distributed storage table according to preset table entries, the storage module 1504 is also used for:

[0176] Convert the data format of the business identifiers and business attribute data of each resource transfer business contained in a target data table;

[0177] Based on the preset table entries, the transformed business attribute data is stored in the corresponding data columns of the pre-set distributed storage table.

[0178] Optionally, when retrieving at least one business attribute data corresponding to the target business identifier and column identifier set from the distributed storage table as the target data, the determining module 1503 is further configured to:

[0179] From the distributed storage table, filter out all data columns corresponding to the target business identifier;

[0180] Extract at least one business attribute data corresponding to the column identifier set from all data columns as the target data.

[0181] Based on the above embodiments, see Figure 16 The diagram shown is a structural schematic of the electronic device in an embodiment of this application.

[0182] This application provides an electronic device that may include a processor 1610 (Center Processing Unit, CPU), a memory 1620, an input device 1630, and an output device 1640. The input device 1630 may include a keyboard, a mouse, a touch screen, etc., and the output device 1640 may include a display device, such as a liquid crystal display (LCD) or a cathode ray tube (CRT).

[0183] The memory 1620 may include read-only memory (ROM) and random access memory (RAM), and provides the processor 1610 with program instructions and data stored in the memory 1620. In this embodiment, the memory 1620 may be used to store a program for any of the business data query methods in this embodiment.

[0184] The processor 1610 executes any of the business data query methods in this application embodiment according to the program instructions stored in the memory 1620.

[0185] Based on the above embodiments, this application provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the business data query method in any of the above method embodiments.

[0186] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0187] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0188] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0189] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0190] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A business data query method, characterized in that, Various resource transfer transactions applied in the banking industry, including: Obtain the source data file set corresponding to each information type. Each source data file set contains: the business identifier and business attribute data of at least one resource transfer business under the corresponding information type. For each obtained source data file set, perform the following operations respectively: batch process the service identifier and service attribute data of at least one resource transfer service contained in a source data file set, and map the obtained service identifier and service attribute data to the target data table associated with the information type corresponding to the source data file set. For each obtained target data table, the following operations are performed: According to preset table entries, map the business attribute data of each resource transfer service contained in a target data table to a column of data in a pre-set distributed storage table, and use the business identifier of each resource transfer service contained in the target data table as the unique identifier of the corresponding row of data in the distributed storage table; the business identifier of each resource transfer service is used as the row key of the distributed storage table, and column families are designed to distribute the business attribute data on the column families; Receive a business data query request, wherein the business data query request carries the target business identifier and column identifier set of the target business to be queried; Obtain a distributed storage table, which contains: the business identifier and business attribute data of each resource transfer service. Each business identifier is a unique identifier for each row of data in the distributed storage table. Each row of data in the distributed storage table contains business attribute data of the corresponding resource transfer service. Each column of data in the distributed storage table contains one type of business attribute data. Obtain at least one business attribute data corresponding to the target business identifier and the column identifier set from the distributed storage table as target data.

2. The method as described in claim 1, characterized in that, Each information type includes at least: basic business types, business number types, and business indicator types; The process of obtaining the source data file set corresponding to each information type includes: Obtain the first source data file set corresponding to each of the basic business types, wherein each first source data file set contains: basic attribute data and business identifier of at least one resource transfer business under the corresponding basic business type; Obtain the second source data file set corresponding to the business number type, wherein the second source data file set includes: number attribute data and business identifier of at least one resource transfer business under the business number type; Obtain the third source data file set corresponding to the business indicator type, wherein the third source data file set contains: indicator attribute data and business identifiers of at least one resource transfer business under the business indicator type.

3. The method as described in claim 2, characterized in that, The step of batch processing the service identifiers and service attribute data of at least one resource transfer service contained in a source data file set, and mapping the obtained service identifiers and service attribute data to the target data table associated with the information type corresponding to the source data file set, includes: For each of the basic business types, the following operations are performed: taking the business identifiers and basic attribute data of the first source data file set corresponding to a basic business type as the first batch of data, and mapping the first batch of data to the target data table associated with the corresponding basic business type based on the first number of processing times and the first processing time. The business identifiers and number attribute data of the second source data file set are used as the second batch data. Based on the second number of processing times and the second processing time, the second batch data is mapped to the target data table associated with the business number type. The business identifiers and indicator attribute data of the third source data file set are used as the third batch data. Based on the third processing number and the third processing time, the third batch data is mapped to the target data table associated with the business indicator type.

4. The method as described in claim 1, characterized in that, The step of mapping a business attribute data of each resource transfer service contained in a target data table to a column of data in a pre-set distributed storage table according to preset table entries includes: Convert the data format of the business identifiers and business attribute data of each resource transfer business contained in a target data table; Based on the preset table entries, the transformed business attribute data is stored in the corresponding data columns of the pre-set distributed storage table.

5. The method as described in claim 1, characterized in that, The step of obtaining at least one business attribute data corresponding to the target business identifier and the column identifier set from the distributed storage table as target data includes: From the distributed storage table, filter out all data columns corresponding to the target business identifier; Extract at least one business attribute data corresponding to the column identifier set from all the data columns as the target data.

6. A business data query device, characterized in that, include: The storage module is used to obtain the source data file set corresponding to each information type. Each source data file set contains: the business identifier of at least one resource transfer service and the business attribute data under the corresponding information type. For each obtained source data file set, perform the following operations respectively: batch process the service identifier and service attribute data of at least one resource transfer service contained in a source data file set, and map the obtained service identifier and service attribute data to the target data table associated with the information type corresponding to the source data file set. For each obtained target data table, the following operations are performed: According to preset table entries, map the business attribute data of each resource transfer service contained in a target data table to a column of data in a pre-set distributed storage table, and use the business identifier of each resource transfer service contained in the target data table as the unique identifier of the corresponding row of data in the distributed storage table; the business identifier of each resource transfer service is used as the row key of the distributed storage table, and column families are designed to distribute the business attribute data on the column families; The receiving module is used to receive a business data query request, wherein the business data query request carries the target business identifier and column identifier set of the target business to be queried; The acquisition module is used to acquire a distributed storage table, which contains: the business identifier and business attribute data of each resource transfer service. Each business identifier is a unique identifier for each row of data in the distributed storage table. Each row of data in the distributed storage table contains business attribute data of the corresponding resource transfer service. Each column of data in the distributed storage table contains one type of business attribute data. The determination module is used to obtain at least one business attribute data corresponding to the target business identifier and the column identifier set from the distributed storage table as target data.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method according to any one of claims 1-5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1-5.

9. A computer program product, characterized in that, The method includes a computer program stored in a computer-readable storage medium; when a processor of an electronic device reads the computer program from the computer-readable storage medium, the processor executes the computer program, causing the electronic device to perform the steps of the method according to any one of claims 1-5.

Images