A data integration method, device, equipment and storage medium thereof

Abstract

This application belongs to the field of financial technology and is applied to the integration of multiple data sources for financial business data. It relates to a data integration method, apparatus, device, and storage medium, including: acquiring financial business data and historical call record documents sent from multiple data sources according to a preset unified receiving interface; identifying the source, storage level, attribute fields, financial business scenario, data value category, and historical call frequency of the financial business data; performing preliminary integration of the financial business data; and adjusting the preliminary integration result according to a preset adjustment strategy to complete the integration of the financial business data. The integration strategy ensures the reasonable storage of financial business data from multiple data sources. The adjustment strategy ensures that when the target financial business data is called and multiple callable partitions exist, the minimum data volume call partition information is provided in advance, facilitating subsequent business calls and enabling reasonable integration of multiple data sources.

Description

Technical Field

[0001] This application relates to the field of financial technology and is applied to the scenario of integrating multiple data sources in financial business. In particular, it relates to a data integration method, apparatus, device and its storage medium. Background Technology

[0002] With the rapid development of the internet, all industries are seeking breakthroughs by leveraging it. In recent years, the financial industry has also been expanding its online business. Currently, as people's awareness of financial services increases, more and more businesses, families, and individuals are insuring their vehicles and purchasing car insurance after buying them. Traditionally, car insurance sales mainly involve agents communicating with car owners and recommending various types of car insurance.

[0003] Currently, the financial industry primarily employs three sales models: telephone / online sales, agent sales, and partner sales. Due to various reasons, customer data is often inconsistent across these models, manifesting in a lack of interoperability, inconsistent information content, and inconsistent customer data management. For example, in insurance, telephone sales primarily involve managing telephone sales agents who communicate with customers; agent sales, on the other hand, rely on expanding and maintaining an agent network to sell insurance services; and car dealer sales are mainly conducted through partnerships with car dealers. While these methods broaden business channels, they also create the problem of inadequate integration of financial business data from multiple data sources. Summary of the Invention

[0004] The purpose of this application is to provide a data integration method, apparatus, device and storage medium to solve the problem that existing financial businesses cannot reasonably integrate financial business data from multiple data sources under multiple business channels.

[0005] To address the aforementioned technical problems, this application provides a data integration method, employing the following technical solution:

[0006] A data integration method includes the following steps:

[0007] According to the preset unified receiving interface, financial business data sent by multiple data sources and historical call record documents are obtained, wherein the historical call record documents include the historical call frequency of all data in the financial business data;

[0008] By analyzing the data, the financial business data is identified by source, storage level, attribute field, financial business scenario, data value category, and historical call frequency to obtain the identification results.

[0009] Based on the identification results and the preset integration strategy, the financial business data is integrated to obtain preliminary integration results;

[0010] The preliminary integration results are adjusted according to the preset adjustment strategy to obtain the target integration result and complete the integration of the financial business data.

[0011] Furthermore, the steps of identifying the source, storage level, attribute field, financial business scenario, data value category, and historical call frequency of the financial business data to obtain the identification results specifically include:

[0012] Based on the business channel identifier of each piece of financial business data, the source is identified, and the financial business data sent by each business channel is identified.

[0013] Based on the storage level identifier of each piece of financial business data, the storage level is identified, and the storage level corresponding to each piece of financial business data is identified.

[0014] Based on the data attribute fields contained in each piece of financial business data, attribute field identification is performed to identify the data values ​​corresponding to each data attribute field.

[0015] Based on the preset financial business scenario identifiers, each piece of financial business data is identified to determine the financial business scenario to which each piece of financial business data belongs.

[0016] Based on preset data value categories, the data value categories of the data attribute fields contained in each piece of financial business data are identified. The preset data value categories include character data categories and numerical data categories.

[0017] Based on the historical call record document, the call frequency of the data attribute fields contained in each piece of financial business data is identified, and the historical call frequency of each data value is identified.

[0018] Furthermore, the step of integrating the financial business data based on the identification results and the preset integration strategy to obtain preliminary integration results specifically includes:

[0019] Based on the different business channel identifiers, the financial business data sent by each business channel is processed into a first partition, the financial business data in all partitions after the first partition processing is obtained, and the business channel identifier is set as the corresponding first partition identifier.

[0020] Based on the storage level corresponding to each piece of financial business data, all financial business data are processed into a second partition, and the financial business data in all partitions after the second partition processing is obtained. The storage level identifier is set as the corresponding second partition identifier, and the storage level identifier is set according to whether the data needs to be encrypted and the encryption level.

[0021] Based on the identification results, the data attribute fields contained in each piece of financial business data are identified. Based on the data attribute fields, all financial business data are processed into a third partition. Financial business data in all partitions after the third partition processing are obtained, and the data attribute fields are set as the corresponding third partition identifiers.

[0022] Based on the financial business scenario to which each piece of financial business data belongs, all financial business data is processed into a fourth partition to obtain the financial business data in all partitions after the fourth partition processing, and the financial business scenario identifier is set as the corresponding fourth partition identifier.

[0023] Based on the data value category of the data attribute fields contained in each piece of financial business data, all financial business data are processed into a fifth partition to obtain the financial business data in all partitions after the fifth partition processing. The fifth partition processing includes character data category partitions, numeric data category partitions, and mixed data category partitions. In the character data category partition, the data value of the data attribute fields contained in each piece of financial business data is all character data. In the numeric data category partition, the data value of the data attribute fields contained in each piece of financial business data is all numeric data. In the mixed data category partition, the data value of the data attribute fields contained in each piece of financial business data contains both character data and numeric data.

[0024] Based on the historical call frequency of the data attribute fields contained in each piece of financial business data, and the preset differences in different call frequencies, all financial business data are processed into a sixth partition to obtain financial business data in all partitions after the sixth partition processing. Among them, different partitions after the sixth partition processing correspond to different preset call frequency intervals.

[0025] The processing results of the first partition, the second partition, the third partition, the fourth partition, the fifth partition, and the sixth partition are obtained as the preliminary integration results.

[0026] Furthermore, after performing the steps of performing first partitioning processing on the financial business data sent by all business channels according to the different business channel identifiers, obtaining the financial business data in all partitions after the first partitioning processing, and setting the business channel identifiers one by one to the corresponding first partition identifiers, the method further includes:

[0027] Based on the unified receiving interface, obtain the latest financial business data sent by the target data source and the historical call record document;

[0028] Use the business channel identifier corresponding to the target data source as the filter field to perform the first partition filtering;

[0029] If the filtering result is empty, a new first partition is added in the target data warehouse, and the business channel identifier corresponding to the target data source is used as the partition identifier of the new first partition.

[0030] If the filtering result is not empty, the latest financial business data sent by the target data source will be added to the first partition corresponding to the filtering field.

[0031] Furthermore, after performing the steps of identifying the data attribute fields contained in each piece of financial business data based on the identification result, performing third partitioning processing on all financial business data based on the data attribute fields, obtaining the financial business data in all partitions after the third partitioning processing, and setting the data attribute fields one by one as the corresponding third partition identifier, the method further includes:

[0032] By using semantic recognition, data attribute fields with the same semantics are filtered out, and a set of attribute fields is constructed.

[0033] Obtain the different partitions corresponding to the data attribute fields with the same semantics, merge them, and obtain the merged partitions;

[0034] The financial business data within the merged partition is obtained as the financial business data corresponding to all data attribute fields in the attribute field set.

[0035] Furthermore, the step of performing a sixth partitioning process on all financial business data based on the historical call frequency of the data attribute fields contained in each piece of financial business data, and preset different call frequency ranges, to obtain the financial business data in all partitions after the sixth partitioning process, specifically includes:

[0036] Obtain all data attribute fields contained in the target financial business data, and identify the data values ​​corresponding to each data attribute field;

[0037] Based on the data values ​​corresponding to the data attribute fields, determine all the data values ​​contained in the target financial business data;

[0038] Based on the historical call record document, identify the historical call frequency corresponding to each of the data values;

[0039] By comparison, the maximum historical call frequency is selected from the historical call frequencies corresponding to all the data values;

[0040] The maximum historical call frequency is taken as the maximum call frequency of the target financial business data;

[0041] The call frequency range corresponding to the target financial business data is determined based on the maximum call frequency;

[0042] Based on the call frequency range corresponding to the target financial business data, the partition after the sixth partition processing corresponding to the target financial business data is determined.

[0043] Furthermore, the step of adjusting the preliminary integration result according to a preset adjustment strategy to obtain the target integration result and complete the integration of the financial business data specifically includes:

[0044] Step 601: Based on the preliminary integration results, identify the partition where the target financial business data is located after being processed by the first partition, calculate the amount of data in the partition, and record it as the first data amount;

[0045] Step 602: Based on the preliminary integration results, identify the partition where the target financial business data is located after being processed by the second partition, calculate the amount of data in the partition, and record it as the second data amount;

[0046] Step 603: Based on the preliminary integration results, identify the partition where the target financial business data is located after being processed by the third partition, calculate the amount of data in the partition, and record it as the third data amount;

[0047] Step 604: Based on the preliminary integration results, identify the partition where the target financial business data is located after being processed by the fourth partition, calculate the amount of data in the partition, and record it as the fourth data amount;

[0048] Step 605: Based on the preliminary integration results, identify the partition where the target financial business data is located after being processed by the fifth partition, calculate the amount of data in the partition, and record it as the fifth data volume;

[0049] Step 606: Based on the preliminary integration results, identify the partition where the target financial business data is located after being processed by the sixth partition, calculate the amount of data in the partition, and record it as the sixth data volume;

[0050] Step 607: Compare the size relationships among the first data volume, the second data volume, the third data volume, the fourth data volume, the fifth data volume, and the sixth data volume, and select the minimum value from them;

[0051] Step 608: Identify the partition corresponding to the minimum value, and use the partition's identification information as the calling partition information for the target financial business data;

[0052] Step 609: Take each piece of financial business data as the target financial business data in sequence, and repeat steps 601 to 608 to obtain the calling partition information of each piece of financial business data as the target integration result.

[0053] To address the aforementioned technical problems, this application also provides a data integration device, which employs the following technical solution:

[0054] A data integration device, comprising:

[0055] The data acquisition module is used to acquire financial business data sent by multiple data sources and historical call record documents according to a preset unified receiving interface. The historical call record documents include the historical call frequency of all data in the financial business data.

[0056] The data parsing and identification module is used to identify the source, storage level, attribute fields, financial business scenarios, data value categories, and historical call frequency of the financial business data through parsing, and obtain the identification results.

[0057] The preliminary integration module is used to integrate the financial business data based on the identification results and the preset integration strategy to obtain preliminary integration results;

[0058] The preliminary integration result adjustment module is used to adjust the preliminary integration result according to a preset adjustment strategy to obtain the target integration result and complete the integration of the financial business data.

[0059] To address the aforementioned technical problems, this application also provides a computer device that employs the following technical solution:

[0060] A computer device includes a memory and a processor, wherein the memory stores computer-readable instructions, and the processor executes the computer-readable instructions to implement the steps of the data integration method described above.

[0061] To address the aforementioned technical problems, this application also provides a computer-readable storage medium, employing the technical solution described below:

[0062] A computer-readable storage medium storing computer-readable instructions that, when executed by a processor, implement the steps of the data integration method described above.

[0063] Compared with the prior art, the embodiments of this application have the following main advantages:

[0064] The data integration method described in this application embodiment acquires financial business data and historical call record documents sent from multiple data sources according to a preset unified receiving interface. The historical call record documents include the historical call frequency of all data in the financial business data. Through parsing, the financial business data is identified by source, storage level, attribute field, financial business scenario, data value category, and historical call frequency to obtain identification results. Based on the identification results and a preset integration strategy, the financial business data is integrated to obtain a preliminary integration result. The preliminary integration result is then adjusted according to a preset adjustment strategy to obtain the target integration result, thus completing the integration of the financial business data. The integration strategy ensures the reasonable storage of financial business data from multiple data sources. The preset adjustment strategy ensures that when the target financial business data is called, if there are multiple corresponding callable partitions, to avoid calling a partition with a large data volume, the call partition information corresponding to each piece of financial business data with the minimum data volume is provided in advance, facilitating subsequent business calls and reasonably integrating data from multiple data sources. Attached Figure Description

[0065] To more clearly illustrate the solutions in this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0066] Figure 1 This is an exemplary system architecture diagram to which this application can be applied;

[0067] Figure 2 This is a flowchart of an embodiment of the data integration method according to this application;

[0068] Figure 3 yes Figure 2 A flowchart of a specific embodiment of step 202 shown;

[0069] Figure 4 yes Figure 2 A flowchart of a specific embodiment of step 203 shown;

[0070] Figure 5 yes Figure 4 A flowchart of a specific embodiment of step 406 shown;

[0071] Figure 6 yes Figure 2A flowchart of a specific embodiment of step 204 shown;

[0072] Figure 7 This is a schematic diagram of a structure of an embodiment of the data integration apparatus according to this application;

[0073] Figure 8 This is a schematic diagram of the structure of one embodiment of the computer device according to this application. Detailed Implementation

[0074] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein in the specification of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "comprising" and "having," and any variations thereof, in the specification, claims, and foregoing drawings of this application, are intended to cover non-exclusive inclusion. The terms "first," "second," etc., in the specification, claims, or foregoing drawings of this application are used to distinguish different objects, not to describe a particular order.

[0075] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0076] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0077] like Figure 1 As shown, system architecture 100 may include terminal devices 101, 102, and 103, a network 104, and a server 105. Network 104 serves as the medium for providing communication links between terminal devices 101, 102, and 103 and server 105. Network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0078] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, and 103, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social media platform software, etc.

[0079] Terminal devices 101, 102, and 103 can be various electronic devices with displays and support web browsing, including but not limited to smartphones, tablets, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III), MP4 players (Moving Picture Experts Group Audio Layer IV), laptops, and desktop computers, etc.

[0080] Server 105 can be a server that provides various services, such as a backend server that supports the pages displayed on terminal devices 101, 102, and 103.

[0081] It should be noted that the data integration method provided in the embodiments of this application is generally executed by a server / terminal device, and correspondingly, the data integration device is generally located in the server / terminal device.

[0082] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0083] Continue to refer to Figure 2 A flowchart of an embodiment of the data integration method according to this application is shown. The data integration method includes the following steps:

[0084] Step 201: Based on the preset unified receiving interface, obtain the financial business data sent by multiple data sources and the historical call record document, wherein the historical call record document includes the historical call frequency of all data in the financial business data.

[0085] In this embodiment, the multiple data sources include financial business data repositories from different business channels.

[0086] Step 202: Through parsing, the financial business data is identified by source, storage level, attribute field, financial business scenario, data value category, and historical call frequency to obtain the identification results.

[0087] Continue to refer to Figure 3 , Figure 3 yes Figure 2 A flowchart of a specific embodiment of step 202 shown includes:

[0088] Step 301: Based on the business channel identifier of each piece of financial business data, identify the source and identify the financial business data sent by each business channel.

[0089] Step 302: Based on the storage level identifier of each piece of financial business data, identify the storage level corresponding to each piece of financial business data;

[0090] Step 303: Based on the data attribute fields contained in each piece of financial business data, perform attribute field identification and identify the data values ​​corresponding to each data attribute field.

[0091] Step 304: Based on the preset financial business scenario identifier, identify the financial business scenario for each piece of financial business data, and identify the financial business scenario to which each piece of financial business data belongs.

[0092] Step 305: Based on the preset data value categories, identify the data value categories of the data attribute fields contained in each piece of financial business data, wherein the preset data value categories include character data categories and numerical data categories;

[0093] Step 306: Based on the historical call record document, identify the call frequency of the data attribute fields contained in each piece of financial business data, and identify the historical call frequency of each data value.

[0094] By identifying the source, storage level, attribute fields, financial business scenario, data value category, and historical call frequency of each piece of financial business data, it is easy to integrate financial business data from multiple data sources.

[0095] Step 203: Based on the identification results and the preset integration strategy, the financial business data is integrated to obtain preliminary integration results.

[0096] Continue to refer to Figure 4 , Figure 4 yes Figure 2 A flowchart of a specific embodiment of step 203 shown includes:

[0097] Step 401: Based on the different business channel identifiers, perform first partitioning on the financial business data sent by all business channels, obtain the financial business data in all partitions after the first partitioning, and set the business channel identifiers as the corresponding first partition identifiers.

[0098] In this embodiment, after performing the steps of performing first partitioning processing on the financial business data sent by all business channels according to the different business channel identifiers, obtaining the financial business data in all partitions after the first partitioning processing, and setting the business channel identifiers one by one as the corresponding first partition identifiers, the method further includes: obtaining the latest financial business data sent by the target data source and the historical call record document according to the unified receiving interface; performing first partition filtering using the business channel identifier corresponding to the target data source as a filtering field; if the filtering result is null, adding a new first partition in the target data warehouse, and using the business channel identifier corresponding to the target data source as the partition identifier of the newly added first partition; if the filtering result is not null, adding the latest financial business data sent by the target data source to the first partition corresponding to the filtering field.

[0099] By performing a first-partition identifier filtering, a new partition is automatically created when a new business channel or a new data source is accessed.

[0100] Step 402: Based on the storage level corresponding to each piece of financial business data, perform second partitioning processing on all financial business data, obtain the financial business data in all partitions after the second partitioning processing, and set the storage level identifier as the corresponding second partition identifier. The storage level identifier is set according to whether the data needs to be encrypted and the encryption level.

[0101] Step 403: Based on the recognition result, identify the data attribute fields contained in each piece of financial business data, perform third partitioning processing on all financial business data according to the data attribute fields, obtain the financial business data in all partitions after the third partitioning processing, and set the data attribute fields as the corresponding third partition identifiers.

[0102] In this embodiment, after performing the steps of identifying the data attribute fields contained in each piece of financial business data according to the recognition result, performing third partitioning processing on all financial business data according to the data attribute fields, obtaining the financial business data in all partitions after the third partitioning processing, and setting each data attribute field as a corresponding third partition identifier, the method further includes: filtering out data attribute fields with the same semantics through semantic recognition to construct an attribute field set; obtaining different partitions corresponding to the data attribute fields with the same semantics, merging them to obtain merged partitions; and obtaining the financial business data in the merged partitions as the financial business data corresponding to all data attribute fields in the attribute field set.

[0103] Since multiple data sources may involve attribute fields with the same semantics when recording data, there may be inconsistencies in the naming of attribute fields. Therefore, semantic recognition is used to filter out data attribute fields with the same semantics, construct an attribute field set, obtain the different partitions corresponding to the data attribute fields with the same semantics, and merge them to obtain the merged partitions. This makes attribute fields with the same semantics correspond to a unified partition, making data integration more reasonable.

[0104] Step 404: Based on the financial business scenario to which each piece of financial business data belongs, perform fourth partitioning on all financial business data to obtain financial business data in all partitions after fourth partitioning, and set the financial business scenario identifier to the corresponding fourth partition identifier.

[0105] Step 405: Based on the data value category of the data attribute field contained in each piece of financial business data, perform fifth partitioning on all financial business data to obtain financial business data in all partitions after fifth partitioning.

[0106] The fifth partition, after processing, includes character data category partitions, numerical data category partitions, and mixed data category partitions. In the character data category partition, the data values ​​of the data attribute fields of each financial business data item are all character data. In the numerical data category partition, the data values ​​of the data attribute fields of each financial business data item are all numerical data. In the mixed data category partition, the data values ​​of the data attribute fields of each financial business data item contain both character data and numerical data.

[0107] Step 406: Based on the historical call frequency of the data attribute fields contained in each piece of financial business data, and the preset differences in different call frequencies, perform sixth partitioning on all financial business data to obtain financial business data in all partitions after the sixth partitioning is processed. The different partitions after the sixth partitioning are respectively corresponding to preset different call frequency ranges.

[0108] Continue to refer to Figure 5 , Figure 5 yes Figure 4 A flowchart of a specific embodiment of step 406 shown includes:

[0109] Step 501: Obtain all data attribute fields contained in the target financial business data, and identify the data values ​​corresponding to each data attribute field;

[0110] Step 502: Determine all data values ​​contained in the target financial business data based on the data values ​​corresponding to the data attribute fields respectively;

[0111] Step 503: Based on the historical call record document, identify the historical call frequency corresponding to each of the data values;

[0112] Step 504: By comparison, the maximum historical call frequency is selected from the historical call frequencies corresponding to all the data values.

[0113] Step 505: Use the maximum historical call frequency as the maximum call frequency of the target financial business data;

[0114] Since each piece of financial business data, regardless of which data value corresponding to its contained data attribute fields is invoked, involves invoking the entire financial business data, the maximum historical call frequency is selected from the historical call frequencies corresponding to all the data values ​​through comparison. This maximum historical call frequency is then used as the maximum call frequency for the target financial business data. This facilitates data integration of financial business data based on historical call frequencies.

[0115] Step 506: Determine the call frequency range corresponding to the target financial business data based on the maximum call frequency;

[0116] Step 507: Based on the call frequency range corresponding to the target financial business data, determine the partition after the sixth partition processing corresponding to the target financial business data.

[0117] Step 407: Obtain the processing results of the first partition, the second partition, the third partition, the fourth partition, the fifth partition, and the sixth partition as the preliminary integration results.

[0118] The integration strategy described above integrates financial business data based on its source, storage level, attribute fields, financial business scenario, data value category, and historical call frequency, ensuring the reasonable storage of financial business data from multiple data sources.

[0119] Step 204: Adjust the preliminary integration result according to the preset adjustment strategy to obtain the target integration result and complete the integration of the financial business data.

[0120] Continue to refer to Figure 6 , Figure 6 yes Figure 2 A flowchart of a specific embodiment of step 204 shown includes:

[0121] Step 601: Based on the preliminary integration results, identify the partition where the target financial business data is located after being processed by the first partition, calculate the amount of data in the partition, and record it as the first data amount;

[0122] Step 602: Based on the preliminary integration results, identify the partition where the target financial business data is located after being processed by the second partition, calculate the amount of data in the partition, and record it as the second data amount;

[0123] Step 603: Based on the preliminary integration results, identify the partition where the target financial business data is located after being processed by the third partition, calculate the amount of data in the partition, and record it as the third data amount;

[0124] Step 604: Based on the preliminary integration results, identify the partition where the target financial business data is located after being processed by the fourth partition, calculate the amount of data in the partition, and record it as the fourth data amount;

[0125] Step 605: Based on the preliminary integration results, identify the partition where the target financial business data is located after being processed by the fifth partition, calculate the amount of data in the partition, and record it as the fifth data volume;

[0126] Step 606: Based on the preliminary integration results, identify the partition where the target financial business data is located after being processed by the sixth partition, calculate the amount of data in the partition, and record it as the sixth data volume;

[0127] Step 607: Compare the size relationships among the first data volume, the second data volume, the third data volume, the fourth data volume, the fifth data volume, and the sixth data volume, and select the minimum value from them;

[0128] Step 608: Identify the partition corresponding to the minimum value, and use the partition's identification information as the calling partition information for the target financial business data;

[0129] Step 609: Take each piece of financial business data as the target financial business data in sequence, and repeat steps 601 to 608 to obtain the calling partition information of each piece of financial business data as the target integration result.

[0130] The preset adjustment strategy involves identifying the partition corresponding to each piece of financial business data and calculating the data volume of that partition. The partition corresponding to each piece of financial business data with the minimum data volume is selected as the calling partition information for the target financial business data. The purpose is to ensure that multiple corresponding callable partitions exist when the target financial business data is called. To avoid calling partitions with large data volumes, the calling partition information corresponding to each piece of financial business data with the minimum data volume is provided in advance, facilitating subsequent business calls and enabling reasonable integration of data from multiple data sources.

[0131] This application acquires financial business data and historical call record documents sent from multiple data sources through a preset unified receiving interface. The historical call record documents include the historical call frequency of all data in the financial business data. Through parsing, the financial business data is identified by source, storage level, attribute fields, financial business scenario, data value category, and historical call frequency to obtain identification results. Based on the identification results and a preset integration strategy, the financial business data is integrated to obtain a preliminary integration result. The preliminary integration result is then adjusted according to a preset adjustment strategy to obtain the target integration result, thus completing the integration of the financial business data. The integration strategy, based on the source, storage level, attribute fields, financial business scenario, data value category, and historical call frequency of the financial business data, ensures reasonable storage of financial business data from multiple data sources. The preset adjustment strategy ensures that when the target financial business data is called, if there are multiple corresponding callable partitions, to avoid calling a partition with a large data volume, the call partition information corresponding to each piece of financial business data with the minimum data volume is provided in advance, facilitating subsequent business calls and reasonably integrating data from multiple data sources.

[0132] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0133] Foundational artificial intelligence technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data integration, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0134] In this embodiment, financial business data and historical call record documents sent from multiple data sources are obtained according to a preset unified receiving interface. The historical call record documents include the historical call frequency of all data in the financial business data. Through parsing, the financial business data is identified by source, storage level, attribute field, financial business scenario, data value category, and historical call frequency to obtain identification results. Based on the identification results and a preset integration strategy, the financial business data is integrated to obtain a preliminary integration result. The preliminary integration result is then adjusted according to a preset adjustment strategy to obtain the target integration result, thus completing the integration of the financial business data. The integration strategy ensures the reasonable storage of financial business data from multiple data sources. The preset adjustment strategy ensures that when the target financial business data is called, if there are multiple corresponding callable partitions, to avoid calling a partition with a large data volume, the call partition information corresponding to each piece of financial business data with the minimum data volume is provided in advance, facilitating subsequent business calls and reasonably integrating data from multiple data sources.

[0135] Further reference Figure 7 As a response to the above Figure 2 To implement the method shown, this application provides an embodiment of a data integration apparatus, which is similar to... Figure 2 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0136] like Figure 7 As shown, the data integration device 700 described in this embodiment includes: a data acquisition module 701, a data parsing and identification module 702, a preliminary integration module 703, and a preliminary integration result adjustment module 704. Wherein:

[0137] The data acquisition module 701 is used to acquire financial business data sent by multiple data sources and historical call record documents according to a preset unified receiving interface, wherein the historical call record documents include the historical call frequency of all data in the financial business data.

[0138] The data parsing and identification module 702 is used to identify the source, storage level, attribute field, financial business scenario, data value category, and historical call frequency of the financial business data through parsing, and obtain the identification results.

[0139] The preliminary integration module 703 is used to integrate the financial business data based on the identification results and the preset integration strategy to obtain preliminary integration results;

[0140] The preliminary integration result adjustment module 704 is used to adjust the preliminary integration result according to a preset adjustment strategy to obtain the target integration result and complete the integration of the financial business data.

[0141] This application acquires financial business data and historical call record documents sent from multiple data sources through a preset unified receiving interface. The historical call record documents include the historical call frequency of all data in the financial business data. Through parsing, the financial business data is identified by source, storage level, attribute fields, financial business scenario, data value category, and historical call frequency to obtain identification results. Based on the identification results and a preset integration strategy, the financial business data is integrated to obtain a preliminary integration result. The preliminary integration result is then adjusted according to a preset adjustment strategy to obtain the target integration result, thus completing the integration of the financial business data. The integration strategy, based on the source, storage level, attribute fields, financial business scenario, data value category, and historical call frequency of the financial business data, ensures reasonable storage of financial business data from multiple data sources. The preset adjustment strategy ensures that when the target financial business data is called, if there are multiple corresponding callable partitions, to avoid calling a partition with a large data volume, the call partition information corresponding to each piece of financial business data with the minimum data volume is provided in advance, facilitating subsequent business calls and reasonably integrating data from multiple data sources.

[0142] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing related hardware through computer-readable instructions. These computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the methods described above. The aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, optical disk, or read-only memory (ROM), or random access memory (RAM).

[0143] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0144] To address the aforementioned technical problems, embodiments of this application also provide a computer device. Please refer to [link / reference needed]. Figure 8 , Figure 8 This is a basic structural block diagram of the computer device in this embodiment.

[0145] The computer device 8 includes a memory 8a, a processor 8b, and a network interface 8c that are interconnected via a system bus. It should be noted that only the computer device 8 with components 8a-8c is shown in the figure; however, it should be understood that it is not required to implement all the shown components, and more or fewer components can be implemented alternatively. Those skilled in the art will understand that the computer device described here is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0146] The computer device can be a desktop computer, laptop, handheld computer, or cloud server, etc. The computer device can interact with the user via a keyboard, mouse, remote control, touchpad, or voice control.

[0147] The memory 8a includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 8a may be an internal storage unit of the computer device 8, such as the hard disk or memory of the computer device 8. In other embodiments, the memory 8a may also be an external storage device of the computer device 8, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computer device 8. Of course, the memory 8a may also include both the internal storage unit and its external storage device of the computer device 8. In this embodiment, the memory 8a is typically used to store the operating system and various application software installed on the computer device 8, such as computer-readable instructions for a data integration method. In addition, the memory 8a can also be used to temporarily store various types of data that have been output or will be output.

[0148] In some embodiments, the processor 8b may be a central processing unit (CPU), controller, microcontroller, microprocessor, or other data integration chip. The processor 8b is typically used to control the overall operation of the computer device 8. In this embodiment, the processor 8b is used to execute computer-readable instructions stored in the memory 8a or to process data, for example, to execute computer-readable instructions for the data integration method.

[0149] The network interface 8c may include a wireless network interface or a wired network interface, which is typically used to establish communication connections between the computer device 8 and other electronic devices.

[0150] The computer device proposed in this embodiment belongs to the field of financial technology and is applied in the scenario of integrating multiple data sources of financial business data. This application acquires financial business data sent from multiple data sources and historical call record documents according to a preset unified receiving interface. The historical call record documents include the historical call frequency of all data in the financial business data. Through parsing, the financial business data is identified by source, storage level, attribute field, financial business scenario, data value category, and historical call frequency to obtain identification results. Based on the identification results and a preset integration strategy, the financial business data is integrated to obtain a preliminary integration result. The preliminary integration result is adjusted according to a preset adjustment strategy to obtain the target integration result, thus completing the integration of the financial business data. The integration strategy described above integrates financial business data based on its source, storage level, attribute fields, financial business scenario, data value category, and historical call frequency. This ensures the reasonable storage of financial business data from multiple data sources. The preset adjustment strategy ensures that when the target financial business data is called, if there are multiple corresponding callable partitions, the system will pre-provide the callable partition information corresponding to each piece of financial business data when the data volume is at its minimum. This facilitates subsequent business calls and enables reasonable integration of data from multiple data sources.

[0151] This application also provides another embodiment, namely, providing a computer-readable storage medium storing computer-readable instructions that can be executed by a processor to cause the processor to perform the steps of the data integration method described above.

[0152] The computer-readable storage medium proposed in this embodiment belongs to the field of financial technology and is applied in the scenario of integrating multiple data sources for financial business data. This application acquires financial business data sent from multiple data sources and historical call record documents according to a preset unified receiving interface. The historical call record documents include the historical call frequencies of all data in the financial business data. Through parsing, the financial business data is identified by source, storage level, attribute field, financial business scenario, data value category, and historical call frequency to obtain identification results. Based on the identification results and a preset integration strategy, the financial business data is integrated to obtain a preliminary integration result. The preliminary integration result is adjusted according to a preset adjustment strategy to obtain a target integration result, thus completing the integration of the financial business data. The integration strategy described above integrates financial business data based on its source, storage level, attribute fields, financial business scenario, data value category, and historical call frequency. This ensures the reasonable storage of financial business data from multiple data sources. The preset adjustment strategy ensures that when the target financial business data is called, if there are multiple corresponding callable partitions, the system will pre-provide the callable partition information corresponding to each piece of financial business data when the data volume is at its minimum. This facilitates subsequent business calls and enables reasonable integration of data from multiple data sources.

[0153] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0154] Obviously, the embodiments described above are only some embodiments of this application, not all embodiments. The accompanying drawings show preferred embodiments of this application, but do not limit the patent scope of this application. This application can be implemented in many different forms; rather, the purpose of providing these embodiments is to provide a more thorough and comprehensive understanding of the disclosure of this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing specific embodiments, or make equivalent substitutions for some of the technical features. Any equivalent structures made using the content of this application's specification and drawings, directly or indirectly applied to other related technical fields, are similarly within the scope of patent protection of this application.

Claims

1. A data integration method, characterized by, Includes the following steps: According to the preset unified receiving interface, financial business data sent by multiple data sources and historical call record documents are obtained, wherein the historical call record documents include the historical call frequency of all data in the financial business data; By analyzing the data, the financial business data is identified by source, storage level, attribute field, financial business scenario, data value category, and historical call frequency to obtain the identification results. Based on the identification results and the preset integration strategy, the financial business data is integrated to obtain preliminary integration results, specifically including: Based on the different business channel identifiers, the financial business data sent by each business channel is processed into a first partition, and the financial business data in all partitions after the first partition processing is obtained. The business channel identifier is then set as the corresponding first partition identifier. Based on the storage level corresponding to each piece of financial business data, all financial business data are processed into a second partition. After the second partition processing, all financial business data in all partitions are obtained, and the storage level identifier is set as the corresponding second partition identifier. The storage level identifier is set according to whether the data needs to be encrypted and the encryption level. Based on the identification results, the data attribute fields contained in each piece of financial business data are identified. Based on the data attribute fields, all financial business data are processed into a third partition. Financial business data in all partitions after the third partition processing are obtained, and the data attribute fields are set as the corresponding third partition identifiers. Based on the financial business scenario to which each piece of financial business data belongs, all financial business data is processed into a fourth partition to obtain the financial business data in all partitions after the fourth partition processing, and the financial business scenario identifier is set as the corresponding fourth partition identifier. Based on the data value category of the data attribute fields contained in each piece of financial business data, all financial business data are processed into a fifth partition to obtain the financial business data in all partitions after the fifth partition processing. The fifth partition processing includes character data category partitions, numerical data category partitions, and mixed data category partitions. In the character data category partition, the data value of the data attribute fields contained in each piece of financial business data is all character data. In the numerical data category partition, the data value of the data attribute fields contained in each piece of financial business data is all numerical data. In the mixed data category partition, the data value of the data attribute fields contained in each piece of financial business data contains both character data and numerical data. Based on the historical call frequency of the data attribute fields contained in each piece of financial business data, and the preset differences in different call frequencies, all financial business data are processed into a sixth partition to obtain financial business data in all partitions after the sixth partition processing. Among them, different partitions after the sixth partition processing correspond to different preset call frequency intervals. The processing results of the first partition, the second partition, the third partition, the fourth partition, the fifth partition, and the sixth partition are obtained as the preliminary integration results. The preliminary integration results are adjusted according to a preset adjustment strategy to obtain the target integration result, thus completing the integration of the financial business data. Specifically, this includes: Step 601: Based on the preliminary integration results, identify the partition where the target financial business data is located after being processed by the first partition, calculate the amount of data in the partition, and record it as the first data amount; Step 602: Based on the preliminary integration results, identify the partition where the target financial business data is located after being processed by the second partition, calculate the amount of data in the partition, and record it as the second data amount; Step 603: Based on the preliminary integration results, identify the partition where the target financial business data is located after being processed by the third partition, calculate the amount of data in the partition, and record it as the third data amount; Step 604: Based on the preliminary integration results, identify the partition where the target financial business data is located after being processed by the fourth partition, calculate the amount of data in the partition, and record it as the fourth data amount; Step 605: Based on the preliminary integration results, identify the partition where the target financial business data is located after being processed by the fifth partition, calculate the amount of data in the partition, and record it as the fifth data volume; Step 606: Based on the preliminary integration results, identify the partition where the target financial business data is located after being processed by the sixth partition, calculate the amount of data in the partition, and record it as the sixth data volume; Step 607: Compare the size relationships among the first data volume, the second data volume, the third data volume, the fourth data volume, the fifth data volume, and the sixth data volume, and select the minimum value from them; Step 608: Identify the partition corresponding to the minimum value, and use the partition's identification information as the calling partition information for the target financial business data; Step 609: Take each piece of financial business data as the target financial business data in sequence, and repeat steps 601 to 608 to obtain the calling partition information of each piece of financial business data as the target integration result.

2. The data integration method according to claim 1, characterized in that, The steps of identifying the source, storage level, attribute field, financial business scenario, data value category, and historical call frequency of the financial business data to obtain the identification results specifically include: Based on the business channel identifier of each piece of financial business data, the source is identified, and the financial business data sent by each business channel is identified. Based on the storage level identifier of each piece of financial business data, the storage level is identified, and the storage level corresponding to each piece of financial business data is identified. Based on the data attribute fields contained in each piece of financial business data, attribute field identification is performed to identify the data values ​​corresponding to each data attribute field. Based on the preset financial business scenario identifiers, each piece of financial business data is identified to determine the financial business scenario to which each piece of financial business data belongs. Based on preset data value categories, the data value categories of the data attribute fields contained in each piece of financial business data are identified. The preset data value categories include character data categories and numerical data categories. Based on the historical call record document, the call frequency of the data attribute fields contained in each piece of financial business data is identified, and the historical call frequency of each data value is identified.

3. The data integration method of claim 1, wherein, After performing the steps of performing first partitioning processing on the financial business data sent by all business channels according to the different business channel identifiers, obtaining the financial business data in all partitions after the first partitioning processing, and setting the business channel identifiers one by one to the corresponding first partition identifiers, the method further includes: Based on the unified receiving interface, obtain the latest financial business data sent by the target data source and the historical call record document; Use the business channel identifier corresponding to the target data source as the filter field to perform the first partition filtering; If the filtering result is empty, a new first partition is added in the target data warehouse, and the business channel identifier corresponding to the target data source is used as the partition identifier of the new first partition. If the filtering result is not empty, the latest financial business data sent by the target data source will be added to the first partition corresponding to the filtering field.

4. The data integration method of claim 1, wherein, After performing the steps of identifying the data attribute fields contained in each piece of financial business data based on the identification result, performing third partitioning processing on all financial business data based on the data attribute fields, obtaining the financial business data in all partitions after the third partitioning processing, and setting each data attribute field as a corresponding third partition identifier, the method further includes: By using semantic recognition, data attribute fields with the same semantics are filtered out, and a set of attribute fields is constructed. Obtain the different partitions corresponding to the data attribute fields with the same semantics, merge them, and obtain the merged partitions; The financial business data within the merged partition is obtained as the financial business data corresponding to all data attribute fields in the attribute field set.

5. The data integration method of claim 1, wherein, The step of performing a sixth partitioning process on all financial business data based on the historical call frequency of the data attribute fields contained in each piece of financial business data, and on preset different call frequency ranges, to obtain the financial business data in all partitions after the sixth partitioning process, specifically includes: Obtain all data attribute fields contained in the target financial business data, and identify the data values ​​corresponding to each data attribute field; Based on the data values ​​corresponding to the data attribute fields, determine all the data values ​​contained in the target financial business data; Based on the historical call record document, identify the historical call frequency corresponding to each of the data values; By comparison, the maximum historical call frequency is selected from the historical call frequencies corresponding to all the data values; The maximum historical call frequency is taken as the maximum call frequency of the target financial business data; The call frequency range corresponding to the target financial business data is determined based on the maximum call frequency; Based on the call frequency range corresponding to the target financial business data, the partition after the sixth partition processing corresponding to the target financial business data is determined.

6. A data integration apparatus, characterized by comprising: The data integration device implements the steps of the data integration method as described in any one of claims 1 to 5, and the data integration device comprises: The data acquisition module is used to acquire financial business data sent by multiple data sources and historical call record documents according to a preset unified receiving interface. The historical call record documents include the historical call frequency of all data in the financial business data. The data parsing and identification module is used to identify the source, storage level, attribute fields, financial business scenarios, data value categories, and historical call frequency of the financial business data through parsing, and obtain the identification results. The preliminary integration module is used to integrate the financial business data based on the identification results and the preset integration strategy to obtain preliminary integration results; The preliminary integration result adjustment module is used to adjust the preliminary integration result according to a preset adjustment strategy to obtain the target integration result and complete the integration of the financial business data.

7. A computer device, characterized by The system includes a memory and a processor, wherein the memory stores computer-readable instructions, and the processor executes the computer-readable instructions to implement the steps of the data integration method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the data integration method as described in any one of claims 1 to 5.

Images