Data Processing Method and Related Equipment

Abstract

Embodiments of the present application disclose a data processing method and related devices. The method includes: determining the first quantity information of each type of to-be-processed credit investigation data at each moment among multiple moments; determining the growth rate of the to-be-processed credit investigation data according to the first quantity information; determining the second quantity information of the to-be-processed credit investigation data with different contents in the same type at multiple moments according to the content information of each piece of to-be-processed credit investigation data in each type of to-be-processed credit investigation data; determining the change frequency of each type of to-be-processed credit investigation data at multiple moments according to the second quantity information; determining the first to-be-processed credit investigation data in the to-be-processed credit investigation data according to the growth rate and the change frequency and performing offline processing to obtain offline credit investigation data and storing it in the database. In this way, data can be processed in batches at different times, reducing the pressure on the server for data processing.

Description

Technical Field

[0001] This application relates to the technical field of data processing, and particularly relates to a data processing method and related devices. Background Art

[0002] Credit investigation data is a type of data in a professional field. The server collects credit investigation data in real time and processes, analyzes, and utilizes the collected credit investigation data. Since the amount of data collected in real time is large and the data is redundant, if immediate processing is required for each piece of credit investigation data collected in real time, it places high requirements on the database's bearing capacity and business data processing capacity. How to distinguish and batch process the collected credit investigation data is an urgent problem to be solved currently.

[0003] Therefore, this application proposes a data processing method to solve the above problems. Summary of the Invention

[0004] The embodiments of this application provide a data processing method and related devices. The first to-be-processed credit investigation data is determined based on the growth rate and change frequency of each type of to-be-processed credit investigation data, and the first to-be-processed credit investigation data is subjected to offline processing and then stored in the database. In this way, data can be processed in batches at different times, reducing the pressure on the server for data processing.

[0005] In a first aspect, the embodiments of this application provide a data processing method applied to a server. The method includes:

[0006] Determine the first quantity information of each type of to-be-processed credit investigation data in each of multiple moments in the to-be-processed credit investigation data;

[0007] Determine the growth rate of the to-be-processed credit investigation data in the multiple moments according to the first quantity information;

[0008] According to the content information of each piece of to-be-processed credit investigation data in each type of to-be-processed credit investigation data, determine the second quantity information of the to-be-processed credit investigation data with different contents in the same type in the multiple moments;

[0009] Determine the change frequency of each type of to-be-processed credit investigation data in the multiple moments according to the second quantity information;

[0010] Determine the first to-be-processed credit investigation data in the to-be-processed credit investigation data according to the growth rate and the change frequency;

[0011] Perform offline processing on the first to-be-processed credit investigation data to obtain offline credit investigation data;

[0012] Store the offline credit investigation data in the database.

[0013] In a second aspect, an embodiment of the present application provides a data processing device, which is applied to a server. The device includes a determination unit, a calculation unit, a processing unit, and a storage unit:

[0014] The determination unit is configured to determine the first quantity information of each type of to-be-processed credit investigation data at each moment in the to-be-processed credit investigation data;

[0015] The calculation unit is configured to determine the growth rate of the to-be-processed credit investigation data at multiple moments according to the first quantity information;

[0016] The determination unit is further configured to determine the second quantity information of the to-be-processed credit investigation data with different contents at multiple moments according to the content information of each piece of to-be-processed credit investigation data in each type of to-be-processed credit investigation data;

[0017] The calculation unit is further configured to determine the change frequency of each type of to-be-processed credit investigation data according to the second quantity information;

[0018] The determination unit is further configured to determine the first to-be-processed credit investigation data in the to-be-processed credit investigation data according to the growth rate and the change frequency;

[0019] The processing unit is configured to perform offline processing on the first to-be-processed credit investigation data to obtain offline credit investigation data;

[0020] The storage unit is configured to store the offline credit investigation data in a database.

[0021] In a third aspect, an embodiment of the present application provides a server, which includes a processor, a memory, a communication interface, and one or more programs. Among them, the above one or more programs are stored in the above memory and are configured to be executed by the above processor. The above programs include instructions for performing the steps in any method of the first aspect of the embodiments of the present application.

[0022] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium. Among them, the above computer-readable storage medium stores a computer program for electronic data exchange. Among them, the above computer program enables a computer to execute some or all of the steps described in any method of the first aspect of the embodiments of the present application.

[0023] In a fifth aspect, an embodiment of the present application provides a computer program product. Among them, the above computer program product includes a non-transitory computer-readable storage medium storing a computer program. The above computer program is operable to enable a computer to execute some or all of the steps described in any method of the first aspect of the embodiments of the present application. This computer program product can be a software installation package.

[0024] It can be seen that in the embodiments of the present application, by determining the first quantity information of each type of to-be-processed credit investigation data at each moment among multiple moments in the to-be-processed credit investigation data; determining the growth rate of the to-be-processed credit investigation data at multiple moments according to the first quantity information; determining the second quantity information of the to-be-processed credit investigation data with different contents in the same type at multiple moments according to the content information of each piece of to-be-processed credit investigation data in each type of to-be-processed credit investigation data; determining the change frequency of each type of to-be-processed credit investigation data at multiple moments according to the second quantity information; determining the first to-be-processed credit investigation data in the to-be-processed credit investigation data according to the growth rate and the change frequency; performing offline processing on the first to-be-processed credit investigation data to obtain offline credit investigation data; and storing the offline credit investigation data in a database. In this way, a set of standardized data processing processes can be established through the solution of the present application, realizing time-division and batch processing of data and reducing the pressure on the server for data processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0026] Figure 1 is a schematic flowchart of a data processing method provided by an embodiment of the present application;

[0027] Figure 2 is a schematic diagram of the process of comparing the similarity of to-be-processed credit investigation data provided by an embodiment of the present application;

[0028] Figure 3 is an overall architecture diagram of data processing provided by an embodiment of the present application;

[0029] Figure 4 is a schematic structural diagram of a server provided by an embodiment of the present application;

[0030] Figure 5 is a block diagram of the functional units of a data processing device provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0031] To enable those skilled in the art to better understand the solution of this application, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of this application.

[0032] The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but optionally further includes steps or units not listed, or optionally further includes other steps or units inherent to these processes, methods, products, or devices.

[0033] Referring to "embodiment" herein means that a specific feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of this application. The phrase appears in various places in the specification and does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art will explicitly and implicitly understand that the embodiments described herein can be combined with other embodiments.

[0034] The server can be a portable server that also includes other functions such as personal digital assistant and / or music player functions, such as mobile phones, tablet computers, wearable servers with wireless communication functions (such as smart watches), etc. Exemplary embodiments of the portable server include, but are not limited to, portable servers running IOS system, Android system, Microsoft system, or other operating systems. The above-mentioned portable server can also be other portable servers, such as laptop computers. It should also be understood that in some other embodiments, the above-mentioned server may not be a portable server, but a desktop computer.

[0035] To better understand the technical solution of this application, the following will specifically describe this application in conjunction with specific embodiments.

[0036] Please refer to Figure 1 , Figure 1 which is a schematic flowchart of a data processing method provided by an embodiment of this application. As Figure 1 shown, the data processing method in this application is applied to a server and specifically includes the following operation steps:

[0037] S101. Determine the first quantity information of each type of credit investigation data to be processed at each moment among multiple moments.

[0038] Specifically, in the embodiments of the present application, the server uses existing data acquisition channels to disassemble public data such as industry and commerce, judiciary, and invoices into credit investigation data with enterprises and individuals as nodes and industrial and commercial relationships, judicial relationships, and invoice purchase and sale relationships as edges, and stores them in a database. The types of credit investigation data to be processed stored in the database include but are not limited to: personal identity information, loan information, credit card information, etc. of users.

[0039] Further, determine the quantity of each type of credit investigation data to be processed at each moment to obtain the first quantity information. Among them, the first quantity information includes the quantity information corresponding to each type of credit investigation data to be processed at each moment.

[0040] Specifically, the above first quantity information may specifically be the quantity information of one type of credit investigation data to be processed at multiple moments, or the quantity information of multiple types of credit investigation data to be processed at multiple moments.

[0041] S102. Determine the growth rate of the credit investigation data to be processed at the multiple moments according to the first quantity information.

[0042] Further, calculate the growth rate of each type of credit investigation data to be processed at multiple moments according to the quantity of each type of credit investigation data to be processed determined in step S101.

[0043] S103. Determine the second quantity information of the credit investigation data with different contents in the same type at the multiple moments according to the content information of each piece of credit investigation data to be processed in each type of credit investigation data to be processed.

[0044] Specifically, in the actual application scenario, it is possible that each type of credit investigation data to be processed changes little or does not change over time. Therefore, there may be duplicate redundant data in the content of each type of credit investigation data obtained at different moments. Therefore, the server determines the quantity corresponding to the credit investigation data with different contents at multiple moments according to the content information of each piece of credit investigation data to be processed in each type of credit investigation data to be processed, and constitutes the second quantity information.

[0045] S104. Determine the change frequency of each type of credit investigation data to be processed at the multiple moments according to the second quantity information.

[0046] Specifically, the server determines the change frequency of each type of to-be-processed credit investigation data at multiple moments based on the quantity information of the to-be-processed credit investigation data corresponding to multiple moments after removing redundant operations on the to-be-processed credit investigation data according to step S103, and the preset time intervals between multiple moments.

[0047] S105. Determine the first to-be-processed credit investigation data in the to-be-processed credit investigation data according to the growth rate and the change frequency.

[0048] Specifically, after an enterprise processes the collected business data (such as the to-be-processed credit investigation data in the solution of the present application) through the server, it applies the processed data to actual business requirements. Among them, some business data needs to be processed in a timely manner. For example: credit card business data. It is necessary to monitor the credit card business data of each user in real time, and confirm the credit card security status of each user, as well as the possible risks according to the credit card business data, and immediately make risk response processing. The demand for real-time data processing is growing rapidly. At the same time, the problem that the demand for real-time data processing in various industries, enterprises and institutions does not match their current project development methods and supporting tools is gradually emerging. It is necessary to load a large amount of information through a real-time data application platform for real-time analysis and processing, resulting in high requirements for the real-time data application platform and high application costs for data real-time processing. How to overcome the difficulties in the data processing process is a compulsory course for current enterprises, institutions, the Internet, the financial field, etc. In actual applications, not every type of business data needs to be processed immediately. If every piece of collected business data is processed through real-time processing, it not only has high requirements for the server but also causes waste of resources. Therefore, before data processing, the to-be-processed business data can be distinguished, and the corresponding processing methods can be used to perform data processing operations on the distinguished to-be-processed business data respectively.

[0049] Specifically, the basis for dividing the to-be-processed credit investigation data collected by the server is to determine the first to-be-processed credit investigation data in the current to-be-processed credit investigation data according to the growth rate and the change frequency calculated in step S102 and step S104.

[0050] S106. Perform offline processing on the first to-be-processed credit investigation data to obtain offline credit investigation data.

[0051] It should be noted that the to-be-processed credit investigation data used by the business party (such as a bank) through the server for risk control using a risk control model includes: personal information submitted by the user during application, such as name, age, gender, income status, etc.; behavioral data generated by the user, such as data input, business module selection, business information browsing, etc.; transaction data accumulated by the user on the platform, such as deposit amount, expenditure amount, etc.; third-party data, such as data from institutions such as the government, public utilities, and banks. The above different types of to-be-processed credit investigation data together constitute the characteristic information description data of the user. These different types of to-be-processed credit investigation data may change, which in turn affects the credit evaluation of the user. To ensure that the risk control model can utilize accurate characteristic data during iteration, it is necessary to update the characteristic information description data corresponding to each user. Therefore, it is necessary to perform offline data processing on the above first to-be-processed credit investigation data and update the processed credit investigation data into the corresponding user characteristic information description data.

[0052] S107. Store the offline credit investigation data in a database.

[0053] It can be seen that in the embodiment of the present application, by determining the first quantity information of each type of to-be-processed credit investigation data in each of multiple moments in the to-be-processed credit investigation data; determining the growth rate of the to-be-processed credit investigation data in multiple moments according to the first quantity information; determining the second quantity information of the to-be-processed credit investigation data with different contents in the same type in multiple moments according to the content information of each to-be-processed credit investigation data in each type of to-be-processed credit investigation data; determining the change frequency of each type of to-be-processed credit investigation data in multiple moments according to the second quantity information; determining the first to-be-processed credit investigation data in the to-be-processed credit investigation data according to the growth rate and the change frequency; performing offline processing on the first to-be-processed credit investigation data to obtain offline credit investigation data; and storing the offline credit investigation data in a database. In this way, a set of standardized data processing processes can be established through the solution of the present application to realize time-division and batch processing of data and reduce the pressure on the server for data processing.

[0054] In a possible example, the time interval between every two of the multiple moments is a preset time period, and the first quantity information includes the quantity corresponding to each type of to-be-processed credit investigation data at each of the moments; for determining the growth rate of the to-be-processed credit investigation data in the multiple moments according to the first quantity information, the method further includes the following steps: determining the quantity difference of each type of to-be-processed credit investigation data between the multiple moments according to the quantity corresponding to each type of to-be-processed credit investigation data at each of the moments; and determining the growth rate of each type of to-be-processed credit investigation data in the multiple moments according to the preset time period between the multiple moments and the quantity difference.

[0055] Specifically, taking the type of credit investigation data to be processed currently as the user's credit card information as an example. The quantity of the user's credit card information at the first moment is 1T, the quantity of the user's credit card information at the second moment is 2.5T, and the quantity of the user's credit card information at the third moment is 3T. The preset time interval between each moment is 1 hour. If the first quantity information currently obtained includes the quantities of the user's credit card information at the first moment and the third moment, the calculation formula for the growth rate of this type of data at this time is (3T - 1T) / 2h, that is, the growth rate is 1T / h. It should be noted that the first quantity information also includes the quantities corresponding to each type of credit investigation data to be processed at one or more other moments, and the calculation method of its growth rate is similar to the above process, which will not be elaborated here.

[0056] It can be seen that through the method proposed in the embodiment of the present application, the server can determine the growth rate of each type of credit investigation data to be processed at multiple moments according to the quantity of each type of credit investigation data to be processed at each moment. Furthermore, according to this growth rate, it is determined whether the current type of credit investigation data to be processed exceeds the growth rate threshold and needs to be processed in real time to reflect the impact of this credit investigation data to be processed on the current user's credit assessment.

[0057] In a possible example, the method for determining the change frequency of each type of credit investigation data to be processed at the multiple moments according to the second quantity information specifically includes the following steps: performing a data deduplication operation on each type of credit investigation data to be processed according to the content information of each piece of credit investigation data to be processed at the multiple moments, to obtain the credit investigation data to be processed with different contents of each type of credit investigation data to be processed at the multiple moments; counting the quantities respectively corresponding to the credit investigation data to be processed with different contents at the multiple moments, to obtain the second quantity information; and determining the change frequency of each type of credit investigation data to be processed according to the preset time interval between the multiple moments and the second quantity information.

[0058] Specifically, in the actual application scenario, it may occur that as time goes by, the change of each type of credit investigation data to be processed with time is small or does not occur, resulting in possible duplicate and redundant data in the content of each type of credit investigation data to be processed obtained at different moments. Therefore, before the server determines whether the current type of credit investigation data to be processed is the first credit investigation data to be processed, it is necessary to determine the quantities corresponding to the credit investigation data to be processed with different contents at multiple moments according to the content information of each piece of credit investigation data to be processed in each type of credit investigation data to be processed.

[0059] Specifically, a data processing operation is performed on each piece of credit investigation data to be processed of each type, including: performing stop word removal processing on each piece of credit investigation data to be processed, which is used to remove unimportant words (such as: de, di, de) or punctuation marks in each piece of credit investigation data to be processed. For example: "Zhang San bought a house in City A in 2018". After performing stop word removal processing, one possible result is: "Zhang San 2018 City A buy a house". It should be noted that the above stop word removal processing operation is based on a stop word dictionary. The acquisition methods of the stop word dictionary include, but are not limited to: directly using an existing stop word dictionary, training based on a large amount of data stored in a database to obtain a stop word dictionary applicable to the current application scenario, and so on. Among them, the stop word dictionary is used to indicate words with a relatively high frequency of occurrence but a relatively small impact on credit investigation data in the current application scenario, or words with an extremely low frequency of occurrence.

[0060] Further, word segmentation processing is performed on each piece of credit investigation data to be processed after stop word removal processing. Specifically, for example, the result after word segmentation processing of the above data may be: "Zhang San 2018 City A buy a house". It should be noted that for word segmentation processing, a pre-trained Chinese word segmentation model can be directly used, such as: Jieba word segmentation model. Or a word segmentation model pre-trained based on a large amount of data stored in a database, which is not specifically limited here.

[0061] Further, specific reference can be made to Figure 2 , such as Figure 2 a schematic diagram of a process for comparing the similarity of credit investigation data to be processed proposed in an embodiment of the present application as shown. By performing vectorization processing on each piece of credit investigation data to be processed after the above-mentioned word segmentation processing operation is completed, a representation vector of each piece of credit investigation data to be processed is obtained, such as Figure 2 the first feature vector, the second feature vector, and the Nth feature vector as shown. And vector similarity calculation is performed based on the representation vector of each piece of credit investigation data to be processed to determine the similarity between each piece of credit investigation data to be processed, and further, different pieces of credit investigation data with different contents in each type of credit investigation data to be processed are determined according to the similarity comparison result.

[0062] Further, the quantities corresponding to different pieces of credit investigation data to be processed at multiple moments are counted to obtain second quantity information. And the data change frequency of each type of credit investigation data to be processed at multiple moments is calculated according to the second quantity information. The data change frequency is used to indicate the amount of content change of the current type of credit investigation data to be processed over time.

[0063] It can be seen that in the implementation manner of the present application, the server can determine the number of the to-be-processed credit investigation data with different contents after de-duplicating the to-be-processed credit investigation data of each type, and then determine the change frequency of the to-be-processed credit investigation data corresponding to each type of the to-be-processed credit investigation data at multiple moments.

[0064] In a possible example, the method for determining the first to-be-processed credit investigation data in the to-be-processed credit investigation data according to the growth rate and the change frequency specifically includes the following steps: If the growth rate is less than a first threshold and the change frequency is less than a second threshold, then determine the to-be-processed credit investigation data of each type corresponding to the current moment as the first to-be-processed credit investigation data, where the multiple moments include the current moment, and the current moment is the latest moment corresponding to the calculation of the growth rate and the change frequency; If the growth rate reaches the first threshold at the current moment, or the change frequency reaches the second threshold at the current moment, then determine the to-be-processed credit investigation data of each type corresponding to the current moment as the second to-be-processed credit investigation data, and perform data processing on the second to-be-processed credit investigation data to obtain online credit investigation data, and store the online credit investigation data in the database.

[0065] Specifically, taking the to-be-processed credit investigation data in the solution of the present application as an example. The server compares the growth rate of the to-be-processed credit investigation data of each type calculated above with the first threshold. If the current growth rate is less than the first threshold, it indicates that the growth rate of the to-be-processed credit investigation data of the current type is relatively slow, and there is no need to adopt the real-time data processing method. Among them, the first threshold is used to indicate that the growth rate of the to-be-processed credit investigation speed of each type reaches the preset growth rate, and the preset growth rate is used to indicate that a large number of data records are received and processed in a short time during real-time data processing, and the throughput requirement reaches dozens of megabytes per second per node. If the growth rate reaches the first threshold at the current moment, then the to-be-processed credit investigation data of the current type needs to be processed in real time. In addition, the server compares the change frequency of the to-be-processed credit investigation data of each type calculated above with the second threshold. If the change frequency of the to-be-processed credit investigation data of the current type is less than the second threshold, there is no need to adopt the real-time data processing method. Among them, the second threshold is used to indicate that the change frequency of the to-be-processed credit investigation data of each type reaches the preset frequency. When the change frequency reaches the second threshold at the current moment, data processing needs to be performed in a timely manner.

[0066] Further, if the growth rate and change frequency of each type of credit investigation data to be processed at the current moment meet the above conditions, then determine each type of credit investigation data to be processed at the current moment as the first credit investigation data to be processed. Herein, the current moment can be the most recent moment or any moment corresponding to the calculated growth rate and change frequency currently. Otherwise, determine each type of credit investigation data to be processed at the current moment as the second credit investigation data to be processed. The above determination basis includes: when the growth rate at the current moment is greater than or equal to the first threshold and / or the change frequency at the current moment is greater than or equal to the second threshold.

[0067] Further, after performing data processing on the second credit investigation data to be processed to obtain the online credit investigation data, store the obtained online credit investigation data in the database.

[0068] It can be seen that in the implementation manner of this application, through the growth rate and change frequency of each type of credit investigation data to be processed, the credit investigation data to be processed is divided, and corresponding methods are adopted for data processing according to the division result, and the processed credit investigation data is stored in the database for subsequent business scenarios. In this way, it is possible to implement time-division and batch processing of data, reduce the data processing pressure on the server, and ensure the reasonable utilization of server resources.

[0069] In a possible example, after determining that each type of credit investigation data to be processed at the current moment is the first credit investigation data to be processed, the method specifically includes the following steps: deposit each of the first credit investigation data to be processed into the target message queue through the data transmission channel; if the number of the first credit investigation data in the target message queue reaches the quantity threshold for offline processing, then obtain all the first credit investigation data in the target message queue and perform the offline processing to obtain the offline credit investigation data; store the offline credit investigation data in the database.

[0070] Exemplarily, as mentioned above, the growth rate and change frequency of each type of credit investigation data in the first credit investigation data to be processed are relatively low. After the server determines that the current credit investigation data to be processed is the first credit investigation data to be processed, it does not immediately perform data offline processing, but adds it to the target message queue for temporary storage. After the number of the first credit investigation data in the current target message queue reaches the quantity requirement for offline processing in batches, obtain all the first credit investigation data from the target message queue and perform offline processing operations to obtain the offline credit investigation data. Among them, the offline processing operations include but are not limited to: data cleaning, data integration, data reduction, and data format conversion.

[0071] Specifically, the purpose of the data cleaning operation is to convert the original unstructured data into structured data. Specifically, if there are missing values ​​in the current data to be processed, the current abnormal data can be deleted by a deletion operation, or the current abnormal data can be repaired by data filling. Among them, the data filling method includes but is not limited to adding missing values ​​as default values, or using Lagrange interpolation to fill missing values.

[0072] Specifically, data integration operation refers to the process of bringing together multiple data sources and putting them into a data warehouse. Among them, each type of credit data to be processed in the first credit data to be processed may be obtained from different data sources. Therefore, it is necessary to identify redundant attributes and solve data value conflicts between data based on the attribute information of the credit data to be processed. For example: there is a field called user_id in table A of the database, and a field called user_num in table B. Then, it is possible to identify whether the current fields in tables A and B correspond to the same attribute based on the identification information that can uniquely identify each piece of credit data to be processed. For example, through user_id comparison and confirmation, if it is confirmed to be a unified attribute, when performing data integration, this section can be used as a condition for linking multiple tables, and one of the values ​​can be retained when generating a new table. Redundant attribute identification refers to whether there is a correlation between certain attributes, or whether an attribute can be derived from other attributes.

[0073] Specifically, data reduction means reducing the amount of data used for analysis while ensuring that the original data information is not lost. The most commonly used method of data reduction is dimension reduction. Dimension reduction means compressing the original high-dimensional data into low-dimensional data in a reasonable way, thereby reducing the amount of data. Feature extraction is to select attributes related to the mining target from massive data into a subtable, excluding irrelevant attributes. For example, in the data mining of credit data about user credit rating, the user's gender is irrelevant to the credit rating, so it can be excluded from the subtable.

[0074] Specifically, data conversion is the operation of normalizing and standardizing the original credit data to be processed. Normalization is to convert the original value into a decimal between (0, 1). The transformation function can use methods such as minimum and maximum normalization. Standardization is to scale the data so that it falls into a small specific interval. The commonly used function is z-score standardization. The mean after processing is 0 and the standard deviation is 1. Generally, standardization requires that the original data approximately conforms to the Gaussian distribution. In the process of data mining, features with larger values ​​will be regarded as having larger weights by the algorithm, but the actual situation is that a larger value does not necessarily mean that the feature is more important. Normalized and standardized data can avoid this problem.

[0075] It can be seen that in the implementation manner of this application, when the server determines that the currently to-be-processed credit investigation data is the first to-be-processed credit investigation data, it adds it to the target message queue, and when the target message queue meets the quantity requirement for offline processing, it performs offline processing on the first to-be-processed credit investigation data in the data queue and then stores it in the database. In this way, the purpose of batch processing and storing data is achieved.

[0076] In a possible example, after storing the offline credit investigation data in the database, the method further includes the following steps: performing user analysis on the user according to the offline credit investigation data to obtain a user portrait, where the user portrait includes parameter information for characterizing the credit investigation level of the user; determining service configuration parameters corresponding to the parameter information of the user's credit investigation level according to credit investigation rules; and associating and saving the service configuration parameters corresponding to the user and the user portrait into the credit investigation data of the corresponding user in the database.

[0077] Specifically, the offline credit investigation data stored in the database includes basic information of at least one user, such as: the user's identity information, occupation information, residence information, etc.; credit information of at least one user, such as the user's credit card, mortgage and other loan repayment information, etc.; the user's public information, such as: social security and provident fund information, court information, tax arrears information, administrative law enforcement information, etc. Through data analysis of the credit investigation data of at least one user, a user credit rating score table and credit labels corresponding to each user can be obtained. An all-round evaluation of the user can be obtained, and the user portrait results of different users can be presented in the form of a radar chart or a table. Among them, the radar chart includes parameter information for characterizing the credit investigation level of the user.

[0078] Further, determine service configuration parameters corresponding to the parameter information of the user's credit investigation level according to the credit investigation rules saved in the database, and save the user's service configuration parameters and user portrait into the credit investigation data of the corresponding user in the database. Among them, the credit investigation rules include, but are not limited to: service configuration parameters corresponding to when the user's credit rating is high, medium, or low, and the service configuration parameters include: information of service personnel provided for the user, available credit limit information of the credit card provided for the user, etc.

[0079] In another possible example, different service plans corresponding to different user credit investigation levels are preset in the database. After performing a user portrait based on the user's credit investigation data, a service plan can also be set for the user according to the user portrait result of the user. The service plan includes the credit limit and repayment interest rate corresponding to the user, basic information of the service personnel corresponding to the user's credit business, etc.

[0080] Specifically, if the user's credit information indicates that the user has no overdue records, the service plan is set to provide the user with the credit limit corresponding to the user's credit rating; if the user's credit information indicates that the user has overdue records and the overdue amount has been repaid, the service plan is set to increase the user's repayment interest rate or reduce the user's credit limit; if the user's credit information indicates that the user has overdue records and the overdue amount has not been repaid, the service plan is set to reject the user's loan application; establish an association relationship between the user and the corresponding service plan, and store the association relationship in the database.

[0081] Specifically, according to the business requirements in the user's credit business and their credit rating, query the preset rules to match the corresponding service personnel for different users, and send the user's credit data and user portrait results to the corresponding service personnel. In this way, the quality and efficiency of the service can be guaranteed.

[0082] It can be seen that in the embodiment of the present application, the user portrait can be performed on the user through the processed offline credit data, and the business configuration parameters adapted to the user can be determined according to the user's credit rating, and then the business configuration parameters are associated with the corresponding user portrait information and saved in the database for storage. In this way, the reasonable utilization of the offline credit data can be realized, and a more comprehensive understanding of the user is beneficial to providing more accurate services for the user.

[0083] In a possible example, the offline credit data includes the credit data of at least one user, and the credit data of the at least one user includes the personal information and credit records of the at least one user, and the at least one user has a unique identification ID in the database; for the user analysis based on the offline credit data, the method further includes the following steps: performing a risk assessment on the at least one user according to the credit data of the at least one user and the preset rules to obtain the risk assessment results of the at least one user; generating risk assessment data of the at least one user according to the risk assessment results of the at least one user to obtain the user portrait of the at least one user; constructing a mapping relationship between the identification ID of the at least one user and the user portrait of the at least one user, and storing the mapping relationship in the credit data of the at least one user in the above database.

[0084] Specifically, when the server saves the user's offline credit data, it sets a user identity identification ID for each user, and the identity identification ID uniquely identifies the corresponding user, and the credit information of each user can be retrieved through the user's identity identification ID.

[0085] Further, at least one user is risk-assessed according to the user's credit data and preset rules stored in the database, and a risk assessment report of the user is generated. Among them, the risk assessment report is used to indicate the user's risk response ability. The obtained risk assessment report of each user is correspondingly associated with the user portrait of the user. And a mapping relationship between the user portrait and the user identification ID is established.

[0086] Further, when the server detects that a businessperson retrieves relevant information of a target user through the user identification ID, while presenting the processed credit data of the target user stored in the database, the user portrait result of the user is displayed for the businessperson to refer to.

[0087] It can be seen that in the embodiment of the present application, the user is risk-assessed through the user's credit data, and the assessment result is saved to the user portrait. The server displays the credit data and the user portrait of the target user corresponding to the search of the target user identification ID by the businessperson, so that the characteristic information of the target user can be presented more comprehensively and flexibly, improving the efficiency and effectiveness of the service.

[0088] For a better understanding of the data processing flow of the solution of the present application, please refer to Figure 3 the overall architecture diagram of a data processing provided by the embodiment of the present application as shown. Specifically as Figure 3 shown: It should be noted that the illustrated Kafka message system is a model similar to the producer-consumer model. The producer sends the data to be processed into the Kafka message system, and Kafka processes the data as a consumer role. Flink is an open-source big data streaming computing engine very commonly used in big data development. It supports both batch processing and stream processing. It can be used as the producer of Kafka or the consumer of Flink. Among them, Figure 3 what is shown is just one of the situations, that is, Flink processes the data transmitted from Kafka as the consumer of Kafka.

[0089] Specifically, the server obtains the credit data to be processed at multiple moments from the relational database. If it is determined that the credit data to be processed at the T+0 moment is the second credit data to be processed, the second credit data to be processed is processed in real time.

[0090] Specifically, the real-time processing can be jointly performed by Kafka + Flink shown in the figure. Kafka obtains the second credit data to be processed from the relational data, and then hands it over to Flink to perform common data engineering, global aggregation and other real-time computing tasks, completing the real-time processing operation of the second credit data to be processed.

[0091] If it is determined that the credit investigation data to be processed at time T+1 is the first credit investigation data to be processed, offline processing is performed on the first credit investigation data to be processed. Specifically, the scheduling operation platform performs job scheduling to select the first credit investigation data to be processed offline. The data operation layer performs tasks such as data cleaning and verification on the first credit investigation data to be processed, and sends the processed first credit investigation data to the data warehouse layer. The data warehouse layer further performs data index processing and then sends it to the data application layer. What the data application layer receives is the standardized offline credit investigation data that has completed offline processing and can be used for data analysis, user profiling, and other operations.

[0092] It can be seen that through the solution of the embodiment of the present application, the credit investigation data to be processed can be divided into the first credit investigation data to be processed and the second credit investigation data to be processed, and then different processing methods and processing processes are respectively adopted to complete the data processing operation. In this way, time-sharing and batch processing of data can be realized, and reasonable allocation and utilization of resources can be achieved.

[0093] Consistent with the above embodiment, please refer to Figure 4 , Figure 4 is a schematic structural diagram of a server provided by an embodiment of the present application. As shown in the figure, it includes a processor, a memory, a communication interface, and one or more programs. Among them, the above one or more programs are stored in the above memory and are configured to be executed by the above processor. In the embodiment of the present application, the above programs include instructions for performing the following steps:

[0094] Determine the first quantity information of each type of credit investigation data to be processed at each moment among multiple moments;

[0095] Determine the growth rate of the credit investigation data to be processed at the multiple moments according to the first quantity information;

[0096] According to the content information of each piece of credit investigation data to be processed in each type of credit investigation data to be processed, determine the second quantity information of the credit investigation data to be processed with different contents in the same type at the multiple moments;

[0097] Determine the change frequency of each type of credit investigation data to be processed at the multiple moments according to the second quantity information;

[0098] Determine the first credit investigation data to be processed in the credit investigation data to be processed according to the growth rate and the change frequency;

[0099] Perform offline processing on the first credit investigation data to be processed to obtain offline credit investigation data;

[0100] Store the offline credit investigation data in the database.

[0101] It can be seen that the server described in the embodiments of the present application determines the first quantity information of each type of to-be-processed credit investigation data at each moment among multiple moments in the to-be-processed credit investigation data; determines the growth rate of the to-be-processed credit investigation data at multiple moments according to the first quantity information; determines the second quantity information of the to-be-processed credit investigation data with different contents in the same type at multiple moments according to the content information of each to-be-processed credit investigation data in each type of to-be-processed credit investigation data; determines the change frequency of each type of to-be-processed credit investigation data at multiple moments according to the second quantity information; determines the first to-be-processed credit investigation data in the to-be-processed credit investigation data and performs offline processing, and stores the offline credit investigation data in the database after obtaining it. In this way, a set of standardized credit investigation data processing processes can be established to facilitate daily credit investigation data processing and index analysis work, reduce the pressure on the server for data processing, and improve data quality and security through the control of the data governance system.

[0102] The above mainly introduces the solution of the embodiments of the present application from the perspective of the execution process on the method side. It can be understood that, in order to implement the above functions, it includes the corresponding hardware structure and / or software module for executing each function. Those skilled in the art should easily realize that, combining the units and algorithm steps of each example described in the embodiments provided in this article, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a certain function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0103] The embodiments of the present application can perform function unit division according to the above method examples. For example, each function unit can be divided corresponding to each function, or two or more functions can be integrated into one processing unit. The above integrated unit can be implemented in the form of hardware or in the form of a software function unit. It should be noted that the division of units in the embodiments of the present application is illustrative, only a logical function division, and there may be other division methods in actual implementation.

[0104] Please refer to Figure 5 , Figure 5 which is a functional unit composition block diagram of a data processing device provided by an embodiment of the present application. The device 500 includes: a determination unit 501, a calculation unit 502, a processing unit 503, and a storage unit 504, where

[0105] The determining unit 501 is configured to determine the first quantity information of each type of credit investigation data to be processed at each moment; and is further configured to determine the second quantity information of the credit investigation data to be processed with different contents at multiple moments according to the content information of each piece of credit investigation data to be processed of each type of credit investigation data to be processed; and is further configured to determine the first credit investigation data to be processed in the credit investigation data to be processed according to the growth rate and the change frequency;

[0106] The calculating unit 502 is configured to determine the growth rate of the credit investigation data to be processed at multiple moments according to the first quantity information; and is further configured to determine the change frequency of each type of credit investigation data to be processed according to the second quantity information;

[0107] The processing unit 503 is configured to perform offline processing on the first credit investigation data to be processed to obtain offline credit investigation data;

[0108] The storage unit 504 is configured to store the offline credit investigation data into a database.

[0109] It can be seen that in the data processing device described in the embodiment of the present application, the determining unit determines the first quantity information of each type of credit investigation data to be processed at each moment among multiple moments; the calculating unit determines the growth rate of the credit investigation data to be processed at multiple moments according to the first quantity information; the determining unit determines the second quantity information of the credit investigation data to be processed with different contents in the same type at multiple moments according to the content information of each piece of credit investigation data to be processed of each type of credit investigation data to be processed; the calculating unit determines the change frequency of each type of credit investigation data to be processed at multiple moments according to the second quantity information; the processing unit determines the first credit investigation data to be processed in the credit investigation data to be processed according to the growth rate and the change frequency and performs offline processing, and after obtaining the offline credit investigation data, the storage unit stores it into the database. In this way, a set of standardized credit investigation data processing processes can be established to facilitate daily credit investigation data processing and index analysis work, and through the control of the data governance system, the pressure on the server data processing can be reduced, and the data quality and security can be improved.

[0110] In a possible example, the interval between every two of the multiple moments is a preset time period, the first quantity information includes the quantity respectively corresponding to each type of credit investigation data to be processed at each moment; for determining the growth rate of the credit investigation data to be processed at the multiple moments according to the first quantity information, the specific steps that the calculating unit 502 further executes are as follows:

[0111] Determine the quantity difference of each type of credit investigation data to be processed between the multiple moments according to the quantity respectively corresponding to each type of credit investigation data to be processed at each moment;

[0112] Determine the growth rate of each type of credit investigation data to be processed at the multiple moments according to the preset time duration between the multiple moments and the quantity difference.

[0113] In a possible example, for the process of determining the change frequency of each type of credit investigation data to be processed according to the second quantity information, the specific steps further executed by the processing unit 503 are as follows:

[0114] Perform data deduplication on each type of credit investigation data to be processed according to the content information of each piece of credit investigation data to be processed at the multiple moments, and obtain the credit investigation data to be processed with different contents of each type of credit investigation data to be processed at the multiple moments;

[0115] Count the quantities corresponding to the credit investigation data to be processed with different contents at the multiple moments, and obtain the second quantity information;

[0116] Determine the change frequency of each type of credit investigation data to be processed according to the preset time duration between the multiple moments and the second quantity information.

[0117] In a possible example, for the process of determining the first credit investigation data to be processed in the credit investigation data to be processed according to the growth rate and the change frequency, the specific steps further executed by the processing unit 503 are as follows:

[0118] If the growth rate is less than the first threshold and the change frequency is less than the second threshold, then determine that each type of credit investigation data to be processed corresponding to the current moment is the first credit investigation data to be processed, where the multiple moments include the current moment, and the current moment is the latest moment corresponding to the calculation of the growth rate and the change frequency;

[0119] If the growth rate reaches the first threshold at the current moment, or the change frequency reaches the second threshold at the current moment, then determine that each type of credit investigation data to be processed corresponding to the current moment is the second credit investigation data to be processed, and perform data processing on the second credit investigation data to be processed to obtain online credit investigation data, and store the online credit investigation data in the database.

[0120] In a possible example, after determining that each type of credit investigation data to be processed corresponding to the current moment is the first credit investigation data to be processed, the specific steps further executed by the processing unit 503 are as follows:

[0121] Store each of the first credit investigation data to be processed into the target message queue through the data transmission channel;

[0122] If the quantity of the first to-be-processed credit investigation data in the target message queue reaches the quantity threshold for offline processing, obtain all the first to-be-processed credit investigation data in the target message queue and perform the offline processing to obtain the offline credit investigation data;

[0123] Store the offline credit investigation data in the database.

[0124] In a possible example, after storing the offline credit investigation data in the database, the processing unit 503 is specifically further configured to perform the following steps:

[0125] Perform user analysis on the user according to the offline credit investigation data to obtain a user portrait, where the user portrait includes parameter information for characterizing the credit level of the user;

[0126] Determine the service configuration parameters corresponding to the parameter information of the credit level of the user according to the credit rules;

[0127] Save the service configuration parameters corresponding to the user and the user portrait to the credit investigation data of the corresponding user in the database.

[0128] In a possible example, the offline credit investigation data includes the credit investigation data of at least one user, the credit investigation data of the at least one user includes the personal information and credit records of the at least one user, and the at least one user has a unique identification ID in the database; for performing user analysis on the user according to the offline credit investigation data, the processing unit 50 is specifically further configured to perform the following steps:

[0129] Perform risk assessment on the at least one user according to the credit investigation data of the at least one user and preset rules to obtain the risk assessment results of the at least one user;

[0130] Generate risk assessment data for the at least one user according to the risk assessment results of the at least one user to obtain the user portrait of the at least one user;

[0131] Construct a mapping relationship between the identification ID of the at least one user and the user portrait of the at least one user, and store the mapping relationship in the credit investigation data of the at least one user in the above database.

[0132] It should be noted that all relevant contents of each step involved in the above method embodiments can be cited in the function descriptions of the corresponding functional modules, and will not be elaborated here.

[0133] The server provided in this embodiment is used to execute the above data processing method, and thus can achieve the same effect as the above implementation method.

[0134] In the case of adopting an integrated unit, the server may include a processing module, a storage module, and a communication module. Among them, the processing module may be used to control and manage the actions of the server. For example, it may be used to support the server to execute the steps performed by the above-mentioned determination unit 501, calculation unit 502, processing unit 503, and storage unit 504. The storage module may be used to support the server to execute storing program codes and data, etc. The communication module may be used to support the communication between the server and other devices.

[0135] Among them, the processing module may be a processor or a controller. It may implement or execute various exemplary logical blocks, modules, and circuits described in combination with the disclosure of this application. The processor may also be a combination that realizes computing functions, such as a combination including one or more microprocessors, a combination of digital signal processing (DSP) and a microprocessor, and so on. The storage module may be a memory. The communication module may specifically be a device for interacting with other servers, such as a radio frequency circuit, a Bluetooth chip, a Wi-Fi chip, etc.

[0136] This application embodiment also provides a computer storage medium. Among them, this computer storage medium stores a computer program for electronic data exchange, and this computer program enables a computer to execute part or all of the steps of any method recorded in the above method embodiment. The above computer includes a control platform.

[0137] This application embodiment also provides a computer program product. The above computer program product includes a non-transitory computer-readable storage medium storing a computer program, and the above computer program is operable to enable a computer to execute part or all of the steps of any method recorded in the above method embodiment. This computer program product may be a software installation package, and the above computer includes a control platform.

[0138] It should be noted that, for the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that this application is not limited by the described action sequence, because according to this application, certain steps may be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0139] In the above embodiments, the descriptions of each embodiment have their own emphases. For the parts not detailed in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

[0140] In several embodiments provided in the present application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the above division of units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces. The indirect coupling or communication connection of the device or unit can be in electrical or other forms.

[0141] The units described above as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they can be located in one place, or they can be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0142] In addition, each functional unit in various embodiments of the present application can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0143] If the above integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the above methods in various embodiments of the present application. And the aforementioned memory includes: USB flash drives, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), mobile hard disks, magnetic disks, or optical discs and other media that can store program codes.

[0144] Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing relevant hardware through a program. This program can be stored in a computer-readable memory. The memory can include: flash drives, read-only memories (abbreviation: ROM), random access memories (abbreviation: RAM), magnetic disks, or optical discs, etc.

[0145] The embodiments of the present application have been introduced in detail above. Specific examples are used herein to elaborate on the principle and implementation manner of the present application. The description of the above embodiments is only used to help understand the method and its core idea of the present application; at the same time, for those of ordinary skill in the art, according to the idea of the present application, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present application.

Claims

1. A data processing method, characterized in that, Applied to a server, the method includes: Determine the first quantity information of each type of credit investigation data to be processed at each moment among multiple moments; Determine the growth rate of the credit investigation data to be processed at the multiple moments according to the first quantity information; Determine the second quantity information of the credit investigation data to be processed with different contents in the same type at the multiple moments according to the content information of each piece of credit investigation data to be processed in each type of credit investigation data to be processed; Determine the change frequency of each type of credit investigation data to be processed at the multiple moments according to the second quantity information; Determine the first credit investigation data to be processed in the credit investigation data to be processed according to the growth rate and the change frequency; Perform offline processing on the first credit investigation data to be processed to obtain offline credit investigation data; Store the offline credit investigation data in a database; Among them, the determining the first credit investigation data to be processed in the credit investigation data to be processed according to the growth rate and the change frequency includes: If the growth rate is less than a first threshold and the change frequency is less than a second threshold, determine that the credit investigation data of each type corresponding to the current moment is the first credit investigation data to be processed, where the multiple moments include the current moment, and the current moment is the latest moment corresponding to the calculation of the growth rate and the change frequency; If the growth rate reaches the first threshold at the current moment, or the change frequency reaches the second threshold at the current moment, determine that the credit investigation data of each type corresponding to the current moment is the second credit investigation data to be processed, and perform data processing on the second credit investigation data to be processed to obtain online credit investigation data, and store the online credit investigation data in the database.

2. The method according to claim 1, characterized in that, There is a preset time interval between every two of the multiple moments, and the first quantity information includes the quantity respectively corresponding to each type of credit investigation data to be processed at each moment; The determining the growth rate of the credit investigation data to be processed at the multiple moments according to the first quantity information includes: Determine the quantity difference of each type of credit investigation data to be processed between the multiple moments according to the quantity respectively corresponding to each type of credit investigation data to be processed at each moment; Determine the growth rate of each type of credit investigation data to be processed at the multiple moments according to the preset time interval between the multiple moments and the quantity difference.

3. The method according to claim 2, characterized in that, The determining the change frequency of each type of credit investigation data to be processed at the multiple moments according to the second quantity information includes: Perform data deduplication on each type of credit investigation data to be processed according to the content information of each piece of credit investigation data to be processed among the multiple moments to obtain the credit investigation data to be processed with different contents of each type of credit investigation data to be processed at the multiple moments; Count the quantities respectively corresponding to the credit investigation data to be processed with different contents at the multiple moments to obtain the second quantity information; Determine the change frequency of the to-be-processed credit investigation data of each type according to the preset duration between the multiple moments and the second quantity information.

4. The method according to claim 1, characterized in that, After determining that the to-be-processed credit investigation data of each type corresponding to the current moment is the first to-be-processed credit investigation data, the method further includes: Deposit each of the first to-be-processed credit investigation data into the target message queue through the data transmission channel; If the quantity of the first to-be-processed credit investigation data in the target message queue reaches the quantity threshold for offline processing, obtain all the first to-be-processed credit investigation data in the target message queue and perform the offline processing to obtain the offline credit investigation data; Deposit the offline credit investigation data into the database.

5. The method according to claim 4, characterized in that, After depositing the offline credit investigation data into the database, the method further includes: Perform user analysis on the user according to the offline credit investigation data to obtain a user portrait, where the user portrait includes parameter information for characterizing the credit rating of the user; Determine the service configuration parameters corresponding to the parameter information of the user's credit rating according to the credit rules; Save the service configuration parameters corresponding to the user and the user portrait into the credit investigation data of the corresponding user in the database.

6. The method according to claim 5, characterized in that, The offline credit investigation data includes the credit investigation data of at least one user, and the credit investigation data of the at least one user includes the personal information and credit records of the at least one user, and the at least one user has a unique identification ID in the database; The performing user analysis on the user according to the offline credit investigation data includes: Perform risk assessment on the at least one user according to the credit investigation data of the at least one user and preset rules to obtain the risk assessment results of the at least one user; Generate risk assessment data for the at least one user according to the risk assessment results of the at least one user to obtain the user portrait of the at least one user; Construct a mapping relationship between the identification ID of the at least one user and the user portrait of the at least one user, and store the mapping relationship into the credit investigation data of the at least one user in the above database.

7. A data processing device, characterized in that, Applied to a server, the device includes a determination unit, a calculation unit, a processing unit, and a storage unit: The determination unit is configured to determine the first quantity information of each type of to-be-processed credit investigation data in each of the multiple moments in the to-be-processed credit investigation data; The calculation unit is configured to determine the growth rate of the to-be-processed credit investigation data in the multiple moments according to the first quantity information; The determination unit is further configured to determine the second quantity information of the to-be-processed credit investigation data with different contents in the same type in the multiple moments according to the content information of each piece of to-be-processed credit investigation data in each type of to-be-processed credit investigation data; The calculation unit is further configured to determine the change frequency of the to-be-processed credit investigation data of each type in the multiple moments according to the second quantity information; The determination unit is further configured to determine the first to-be-processed credit investigation data in the to-be-processed credit investigation data according to the growth rate and the change frequency; The processing unit is configured to perform offline processing on the first credit investigation data to be processed, so as to obtain offline credit investigation data; The storage unit is configured to store the offline credit investigation data into a database; Wherein, determining the first credit investigation data to be processed in the credit investigation data to be processed according to the growth rate and the change frequency includes: If the growth rate is less than a first threshold and the change frequency is less than a second threshold, then determine the credit investigation data to be processed of each type corresponding to the current moment as the first credit investigation data to be processed, where the multiple moments include the current moment, and the current moment is the latest moment corresponding to the calculation of the growth rate and the change frequency; If the growth rate reaches the first threshold at the current moment, or the change frequency reaches the second threshold at the current moment, then determine the credit investigation data to be processed of each type corresponding to the current moment as the second credit investigation data to be processed, and perform data processing on the second credit investigation data to be processed to obtain online credit investigation data, and store the online credit investigation data into the database.

8. A server, characterized in that, It includes a processor, a memory, a communication interface, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the processor, and the one or more programs include instructions for performing the steps in the method according to any one of claims 1-6.

9. A readable computer storage medium, characterized in that, The computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the method according to any one of claims 1-6.

Images