Multi-level processing method and system for credit big data based on data warehouse modeling

Abstract

The invention discloses a multi-level processing method and system for credit big data based on data warehouse modeling, and the method comprises the steps: carrying out the classification according to the channel source and characteristics of original data, and employing different uploading modes to a data platform; the platform performs standardization processing and dirty data cleaning on the uploaded original data to construct standard data; classifying standard data based on a business process, establishing corresponding business models, analyzing relevance between the models, and performing multi-level development; carrying out standardized definition on credit indexes and binding the credit indexes to a business model to realize credit analysis and monitoring; and matching a data output mode according to the service scene, constructing a full-link data quality assurance system, and generating a data quality report. Therefore, the data processing efficiency can be improved, repeated processing can be avoided, the data reusability is improved through a multi-level development model, intuitive and visual data analysis and monitoring services can be provided, the business processing efficiency is improved, and data service requirements can be responded more quickly.

Description

Technical Field

[0001] The present invention relates to the technical field of big data processing and analysis, and in particular to a multi-level processing method and system for credit big data based on data warehouse modeling. Background Art

[0002] In order to prevent systemic financial risks, all-round monitoring of credit loans has become a key focus and development for major financial institutions. With the gradual improvement of digital collection systems, massive amounts of credit loan data are efficiently acquired, providing financial institutions with a rich basis for credit risk assessment.

[0003] The existing risk control system still has many deficiencies in data processing and analysis, such as data link chimney development, low reusability and redundancy, decentralized data processing, and difficulty in data monitoring and tracing. These problems have caused the data processing and analysis process to be out of touch with the actual business scenario, increased time and governance costs, affected the efficiency and accuracy of data processing and analysis, and also restricted the stability of credit business and the improvement of risk control capabilities.

[0004] As disclosed in publication number CN115187388A, a pre-set credit data model construction requirement is obtained, and the specified configured data is processed according to the credit data model construction requirement to obtain data processing indicators; the data processing indicators are input into the credit data model to obtain calculation results of the credit data model. However, this technical solution still has the problem of repeated construction of multiple business scenarios, a large increase in time costs, and inevitable data and business confusion, and cannot meet the current actual business needs for high data reuse capabilities and processing efficiency.

[0005] Therefore, the existing technology has obvious deficiencies in data reuse capability and processing efficiency, and there is an urgent need for a management method and device that can improve data reuse and processing efficiency. Summary of the invention

[0006] One object of the present invention is to propose a multi-level processing method for credit big data based on data warehouse modeling. The present invention improves the reusability of data and can respond to data service needs more quickly by uploading the original data in a classified manner and performing multi-level development of data models according to business relevance.

[0007] To achieve the above objectives, the technical solutions adopted by the present invention include:

[0008] A multi-level processing method for credit big data based on data warehouse modeling, characterized by comprising the following steps:

[0009] S1. Obtaining original data, classifying the original data according to its channel source and characteristics, and uploading it to the data platform using different uploading methods according to its classification;

[0010] S2. The data platform performs standardization processing and dirty data cleaning on the uploaded raw data to construct standard data;

[0011] S3. Classify the standard data based on the business process, establish corresponding business models according to each stage in the business process, analyze the correlation between the business models, and establish the correlation between business models within each data classification and between data classifications, and perform multi-level development;

[0012] S4. By analyzing and monitoring the credit indicators in the business process, a standardized definition of the credit indicators is performed, and the credit indicators are bound to the corresponding business models;

[0013] S5. Match the corresponding data output method according to different business scenarios;

[0014] S6. Build a full-link data quality assurance system and generate a data quality report.

[0015] Further, the S1 specifically includes:

[0016] S11. Obtaining original data, and classifying the original data into actively provided data, internal system data, and external system data according to the channel source and characteristics of the original data;

[0017] S12, uploading the actively provided data to a server using a secure and encrypted transmission protocol;

[0018] Transmitting the internal system data to the data platform in real time via an internal data bus system;

[0019] The external system data is pulled and uploaded to the data platform in a scheduled or real-time manner according to its API interface specifications.

[0020] Further, the S2 specifically includes:

[0021] S21. Use a data quality check tool to explore the raw data, identify and mark abnormal values, missing values, duplicate values, etc.;

[0022] S22. Clean the raw data according to preset cleaning rules, where the preset cleaning rules include but are not limited to removing duplicate data, filling or deleting missing values, correcting outliers, etc.;

[0023] S23, constructing the cleaned original data into standard data after verification.

[0024] Further, the S3 specifically includes:

[0025] S31. Classifying the standard data according to the business processes before, during and after the loan;

[0026] S32. Establish a corresponding business model for each business stage, wherein the business model includes a pre-loan business model, a mid-loan business model, and a post-loan business model;

[0027] S33, analyzing the correlation between the business models, and establishing data correlation of the business models, and performing multi-level development.

[0028] Further, the S4 specifically includes:

[0029] S41. defining its credit indicators based on the business process;

[0030] S42: Bind the credit indicator to the corresponding business model, and evaluate the business model by monitoring the credit indicator.

[0031] Further, the S5 specifically includes:

[0032] S51. Determine the format and content of data output according to the needs of the business scenario;

[0033] S52. Matching a corresponding data output method based on the format and content of the data output, wherein the data output method includes a data report, a visual chart, and an API interface.

[0034] Further, the S6 specifically includes:

[0035] S61. Establish a full-link data quality monitoring system, which includes data source quality monitoring, data processing process quality monitoring, and data output quality monitoring;

[0036] S62. Generate a data quality report according to the data quality monitoring system to evaluate and report the quality status of the data.

[0037] In addition, the present invention also relates to a multi-level credit big data processing system based on data warehouse modeling, which is characterized by comprising:

[0038] The data acquisition module is used to acquire the original data, classify the original data according to its channel source and characteristics, and upload it to the data platform using different uploading methods according to its classification;

[0039] A data processing module is used to perform standardization processing on the uploaded raw data and clean dirty data to construct standard data;

[0040] A model building module is used to classify the standard data based on the business process, establish corresponding business models according to each stage in the business process, analyze the correlation between the business models, and establish the correlation between business models within each data classification and between data classifications, and perform multi-level development;

[0041] The indicator monitoring module is used to analyze and monitor the credit indicators in the business process to standardize the definition of credit indicators and bind the credit indicators to the corresponding business model;

[0042] Data output module, used to match corresponding data output methods according to different business scenarios;

[0043] The quality assurance module is used to build a full-link data quality assurance system and generate data quality reports.

[0044] In addition, the present invention also relates to an electronic device, characterized in that it comprises a processor and a memory;

[0045] The memory is used to store operation instructions;

[0046] The processor is used to execute the above method by calling the operation instruction.

[0047] In addition, the present invention also relates to a computer-readable storage medium, characterized in that a computer program is stored on the storage medium, and the computer program implements the above method when executed by a processor.

[0048] Adopting the scheme provided by this application, the original data is classified according to its channel source and characteristics, and different upload methods are adopted to the data platform; the platform standardizes the uploaded original data and cleans the dirty data to construct standard data; classifies the standard data based on the business process, establishes the corresponding business model and analyzes the correlation between models, and conducts multi-level development; standardizes the definition of credit indicators and binds them to the business model to realize credit analysis and monitoring; matches the data output method according to the business scenario, builds a full-link data quality assurance system, and generates a data quality report. Therefore, the above technical scheme can improve the efficiency of data processing, avoid repeated processing, and improve data reusability through multi-level development models. Through the output method matching the data, it can provide intuitive and visual data analysis and monitoring services, improve the efficiency of business processing and respond to data service needs more quickly. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 A flowchart of a multi-level credit big data processing method based on data warehouse modeling provided in an embodiment of the present application;

[0050] Figure 2This is a schematic diagram of classifying original data in an embodiment of the present application;

[0051] Figure 3 An architecture diagram of a multi-level credit big data processing system based on data warehouse modeling provided in an embodiment of the present application;

[0052] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0053] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be interpreted as limiting the present invention.

[0054] Those skilled in the art can understand that, unless otherwise stated, the singular forms "one", "an", "said" and "the" used herein may also include plural forms, and the "first", "second", etc., are only used to distinguish and define objects for the purpose of introducing the scheme clearly, and do not limit the objects themselves. Of course, the objects defined by "first" and "second" may be the same terminal, device, user, etc., or the same type of terminal, device and user. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. The term "and / or" used herein includes all or any unit and all combinations of one or more associated listed items.

[0055] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following embodiments may be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

[0056] Figure 1 The flowchart of a multi-level credit big data processing method based on data warehouse modeling provided by the embodiment of the present application is shown, which specifically includes:

[0057] S1. Obtaining original data, classifying the original data according to its channel source and characteristics, and uploading it to the data platform using different uploading methods according to its classification;

[0058] In this implementation, S1 specifically includes:

[0059] S11, obtain the original data, such as Figure 2As shown, the original data is divided into actively provided data, internal system data, and external system data according to the channel source and characteristics of the original data;

[0060] Specifically, the data actively provided includes personal information, contact information, device information, etc.;

[0061] Internal system data includes user behavior, order details, risk lists, middle office derivatives, etc.

[0062] External system data includes personal identity, credit data, consumption data, travel data, social data and other information;

[0063] S12, uploading the actively provided data to a server using a secure and encrypted transmission protocol;

[0064] Transmitting the internal system data to the data platform in real time via an internal data bus system;

[0065] The external system data is pulled and uploaded to the data platform in a scheduled or real-time manner according to its API interface specifications.

[0066] By disassembling and organizing various types of raw data and selecting different uploading methods according to their characteristics, repeated processing caused by data mismatch can be avoided, thereby improving the overall efficiency of modeling.

[0067] S2. The data platform performs standardization processing and dirty data cleaning on the uploaded raw data to construct standard data;

[0068] In this implementation, S2 specifically includes:

[0069] S21. Use data quality inspection tools to explore the raw data, identify and mark abnormal values, missing values, duplicate values, etc., establish data awareness of related businesses in advance, and understand the basic details of each dimension, indicator and fact;

[0070] S22. Clean the raw data according to preset cleaning rules. The preset cleaning rules include but are not limited to removing duplicate data, filling or deleting missing values, correcting abnormal values, etc., and also include decomposing complex structures into simple structures as much as possible. Specifically, key, value information, IP address, user Agent and other information can be further decomposed into basic information such as address, model, version, etc.;

[0071] S23, constructing the cleaned original data into standard data after verification.

[0072] S3. Classify the standard data based on the business process, establish corresponding business models according to each stage in the business process, analyze the correlation between the business models, and establish the correlation between business models within each data classification and between data classifications, and perform multi-level development;

[0073] In this implementation, S3 specifically includes:

[0074] S31. Classifying the standard data according to the business processes before, during and after the loan;

[0075] S32. Establish a corresponding business model for each business stage, wherein the business model includes a pre-loan business model, a mid-loan business model, and a post-loan business model;

[0076] S33, analyzing the correlation between the business models, and establishing data correlation of the business models, and performing multi-level development.

[0077] Specifically, in the pre-loan business stage, comprehensive information of borrowers is mainly collected for credit rating. By aggregating personal information data and associating other basic information through the Left Join method of SQL statements, a pre-loan business model is formed;

[0078] During the loan business phase, we mainly focus on loan review, remove redundant dimensions from behavioral data, and set up indexes to form a loan business model.

[0079] In the post-loan business stage, the main task is to monitor loan usage and be responsible for loan recovery. It is also based on behavioral data, but unnecessary dimensions will be removed to form a post-loan business model. The post-loan business model will be associated with the pre-loan and mid-loan data models to achieve comprehensive tracking of various data throughout the loan life cycle.

[0080] Based on the joint model, multi-level development is carried out. First, the light summary layer is developed. By initially summarizing and aggregating the data, the light summary layer effectively removes redundant information, provides a more refined data set for subsequent analysis, improves the efficiency of data query, and lays the foundation for deeper data mining. Secondly, the business summary layer is developed. The business summary layer focuses on the business logic and association of the data, and is committed to building a data model that can support complex business decisions. Finally, the business application layer is developed. The business application layer is directly oriented to specific business scenarios and application needs, such as risk control scoring, post-loan monitoring, etc.

[0081] S4. By analyzing and monitoring the credit indicators in the business process, a standardized definition of the credit indicators is performed, and the credit indicators are bound to the corresponding business models;

[0082] In this implementation, S4 specifically includes:

[0083] S41. defining its credit indicators based on the business process;

[0084] Specifically, the basic information of the credit indicator, such as name, description, data type, etc., is specified based on the indicator template. According to the specific needs of the credit business, the various dimensions of the indicator are defined, such as credit limit, loan term, etc., and the specific meaning and value range of each dimension are clarified. By defining credit indicators, data quality and analysis accuracy are ensured.

[0085] S42: Bind the credit indicator to the corresponding business model, and evaluate the business model by monitoring the credit indicator.

[0086] Specifically, the defined credit indicators are bound to the corresponding business models to ensure that each credit indicator accurately corresponds to the specific business data and model. The indicator layer model is constructed and verified, and the accuracy of the indicator value is checked by comparing and analyzing the test data or historical data.

[0087] Configure the monitoring strategy of credit indicators, including setting the monitoring cycle, monitoring threshold and alarm method. When the value of the credit indicator exceeds the set threshold, the system will automatically send an alarm message;

[0088] Deploy the configured monitoring strategy to the system and ensure that the system can automatically perform monitoring tasks and provide real-time feedback on monitoring results. This allows for timely discovery and handling of abnormal situations, thereby adjusting business strategies or optimizing data models based on monitoring results. Therefore, by accurately binding and effectively monitoring credit indicators with business models, the risk management capabilities and decision-making support levels of credit business can be improved.

[0089] S5. Match the corresponding data output method according to different business scenarios;

[0090] In this implementation, S5 specifically includes:

[0091] S51. Determine the format and content of data output according to the needs of the business scenario;

[0092] S52. Matching a corresponding data output method based on the format and content of the data output, wherein the data output method includes a data report, a visual chart, and an API interface.

[0093] Specifically, we select the appropriate data output method according to the business scenario. By analyzing the data usage requirements of different users and business scenarios, we can choose to provide real-time data access through the API interface, or export data to data reports and visual charts to support batch processing and efficient query. We can also save data to the file storage system for offline processing or long-term storage. After selecting the output method, we perform necessary format conversion and adaptation according to the requirements of the target system, and ensure accurate and timely data transmission through automated deployment and monitoring.

[0094] S6. Build a full-link data quality assurance system and generate data quality reports;

[0095] In this implementation, S6 specifically includes:

[0096] S61. Establish a full-link data quality monitoring system, which includes data source quality monitoring, data processing process quality monitoring, and data output quality monitoring;

[0097] S62. Generate a data quality report according to the data quality monitoring system to evaluate and report the quality status of the data.

[0098] Specifically, we use the metadata and factual data left in the system by the data link to build a data quality monitoring system at the field, model, and indicator levels, set up month-on-month and year-on-year comparisons, analyze trend charts and set thresholds, etc., to monitor each link in the overall data flow process, avoid data pollution caused by accidental reasons, and reduce maintenance costs; at the same time, we establish business monitoring in combination with business scenarios, automatically monitor the progress of the business while conducting data analysis, configure DQC monitoring of each model, grasp the data abnormalities, set up reports, robot push and other links, understand the dynamic changes of business indicators, and obtain basic business information.

[0099] Adopting the scheme provided by this application, the original data is classified according to its channel source and characteristics, and different upload methods are adopted to the data platform; the platform standardizes the uploaded original data and cleans the dirty data to construct standard data; classifies the standard data based on the business process, establishes the corresponding business model and analyzes the correlation between models, and conducts multi-level development; standardizes the definition of credit indicators and binds them to the business model to realize credit analysis and monitoring; matches the data output method according to the business scenario, builds a full-link data quality assurance system, and generates a data quality report. Therefore, the above technical scheme can improve the efficiency of data processing, avoid repeated processing, and improve data reusability through multi-level development models. Through the output method matching the data, it can provide intuitive and visual data analysis and monitoring services, improve the efficiency of business processing and respond to data service needs more quickly.

[0100] based on Figure 1 The multi-level credit big data processing method based on data warehouse modeling is shown in the figure. On the other hand, it also relates to a multi-level credit big data processing system based on data warehouse modeling, which is used to execute the above method. Its structure is as follows Figure 3 As shown, including:

[0101] The data acquisition module 301 is used to acquire original data, classify the original data according to its source and characteristics, and upload it to the data platform using different uploading methods according to its classification;

[0102] The data processing module 302 is used to perform standardization processing on the uploaded raw data and clean dirty data to construct standard data;

[0103] The model building module 303 is used to classify the standard data based on the business process, establish corresponding business models according to each stage in the business process, analyze the correlation between the business models, and establish the correlation between the business models within each data classification and between data classifications, and perform multi-level development;

[0104] The indicator monitoring module 304 is used to analyze and monitor the credit indicators in the business process to standardize the definition of credit indicators and bind the credit indicators to the corresponding business model;

[0105] The data output module 305 is used to match the corresponding data output mode according to different business scenarios;

[0106] The quality assurance module 306 is used to build a full-link data quality assurance system and generate a data quality report.

[0107] Furthermore, the data acquisition module 301 is also used for:

[0108] Obtaining raw data, and classifying the raw data into actively provided data, internal system data, and external system data according to the source and characteristics of the raw data;

[0109] Uploading the actively provided data to the server using a secure and encrypted transmission protocol;

[0110] Transmitting the internal system data to the data platform in real time via an internal data bus system;

[0111] The external system data is pulled and uploaded to the data platform in a scheduled or real-time manner according to its API interface specifications.

[0112] Furthermore, the data processing module 302 is also used for:

[0113] Use data quality checking tools to explore the raw data, identify and mark outliers, missing values, duplicate values, etc.;

[0114] Cleaning the raw data according to preset cleaning rules, which may include but are not limited to removing duplicate data, filling or deleting missing values, correcting outliers, etc.;

[0115] The cleaned raw data is verified and then constructed into standard data.

[0116] Furthermore, the model building module 303 is also used for:

[0117] Classify the standard data according to the business processes before, during and after lending;

[0118] For each business stage, a corresponding business model is established, wherein the business model includes a pre-loan business model, a mid-loan business model, and a post-loan business model;

[0119] Analyze the correlation between the business models, establish data correlation of the business models, and carry out multi-level development.

[0120] Furthermore, the indicator monitoring module 304 is also used for:

[0121] Define its credit indicators based on the business process;

[0122] The credit indicators are bound to corresponding business models, and the business models are evaluated by monitoring the credit indicators.

[0123] Furthermore, the data output module 305 is also used for:

[0124] Determine the format and content of data output based on business scenario requirements;

[0125] Matching the corresponding data output mode based on the format and content of the data output, the data output mode includes data reports, visual charts, and API interfaces;

[0126] Furthermore, the quality assurance module 306 is also used for:

[0127] Establish a full-link data quality monitoring system, which includes data source quality monitoring, data processing process quality monitoring and data output quality monitoring;

[0128] Generate a data quality report based on the data quality monitoring system to evaluate and report the quality status of the data.

[0129] An embodiment of the present invention further provides an electronic device for executing the above method, as an implementation device of the method, comprising a processor and a memory;

[0130] A memory, used for storing operation instructions;

[0131] The processor is used to execute the multi-level credit big data processing method based on data warehouse modeling provided in any embodiment of the present application by calling operation instructions.

[0132] As an example, Figure 4 A schematic diagram of the structure of an electronic device applicable to an embodiment of the present application is shown, and the electronic device 400 includes: a processor 401 and a memory 403. The processor 401 and the memory 403 are connected, such as being connected through a bus 402. Optionally, the electronic device 400 may also include a transceiver 404. It should be noted that in actual applications, the transceiver 404 is not limited to one. It is understandable that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the specific structure of the electronic device 400. In other embodiments of the present application, the electronic device 400 may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware. Optionally, the electronic device may also include a display screen 405 for displaying images, or receiving user operation instructions when necessary.

[0133] The processor 401 is applied in the embodiment of the present application to implement the method shown in the above method embodiment. The transceiver 404 may include a receiver and a transmitter. The transceiver 404 is applied in the embodiment of the present application to implement the function of the electronic device of the embodiment of the present application communicating with other devices when executed.

[0134] Processor 401 may be a CPU (Central Processing Unit), a general purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. Processor 401 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0135] The processor 401 may also include one or more processing units, for example, the processor 401 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processor (NPU). Different processing units may be independent devices or integrated into one or more processors. Among them, the controller may be the nerve center and command center of the electronic device 400. The controller may generate an operation control signal according to the instruction opcode and the timing signal to complete the control of fetching and executing instructions. A memory may also be provided in the processor 401 for storing instructions and data. In some embodiments, the memory in the processor 401 is a high-speed cache memory. The memory may store instructions or data that the processor 401 has just used or circulated. If the processor 401 needs to use the instruction or data again, it may be directly called from the memory. Repeated access is avoided, the waiting time of the processor 401 is reduced, and the efficiency of the system is improved.

[0136] Processor 401 can run the multi-level processing method for credit big data based on data warehouse modeling provided in the embodiment of the present application. Processor 401 can include different devices. For example, when integrating CPU and GPU, the CPU and GPU can cooperate to execute the multi-level processing method for credit big data based on data warehouse modeling provided in the embodiment of the present application. Part of the algorithm is executed by the CPU, and the other part of the algorithm is executed by the GPU to obtain faster processing efficiency.

[0137] The bus 402 may include a path to transmit information between the above components. The bus 402 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus 402 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0138] The memory 403 may be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory), or may include a high-speed random access memory, or may include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), a disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited to this.

[0139] Optionally, the memory 403 is used to store application code for executing the solution of the present application, and the execution is controlled by the processor 401. The processor 401 is used to execute the application code stored in the memory 403 to implement the multi-level credit big data processing method based on data warehouse modeling provided in any embodiment of the present application.

[0140] The memory 403 can be used to store computer executable program codes, which include instructions. The processor 401 executes various functional applications and data processing of the electronic device 400 by running the instructions stored in the memory 403. The memory 403 may include a program storage area and a data storage area. Among them, the program storage area may store the code of the operating system, application program, etc. The data storage area may store data created during the use of the electronic device 400 (such as images, videos, etc. collected by the camera application), etc.

[0141] The memory 403 may also store one or more computer programs corresponding to the multi-level credit big data processing method based on data warehouse modeling provided in the embodiment of the present application. The one or more computer programs are stored in the above-mentioned memory 403 and are configured to be executed by the one or more processors 401. The one or more computer programs include instructions, and the above-mentioned instructions can be used to execute each step in the above-mentioned corresponding embodiment.

[0142] Of course, the code of the multi-level processing method of credit big data based on data warehouse modeling provided in the embodiment of the present application can also be stored in an external memory.

[0143] The display screen 405 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Miniled, a MicroLed, a Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 400 may include 1 or N display screens 405, where N is a positive integer greater than 1. The display screen 405 may be used to display information input by a user or information provided to a user and various graphical user interfaces (GUIs). For example, the display screen 405 may display photos, videos, web pages, or files, etc.

[0144] The electronic device provided in the embodiment of the present application is applicable to any embodiment of the above method. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above, which will not be repeated here.

[0145] An embodiment of the present invention also provides a computer-readable storage medium capable of implementing all the steps in the method in the above embodiment, wherein a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, all the steps in the method in the above embodiment are implemented.

[0146] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0147] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A system that specifies the functions of a box or boxes.

[0148] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction system, which is implemented in the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0149] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps of the functions specified in a box or multiple boxes. Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they know the basic creative concept. Therefore, the attached claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0150] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A multi-level processing method for credit big data based on data warehouse modeling, characterized in that: The following steps are involved: S1. Obtaining original data, classifying the original data according to its channel source and characteristics, and uploading it to the data platform using different uploading methods according to its classification; S2. The data platform performs standardization processing and dirty data cleaning on the uploaded raw data to construct standard data; S3. Classify the standard data based on the business process, establish corresponding business models according to each stage in the business process, analyze the correlation between the business models, and establish the correlation between business models within each data classification and between data classifications, and perform multi-level development; S4. By analyzing and monitoring the credit indicators in the business process, a standardized definition of the credit indicators is performed, and the credit indicators are bound to the corresponding business models; S5. Match the corresponding data output method according to different business scenarios; S6. Build a full-link data quality assurance system and generate a data quality report.

2. The method according to claim 1, characterized in that The S1 further comprises: S11. Obtaining original data, and classifying the original data into actively provided data, internal system data, and external system data according to the channel source and characteristics of the original data; S12, uploading the actively provided data to a server using a secure and encrypted transmission protocol; Transmitting the internal system data to the data platform in real time via an internal data bus system; The external system data is pulled and uploaded to the data platform in a scheduled or real-time manner according to its API interface specifications.

3. The method according to claim 1, characterized in that The S2 further comprises: S21. Use a data quality check tool to explore the raw data, identify and mark abnormal values, missing values, and duplicate values; S22. Clean the raw data according to preset cleaning rules, where the preset cleaning rules include but are not limited to removing duplicate data, filling or deleting missing values, and correcting abnormal values; S23, constructing the cleaned original data into standard data after verification.

4. The method according to claim 1, characterized in that: The S3 further comprises: S31. Classifying the standard data according to the business processes before, during and after the loan; S32. Establish a corresponding business model for each business stage, wherein the business model includes a pre-loan business model, a mid-loan business model, and a post-loan business model; S33, analyzing the correlation between the business models, and establishing data correlation of the business models, and performing multi-level development.

5. The method according to claim 1, characterized in that The S4 further comprises: S41. defining its credit indicators based on the business process; S42: Bind the credit indicator to the corresponding business model, and evaluate the business model by monitoring the credit indicator.

6. The method according to claim 1, characterized in that The S5 further comprises: S51. Determine the format and content of data output according to the needs of the business scenario; S52. Matching a corresponding data output method based on the format and content of the data output, wherein the data output method includes a data report, a visual chart, and an API interface.

7. The method according to claim 1, characterized in that The S6 further comprises: S61. Establish a full-link data quality monitoring system, which includes data source quality monitoring, data processing process quality monitoring, and data output quality monitoring; S62. Generate a data quality report according to the data quality monitoring system to evaluate and report the quality status of the data.

8. A multi-level credit big data processing system based on data warehouse modeling, characterized in that: include: The data acquisition module is used to acquire the original data, classify the original data according to its channel source and characteristics, and upload it to the data platform using different uploading methods according to its classification; A data processing module is used to perform standardization processing on the uploaded raw data and clean dirty data to construct standard data; A model building module is used to classify the standard data based on the business process, establish corresponding business models according to each stage in the business process, analyze the correlation between the business models, and establish the correlation between business models within each data classification and between data classifications, and perform multi-level development; The indicator monitoring module is used to analyze and monitor the credit indicators in the business process to standardize the definition of credit indicators and bind the credit indicators to the corresponding business model; Data output module, used to match corresponding data output methods according to different business scenarios; The quality assurance module is used to build a full-link data quality assurance system and generate data quality reports.

9. An electronic device, characterized in that: including a processor and a memory; The memory is used to store operation instructions; The processor is used to execute the method according to any one of claims 1 to 7 by calling the operation instruction.

10. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images