Data processing method and related hardware
By classifying business objects by feature and extracting features from the target data processing model, the problem of inaccurate business data analysis for different business objects is solved, and the accuracy of targeted analysis is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-04
- Publication Date
- 2026-04-03
AI Technical Summary
Existing technologies use fixed analytical methods to analyze business data, which may not yield accurate results and cannot adapt to the business characteristics of different business entities.
By extracting the first feature of the business objects to be processed and classifying them, a set of business objects to be processed is obtained. Then, according to the second feature category of the target data processing model, the corresponding features are extracted and input into the target data processing model for processing to generate analysis information.
It enables targeted analysis of business objects with different characteristics, which can obtain correct analysis results and avoid errors caused by using the same method to process them.
Smart Images
Figure CN121786362A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of data processing technology, and more particularly to a data processing method and related hardware. Background Technology
[0002] For businesses, it is necessary to analyze and process the business data corresponding to their business entities in order to make corresponding business decisions based on the analysis results. However, the same category of business data corresponding to different business entities may have different business characteristics. If a fixed analysis method is used to analyze the business data, the correct analysis results may not be obtained. Summary of the Invention
[0003] This application provides a data processing method and related hardware to solve the problem in the prior art that using a fixed analysis method to analyze business data may not yield correct analysis results.
[0004] In a first aspect, an embodiment of this application provides a data processing method, comprising:
[0005] Extract at least one first feature of each category from each pending business object;
[0006] Based on the first feature, each of the pending business objects is classified to obtain at least one set of pending business objects.
[0007] For any pending business object in any set of pending business objects, perform the following steps:
[0008] Based on the second feature category corresponding to the target data processing model, the corresponding second feature is extracted from the business object to be processed; wherein the target data processing model corresponds to the set of business objects to be processed, and the second feature categories corresponding to different target data processing models are at least partially different.
[0009] The second feature is input into the target data processing model to obtain the reasoning result corresponding to the business object to be processed;
[0010] Based on the reasoning results, analytical information is generated for the business object to be processed.
[0011] Optionally, after classifying each business object to be processed according to the first feature to obtain at least one set of business objects to be processed, the method further includes:
[0012] For any set of business objects to be processed, the set of business objects to be processed is divided into at least one subset of business objects to be processed according to a preset rule;
[0013] Each subset of pending business objects is sent to a corresponding target data processing node, so that each target data processing node performs the following steps on any pending business object in the received subset:
[0014] Based on the second feature category corresponding to the target data processing model, extract the corresponding second feature of the business object to be processed;
[0015] The second feature is input into the target data processing model to obtain the reasoning result corresponding to the business object to be processed;
[0016] Based on the reasoning results, analytical information is generated for the business object to be processed.
[0017] Alternatively, for any set of business objects to be processed, the target data processing model can be trained in the following way:
[0018] Obtain the target historical sample set corresponding to the target data processing model; wherein, the target historical sample set is obtained by extracting at least one category of first features from each historical sample, classifying each historical sample according to the first features, and obtaining at least one historical sample set, which is the historical sample set corresponding to the target first feature; the target first feature is the first feature corresponding to the target data processing model; the historical sample is a sample constructed based on historical business objects and corresponding inference results;
[0019] For any historical sample in the target historical sample set, extract the corresponding second feature of the historical sample according to the second feature category corresponding to the target data processing model;
[0020] Using the second feature of the test sample in the target historical sample set as input and the sample label corresponding to the test sample as output as the training target, the initial target data processing model is trained for at least one round using the training samples in the target historical sample set until the training ends when the training termination condition is met, thus obtaining the final target data processing model.
[0021] Optionally, before training the initial target data processing model for at least one round using the second feature of the test sample in the target historical sample set as input and the sample label corresponding to the test sample as output as the training target, the method further includes:
[0022] The target historical sample set is divided into multiple target historical sample subsets;
[0023] During the process of training the initial target data processing model using training samples from the target historical sample set for at least one round, in each round of training, historical samples from one subset of the target historical sample set are used as test samples for the current training round, and historical samples from the remaining subsets of the target historical sample set are used as training samples for the current training round.
[0024] Optionally, generating analysis information for the business object to be processed based on the reasoning result includes:
[0025] Based on the reasoning results, determine the corresponding target mind chain CoT prompt engineering template;
[0026] The analysis instructions are determined based on the target CoT prompt engineering template, the business object to be processed, and / or the second feature, and then input into the Large Language Model (LLM) to obtain the analysis information for the business object to be processed output by the LLM.
[0027] Different inference results correspond to different CoT hint engineering templates.
[0028] Optionally, the target data processing model includes an iterative decision tree model.
[0029] Secondly, based on the same inventive concept, embodiments of this application also provide a data processing apparatus, including:
[0030] The first feature extraction module is used to extract at least one category of first features from each business object to be processed.
[0031] The business object classification module is used to classify each of the business objects to be processed according to the first feature, so as to obtain at least one set of business objects to be processed.
[0032] The business object processing module is used to perform the following steps on any business object in any set of business objects to be processed: extracting a corresponding second feature for the business object to be processed according to the second feature category corresponding to the target data processing model; wherein the target data processing model corresponds to the set of business objects to be processed, and the second feature categories corresponding to different target data processing models are at least partially different; inputting the second feature into the target data processing model to obtain the inference result corresponding to the business object to be processed; and generating analysis information for the business object to be processed based on the inference result.
[0033] Thirdly, based on the same inventive concept, embodiments of this application also provide an electronic device including: a processor and a memory for storing processor-executable instructions;
[0034] The processor is configured to execute the instructions to implement the data processing method described in the first aspect.
[0035] Fourthly, based on the same inventive concept, embodiments of this application also provide a computer-readable storage medium storing computer program code, which, when run on a computer, causes the computer to perform the data processing method as described in the first aspect.
[0036] Fifthly, based on the same inventive concept, embodiments of this application also provide a computer program product, the computer program product comprising: computer program code, which, when run on a computer, causes the computer to perform the data processing method as described in the first aspect.
[0037] The beneficial effects of this application are as follows:
[0038] The data processing method and related hardware provided in this application first classify the business objects to be processed according to a first feature to obtain at least one set of business objects to be processed. Then, corresponding features are extracted from the business objects in different sets of business objects to be processed and input into the target data processing model corresponding to the set of business objects to be processed for processing. The analysis information corresponding to the business objects to be processed is determined according to the final inference result. It can analyze the business objects to be processed based on different feature categories with different characteristics, instead of using the same data processing method for all business objects to be processed. In this way, the characteristics of the business data can be targeted to obtain the correct analysis results. Attached Figure Description
[0039] Figure 1 One of the flowcharts for the data processing method provided in the embodiments of this application;
[0040] Figure 2 This is one of the flowcharts illustrating the training process of the target data processing model in the data processing method provided in the embodiments of this application.
[0041] Figure 3 This is one of the flowcharts showing the training process of the target data processing model in the data processing method provided in the embodiments of this application;
[0042] Figure 4 The second flowchart of the data processing method provided in the embodiments of this application;
[0043] Figure 5 The second flowchart illustrates the training process of the target data processing model in the data processing method provided in this application embodiment.
[0044] Figure 6The second part of the flowchart of the training process of the target data processing model in the data processing method provided in the embodiments of this application;
[0045] Figure 7 The third flowchart of the data processing method provided in the embodiments of this application;
[0046] Figure 8 The third flowchart illustrates the training process of the target data processing model in the data processing method provided in this application embodiment;
[0047] Figure 9 The third part of the flowchart of the training process of the target data processing model in the data processing method provided in the embodiments of this application;
[0048] Figure 10 This is a schematic diagram of the structure of the data processing apparatus provided in the embodiments of this application;
[0049] Figure 11 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation
[0050] To make the above-mentioned objectives, features, and advantages of this application more apparent and understandable, the application will be further described below in conjunction with the accompanying drawings and embodiments. However, the exemplary embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided to make the application more comprehensive and complete, and to fully convey the concept of the exemplary embodiments to those skilled in the art. The same reference numerals in the figures denote the same or similar structures, and therefore repeated descriptions of them will be omitted. Terms describing position and direction described in this application are illustrative based on the accompanying drawings, but changes can be made as needed, and all such changes are included within the scope of protection of this application. The accompanying drawings of this application are for illustrating relative positional relationships only and do not represent actual scale.
[0051] It should be noted that specific details are set forth in the following description to provide a full understanding of this application. However, this application can be implemented in many ways other than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this application. Therefore, this application is not limited to the specific embodiments disclosed below. The following descriptions are preferred embodiments for carrying out this application; however, these descriptions are for the purpose of illustrating the general principles of this application and are not intended to limit the scope of this application. The scope of protection of this application shall be determined by the appended claims.
[0052] It should be noted that the acquisition, transmission, storage, and use of data in this application comply with the requirements of relevant national laws and regulations.
[0053] The data processing method and related hardware provided in the embodiments of this application will be described in detail below with reference to the accompanying drawings.
[0054] This application provides a data processing method, such as... Figure 1 As shown, it includes:
[0055] S100: Obtain each pending business object.
[0056] In practice, the data contained in the business object to be processed can be obtained from multiple data sources to construct the business object.
[0057] In practice, the original business data can be preprocessed to obtain the business objects to be processed. The preprocessing of the original business data may include data cleaning (such as data format conversion (including numerical format conversion, data type conversion, etc.)) and missing value filling, etc., which are not limited in detail in this embodiment.
[0058] S110. Extract at least one first feature of each category from each business object to be processed.
[0059] S120. Classify each business object to be processed according to the first feature to obtain at least one set of business objects to be processed.
[0060] In the specific implementation process, since different business objects may have different characteristics, it is necessary to process data in different ways according to the characteristics of different business objects. Therefore, at least one first feature of a category can be set in advance, and the business objects to be processed can be classified using the first feature. Business objects to be processed with the same data processing method can be classified into the same set of business objects to be processed, so that subsequent steps can process the data in the same set of business objects to be processed using the same data processing method.
[0061] Subsequently, for any business object in any set of business objects to be processed, analysis information for that business object is determined based on the data processing method corresponding to the set of business objects to which it belongs. Specifically, this includes:
[0062] S130. For any business object in any set of business objects to be processed, extract the corresponding second feature of the business object according to the second feature category corresponding to the target data processing model. The target data processing model is the data processing model corresponding to the set of business objects to be processed. Different target data processing models have at least partially different second feature categories.
[0063] That is, there is a correspondence between the target data processing model, the second feature category, and the set of business objects to be processed.
[0064] In the specific implementation process, suitable data items can be selected from the data items of the business object as the first feature category and / or the second feature category based on indicators such as the missing proportion of corresponding data items in the business object, the homogeneity rate of data items, information value (IV), the correlation between data items, and the Population Stability Index (PSI).
[0065] S140. For any business object in any set of business objects to be processed, input the second feature corresponding to the business object to be processed into the target data processing model corresponding to the set of business objects to be processed, and obtain the reasoning result corresponding to the business object to be processed.
[0066] In practical implementation, the target data processing model can be provided as a locally loaded service or as a microservice interface, enabling high-concurrency calls to the target data processing model to batch process multiple pending business objects and determine the corresponding inference results. Furthermore, the execution status, runtime, and exception information of each task can be recorded during the call to each target data processing model, thereby enabling monitoring of the data processing process of each target data processing model.
[0067] In the specific implementation process, the target data processing model can be a classification model, then the inference result corresponding to the business object to be processed is the classification category corresponding to the business object to be processed; or the target data processing model can be a regression model, then the inference result corresponding to the business object to be processed is the regression prediction value corresponding to the business object to be processed.
[0068] In practical implementation, the target data processing model can include Convolutional Neural Networks (CNNs), Support Vector Machines (SVMs), decision trees, random forests, and other implementations. For example, the target data processing model can be an iterative decision tree model, such as Gradient Boosting Decision Tree (GBDT), Adaptive Boosting (Adaboost), or Extreme Gradient Boosting (XGBoost). This paper primarily uses the XGBoost model as an example to illustrate the target data processing model.
[0069] S150. For any business object in any set of business objects to be processed, generate analysis information for the business object to be processed based on the reasoning result corresponding to the business object to be processed.
[0070] In the specific implementation process, corresponding analysis information can be generated for the business objects to be processed as needed. For example, the analysis information may be for a single business object to be processed, or it may be for multiple business objects to be processed as a whole. This application embodiment does not impose any further limitations on this.
[0071] In the specific implementation process, in step S140, the inference results corresponding to the business objects to be processed in different sets can be stored in different files. Then, in step S150, the inference result files under different storage paths are integrated to generate overall corresponding analysis information for multiple business objects to be processed.
[0072] In some possible implementations, for at least a portion of the set of business objects to be processed, different analysis information can be pre-set for different inference results and a mapping relationship can be established, and then the corresponding analysis information can be determined according to the inference result corresponding to the business object to be processed.
[0073] In some possible implementations, for at least a portion of the set of business objects to be processed, a corresponding target Chain of Thought (CoT) hint template can be determined based on the inference results corresponding to the business objects. Analysis instructions are then determined based on the business object itself, at least one of its second features, and the target CoT hint template. These analysis instructions are input into a Large Language Model (LLM) to obtain LLM output analysis information for the business objects to be processed. Different inference results correspond to different CoT templates. The LLM can be models such as ChatGPT, Gemini, Claude, Grok, Llama, Falcon, Kimi, DeepSeek, Wenxin Yiyan, Doubao, Qwen, Yuanbao, and Xinghuo, etc. This application does not impose further limitations on these models. CoT is a technique that enhances the ability of large language models to solve complex problems through step-by-step logical reasoning. It guides the LLM to generate intermediate reasoning processes for the problem by adding prompts to the questions input to the LLM to break down the question into sub-questions and / or steps with corresponding logical connections, or by adding exemplary few-shot prompts to the questions input to the LLM to refer to the step-by-step reasoning of the question. Finally, it integrates these into a complete problem-solving chain, thereby guiding the LLM to output more accurate answers.
[0074] In the specific implementation process, after obtaining the analysis information of each business object to be processed, the analysis information of each business object can be integrated into a single file for storage. During the storage of the analysis information, in order to ensure the accuracy and completeness of the stored analysis information, a corresponding verification file (such as a hash value file) can be generated for the analysis information file, so as to facilitate subsequent consistency verification of the stored analysis information file.
[0075] In the specific implementation process, the analysis information of the business objects to be processed can be converted into structured files and stored in the corresponding database tables, so that other business can be implemented based on the analysis information of the business objects to be processed.
[0076] Thus, the data processing method provided in this application first classifies the business objects to be processed according to a first feature to obtain at least one set of business objects to be processed. Then, it extracts the corresponding features of the business objects in different sets of business objects to be processed and inputs them into the target data processing model corresponding to the set of business objects to be processed for processing. Based on the final inference result, it determines the analysis information corresponding to the business objects to be processed. It can analyze the business objects to be processed based on different feature categories with different characteristics, instead of using the same data processing method for all business objects to be processed. In this way, it can achieve targeted processing based on the characteristics of business data to obtain the correct analysis results.
[0077] In practical implementation, if the amount of data for the business objects to be processed is large, the computational pressure of having a single computing node handle all the data processing tasks would be too high. Therefore, the data processing tasks can be distributed across multiple computing nodes for distributed processing to reduce the computational pressure on a single computing node and avoid skewed computational load onto a single node. For example, the method can be implemented using a distributed computing framework such as Apache Spark. When obtaining each business object to be processed, the processed business object table can be read from the big data platform using Apache Spark. Accordingly, after step S120, the method further includes:
[0078] S121. For any set of business objects to be processed, divide the set of business objects to be processed into at least one subset of business objects to be processed according to a preset rule.
[0079] As an optional implementation, for any set of business objects to be processed, the set is divided into multiple subsets according to a preset rule to achieve load balancing across computing nodes. As another optional implementation, based on the data volume of the set of business objects, sets with data volumes greater than a preset data volume threshold are further divided into multiple subsets, while sets with data volumes less than or equal to the preset data volume threshold are not further divided, i.e., they are only divided into one subset.
[0080] In practical implementation, the business to which the business objects to be processed belong, their creation time, and unique identifiers (such as the ID or data key of the business object) can be used as criteria for subset division to divide the set of business objects to be processed into multiple subsets. Furthermore, when dividing based on subset division criteria (such as using the unique identifier of the business object as the subset division criterion), consistent hashing algorithms, modular arithmetic algorithms, etc., can be used to divide the business objects in the set of business objects to be processed into multiple different subsets. This embodiment of the application does not impose further limitations on these methods.
[0081] S122. Send each subset of business objects to be processed to the corresponding target data processing node.
[0082] In the specific implementation process, after each target data processing node receives the corresponding subset of business objects to be processed, it persists the data to a specified local path to form an independent data input unit, which facilitates the scheduling of subsequent steps to use the subset of business objects to be processed to perform corresponding data processing tasks and troubleshooting.
[0083] Correspondingly, for any target data processing node, the target data processing node determines the analysis information for any business object in the received subset of business objects to be processed, based on the data processing method corresponding to the set of business objects to be processed to which the business object belongs. That is, for any business object in the received subset of business objects to be processed, the target data processing node extracts the corresponding second feature of the business object to be processed according to the second feature category corresponding to the target data processing model; inputs the second feature corresponding to the business object to be processed into the target data processing model corresponding to the set of business objects to be processed to obtain the inference result corresponding to the business object to be processed; and generates the analysis information for the business object to be processed based on the inference result corresponding to the business object to be processed.
[0084] In practical implementation, the process of sending each subset of pending business objects to corresponding target data processing nodes for data processing can also be scheduled using distributed computing frameworks such as Apache Spark. This effectively improves the overall data processing efficiency of the system, significantly shortens the time required for full data processing, reduces single-machine computing load and memory pressure, and supports stable and efficient operation in production environments. Furthermore, the use of a distributed computing framework for scheduling provides the solution with good scalability and fault tolerance, enabling it to adapt to future growth in the volume of pending business object data and the need for model iteration and upgrades.
[0085] Furthermore, such as Figure 2 As shown, the target data processing model for any set of business objects to be processed is trained in the following way:
[0086] S210. Obtain the target historical sample set corresponding to the target data processing model.
[0087] Specifically, such as Figure 3 As shown, the target historical sample set corresponding to any target data processing model is obtained in the following way:
[0088] S300: Obtain historical business objects and corresponding inference results to construct historical samples.
[0089] In the specific implementation process, the original historical business data can be preprocessed to obtain historical business objects. The preprocessing process of the original historical business data may include data sampling, data cleaning (such as data format conversion (including numerical format conversion, data type conversion, etc.), missing value imputation, data filtering (such as removing historical business data with a missing data ratio greater than the acceptable missing data ratio), etc. The embodiments of this application do not impose too many limitations here.
[0090] In the specific implementation process, the reasoning results corresponding to historical business objects can be used as sample labels to construct historical samples.
[0091] S310. Extract at least one first feature of each category from each historical sample.
[0092] S320. Classify each historical sample according to the first feature to obtain at least one historical sample set. Different historical sample sets correspond to different target data processing models.
[0093] S220. For any historical sample in the target historical sample set, extract the corresponding second feature of the historical sample according to the second feature category corresponding to the target data processing model.
[0094] S230. Using the second feature of the test sample in the target historical sample set as input and the sample label corresponding to the test sample as output as the training target, the initial target data processing model is trained for at least one round using the training samples in the target historical sample set until the training ends when the training termination condition is met, thus obtaining the final target data processing model.
[0095] Furthermore, after determining the target historical sample set, it is necessary to further divide the historical samples within the target historical sample set into training samples and test samples. As an optional implementation, a hold-out method can be used to divide the historical samples, that is, dividing the historical samples in the target historical sample set into a training sample set and a test sample set according to a certain ratio (e.g., a training sample to test sample ratio of 5:1). As another optional implementation, cross-validation can be used to divide the historical samples, dividing the target historical sample set into multiple target historical sample subsets. In each training round, one of these target historical sample subsets is selected as the test sample set for the current round, and the remaining target historical sample subsets form the training sample set for the current round. At least some different training rounds will have different test sample sets.
[0096] The following are several possible embodiments to illustrate the above scheme in detail.
[0097] Example 1:
[0098] This embodiment describes the credit card application approval process of financial institutions in the financial sector. In this embodiment, the business object is the pending credit card approval data corresponding to users involved in credit card application approval processes. It should be noted that in this embodiment, the acquisition, transmission, storage, and use of data must be authorized by the user, and all such acquisition, transmission, storage, and use must comply with relevant national laws and regulations.
[0099] Accordingly, such as Figure 4 As shown, the data processing method specifically includes:
[0100] S100A: Obtain pending credit card approval data for each user awaiting approval.
[0101] In practice, credit card approval data may include, but is not limited to, the following data items:
[0102] ① Describe the user's basic information about static attributes, such as the user's identity information (e.g., age, residential address, etc.) and social relationship information (e.g., education level, type of work, marital status, etc.).
[0103] ② User's asset information, such as the total value of the user's assets, the level of the assets held by the user, the amount of the user's RMB current deposits, and the available balance of the user's personal pension, etc.
[0104] ③ User's financial business information, such as the types of financial products held by the user and the value of the financial products held by the user.
[0105] ④ User's loan information, such as the user's loan amount and the user's overdue loan amount.
[0106] ⑤ User income information, such as average income and average salary.
[0107] ⑥ User's fund flow information, such as the user's average transaction amount and the user's cumulative debit amount within a preset period.
[0108] ⑦ User behavioral preference information, such as the frequency with which users conduct financial transactions and the frequency with which users use financial service programs.
[0109] ⑧ User credit information, such as user financial credit rating, etc.
[0110] 9. User's credit card information, such as the user's current credit card holding status and the user's historical credit card holding status.
[0111] By analyzing multiple dimensions such as asset status, behavioral preferences, product holding information, personal loan information, payroll processing, transaction settlement, customer basic information, and risk rating information, comprehensive credit card approval business data is generated for each user.
[0112] In the specific implementation process, when acquiring the pending credit card approval business data for each user, Apache Spark can be used to read the processed credit card approval business data table from the big data platform. The credit card approval business data covers multi-dimensional information such as customer assets, transactions, income, and product holdings, with a monthly data volume of approximately tens of millions of records.
[0113] S110A: Extract at least one category of first features from the pending credit card approval business data corresponding to each user.
[0114] In the specific implementation process, based on the characteristics of credit card application and approval business, the first feature category can be set as: ① the user's current credit card holding status (including the current holding of a credit card and the current not holding a credit card), ② the user's asset level (including high asset level, medium asset level, and low asset level), ③ the user's historical credit card holding status (including never holding a credit card, previously holding a credit card and currently not holding a credit card, currently holding a credit card and inactive in using the credit card, etc.).
[0115] S120A. Classify the pending credit card approval business data corresponding to each user according to the first feature to obtain at least one set of pending credit card approval business data.
[0116] For example, based on the first feature category mentioned above, the pending credit card approval business data corresponding to each user can be divided into multiple different pending credit card approval business data sets. In subsequent steps, different methods of analysis will be carried out according to the characteristics of the data in different pending credit card approval business data sets.
[0117] S130A: For any user's pending credit card approval business data in any pending credit card approval business dataset, extract the corresponding second feature of the pending credit card approval business data according to the second feature category corresponding to the target data processing model. The target data processing model is the XGboost data processing model corresponding to the pending credit card approval business dataset. Different target data processing models correspond to at least partially different second feature categories.
[0118] For example, for a dataset of pending credit card approval transactions where users hold mid-level assets, the second feature category could include the total value of the user's assets and the user's cumulative loan amount within a preset period, focusing on the user's asset holdings and financial activity. Conversely, for a dataset of pending credit card approval transactions where users hold low-level assets, the second feature category could include the user's average income and the user's cumulative debit amount within a preset period, focusing on the user's financial activity and historical behavioral preferences. This allows for more targeted analysis of pending credit card approval data from users with different characteristics.
[0119] S140A: For any user's pending credit card approval business data in any pending credit card approval business data set, input the second feature corresponding to the user's pending credit card approval business data into the target data processing model corresponding to the pending credit card approval business data set to obtain the inference result corresponding to the user's pending credit card approval business data.
[0120] Accordingly, different target data processing models (e.g., different model parameters) are used for different sets of credit card approval business data to conduct targeted analysis on the different extracted second feature categories and give corresponding inference results.
[0121] In the specific implementation process, the reasoning results corresponding to the credit card approval business data to be processed include the classification results of whether the credit card application has been approved and the confidence level corresponding to the classification results. That is, the target data processing model in this embodiment is a binary classification task model.
[0122] S150A: For any user's pending credit card approval business data in any pending credit card approval business data set, generate analysis information for the user's pending credit card approval business data based on the inference results corresponding to the user's pending credit card approval business data.
[0123] In practice, the analytical information can be analytical information corresponding to a single user, and the analytical information includes, but is not limited to, at least one of the following:
[0124] ① The classification results of whether a credit card application for a single user is approved (i.e., approved and rejected results).
[0125] ② Confidence level of the classification results of whether a credit card application for a single user is approved.
[0126] ③ If the classification result is "passed", it also includes the individual user's credit card limit information.
[0127] ④ If the classification result is "passed" and the user has not actively submitted a credit card application, then the category also includes a referral user tag for inviting the user to open a credit card account.
[0128] ⑤ If the classification result is "passed" and the user currently holds a credit card that is frozen, then the user tag for unfreezing the credit card is also included.
[0129] ⑥ If the classification result is "pass" and the user's credit rating is "excellent", then the "instant approval" user tag, which provides the user with a fast credit card application process, is also included.
[0130] The analysis information can also be analysis information corresponding to multiple users as a whole. For example, based on the classification results of all users in the pending credit card approval business data set, statistical information on the overall approval results of these users can be generated (i.e., the overall approval result pass rate, the overall approval result confidence level, the average credit card credit limit of users whose approval result is approved, etc.).
[0131] After processing all users' pending credit card approval data in the entire pending credit card approval data set, and obtaining the analysis information of all users' pending credit card approval data, the analysis information can be generated into structured data files and stored in the corresponding database tables for subsequent business (such as credit card approval business engine, credit card opening invitation marketing system) to call, supporting the user-facing front-end business system to carry out precise and automated credit card application approval services.
[0132] Furthermore, such as Figure 5 As shown, the target data processing model for any set of pending credit card approval business data is trained in the following way:
[0133] S210A, Obtain the target historical credit card approval business data set corresponding to the target data processing model.
[0134] Specifically, such as Figure 6 As shown, the target historical credit card approval business data set corresponding to any target data processing model is obtained in the following way:
[0135] S300A: Obtain historical credit card approval business data and corresponding inference results to construct historical samples. The approval results corresponding to the historical credit card approval business data serve as the sample labels for the historical samples.
[0136] In practice, the original historical business data can be preprocessed to obtain historical credit card approval business data. This preprocessing can include data sampling, data cleaning (e.g., data format conversion (including numerical format conversion, data type conversion, etc.), missing value imputation, and data filtering (e.g., removing historical business data with a missing percentage greater than an acceptable percentage). This embodiment does not impose further limitations on these methods.
[0137] S310A: Extract at least one first feature of each category from each historical sample.
[0138] S320A. Classify each historical sample according to the first feature to obtain at least one historical sample set. Different historical sample sets correspond to different target data processing models.
[0139] S220A. For any historical sample in the target historical sample set, extract the corresponding second feature of the historical sample according to the second feature category corresponding to the target data processing model.
[0140] S230A: Using the second feature of the test sample in the target historical sample set as input and the inference result sample label corresponding to the test sample as output as the training target, the initial target data processing model is trained for at least one round using the training samples in the target historical sample set until the training ends when the training termination condition is met, thus obtaining the final target data processing model.
[0141] In each training round, cross-validation is used to divide the historical samples, dividing the target historical sample set into multiple target historical sample subsets. In each training round, one of the target historical sample subsets is selected as the test sample set for the current round, and the remaining target historical sample subsets are used to form the training sample set for the current round. At least some of the test sample sets are different in different training rounds.
[0142] During the training of the target data processing model, a targeted second feature is set as the sample feature based on the data characteristics of the target historical sample set. The hyperparameters of the target data processing model are optimized through cross-validation, and the predictive performance and interpretability of the model are ensured.
[0143] In this way, the above scheme can be used to analyze and model users with different characteristics from different secondary features, build a differentiated analysis system, break through the limitations of the traditional unified modeling of credit card approval business data, and effectively improve the prediction accuracy and applicability of the target analysis model.
[0144] Example 2:
[0145] This embodiment analyzes the product production and sales strategies of businesses in the e-commerce field. In this embodiment, the business object is product sales parameters. Accordingly, as... Figure 7 As shown, the data processing method specifically includes:
[0146] S100B: Obtain the sales parameters of the products to be processed corresponding to each product.
[0147] In the specific implementation process, product sales parameters may include, but are not limited to, the following data:
[0148] ① Product category (e.g., clothing, consumer electronics, food, pharmaceuticals, daily chemical products, etc.).
[0149] ② Product brand information, including the specific brand of the product and whether the brand is a well-known brand.
[0150] ③ Product hardware parameters, including weight, volume, production date, shelf life, etc.
[0151] ④ Product sales time characteristics information, such as clothing sales time characteristics information being seasonal, food sales time characteristics information being seasonal holidays, and consumer electronics sales time characteristics information being student holidays, etc.
[0152] ⑤ Product price information.
[0153] ⑥ User feedback information for the product, including positive review rate, return rate, etc.
[0154] ⑦ Target user information of the product, such as teenagers, middle-aged people, and the elderly.
[0155] S110B: Extract at least one category of first features from the sales parameters of the products to be processed corresponding to each product.
[0156] In the specific implementation process, based on the characteristics of product sales, the first feature category can be set as the product category and the product's target user information.
[0157] S120B: Classify the sales parameters of the products to be processed according to the first feature to obtain at least one set of sales parameters of the products to be processed.
[0158] For example, based on the first feature category mentioned above, the sales parameters of the products to be processed corresponding to each product can be divided into multiple different sets of sales parameters of the products to be processed. In subsequent steps, different methods of analysis will be carried out according to the characteristics of the data in the different sets of sales parameters of the products to be processed.
[0159] S130B: For the sales parameters of any product in any set of sales parameters to be processed, extract the corresponding second feature based on the second feature category of the target data processing model. The target data processing model is the XGboost data processing model corresponding to the set of sales parameters to be processed. Different target data processing models will have at least partially different second feature categories.
[0160] For example, for a set of product parameters to be processed that falls under the category of clothing and targets middle-aged users, the second feature category could include user reviews, price information, and sales time characteristics. Conversely, for a set of product parameters to be processed that falls under the category of electronic products and targets teenagers, the second feature category could include price information, hardware specifications, and brand information. This allows for more targeted analysis of the sales parameters of products with different characteristics.
[0161] S140B: For any product in any set of product sales parameters to be processed, input the second feature corresponding to the product sales parameters to be processed into the target data processing model corresponding to the set of product sales parameters to be processed, and obtain the inference result corresponding to the product sales parameters to be processed.
[0162] Accordingly, the target data processing models corresponding to different sets of sales parameters of products to be processed are different, so as to conduct targeted analysis for different extracted second feature categories and give corresponding inference results.
[0163] In the specific implementation process, the inference results corresponding to the sales parameters of the product to be processed can be the production and sales strategy categories of the corresponding product, such as suggesting to increase production and raise the selling price, suggesting to increase production and maintain the selling price, suggesting to stop production and maintain the selling price, suggesting to stop production and lower the selling price, etc.
[0164] S150B: For any product in any set of pending product sales parameters, generate analysis information for the pending product sales parameters of the product based on the reasoning results corresponding to the pending product sales parameters of the product.
[0165] For example, for products whose reasoning suggests increasing production and raising prices, specific recommendations for increasing production volume and the recommended increase in price can be provided based on the product's competitors, in order to assist businesses in adjusting the product's production and sales.
[0166] Furthermore, such as Figure 8 As shown, the target data processing model for any set of pending credit card approval business data is trained in the following way:
[0167] S210B: Obtain the set of target historical product sales parameters corresponding to the target data processing model.
[0168] Specifically, such as Figure 9 As shown, the set of target historical product sales parameters corresponding to any target data processing model is obtained in the following way:
[0169] S300B: Obtain historical product sales parameters and corresponding inference results to construct historical samples. The inference results corresponding to the product sales parameters serve as the sample labels for the historical samples.
[0170] In the specific implementation process, the original historical business data can be preprocessed to obtain historical product sales parameters. The preprocessing of the original historical business data can include data sampling, data cleaning (such as data format conversion (including numerical format conversion, data type conversion, etc.), missing value imputation, data filtering (such as removing historical business data with a missing data ratio greater than the acceptable missing data ratio), etc. The embodiments of this application do not impose too many limitations here.
[0171] S310B: Extract at least one first feature of each category from each historical sample.
[0172] S320B. Classify each historical sample according to the first feature to obtain at least one historical sample set. Different historical sample sets correspond to different target data processing models.
[0173] S220B: For any historical sample in the target historical sample set, extract the corresponding second feature of the historical sample according to the second feature category corresponding to the target data processing model.
[0174] S230B: Using the second feature of the test sample in the target historical sample set as input and the inference result sample label corresponding to the test sample as output as the training target, the initial target data processing model is trained for at least one round using the training samples in the target historical sample set until the training ends when the training termination condition is met, thus obtaining the final target data processing model.
[0175] In each training round, cross-validation is used to divide the historical samples, dividing the target historical sample set into multiple target historical sample subsets. In each training round, one of the target historical sample subsets is selected as the test sample set for the current round, and the remaining target historical sample subsets are used to form the training sample set for the current round. At least some of the test sample sets are different in different training rounds.
[0176] During the training of the target data processing model, a targeted second feature is set as the sample feature based on the data characteristics of the target historical sample set. The hyperparameters of the target data processing model are optimized through cross-validation, and the predictive performance and interpretability of the model are ensured.
[0177] In this way, the above scheme can be used to analyze and model products with different characteristics from different secondary features, build a differentiated analysis system, and provide more accurate suggestions for product production and sales strategies.
[0178] Based on the same inventive concept, embodiments of this application also provide a data processing apparatus, such as... Figure 10 As shown, it includes:
[0179] The first feature extraction module M1 is used to extract at least one category of first features from each business object to be processed.
[0180] The business object classification module M2 is used to classify each of the business objects to be processed according to the first feature, so as to obtain at least one set of business objects to be processed.
[0181] The business object processing module M3 is used to perform the following steps on any business object in any set of business objects to be processed: extracting the corresponding second feature of the business object to be processed according to the second feature category corresponding to the target data processing model; wherein the target data processing model corresponds to the set of business objects to be processed, and the second feature categories corresponding to different target data processing models are at least partially different; inputting the second feature into the target data processing model to obtain the inference result corresponding to the business object to be processed; and generating analysis information for the business object to be processed based on the inference result.
[0182] Alternatively, the device may further include:
[0183] The distributed scheduling module M4 is used to divide any set of pending business objects into at least one subset of pending business objects according to a preset rule.
[0184] Each subset of pending business objects is sent to a corresponding target data processing node, so that each target data processing node performs the following steps on any pending business object in the received subset:
[0185] Based on the second feature category corresponding to the target data processing model, extract the corresponding second feature of the business object to be processed;
[0186] The second feature is input into the target data processing model to obtain the reasoning result corresponding to the business object to be processed;
[0187] Based on the reasoning results, analytical information is generated for the business object to be processed.
[0188] Alternatively, for any set of business objects to be processed, the target data processing model can be trained in the following way:
[0189] Obtain the target historical sample set corresponding to the target data processing model; wherein, the target historical sample set is obtained by extracting at least one category of first features from each historical sample, classifying each historical sample according to the first features, and obtaining at least one historical sample set, which is the historical sample set corresponding to the target first feature; the target first feature is the first feature corresponding to the target data processing model; the historical sample is a sample constructed based on historical business objects and corresponding inference results;
[0190] For any historical sample in the target historical sample set, extract the corresponding second feature of the historical sample according to the second feature category corresponding to the target data processing model;
[0191] Using the second feature of the test sample in the target historical sample set as input and the sample label corresponding to the test sample as output as the training target, the initial target data processing model is trained for at least one round using the training samples in the target historical sample set until the training ends when the training termination condition is met, thus obtaining the final target data processing model.
[0192] Alternatively, before training the initial target data processing model with the second feature of the test sample in the target historical sample set as input and the sample label corresponding to the test sample as output as the training target, the target historical sample set is divided into multiple target historical sample subsets.
[0193] During the process of training the initial target data processing model using training samples from the target historical sample set for at least one round, in each round of training, historical samples from one subset of the target historical sample set are used as test samples for the current training round, and historical samples from the remaining subsets of the target historical sample set are used as training samples for the current training round.
[0194] Optionally, generating analysis information for the business object to be processed based on the reasoning result includes:
[0195] Based on the reasoning results, determine the corresponding target mind chain CoT prompt engineering template;
[0196] The analysis instructions are determined based on the target CoT prompt engineering template, the business object to be processed, and / or the second feature, and then input into the Large Language Model (LLM) to obtain the analysis information for the business object to be processed output by the LLM.
[0197] Different inference results correspond to different CoT hint engineering templates.
[0198] Alternatively, the target data processing model may include an iterative decision tree model.
[0199] It should be understood that the device embodiments described above are merely illustrative. For example, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between devices or modules, and may be electrical, mechanical, or other forms.
[0200] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.
[0201] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium.
[0202] Since the specific methods of operation of each module of the data processing device have been described in detail in the corresponding data processing methods, they will not be repeated here.
[0203] Based on the same inventive concept, embodiments of this application also provide an electronic device, such as... Figure 11 As shown, it includes: a processor 110 and a memory 120 for storing executable instructions of the processor 110; wherein the processor 110 is configured to execute the instructions to implement the data processing method.
[0204] In specific implementations, the device may vary significantly due to differences in configuration or performance. It may include one or more processors 110, memory 120, and computer-readable storage media 130. The memory 120 and / or computer-readable storage media 130 may contain one or more application programs 131 or data 132. The memory 120 and / or computer-readable storage media 130 may also contain one or more operating systems 133, such as Windows, Mac OS, Linux, iOS, Android, Unix, FreeBSD, etc. The memory 120 and computer-readable storage media 130 may be temporary or persistent storage. The application program 131 may include one or more of the aforementioned modules (…). Figure 11 (Not shown in the image), each module may include a series of instruction operations. Furthermore, the processor 110 may be configured to communicate with the computer-readable storage medium 130 and execute a series of instruction operations in the computer-readable storage medium 130 on the device. The device may also include one or more power supplies (…). Figure 11 (not shown in the image); one or more network interfaces 140, including wired network interface 141 and / or wireless network interface 142; one or more input / output interfaces 143.
[0205] Based on the same inventive concept, embodiments of this application provide a computer-readable storage medium storing computer program code, which, when executed on a computer, enables the computer to implement the data processing method described above.
[0206] The computer-readable storage medium can be any available medium that a computer can store, or a data storage device such as a server or data center that integrates one or more available media. The available medium can be magnetic media (e.g., floppy disks, hard disks, USB flash drives, magnetic tapes, read-only memory (ROM), random access memory (RAM)), optical media (e.g., high-density digital video discs (DVDs), video compact discs (VCDs)), or semiconductor media (e.g., solid-state drives (SSDs)).
[0207] Since the principle of the computer-readable storage medium in solving the problem is the same as the data processing method described above, the implementation of the computer-readable storage medium can be found in the implementation of the method, and the repeated parts will not be described again.
[0208] Based on the same inventive concept, this application also provides a computer program product, which includes computer program code, which, when run on a computer, causes the computer to implement the data processing method.
[0209] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from a website site, computer, server, or data center to another website site, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means.
[0210] Since the principle behind the problem-solving of the above-mentioned computer program product is the same as that of the data processing method described above, the implementation of the above-mentioned computer program product can be referred to the implementation of the method, and the repeated parts will not be described again.
[0211] The data processing method and related hardware provided in this application first classify the business objects to be processed according to a first feature to obtain at least one set of business objects to be processed. Then, corresponding features are extracted from the business objects in different sets of business objects to be processed and input into the target data processing model corresponding to the set of business objects to be processed for processing. The analysis information corresponding to the business objects to be processed is determined according to the final inference result. It can analyze the business objects to be processed based on different feature categories with different characteristics, instead of using the same data processing method for all business objects to be processed. In this way, the characteristics of the business data can be targeted to obtain the correct analysis results.
[0212] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0213] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0214] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0215] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0216] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.
Claims
1. A data processing method, characterized in that, include: Extract at least one first feature of each category from each pending business object; Based on the first feature, each of the pending business objects is classified to obtain at least one set of pending business objects. For any pending business object in any set of pending business objects, perform the following steps: Based on the second feature category corresponding to the target data processing model, the corresponding second feature is extracted from the business object to be processed; wherein the target data processing model corresponds to the set of business objects to be processed, and the second feature categories corresponding to different target data processing models are at least partially different. The second feature is input into the target data processing model to obtain the reasoning result corresponding to the business object to be processed; Based on the reasoning results, analytical information is generated for the business object to be processed.
2. The method as described in claim 1, characterized in that, After classifying each pending business object according to the first feature to obtain at least one set of pending business objects, the method further includes: For any set of business objects to be processed, the set of business objects to be processed is divided into at least one subset of business objects to be processed according to a preset rule; Each subset of pending business objects is sent to a corresponding target data processing node, so that each target data processing node performs the following steps on any pending business object in the received subset: Based on the second feature category corresponding to the target data processing model, extract the corresponding second feature of the business object to be processed; The second feature is input into the target data processing model to obtain the reasoning result corresponding to the business object to be processed; Based on the reasoning results, analytical information is generated for the business object to be processed.
3. The method as described in claim 1, characterized in that, For any set of business objects to be processed, the target data processing model is trained as follows: Obtain the target historical sample set corresponding to the target data processing model; wherein, the target historical sample set is obtained by extracting at least one category of first features from each historical sample, classifying each historical sample according to the first features, and obtaining at least one historical sample set, which is the historical sample set corresponding to the target first feature; the target first feature is the first feature corresponding to the target data processing model; the historical sample is a sample constructed based on historical business objects and corresponding inference results; For any historical sample in the target historical sample set, extract the corresponding second feature of the historical sample according to the second feature category corresponding to the target data processing model; Using the second feature of the test sample in the target historical sample set as input and the sample label corresponding to the test sample as output as the training target, the initial target data processing model is trained on the training samples in the target historical sample set for at least one round until the training ends when the training termination condition is met, thus obtaining the final target data processing model.
4. The method as described in claim 1, characterized in that, Before training the initial target data processing model for at least one round using the second feature of the test sample in the target historical sample set as input and the sample label corresponding to the test sample as output as the training target, the method further includes: The target historical sample set is divided into multiple target historical sample subsets; During the process of training the initial target data processing model using training samples from the target historical sample set for at least one round, in each round of training, historical samples from one subset of the target historical sample set are used as test samples for the current training round, and historical samples from the remaining subsets of the target historical sample set are used as training samples for the current training round.
5. The method as described in claim 1, characterized in that, The step of generating analysis information for the business object to be processed based on the reasoning result includes: Based on the reasoning results, determine the corresponding target mind chain CoT prompt engineering template; The analysis instructions are determined based on the target CoT prompt engineering template, the business object to be processed, and / or the second feature, and then input into the Large Language Model (LLM) to obtain the analysis information for the business object to be processed output by the LLM. Different inference results correspond to different CoT hint engineering templates.
6. The method according to any one of claims 1-5, characterized in that, The target data processing model includes an iterative decision tree model.
7. A data processing apparatus, characterized in that, include: The first feature extraction module is used to extract at least one category of first features from each business object to be processed. The business object classification module is used to classify each of the business objects to be processed according to the first feature, so as to obtain at least one set of business objects to be processed. The business object processing module is used to perform the following steps on any business object in any set of business objects to be processed: extracting a corresponding second feature for the business object to be processed according to the second feature category corresponding to the target data processing model; wherein the target data processing model corresponds to the set of business objects to be processed, and the second feature categories corresponding to different target data processing models are at least partially different; inputting the second feature into the target data processing model to obtain the inference result corresponding to the business object to be processed; and generating analysis information for the business object to be processed based on the inference result.
8. An electronic device, characterized in that, include: A processor and a memory for storing processor-executable instructions; The processor is configured to execute the instructions to implement the data processing method as described in any one of claims 1-6.
9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program code that, when executed on a computer, causes the computer to perform the data processing method as described in any one of claims 1-6.
10. A computer program product, characterized in that, The computer program product includes: computer program code, which, when run on a computer, causes the computer to perform the data processing method as described in any one of claims 1-6.