Data processing method, device, computer equipment and storage medium
By acquiring and utilizing key model processing results for business scenarios, the results of data processing tasks are dynamically determined, which solves the inefficiency problem caused by the solidification of model processing sequence in traditional data processing, and improves data processing efficiency.
Patent Information
- Application Number
- CN202410357099.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-03-27
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2044-03-27
AI Technical Summary
In traditional data processing, due to the solidification of the processing sequence of the model, the data processing task needs to wait for the data processing results of the non-correlated model, resulting in inefficient processing, especially when business scenarios are diversified.
By obtaining the key models of the current business scenario, determining the processing results of the data processing task based on its processing results, avoiding waiting for the processing results of the non-dependent model, thereby improving data processing efficiency.
It realizes the flexibility to adjust key models in the interaction between the data platform and the model platform, dynamically determine the data processing results, reduces blocking waiting time, and improves data processing efficiency.
Smart Images

Figure CN118193164B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of data processing technology, and in particular to a data processing method, apparatus, computer equipment and storage medium. Background Art
[0002] With the rapid development of technologies in big data related fields, the transmission and processing of user data in data systems has become increasingly complex. In this context, multiple models are often provided for data to complete data calculations in parallel to improve performance and efficiency.
[0003] In the traditional data processing process, a certain strategy is often formulated based on the business, and the processing order of the model is set in advance. The problem that this method is prone to is that when the business scenarios are diversified, due to the relatively fixed processing order of the model, the execution process of the processing flow of data processing tasks in many business scenarios needs to be blocked to wait for the data processing results of some irrelevant models, resulting in low data processing efficiency. Summary of the invention
[0004] Based on this, it is necessary to provide a data processing method, device, computer equipment and storage medium that can improve data processing efficiency in response to the above technical problems.
[0005] A data processing method is applied to a data platform, and the method includes:
[0006] Get the current data processing task, which is the data processing task corresponding to the current business scenario;
[0007] Sending the current data processing task to the model platform, the model platform includes at least one model, and the at least one model is used to process the current data processing task;
[0008] Get the key model pre-configured for the current business scenario. The key model is the model in the model platform that the current business scenario depends on.
[0009] Obtain the processing results of the key model corresponding to the current business scenario from the model platform, and determine the processing results of the current data processing task based on the processing results of the key model corresponding to the current business scenario.
[0010] In one embodiment, obtaining a key model pre-configured for a current business scenario includes:
[0011] Get the policies pre-configured for the current business scenario;
[0012] Analyze the strategy to obtain the models that the current business scenario depends on and the priority of each model;
[0013] Determine the key model corresponding to the current business scenario based on the priority of each model that the current business scenario depends on.
[0014] In one embodiment, obtaining the processing result of the key model corresponding to the current business scenario from the model platform, and determining the processing result of the current data processing task according to the processing result of the key model corresponding to the current business scenario, includes:
[0015] Receive the processing results of each model sent by the model platform in sequence;
[0016] When the processing result of the key model corresponding to the current business scenario is received, the processing result of the current data processing task is determined according to the processing result of the key model corresponding to the current business scenario.
[0017] In one embodiment, the data processing method further includes:
[0018] Obtain new data processing tasks, which are data processing tasks corresponding to new business scenarios;
[0019] The new data processing task is used as the current data processing task, and the step of sending the current data processing task to the model platform is returned.
[0020] In one embodiment, the data processing method further includes:
[0021] Get a new data processing task, which is the next data processing task for the current business scenario;
[0022] Send new data processing tasks to the model platform;
[0023] The new data processing task is used as the current data processing task, and the step of returning to receive the processing result of the key model corresponding to the current business scenario sent by the model platform is performed.
[0024] In one embodiment, the key models corresponding to the current business scenario include multiple ones, obtaining the processing results of the key models corresponding to the current business scenario from the model platform, and determining the processing results of the current data processing task according to the processing results of the key models corresponding to the current business scenario, including:
[0025] When the key models corresponding to the current business scenario include an empty run model, the processing results of each key model except the empty run model are obtained from the model platform;
[0026] According to the processing results of each key model except the empty run model, the processing result of the current data processing task is obtained.
[0027] In one embodiment, the data processing method further includes:
[0028] Receive a configuration request for a new business scenario, where the configuration request includes each model that the new business scenario depends on and the priority of each model that the new business scenario depends on;
[0029] Configure the new business scenarios according to the models they rely on and the priorities of the models they rely on.
[0030] A data processing device, the data processing device is applied to a data platform, and the data processing device includes:
[0031] The first acquisition module is used to acquire the current data processing task, which is the data processing task corresponding to the current business scenario;
[0032] A sending module, used to send the current data processing task to the model platform, the model platform includes at least one model, and the at least one model is used to process the current data processing task;
[0033] The second acquisition module is used to acquire the key model pre-configured for the current business scenario, where the key model is a model in the model platform that the current business scenario depends on;
[0034] The processing module is used to obtain the processing results of the key model corresponding to the current business scenario from the model platform, and determine the processing results of the current data processing task based on the processing results of the key model corresponding to the current business scenario.
[0035] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:
[0036] Get the current data processing task, which is the data processing task corresponding to the current business scenario;
[0037] Sending the current data processing task to the model platform, the model platform includes at least one model, and the at least one model is used to process the current data processing task;
[0038] Get the key model pre-configured for the current business scenario. The key model is the model in the model platform that the current business scenario depends on.
[0039] Obtain the processing results of the key model corresponding to the current business scenario from the model platform, and determine the processing results of the current data processing task based on the processing results of the key model corresponding to the current business scenario.
[0040] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:
[0041] Get the current data processing task, which is the data processing task corresponding to the current business scenario;
[0042] Sending the current data processing task to the model platform, the model platform includes at least one model, and the at least one model is used to process the current data processing task;
[0043] Get the key model pre-configured for the current business scenario. The key model is the model in the model platform that the current business scenario depends on.
[0044] Obtain the processing results of the key model corresponding to the current business scenario from the model platform, and determine the processing results of the current data processing task based on the processing results of the key model corresponding to the current business scenario.
[0045] The above-mentioned data processing method, device, computer equipment and storage medium, the data processing method is applied to the data platform, and the data processing method includes: obtaining the current data processing task, the current data processing task is the data processing task corresponding to the current business scenario; sending the current data processing task to the model platform, the model platform includes at least one model, and at least one model is used to process the current data processing task; obtaining the key model pre-configured for the current business scenario, the key model is the model in the model platform that the current business scenario depends on; obtaining the processing result of the key model corresponding to the current business scenario from the model platform, and determining the processing result of the current data processing task according to the processing result of the key model corresponding to the current business scenario. The present application applies the data processing method to the interaction between the data platform and the model platform, and flexibly adjusts the key model that the current business scenario depends on, so as to determine the processing result of the current data processing task based on the processing result of the key model, without blocking and waiting for the processing result of the non-dependent model, so as to solve the problem of blocking and waiting between the data platform and the model platform during data interaction, and improve data processing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] Figure 1 An application environment diagram of a data processing method in an embodiment;
[0047] Figure 2 is a flow chart of a data processing method in one embodiment;
[0048] Figure 3 A flowchart of obtaining key model steps pre-configured for the current business scenario in one embodiment;
[0049] Figure 4 is a structural block diagram of a data processing device in one embodiment;
[0050] Figure 5 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION
[0051] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0052] Please refer to Figure 1 , Figure 1 A schematic diagram of an application environment of a data processing method provided by an exemplary embodiment of the present application. Figure 1 As shown, the application environment includes a data platform 100 and a model platform 101. The data platform 100 and the model 101 can be communicatively connected via a network 102 to implement the data processing method of the present application.
[0053] The data platform 100 is used to obtain the current data processing task, which is the data processing task corresponding to the current business scenario, and send the current data processing task to the model platform 101. The model platform 101 includes at least one model, and at least one model is used to process the current data processing task, obtain the key model pre-configured for the current business scenario, which is the model in the model platform 101 that the current business scenario depends on, obtain the processing result of the key model corresponding to the current business scenario from the model platform 101, and determine the processing result of the current data processing task according to the processing result of the key model corresponding to the current business scenario.
[0054] The model platform 101 is used to receive the current data processing task sent by the data platform 100, process the current data processing task through each model in the model platform 101, and feed back the processing results of each model to the data platform 100.
[0055] The data platform 100 and the model platform 101 may be implemented using independent servers or a server cluster consisting of multiple servers.
[0056] The network 102 is used to realize the network connection between the terminal 101 and the server 100. Specifically, the network 102 may include various types of wired or wireless networks.
[0057] In one embodiment, Figure 2 As shown, a data processing method is provided, which is applied to Figure 1 The data platform in the example is used to illustrate the process, including the following steps:
[0058] Step S21, obtaining the current data processing task, where the current data processing task is the data processing task corresponding to the current business scenario.
[0059] Among them, the current business scenario may include but is not limited to pre-loan credit, mid-loan transactions, anti-fraud and post-loan risk control scenarios. The current data processing task refers to the task that needs to be processed in the current business scenario. For example, the current data processing task can be a feature value calculation task, where the feature values may include the number of orders, order amount and user credit limit information before adjustment in transaction features, or it can be a browsing record of the product such as the number of views, the number of purchases and the purchase amount. Specifically, the feature values here can also be features that users in other systems need to pay attention to, which are set according to actual needs and are not specifically limited here.
[0060] The data platform here refers to the platform used to process data, which is mainly used to calculate the characteristic values required by other systems.
[0061] Step S22, sending the current data processing task to the model platform, the model platform includes at least one model, and the at least one model is used to process the current data processing task.
[0062] The model platform here refers to a platform responsible for processing the original data sent by the data platform. The model platform includes at least one model, and each model is used to process the data processing task sent by the data platform.
[0063] Specifically, the current data processing task can be specifically assigned to one or more models for processing.
[0064] Step S23, obtaining a key model pre-configured for the current business scenario, where the key model is a model in a model platform that the current business scenario depends on.
[0065] Among them, the key model refers to the important model required for the data processing task of the current business scenario. In this application, multiple business scenarios are pre-configured, and the model information of the key model corresponding to each business scenario is configured, for example, the mapping relationship between each business scenario and the model identifier of the key model is pre-configured.
[0066] In this application, each business scenario relies on different key models and supports flexible and dynamic configuration.
[0067] Step S24, obtaining the processing result of the key model corresponding to the current business scenario from the model platform, and determining the processing result of the current data processing task according to the processing result of the key model corresponding to the current business scenario.
[0068] Among them, the data platform of the present application can obtain the processing results of the key model from the model platform, or the model platform can feed back the processing results of the key model to the data platform. After obtaining the data results of the key model, it is determined that the current data processing task is completed, and the next data processing task can be obtained for processing without blocking and waiting for the processing results of other non-dependent models, thereby improving data processing efficiency.
[0069] In some embodiments, Figure 3 As shown, the above-mentioned acquisition of the key model pre-configured for the current business scenario may include:
[0070] Step S31, obtaining a policy pre-configured for the current business scenario.
[0071] Step S32, parsing the policy to obtain the various models that the current business scenario depends on and the priority of each model.
[0072] Step S33, determining the key model corresponding to the current business scenario according to the priority of each model that the current business scenario depends on.
[0073] Among them, in this application, corresponding strategies are pre-configured for each business scenario, and each strategy specifically includes task processing details information corresponding to the data processing task. Exemplarily, the strategy may include task information and model dependency information for each data processing task. Among them, the task information may include task identification, task content, and execution time, etc. The task content may include counting the number of transaction orders, counting the transaction amount, etc. Model dependency information refers to the identification information of the model that each data processing task needs to rely on.
[0074] Specifically, the present application may also include a policy platform, in which policies corresponding to various business scenarios are set, and the policies include corresponding data processing tasks. The data platform interacts with the policy platform to obtain various data processing tasks. After obtaining the data processing tasks, the model dependency information corresponding to the data processing tasks is further parsed from the policies, so as to obtain the processing results of the data processing tasks according to the model dependency information, without blindly waiting for the processing results of each model.
[0075] Specifically, when configuring the strategy, the present application may configure the key model corresponding to each business scenario, or may configure the priority of each dependent model corresponding to each business scenario, and then select other preset digits with higher priority as the key model according to the priority.
[0076] For example, please refer to Table 1 below, which is a configuration information table of a business scenario in an embodiment.
[0077] Table 1
[0078] Business Scenario Key Models Scenario 1 Model A, Model B, and Model C Scenario 2 Model D, Model E and Model F Scene 3 Model A and Model E
[0079] Table 1 shows that each business scenario is configured with a corresponding key model. For example, the key models corresponding to scenario 1 are model A, model B, and model C.
[0080] Please refer to Table 2 below, which is a configuration information table of a business scenario in an embodiment.
[0081] Table 2
[0082] Business Scenario Dependent models and priority ranking Key Models Scenario 1 Model A, Model B, Model C, and Model D Model A, Model B, and Model C Scenario 2 Model D, Model E, Model F, and Model G Model D, Model E and Model F Scene 3 Model A, Model E, and Model C Model A and Model E
[0083] Table 2 lists the models that each business scenario is dependent on and their priorities, as well as the key models selected from each model. For example, the models that scenario 1 is dependent on and their priorities are: model A, model B, model C, and model D. The key models configured are the top 3 models, namely model A, model B, and model C.
[0084] This application can be used to configure the key models that data processing tasks in various business scenarios rely on. When data processing tasks in different business scenarios are received, the key models that are relied on can be dynamically adjusted based on the configuration information, and the final data processing results can be determined based on the processing results of the key models. There is no need to block and wait for the processing results of non-associated models, thereby improving data processing efficiency.
[0085] In some embodiments, obtaining the processing result of the key model corresponding to the current business scenario from the model platform, and determining the processing result of the current data processing task according to the processing result of the key model corresponding to the current business scenario, includes:
[0086] Receive the processing results of each model sent by the model platform in sequence;
[0087] When the processing result of the key model corresponding to the current business scenario is received, the processing result of the current data processing task is determined according to the processing result of the key model corresponding to the current business scenario.
[0088] Among them, the model platform can sequentially process the results of each model associated with the current business scenario (each model on which the current business scenario depends). Correspondingly, the data platform sequentially receives the processing results of each associated model sent by the model platform. When the data platform receives the processing results of each key model corresponding to the current business scenario, the current data processing result is obtained according to the processing results of each key model, or in one embodiment, the processing results of each key model are used as the current data processing result.
[0089] In this application, there is no need to block and wait for the processing results of all models. Instead, after receiving the processing results of the key model, the next data processing task can be obtained.
[0090] In some embodiments, the above data processing method may further include:
[0091] Obtain identification information of the key model corresponding to the current business scenario, and send the identification information of the key model corresponding to the current business scenario to the model platform;
[0092] Obtaining the processing results of the key model corresponding to the current business scenario from the model platform, and determining the processing results of the current data processing task according to the processing results of the key model corresponding to the current business scenario, which may include:
[0093] The receiving model platform filters out the processing results of the key model corresponding to the current business scenario according to the identification information of the key model corresponding to the current business scenario and then sends the processing results of the key model corresponding to the current business scenario.
[0094] Among them, after the data platform obtains the key model that the current business scenario depends on, it can send the identification information of the key model to the model platform. The model platform will filter out the processing results of the key model based on the identification of the key model and send them to the data platform. The data platform obtains the processing results of the current data based on the processing results of the key model.
[0095] In some embodiments, the above data processing method may further include:
[0096] Obtain new data processing tasks, which are data processing tasks corresponding to new business scenarios;
[0097] The new data processing task is used as the current data processing task, and the step of sending the current data processing task to the model platform is returned.
[0098] In this application, different business scenarios correspond to different key models. When the business scenario switches (that is, when the acquired data processing task is a task under a new business scenario), the data platform will obtain the key model corresponding to the new business scenario, obtain the processing results of the key model corresponding to the new business scenario from the model platform, and determine the processing results of the current data processing task based on the processing results of the key model corresponding to the current business scenario.
[0099] This application can be used to switch to new business scenarios and dynamically adjust the key models corresponding to the current data processing tasks in real time according to the changes in business scenarios, so as to obtain the current data processing results based on the key models, realize the dynamic adjustment of the key models for data processing, reduce the blocking waiting time, and improve the data processing efficiency.
[0100] In some embodiments, the above data processing method may further include:
[0101] Get a new data processing task, which is the next data processing task for the current business scenario;
[0102] Send new data processing tasks to the model platform;
[0103] The new data processing task is used as the current data processing task, and the step of returning to receive the processing result of the key model corresponding to the current business scenario sent by the model platform is performed.
[0104] In this application, when the business scenario to which the next data processing task belongs is still the current business scenario, there is no need to re-acquire the key model that the current business scenario depends on.
[0105] In some embodiments, the key models corresponding to the current business scenario include multiple ones, obtaining the processing results of the key models corresponding to the current business scenario from the model platform, and determining the processing results of the current data processing task according to the processing results of the key models corresponding to the current business scenario, including:
[0106] When the key models corresponding to the current business scenario include an empty run model, the processing results of each key model except the empty run model are obtained from the model platform;
[0107] According to the processing results of each key model except the empty run model, the processing result of the current data processing task is obtained.
[0108] In this application, the above-mentioned dry run model refers to the process for performing dry run verification. When there is a dry run model in the key model, when executing the data processing task, there is no need to wait for the processing result of the dry run model. When other models in the key model except the dry run model have been executed, the execution result can be returned, and then the next data processing task can be obtained for further processing.
[0109] By adopting this implementation of the present application, in some special cases, such as dry run verification, the blocking waiting time can be further shortened and the data processing efficiency can be improved.
[0110] In some embodiments, the above data processing method may further include:
[0111] Receive a configuration request for a new business scenario, where the configuration request includes each model that the new business scenario depends on and the priority of each model that the new business scenario depends on;
[0112] Configure the new business scenarios according to the models they rely on and the priorities of the models they rely on.
[0113] In the present application, the data platform may receive a configuration request for a new business scenario, which may include identification information of a key model, or may include each model on which the newly added business scenario depends and the priority of each model on which the newly added business scenario depends. The data platform establishes a mapping relationship between the new business scenario and the key model based on the identification information of the key model corresponding to the newly added business scenario, or establishes a mapping relationship between the new business scenario and each model on which the newly added business scenario depends and the priority of each model on which the new business scenario depends.
[0114] Through this implementation, new business scenarios can be flexibly configured, and configuration information of existing business scenarios can be flexibly modified to achieve expansion and flexible adjustment of solutions.
[0115] In the present application, on the one hand, the key models corresponding to the business scenarios can be flexibly configured. Based on the configuration of the business scenarios, the processing priority of the model can be dynamically adjusted in different business scenarios, thereby achieving data processing based on different model processing priorities in different business scenarios, which solves the problem of rigid configuration of model processing in traditional technologies. On the other hand, data processing tasks in different business scenarios in the present application can be processed based on different model processing priorities, without blocking and waiting for the processing results of non-dependent models, thereby improving data processing efficiency.
[0116] In one embodiment, Figure 4 As shown, a data processing device is provided, comprising: a first acquisition module 41, a sending module 42, a second acquisition module 43, and a processing module 44, wherein:
[0117] A first acquisition module 41 is used to acquire a current data processing task, where the current data processing task is a data processing task corresponding to a current business scenario;
[0118] A sending module 42, used to send the current data processing task to the model platform, the model platform includes at least one model, and the at least one model is used to process the current data processing task;
[0119] The second acquisition module 43 is used to acquire a key model pre-configured for the current business scenario, where the key model is a model in a model platform that the current business scenario depends on;
[0120] The processing module 44 is used to obtain the processing results of the key model corresponding to the current business scenario from the model platform, and determine the processing results of the current data processing task according to the processing results of the key model corresponding to the current business scenario.
[0121] In one of the embodiments, the second acquisition module 43 can obtain the policy pre-configured for the current business scenario, parse the policy, obtain the various models that the current business scenario depends on and the priority of each model, and determine the key model corresponding to the current business scenario based on the priority of each model that the current business scenario depends on.
[0122] In one embodiment, the above-mentioned processing module 44 can receive the processing results of each model sent by the model platform in sequence. When the processing result of the key model corresponding to the current business scenario is received, the processing result of the current data processing task is determined according to the processing result of the key model corresponding to the current business scenario.
[0123] In one embodiment, the above-mentioned processing module 44 can also obtain a new data processing task, which is a data processing task corresponding to a new business scenario, and take the new data processing task as the current data processing task, and return to the step of sending the current data processing task to the model platform.
[0124] In one embodiment, the above-mentioned processing module 44 can also obtain a new data processing task, which is the next data processing task of the current business scenario, send the new data processing task to the model platform, take the new data processing task as the current data processing task, and return to the step of receiving the processing results of the key model corresponding to the current business scenario sent by the model platform.
[0125] In one embodiment, the key models corresponding to the current business scenario include multiple ones. The processing module 44 can also obtain the processing results of each key model except the empty run model from the model platform when the key models corresponding to the current business scenario include the empty run model, and obtain the processing result of the current data processing task according to the processing results of each key model except the empty run model.
[0126] In one of the embodiments, the above-mentioned processing module 44 can also receive a configuration request for a newly added business scenario, the configuration request including various models on which the newly added business scenario depends and the priorities of various models on which the newly added business scenario depends, and the newly added business scenario is configured according to the various models on which the newly added business scenario depends and the priorities of various models on which the new business scenario depends.
[0127] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 5As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the data processing method is implemented.
[0128] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the following steps are implemented: obtaining a current data processing task, where the current data processing task is a data processing task corresponding to a current business scenario; sending the current data processing task to a model platform, where the model platform includes at least one model, where the at least one model is used to process the current data processing task; obtaining a key model pre-configured for the current business scenario, where the key model is a model in the model platform that the current business scenario depends on; obtaining a processing result of the key model corresponding to the current business scenario from the model platform, and determining a processing result of the current data processing task based on the processing result of the key model corresponding to the current business scenario.
[0129] In one embodiment, when the processor executes the computer program to implement the above-mentioned step of obtaining the key model pre-configured for the current business scenario, the following steps are specifically implemented:
[0130] Get the policies pre-configured for the current business scenario;
[0131] Analyze the strategy to obtain the models that the current business scenario depends on and the priority of each model;
[0132] Determine the key model corresponding to the current business scenario based on the priority of each model that the current business scenario depends on.
[0133] In one embodiment, when the processor executes the computer program to implement the above-mentioned steps of obtaining the processing result of the key model corresponding to the current business scenario from the model platform and determining the processing result of the current data processing task according to the processing result of the key model corresponding to the current business scenario, the following steps are specifically implemented:
[0134] Receive the processing results of each model sent by the model platform in sequence;
[0135] When the processing result of the key model corresponding to the current business scenario is received, the processing result of the current data processing task is determined according to the processing result of the key model corresponding to the current business scenario.
[0136] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:
[0137] Obtain new data processing tasks, which are data processing tasks corresponding to new business scenarios;
[0138] The new data processing task is used as the current data processing task, and the step of sending the current data processing task to the model platform is returned.
[0139] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:
[0140] Get a new data processing task, which is the next data processing task for the current business scenario;
[0141] Send new data processing tasks to the model platform;
[0142] The new data processing task is used as the current data processing task, and the step of returning to receive the processing result of the key model corresponding to the current business scenario sent by the model platform is performed.
[0143] In one embodiment, the key models corresponding to the current business scenario include multiple ones. When the processor executes the computer program to implement the above-mentioned steps of obtaining the processing result of the key model corresponding to the current business scenario from the model platform and determining the processing result of the current data processing task according to the processing result of the key model corresponding to the current business scenario, the following steps are specifically implemented:
[0144] When the key models corresponding to the current business scenario include an empty run model, the processing results of each key model except the empty run model are obtained from the model platform;
[0145] According to the processing results of each key model except the empty run model, the processing result of the current data processing task is obtained.
[0146] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:
[0147] Receive a configuration request for a new business scenario, where the configuration request includes each model that the new business scenario depends on and the priority of each model that the new business scenario depends on;
[0148] Configure the new business scenarios according to the models they rely on and the priorities of the models they rely on.
[0149] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented: obtaining a current data processing task, the current data processing task is a data processing task corresponding to a current business scenario, sending the current data processing task to a model platform, the model platform includes at least one model, and the at least one model is used to process the current data processing task, obtaining a key model pre-configured for the current business scenario, the key model is a model in the model platform that the current business scenario depends on, obtaining a processing result of the key model corresponding to the current business scenario from the model platform, and determining a processing result of the current data processing task according to the processing result of the key model corresponding to the current business scenario.
[0150] In one embodiment, when the computer program is executed by the processor to implement the above-mentioned step of obtaining the key model pre-configured for the current business scenario, the following steps are specifically implemented:
[0151] Get the policies pre-configured for the current business scenario;
[0152] Analyze the strategy to obtain the models that the current business scenario depends on and the priority of each model;
[0153] Determine the key model corresponding to the current business scenario based on the priority of each model that the current business scenario depends on.
[0154] In one embodiment, when the computer program is executed by the processor to implement the above-mentioned steps of obtaining the processing result of the key model corresponding to the current business scenario from the model platform and determining the processing result of the current data processing task according to the processing result of the key model corresponding to the current business scenario, the following steps are specifically implemented:
[0155] Receive the processing results of each model sent by the model platform in sequence;
[0156] When the processing result of the key model corresponding to the current business scenario is received, the processing result of the current data processing task is determined according to the processing result of the key model corresponding to the current business scenario.
[0157] In one embodiment, when the computer program is executed by a processor, the following steps are specifically implemented:
[0158] Obtain new data processing tasks, which are data processing tasks corresponding to new business scenarios;
[0159] The new data processing task is used as the current data processing task, and the step of sending the current data processing task to the model platform is returned.
[0160] In one embodiment, when the computer program is executed by a processor, the following steps are specifically implemented:
[0161] Get a new data processing task, which is the next data processing task for the current business scenario;
[0162] Send new data processing tasks to the model platform;
[0163] The new data processing task is used as the current data processing task, and the step of returning to receive the processing result of the key model corresponding to the current business scenario sent by the model platform is performed.
[0164] In one embodiment, the key models corresponding to the current business scenario include multiple ones. When the computer program is executed by the processor to implement the above-mentioned steps of obtaining the processing result of the key model corresponding to the current business scenario from the model platform and determining the processing result of the current data processing task according to the processing result of the key model corresponding to the current business scenario, the following steps are specifically implemented:
[0165] When the key models corresponding to the current business scenario include an empty run model, the processing results of each key model except the empty run model are obtained from the model platform;
[0166] According to the processing results of each key model except the empty run model, the processing result of the current data processing task is obtained.
[0167] In one embodiment, when the computer program is executed by a processor, the following steps are specifically implemented:
[0168] Receive a configuration request for a new business scenario, where the configuration request includes each model that the new business scenario depends on and the priority of each model that the new business scenario depends on;
[0169] Configure the new business scenarios according to the models they rely on and the priorities of the models they rely on.
[0170] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing related hardware through a computer program, and the above-mentioned computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct RAMbus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.
[0171] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0172] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.
Claims
1. A data processing method, characterized in that: The method is applied to a data platform, and the method includes: Obtaining a current data processing task, where the current data processing task is a data processing task corresponding to the current business scenario; Sending the current data processing task to a model platform, wherein the model platform includes at least one model, and the at least one model is used to process the current data processing task; Acquire a key model pre-configured for the current business scenario according to a mapping relationship between each pre-configured business scenario and a key model, wherein the key model is a model in the model platform on which the current business scenario depends, and the key model refers to a model required to be used in the process of executing the current data processing task; The processing result of the key model corresponding to the current business scenario is obtained from the model platform, and the processing result of the current data processing task is determined according to the processing result of the key model corresponding to the current business scenario.
2. The method according to claim 1, characterized in that The obtaining of a key model pre-configured for the current business scenario includes: Obtaining a policy pre-configured for the current business scenario; Parse the strategy to obtain the models that the current business scenario depends on and the priority of each model; According to the priorities of the various models that the current business scenario depends on, the key model corresponding to the current business scenario is determined.
3. The method according to claim 1, characterized in that The acquiring, from the model platform, the processing result of the key model corresponding to the current business scenario, and determining the processing result of the current data processing task according to the processing result of the key model corresponding to the current business scenario, includes: receiving the processing results of each model sent by the model platform in sequence; When the processing result of the key model corresponding to the current business scenario is received, the processing result of the current data processing task is determined according to the processing result of the key model corresponding to the current business scenario.
4. The method according to claim 1, characterized in that: The method further comprises: Obtaining a new data processing task, where the new data processing task is a data processing task corresponding to a new business scenario; Taking the new data processing task as the current data processing task, returning to the step of sending the current data processing task to the model platform.
5. The method according to claim 1, characterized in that The method further comprises: Obtaining a new data processing task, where the new data processing task is the next data processing task of the current business scenario; Sending the new data processing task to the model platform; Taking the new data processing task as the current data processing task, returning to the step of sending the current data processing task to the model platform.
6. The method according to claim 1, characterized in that The key models corresponding to the current business scenario include multiple ones, and the processing result of the key model corresponding to the current business scenario is obtained from the model platform, and the processing result of the current data processing task is determined according to the processing result of the key model corresponding to the current business scenario, including: When the key models corresponding to the current business scenario include an empty run model, obtaining processing results of each key model except the empty run model from the model platform; The processing result of the current data processing task is obtained according to the processing results of each key model except the empty run model.
7. The method according to claim 2, characterized in that The method further comprises: Receive a configuration request for a newly added business scenario, wherein the configuration request includes each model on which the newly added business scenario depends and each model priority on which the newly added business scenario depends; The newly added business scenario is configured according to each model on which the newly added business scenario depends and the priority of each model on which the newly added business scenario depends.
8. A data processing device, characterized in that: The device is applied to a data platform, and comprises: A first acquisition module is used to acquire a current data processing task, where the current data processing task is a data processing task corresponding to a current business scenario; A sending module, used for sending the current data processing task to a model platform, wherein the model platform includes at least one model, and the at least one model is used for processing the current data processing task; A second acquisition module is used to acquire a key model pre-configured for the current business scenario according to a mapping relationship between each pre-configured business scenario and a key model, wherein the key model is a model in the model platform on which the current business scenario depends, and the key model refers to a model required to be used in the process of executing the current data processing task; A processing module is used to obtain the processing results of the key model corresponding to the current business scenario from the model platform, and determine the processing results of the current data processing task according to the processing results of the key model corresponding to the current business scenario.
9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.
