Decision engine updating method and apparatus based on strategy large model

Through the decision engine update method based on the strategy big model, the decision engine strategy is automatically tested and optimized, and the problem of difficult strategy writing and testing in complex decision engines is solved, which improves testing efficiency and accuracy and shortens the optimization cycle.

WO2024239577A1PCT designated stage expired Publication Date: 2024-11-28SHANGHAI QIYUE INFORMATION TECH CO LTD

Patent Information

Application Number
PCT/CN2023/135340
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-05-25
Filing Date
2023-11-30
Publication Date
2024-11-28

AI Technical Summary

Technical Problem

In complex decision-making engines, human errors are prone to occur during strategy writing and testing, resulting in production accidents. The online work after each modification requires multiple tests, which is time-consuming and labor-intensive.

Method used

Adopt the decision engine update method based on the strategy big model, by obtaining the updated running data packets and initial running data packets of the terminal decision engine, generating an observation parameter set, obtaining matching actual decision data, executing preset strategies, iterating and adjusting rules until they meet the pre-pre-conformity Set decision-making expectations, and automatically conduct testing and optimization.

Benefits of technology

It greatly improves testing efficiency and accuracy, shortens the decision engine performance optimization cycle, and reduces manual operation processes.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN2023135340_28112024_PF_FP_ABST
    Figure CN2023135340_28112024_PF_FP_ABST
Patent Text Reader

Abstract

The present application relates to a decision engine updating method and apparatus based on a strategy large model. The method comprises: generating an observation parameter set for differences between an updated operation data packet and an initial operation data packet on the basis of a strategy large model; acquiring sample data matched with the observation parameter set; testing the updated operation data packet by means of the sample data; calling the strategy large model to generate an iteration reference trend according to a deviation between a strategy execution result obtained in each iteration and a preset decision expectation when the strategy execution result does not meet the preset decision expectation; generating a plurality of reference rules according to the iteration reference trend to iteratively re-execute a preset strategy for the updated operation data packet; and updating a terminal decision engine according to the updated operation data packet after the iteration. According to the present application, a large model is used to assist in determining the performance change of a decision engine caused by the change of operation parameters, thereby greatly improving the testing efficiency and accuracy and shortening the performance optimization cycle of the decision engine.
Need to check novelty before this filing date? Find Prior Art

Description

Decision engine updating method and device based on strategic big model Technical Field

[0001] The present application relates to the field of computer information processing, and more specifically, to a decision engine update method, device, electronic device, and computer-readable medium based on a policy big model. Background Art

[0002] A decision engine refers to a platform that enables an enterprise to provide personalized service decisions to its customers. The decision engine is a tool that can support the enterprise's various decisions in customer management (CRM), and various solutions can be developed based on the decision engine.

[0003] The decision engine requires interfaces to obtain various data from other systems (such as data warehouses). The specific data input depends on the specific business problem being solved. After calculation, the decision engine outputs a decision result. For example, in a single-user application scenario, the decision result for user A; in a batch user application scenario, the decision result for a batch of users (user group A) is output. Business personnel can design the decision process in a visual interface.

[0004] As empirical strategies and model-based strategies are increasingly combined in automated decision-making scenarios, the decision engine supports the import, export, and invocation of algorithmic models. Before being applied, strategies in the decision engine undergo three steps: strategy writing, strategy testing, and strategy deployment. When the decision process is complex or the rules are significantly adjusted, human errors in strategy writing and testing are inevitable, leading to production accidents. Because the decision engine involves complex interface configuration and logical processing, each change requires multiple tests before deployment, which is time-consuming and labor-intensive.

[0005] Therefore, a new decision engine update method, device, electronic device and computer-readable medium based on a policy big model are needed.

[0006] The above information disclosed in this Background section is only for enhancement of understanding of the background of the application and therefore it may contain information that does not form the prior art that is already known to a person of ordinary skill in the art.

[0007] Summary of the Invention

[0008] In view of this, the present application provides a decision engine update method, device, electronic device and computer-readable medium based on a policy big model, which can automatically test the policies in the decision engine, intelligently produce test reports, summarize the changes in key indicators, and use big models to assist in determining the impact of changes in operating parameters on the performance of the decision engine, greatly improving test efficiency and accuracy, and accelerating the performance optimization cycle of the decision engine.

[0009] Other features and advantages of the present application will become apparent from the following detailed description, or may be learned in part by practice of the present application.

[0010] According to one aspect of the present application, a decision engine update method based on a policy big model is proposed, the method comprising: obtaining an updated operation data packet to be tested in a terminal decision engine and its corresponding initial operation data packet; generating an observation parameter set for the difference between the updated operation data packet and the initial operation data packet; obtaining actual decision data corresponding to each operation scenario that matches the observation parameter set within a predetermined time as sample data; executing a preset policy on the updated operation data packet through the sample data; when the policy execution result does not meet the preset decision expectation, generating an iterative reference trend based on the deviation between the policy execution result obtained in each iteration and the preset decision expectation; generating multiple reference rules through the iterative reference trend to iteratively re-execute the preset policy on the updated operation data packet until the policy execution result meets the preset decision expectation; and updating the terminal decision engine through the iterated updated operation data packet.

[0011] Optionally, a preset strategy is executed on the updated operation data packet through the sample data, including: matching the operation data in the updated operation data packet with the operation data corresponding to each operation scenario in the sample data; predicting the predicted decision data when the updated operation data packet is put into operation online based on the matching result; and generating a strategy execution result through a comparison result between the predicted decision data and the initial decision data generated when the initial operation data packet is running.

[0012] Optionally, a strategy execution result is generated by comparing the predicted decision data with the initial decision data generated when the initial running data packet is running, including: generating a change comparison curve based on the difference between the predicted decision data and the initial decision data; and generating the strategy execution result by matching the difference between the running rules of the observation parameter set and the change comparison curve.

[0013] Optionally, the updated operation data packet is iterated by generating multiple reference rules through the iterative reference trend, including: the strategy big model determines the contribution of different operation rules in the updated operation data packet to the change of the predicted decision data and the initial decision data according to the iterative reference trend; the strategy big model generates multiple reference rules based on the deviation between the strategy execution result and the preset decision expectation and the change contribution corresponding to the different operation rules; and the updated operation data packet is corrected through the multiple reference rules.

[0014] Optionally, obtaining an updated operating data packet to be tested and its corresponding initial operating data packet in the terminal decision engine includes: real-time monitoring of multiple operating rules of the initial operating data packet in the terminal decision engine; when the change of the initial operating data packet meets the modification policy, generating the updated operating data packet according to the modified operating data packet.

[0015] Optionally, an observation parameter set is generated for the differences between the updated operation data packet and the initial operation data packet, including: matching multiple observation parameters of the updated operation data packet and the initial operation data packet through the strategy big model, and combining the observation parameters that are not successfully matched to generate the observation parameter set.

[0016] Optionally, generating an observation parameter set for the difference between the updated operation data packet and the initial operation data packet includes: matching multiple operation rules of the updated operation data packet and the initial operation data packet through the policy big model; and generating an operation rule set through the operation rules that are not successfully matched.

[0017] Optionally, actual decision data corresponding to each operating scenario that matches the observation parameter set within a predetermined time is obtained as sample data, including: determining a predetermined time range, target sample category and target sample quantity based on each operating scenario corresponding to the initial operating data packet; based on the strategy big model, historical samples are screened according to the predetermined time range, the target sample category and the target sample quantity to extract the sample data.

[0018] Optionally, based on the strategy big model, historical samples are screened according to the predetermined time range, the target sample category and the target sample quantity, and the sample data is extracted, including: inputting the historical samples within the predetermined time range into the strategy big model, classifying the historical samples to obtain sample categories; and extracting historical samples that meet the target sample category and the target sample quantity through the strategy big model to obtain the sample data.

[0019] Optionally, executing a preset strategy on the update operation data package using the sample data includes: distributing the sample data to multiple servers; and the multiple servers executing the preset strategy on the update operation data package in parallel using the sample data.

[0020] According to one aspect of the present application, a decision engine update device based on a policy big model is proposed, which includes: a data module for obtaining an updated operation data packet to be tested in a terminal decision engine and its corresponding initial operation data packet; a parameter module for generating an observation parameter set for the difference between the updated operation data packet and the initial operation data packet based on the policy big model; a sample module for obtaining actual decision data corresponding to each operation scenario that matches the observation parameter set within a predetermined time as sample data; a test module for executing a preset policy on the updated operation data packet through the sample data; an iteration module for calling the policy big model to generate an iterative reference trend based on the deviation between the policy execution result obtained in each iteration and the preset decision expectation when the policy execution result does not meet the preset decision expectation; generating multiple reference rules through the iterative reference trend to iteratively re-execute the preset policy on the updated operation data packet until the policy execution result meets the preset decision expectation; and an update module for updating the terminal decision engine through the iterated updated operation data packet.

[0021] According to one aspect of the present application, an electronic device is proposed, which includes: one or more processors; a storage device for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the method as described above.

[0022] According to one aspect of the present application, a computer-readable medium is provided, on which a computer program is stored. When the program is executed by a processor, the method described above is implemented.

[0023] According to the decision engine update method, device, electronic device and computer-readable medium based on the policy big model of the present application, an updated operation data packet to be tested and its corresponding initial operation data packet are obtained in the terminal decision engine; based on the policy big model, an observation parameter set is generated for the difference between the updated operation data packet and the initial operation data packet; actual decision data corresponding to each operation scenario matching the observation parameter set within a predetermined time is obtained as sample data; a preset policy is executed on the updated operation data packet using the sample data; and when the policy execution result does not meet the preset decision expectation, the policy big model is called to generate an iterative reference trend based on the deviation between the policy execution result obtained in each iteration and the preset decision expectation; multiple reference rules are generated based on the iterative reference trend to iteratively re-execute the preset policy on the updated operation data packet until the policy execution result meets the preset decision expectation; and the terminal decision engine is updated by the iterative updated operation data packet. This method can automatically test the policy in the decision engine, intelligently produce a test report, summarize the changes in key indicators, and use the big model to assist in determining the impact of changes in operating parameters on the performance of the decision engine, greatly improving test efficiency and accuracy and accelerating the performance optimization cycle of the decision engine.

[0024] It should be understood that the foregoing general description and the following detailed description are merely illustrative and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The above and other objects, features, and advantages of the present application will become more apparent by describing in detail exemplary embodiments thereof with reference to the accompanying drawings. The drawings described below are merely some embodiments of the present application, and it is apparent to those skilled in the art that other drawings can be derived from these drawings without inventive effort.

[0026] Fig. 1 is a flow chart showing a decision engine update method based on a policy big model according to an exemplary embodiment.

[0027] Fig. 2 is a flow chart showing a decision engine update method based on a policy big model according to another exemplary embodiment.

[0028] Fig. 3 is a flow chart showing a decision engine update method based on a policy big model according to another exemplary embodiment.

[0029] Fig. 4 is a flow chart showing a decision engine update method based on a policy big model according to another exemplary embodiment.

[0030] Fig. 5 is a block diagram showing a decision engine updating device based on a policy big model according to an exemplary embodiment.

[0031] Fig. 6 is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0032] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be embodied in many forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete and will fully convey the concepts of the example embodiments to those skilled in the art. Like reference numerals in the drawings represent like or similar parts, and thus repetitive description thereof will be omitted.

[0033] Fig. 1 is a flow chart showing a decision engine update method based on a policy big model according to an exemplary embodiment. The decision engine update method 10 based on a policy big model comprises at least steps S102 to S112.

[0034] As shown in Figure 1, in S102, an updated operational data packet to be tested and its corresponding initial operational data packet are obtained from the terminal decision engine. For example, multiple operational rules of the initial operational data packet in the terminal decision engine can be monitored in real time. When a change to the initial operational data packet satisfies a modification policy, the updated operational data packet is generated based on the modified operational data packet.

[0035] In a real-world application, a terminal decision engine can be used to analyze the real-time characteristics of a node and assign security or traffic policies to it. The decision engine's operational data package includes the rules, parameters, and logical relationships between the rules required for decision-making.

[0036] More specifically, in an actual application, a decision rule in the initial operation data packet of the decision engine may be: reduce the task processing load for nodes whose data transmission volume exceeds 10G in the current time period. The current time period may be 10 minutes, and the task processing load may be reduced by 50%.

[0037] At some point, the management user changes the decision rule in the initial operation data packet to reduce the task processing capacity for nodes with data transmission volume greater than 10G in the current time period. The current time period may be 50 minutes, and the task processing capacity may be reduced by 70%.

[0038] In the process of real-time monitoring of decision rules, after the management user changes the rules, a new update strategy is automatically generated, and then an update operation data package is generated according to the update strategy. In subsequent processing, the update operation data package is automatically tested.

[0039] During real-time monitoring, a change policy threshold can be set. For example, if the user changes a parameter range by less than 10%, automatic testing may not be performed. In the above embodiment, the user change decision rule is: if the current time period is 10.5 minutes, the change is small, and the automatic testing process may not be initiated.

[0040] More specific modification strategies for starting automatic testing can be set according to different application scenarios, and this application is not limited thereto.

[0041] In S104 , based on the policy macro model, an observation parameter set is generated for the differences between the updated operation data packet and the initial operation data packet.

[0042] In one embodiment, the policy macro model can be used to match multiple observation parameters of the updated operation data packet and the initial operation data packet, and the observation parameters that did not successfully match can be combined to generate the observation parameter set. Continuing with the above embodiment, the observation parameters of the updated operation data packet and the initial operation data packet can be compared. The observation parameters can include time period, data transmission volume, task processing reduction coefficient, etc. Since the current time period and task processing volume have been changed, these two observation parameters do not successfully match, and thus the observation parameter set is generated based on these two parameters.

[0043] In one embodiment, the strategy big model in the cost scheme can be formed by combining multiple sub-models constructed based on the architectural form of the multimodal big model, combined with the knowledge of the data screening process, the decision-making operation process, and the correlation between the operation data package and the actual decision data in this application. Alternatively, the multimodal big model directly learns the above knowledge in this application and trains to obtain the strategy big model in this scheme.

[0044] Separately constructed sub-models can avoid interference between the knowledge provided, resulting in the final large model being able to understand the instructions issued by the management user, but the execution process and results may not be consistent with expectations. The sub-models built by learning knowledge separately can only execute a certain plan, but the instructions given by the management personnel need to be more accurate to ensure that the strategy large model calls the accurate sub-model to execute the plan corresponding to the instruction.

[0045] Directly input knowledge data into the multimodal big model to build the strategy big model of this solution. When the amount of input knowledge data is sufficient, the multimodal big model can not only execute multiple solutions separately, but also learn the connection between each solution. Directly in a dialogue process, it continuously performs corresponding work according to the instructions issued by the management user, reducing manual operation processes and improving work efficiency and accuracy.

[0046] In one embodiment, the policy big model can be used to match the multiple operation rules of the updated operation data packet and the initial operation data packet; and an observation parameter set is generated based on the operation rules that have not been successfully matched. In an actual application scenario, the administrator can add new operation rules, for example, the new operation rule can be: nodes with a failure rate that reaches a threshold will be inspected on a daily basis. There were no similar operation rules in the previous system. After the new operation rules are added, the updated operation data packet and the initial operation data packet are input into the policy big model, and an instruction is sent to the policy big model to match the updated operation data packet and the initial operation data packet to generate an observation parameter set, and the operation rule is matched with the existing rules. If there are similar operation rules, then as described above, an observation parameter set is generated based on the observation parameters that have not been successfully matched. Otherwise, an observation parameter set is generated based on the newly added operation rules.

[0047] In S106, actual decision data corresponding to each operational scenario that matches the observation parameter set within a predetermined time period is obtained as sample data. For example, after the policy macromodel generates the observation parameter set, the management user inputs an instruction to the policy macromodel to obtain the actual decision data corresponding to the observation parameter set. The predetermined time range, target sample category, and target sample quantity are determined based on each operational scenario corresponding to the initial operational data packet. Because the policy macromodel has learned the association between operational data packets and actual decision data, the policy macromodel can be used to filter historical samples based on the predetermined time range, target sample category, and target sample quantity to extract the sample data.

[0048] Management users can set the time range, target sample category or number of samples for sample sampling in the system. They can also configure corresponding rules according to different operating scenarios and automatically generate the time range, target sample category or number of samples for sample sampling through the strategy model.

[0049] In S108, the preset policy is executed on the update execution data package using the sample data. The sample data can be distributed to multiple servers, and a large policy model can be deployed on each of the servers. Based on the large policy model, the multiple servers execute the preset policy on the update execution data package in parallel using the sample data, thereby executing different rules in the update execution data package separately and improving operational efficiency. The sample data can be distributed to multiple machines using MySQL to execute the preset policy simultaneously.

[0050] In one embodiment, after the policy big model obtains the sample data, the management user can issue an instruction to the policy big model to predict the decision result of the updated operation data package through the sample data, so as to match the operation data in the updated operation data package with the operation data corresponding to each operation scenario in the sample data through the policy big model; predict the predicted decision data when the updated operation data package is put into operation online based on the matching result; and generate the policy execution result through the comparison result of the predicted decision data and the initial decision data generated when the initial operation data package is running.

[0051] In S110, when the policy execution result does not meet the preset decision expectation, the management user can send an instruction to the policy big model to update the update operation data package according to the policy execution result obtained in the iterative process, and re-execute the preset policy. The policy big model executes the deviation between the policy execution result obtained in each iteration and the preset decision expectation to generate an iterative reference trend; multiple reference rules are generated through the iterative reference trend to iteratively re-execute the preset policy on the updated operation data package until the policy execution result meets the preset decision expectation.

[0052] In one embodiment, for example, the strategy big model can determine the contribution of different operating rules in the updated operation data package to the change of the predicted decision data and the initial decision data based on the iterative reference trend; the strategy big model generates multiple reference rules based on the deviation between the strategy execution result and the preset decision expectation and the contribution of the change corresponding to the different operation rules; and the updated operation data package is corrected using the multiple reference rules.

[0053] More specifically, in this embodiment, the different operating rules are the rules corresponding to the observation parameter set, that is, the different rules in the updated operating data package and the initial operating data package. Only this part of the rules has changed, and has caused the predicted decision data to change. At this time, according to the changes in parameters during the iteration process, the contribution of the changes in different operating rules to the changes in the predicted decision data and the initial decision data can be determined, and then the deviation between the strategy execution result and the preset decision expectation can be further determined. Finally, based on the deviation and contribution, reference rules are generated to correct the rules in the updated operating data package, so as to achieve rapid optimization of the deviation between the predicted decision data corresponding to the updated operating data package and the preset decision expectation, and more efficiently obtain the updated operating data package that meets the preset decision expectations, thereby improving the operating data update efficiency of the decision engine.

[0054] In S112, the terminal decision engine is updated by the iterative update operation data package. After the preset policy is satisfied, the terminal decision engine can be updated by the update operation data package and automatically put into operation.

[0055] According to the decision engine update method based on the policy big model of the present application, an updated operation data packet to be tested and its corresponding initial operation data packet are obtained in the terminal decision engine; an observation parameter set is generated based on the policy big model for the difference between the updated operation data packet and the initial operation data packet; actual decision data corresponding to each operation scenario that matches the observation parameter set within a predetermined time is obtained as sample data; a preset policy is executed on the updated operation data packet through the sample data; when the policy execution result does not meet the preset decision expectation, the policy big model is called to generate an iterative reference trend based on the deviation between the policy execution result obtained in each iteration and the preset decision expectation; multiple reference rules are generated through the iterative reference trend to iteratively re-execute the preset policy on the updated operation data packet until the policy execution result meets the preset decision expectation; the terminal decision engine is updated by the iterative updated operation data packet, which can automatically test the policy in the decision engine, intelligently produce a test report, summarize the changes in key indicators, and use the big model to assist in determining the impact of changes in operation parameters on the performance of the decision engine, greatly improving test efficiency and accuracy, and accelerating the performance optimization cycle of the decision engine.

[0056] It should be clearly understood that this application describes how to form and use specific examples, but the principles of this application are not limited to any details of these examples. On the contrary, based on the teaching of the content disclosed in this application, these principles can be applied to many other embodiments.

[0057] Figure 2 is a flow chart of a decision engine update method based on a policy big model according to another exemplary embodiment. The process 20 shown in Figure 2 is a detailed description of S108 "executing a preset policy on the update operation data package using the sample data" in the process shown in Figure 1.

[0058] As shown in Figure 2, in S202, the policy model matches the operational data in the updated operational data package with the operational data corresponding to each operational scenario in the sample data. After the management user issues an instruction to the policy model to determine the predicted decision data for the updated operational data package based on the sample data, the policy model matches the operational data required in the updated operational data package with the actual performance data of the sample in each scenario.

[0059] In S204, the prediction decision data of the updated operation data package when it is online is predicted based on the matching results. More specifically, the strategy model performs decision matching based on the sample data according to the operation rules in the updated operation data package, and generates prediction decision data from the decision data that matches the sample data.

[0060] Among them, the prediction decision data can be whether the node meets the security rules, the category of the user portrait, the security level of the user, etc.

[0061] In S206, the policy big model generates a policy execution result by comparing the predicted decision data with the initial decision data generated when the initial operation data package is running.

[0062] The policy model is invoked to compare the predicted solution data with the initial decision data, and the policy execution results are generated based on the comparison results. Historical decision results prior to the sample data can be directly extracted from the business system and considered to be the decision results of the initial execution data package. Furthermore, the policy model can also be used to determine the decision results corresponding to the initial execution data package based on the sample data.

[0063] In one embodiment, a change comparison curve can be generated based on the difference between the predicted decision data and the initial decision data through the strategy big model; and the difference between the running rules of the observation parameter set and the change comparison curve can be matched to generate the strategy execution result.

[0064] The strategy model can be used to generate a change comparison curve for each observation parameter, and a decision result change curve can also be generated, etc. This application is not limited to this.

[0065] The decision engine update method based on the policy big model of this application is based on production data instances, and integrates functions such as automatic data extraction, rule execution, result comparison, and intelligent output test reports. It automatically updates the terminal decision engine and reduces 80% of manual operation processes during the automated testing process. In actual applications, the execution time of 20,000 operating rule data does not exceed 3 minutes. Compared with the manual testing method in the existing technology, this application greatly improves the operating efficiency and accuracy, and can escort the safe launch of the strategy.

[0066] FIG3 is a flowchart illustrating a decision engine update method based on a policy macro model, according to another exemplary embodiment. The process 30 shown in FIG3 is a detailed description of S106 in the process shown in FIG1 , "obtaining actual decision data corresponding to each operational scenario that matches the observed parameter set within a predetermined time period, as sample data."

[0067] As shown in FIG3 , in S302 , a predetermined time range, target sample category, and target sample quantity are determined according to each operation scenario corresponding to the initial operation data packet.

[0068] Parameters such as the time range, sample category, and target sample quantity are determined based on the various operating scenarios and business processing data corresponding to the operating rules. In a specific embodiment, in a user operation scenario, the rule to be tested may be a user profiling rule, the time range may be determined to be user data within one month, the target sample category may be users who purchased a certain service, and the target sample quantity may be 10,000 users. After determining the sample conditions, the manager inputs the corresponding conditions into the policy model, and uses the policy model to filter samples from the knowledge data it has learned. For example, the user who meets the conditions is filtered in the database to generate sample data.

[0069] In S304, historical samples within the predetermined time range are obtained.

[0070] In one embodiment, when the decision target of the decision engine is a user, the historical sample may be a historical user sample. More specifically, the user may be an individual user or a corporate user. User information may include basic information authorized by the user, such as business account information, user terminal device identification information, user location information, etc.; user information may also include behavioral information, such as user page operation data, user service access duration, user service access frequency, etc. The specific content of user information can be determined according to the actual application scenario and is not limited here. Remote information may be user data of the user on other trading platforms or other business departments.

[0071] In one embodiment, when the decision engine's decision target is a computer node or other device node, the historical samples may be historical device node samples. The nodes may be individual electronic product terminals in a cluster of electronic product terminals. The security identification model is used to determine the terminal's security level based on the terminal's node information. The node information of the electronic product terminal may include: data transmission information, data acquisition information, data download information, information on data transmission violations, information on illegal acquisition of data within the terminal, data processing volume information, and so on.

[0072] In S306, historical samples within the predetermined time range are input into the strategic model and classified to obtain sample categories. A sample classification model can be trained using historical samples and then embedded into the strategic model. In actual use, the strategic model can assign category labels to samples by calling the sample classification model.

[0073] In S308 , historical samples that meet the target sample category and the target sample quantity are extracted through the strategic macro model to obtain the sample data.

[0074] In this embodiment, instructions for the target sample category and target sample quantity are input into the strategy big model. The strategy big model processes the historical samples input into the strategy big model in the previous step based on the input target sample category and target sample quantity, and extracts historical samples whose sample categories meet the target sample category and target sample quantity as sample data in this solution.

[0075] Fig. 4 is a flow chart showing a decision engine update method based on a policy big model according to another exemplary embodiment.

[0076] As shown in FIG4 , in S402 , a policy is selected.

[0077] In S404 , a sample is designated.

[0078] In S406, data is extracted.

[0079] In S408, observation parameters are analyzed.

[0080] In S410 , a test is customized.

[0081] In S412 , the scene is customized.

[0082] In S414, the case is distributed and executed.

[0083] In S416, the results are compared.

[0084] In S418, the output is reported.

[0085] In S420, manual acceptance is performed.

[0086] In S422, the rules are put online.

[0087] In an actual application, the business personnel can modify the policy package to which the policy rules belong. The business personnel can also set the sample time and sample size to be tested. The policy model will automatically bring out the observation parameters involved in this change. The observation parameters may include input parameters and output parameters, including the observation data required for the automated test report automatically generated by the policy model, as well as the data that needs to be observed manually.

[0088] In actual applications, you can also manually adjust input parameter values ​​to customize tests and cover more test scenarios. Using MySQL, cases can be distributed to multiple machines, and large policy models can be configured on each machine to execute rules simultaneously. Pre-execution distribution can be used to distribute large batches of cases across multiple servers, improving overall execution efficiency and reducing test backend execution time.

[0089] After a preset strategy is executed, the system's master strategy model saves the results and automatically compares them with those that require observation. This intelligently generates a test report summarizing changes in key indicators, helping business personnel quickly and efficiently execute rules. After submitting an execution case, the system automatically compares and generates a report on the results, reducing manual comparison time.

[0090] In actual applications, decision engines often contain a large number of policy packages, and each policy package contains many policy rules. According to the decision engine update method based on the policy big model of this application, when business personnel modify the rules in the policy package, the policy can be automatically executed to verify whether the modified policy execution result is in line with expectations. It can also compare the input and output parameters of the two versions, and automatically bring out the changed input or output parameters, thereby improving testing efficiency and facilitating R&D review. The method of this application uses a big model to assist in determining the performance changes of the decision engine due to changes in operating parameters, greatly improving test efficiency and accuracy, and accelerating the performance optimization cycle of the decision engine.

[0091] Those skilled in the art will appreciate that all or part of the steps implementing the above embodiments can be implemented as a computer program executed by a CPU. When executed by the CPU, the computer program performs the functions defined in the above method provided herein. The program can be stored in a computer-readable storage medium, such as a read-only memory, a magnetic disk, or an optical disk.

[0092] Furthermore, it should be noted that the aforementioned figures are merely illustrative of the processes included in the methods according to exemplary embodiments of the present application and are not intended to be limiting. It is readily understood that the processes illustrated in the aforementioned figures do not indicate or limit the temporal order of these processes. Furthermore, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0093] The following are device embodiments of the present application, which can be used to implement the method embodiments of the present application. For details not disclosed in the device embodiments of the present application, please refer to the method embodiments of the present application.

[0094] FIG5 is a block diagram illustrating a decision engine update device based on a policy big model, according to another exemplary embodiment. As shown in FIG5 , the decision engine update device 50 based on a policy big model includes: a data module 502, a parameter module 504, a sample module 506, a test module 508, an iteration module 510, and an update module 512.

[0095] The data module 502 is used to obtain the updated operating data packet to be tested and its corresponding initial operating data packet in the terminal decision engine; the data module 502 is also used to monitor multiple operating rules of the initial operating data packet in the terminal decision engine in real time; when the change of the initial operating data packet meets the modification policy, the updated operating data packet is generated according to the modified operating data packet.

[0096] The parameter module 504 is configured to generate an observation parameter set based on the differences between the updated operation data packet and the initial operation data packet based on the policy macromodel. The parameter module 504 is further configured to match the multiple operation rules of the updated operation data packet and the initial operation data packet using the policy macromodel, and to generate an operation rule set based on the unmatched operation rules. The parameter module 504 is further configured to match the multiple observation parameters of the updated operation data packet and the initial operation data packet using the policy macromodel, and to combine the unmatched observation parameters to generate the observation parameter set.

[0097] The sample module 506 is used to obtain the actual decision data corresponding to each operation scenario that matches the observation parameter set within a predetermined time through the strategy big model as sample data; the sample module 506 is also used to determine the predetermined time range, target sample category and target sample quantity according to each operation scenario corresponding to the initial operation data packet; based on the strategy big model, historical samples are screened according to the predetermined time range, the target sample category and the target sample quantity to extract the sample data.

[0098] The testing module 508 is configured to execute a preset policy on the updated operational data package using the sample data based on the policy macromodel. The testing module 508 is further configured to distribute the sample data to multiple servers; the multiple servers execute the preset policy on the updated operational data package in parallel using the sample data. The testing module 508 is further configured to match the operational data in the updated operational data package with the operational data corresponding to each operational scenario in the sample data using the policy macromodel; predict the predicted decision data when the updated operational data package is put into operation based on the matching results; the policy macromodel generates a policy execution result by comparing the predicted decision data with the initial decision data generated when the initial operational data package is executed.

[0099] The iteration module 510 is used to call the strategy big model to generate an iterative reference trend based on the deviation between the strategy execution result and the preset decision expectation obtained in each iteration when the strategy execution result does not meet the preset decision expectation; generate multiple reference rules through the iterative reference trend to iteratively re-execute the preset strategy on the updated operation data packet until the strategy execution result meets the preset decision expectation; the iteration module 510 is also used for the strategy big model to determine the contribution of different operation rules in the updated operation data packet to the change of the predicted decision data and the initial decision data based on the iterative reference trend; the strategy big model generates multiple reference rules based on the deviation between the strategy execution result and the preset decision expectation and the change contribution corresponding to the different operation rules; and correct the updated operation data packet through multiple reference rules.

[0100] The updating module 512 is configured to update the terminal decision engine through the iterated updating operation data packet.

[0101] According to the decision engine updating device based on the policy big model of the present application, an updated operation data packet to be tested and its corresponding initial operation data packet are obtained in the terminal decision engine; an observation parameter set is generated based on the policy big model for the difference between the updated operation data packet and the initial operation data packet; actual decision data corresponding to each operation scenario that matches the observation parameter set within a predetermined time is obtained as sample data; a preset policy is executed on the updated operation data packet through the sample data; when the policy execution result does not meet the preset decision expectation, the policy big model is called to generate an iterative reference trend based on the deviation between the policy execution result obtained in each iteration and the preset decision expectation; multiple reference rules are generated through the iterative reference trend to iteratively re-execute the preset policy on the updated operation data packet until the policy execution result meets the preset decision expectation; by updating the terminal decision engine through the updated operation data packet after iteration, the policy in the decision engine can be automatically tested, a test report can be intelligently produced, and changes in key indicators can be summarized. The big model can be used to assist in determining the impact of changes in operation parameters on the performance of the decision engine, greatly improving test efficiency and accuracy, and accelerating the performance optimization cycle of the decision engine.

[0102] As shown in FIG6 , an embodiment of the present application provides an electronic device, including a processor 610 , a communication interface 620 , a memory 630 , and a communication bus 640 , wherein the processor 610 , the communication interface 620 , and the memory 630 communicate with each other via the communication bus 640 ;

[0103] Memory 630, for storing computer programs;

[0104] The processor 610 is configured to implement the decision engine update method based on the policy big model according to any of the above embodiments when executing the program stored in the memory 630 .

[0105] The communication interface 620 is used for communication between the electronic device and other devices.

[0106] The memory 630 may include a random access memory 630 (RAM) or a non-volatile memory 630, such as at least one disk storage 630. Alternatively, the memory 630 may be at least one storage device located away from the processor 610.

[0107] The embodiment of the present application provides a computer-readable storage medium, which stores one or more programs, and the one or more programs can be executed by one or more processors to implement the decision engine update method based on the policy big model of any of the above embodiments. For example, an updated operation data packet to be tested and its corresponding initial operation data packet are obtained in the terminal decision engine; based on the policy big model, an observation parameter set is generated for the difference between the updated operation data packet and the initial operation data packet; actual decision data corresponding to each operation scenario that matches the observation parameter set within a predetermined time is obtained as sample data; a preset policy is executed on the updated operation data packet through the sample data; when the policy execution result does not meet the preset decision expectation, the policy big model is called to generate an iterative reference trend based on the deviation between the policy execution result obtained in each iteration and the preset decision expectation; multiple reference rules are generated through the iterative reference trend to iteratively re-execute the preset policy on the updated operation data packet until the policy execution result meets the preset decision expectation; and the terminal decision engine is updated through the iterative updated operation data packet.

[0108] While the exemplary embodiments of the present application have been specifically illustrated and described above, it should be understood that the present application is not limited to the detailed structures, configurations, or implementations described herein; rather, the present application is intended to encompass various modifications and equivalent configurations within the spirit and scope of the appended claims.

Claims

1. A decision engine update method based on a big policy model, characterized in that: include: Obtaining the updated running data packet to be tested and its corresponding initial running data packet in the terminal decision engine; Based on the policy big model, generating an observation parameter set for the difference between the updated operation data packet and the initial operation data packet; Acquire actual decision data corresponding to each operation scenario matching the observation parameter set within a predetermined time as sample data; Executing a preset strategy on the update operation data packet through the sample data; When the strategy execution result does not meet the preset decision expectation, the strategy big model is called to generate an iterative reference trend according to the deviation between the strategy execution result obtained in each iteration and the preset decision expectation; Generate multiple reference rules through the iterative reference trend to iteratively re-execute the preset strategy on the updated running data packet until the strategy execution result meets the preset decision expectation; The terminal decision engine is updated through the iterative update operation data packet.

2. The method according to claim 1, characterized in that Executing a preset strategy on the update operation data packet through the sample data includes: The strategy big model matches the operation data in the updated operation data packet with the operation data corresponding to each operation scenario in the sample data; Predicting the prediction decision data when the update operation data package is online and running according to the matching result; The strategy big model generates a strategy execution result by comparing the predicted decision data with the initial decision data generated when the initial running data packet is running.

3. The method according to claim 2, characterized in that The strategy big model generates a strategy execution result by comparing the predicted decision data with the initial decision data generated when the initial operation data packet is running, including: The strategy big model generates a variation comparison curve according to the difference between the predicted decision data and the initial decision data; The difference in the running rules of the observed parameter set and the change amount comparison curve are matched to generate the strategy execution result.

4. The method according to claim 3, characterized in that: Generating a plurality of reference rules through the iterative reference trend to iterate the update operation data packet includes: The strategy big model determines the contribution of different operation rules in the updated operation data packet to the change of the predicted decision data and the initial decision data according to the iterative reference trend; The strategy big model generates a plurality of reference rules based on the deviation between the strategy execution result and the preset decision expectation and the change contribution corresponding to the different operation rules; The update operation data packet is modified by using a plurality of the reference rules.

5. The method according to claim 1, characterized in that Obtain the updated running data packet to be tested and its corresponding initial running data packet in the terminal decision engine, including: Real-time monitoring of multiple operation rules of the initial operation data packet in the terminal decision engine; When the change of the initial operation data packet satisfies the modification policy, the updated operation data packet is generated according to the modified operation data packet.

6. The method according to claim 1, characterized in that Generating an observation parameter set according to the difference between the updated operation data packet and the initial operation data packet, including: The multiple observation parameters of the updated operation data packet and the initial operation data packet are matched through the strategy big model, and the observation parameters that are not successfully matched are combined to generate the observation parameter set.

7. The method according to claim 1, characterized in that Generating an observation parameter set according to the difference between the updated operation data packet and the initial operation data packet, including: The multiple operation rules of the updated operation data packet and the initial operation data packet are matched through the strategy big model, and the operation rules that are not successfully matched are combined to generate an observation parameter set.

8. The method according to claim 1, characterized in that Acquiring actual decision data corresponding to each operation scenario matching the observation parameter set within a predetermined time as sample data, including: Determine a predetermined time range, a target sample category, and a target sample quantity according to each operation scenario corresponding to the initial operation data packet; Based on the strategic macro model, historical samples are screened according to the predetermined time range, the target sample category and the target sample quantity to extract the sample data.

9. The method according to claim 8, characterized in that Based on the strategy macro model, historical samples are screened according to the predetermined time range, the target sample category and the target sample quantity to extract the sample data, including: Inputting historical samples within the predetermined time range into the strategy macro model, and classifying the historical samples to obtain sample categories; The sample data is obtained by extracting historical samples that meet the target sample category and the target sample quantity through the strategy macro model.

10. The method according to claim 1, characterized in that Executing a preset strategy on the update operation data packet through the sample data includes: Distributing the sample data to multiple servers; The multiple servers execute preset strategies on the update operation data packet in parallel through the sample data.

11. A decision engine updating device based on a big policy model, characterized in that: include: A data module, used to obtain an updated operation data packet to be tested and its corresponding initial operation data packet in the terminal decision engine; A parameter module, for generating an observation parameter set for the difference between the updated operation data packet and the initial operation data packet based on the strategy big model; A sample module, used to obtain actual decision data corresponding to each operation scenario matching the observation parameter set within a predetermined time through the strategy big model as sample data; A testing module, used for executing a preset strategy on the update operation data packet through the sample data based on the strategy macro model; An iteration module, used for calling the strategy big model to generate an iteration reference trend according to the deviation between the strategy execution result obtained in each iteration and the preset decision expectation when the strategy execution result does not meet the preset decision expectation; Generate multiple reference rules through the iterative reference trend to iteratively re-execute the preset strategy on the updated running data packet until the strategy execution result meets the preset decision expectation; An updating module is used to update the terminal decision engine through the iterated update operation data packet.

12. An electronic device, characterized in that: include: one or more processors; A storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 10.

13. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 10 is implemented.

Citation Information

Patent Citations

  • Service decision logic updating method based on decision engine and model platform

    CN110929879A

  • Strategy iteration method, device and system based on strategy robot

    CN115599835A

  • Main and bypass switching method and device for decision engine upgrading, equipment and storage medium

    CN115981744A

  • Decision engine updating method and device based on large strategy model

    CN116880867A

  • Computer system decision engine

    US20180365578A1

Cited By

  • Anti-money laundering risk rule generation method and system based on multi-modal large language model

    CN120598567A

  • Method and system for generating open source component repair opinions and storage medium

    CN120611388A

  • Method and system for carrying out version switching on risk control rule, and electronic equipment

    CN121658063A

  • Optimization method, device and equipment for agent based on large language model

    CN121683853A