Causal relationship determination method based on large language model and related equipment
By updating and correcting the causal relationship in business scenarios on large language models and generating more accurate causal data, the problem of inaccurate screening of modeling features in the existing technology is solved, and the performance and modeling efficiency of the business model are improved.
Patent Information
- Application Number
- CN202311370497.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-10-20
- Publication Date
- 2025-04-22
AI Technical Summary
The prior art lacks reliable methods for screening modeling features, resulting in poor operational results of business models.
By obtaining business information of the target business scenario, determining the causal relationship between feature data, and using a large language model to update and correct these causal relationships to generate more accurate causal data to build a business model.
It improves the reliability and accuracy of modeling characteristics, improves the performance of business models, helps modelers understand business logic faster, and lowers the threshold for business foundation.
Smart Images
Figure CN119862948A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the technical field of computers, and in particular to a causal relationship determination method based on a large language model and related equipment. Background Art
[0002] "Large language model" refers to a large-scale language model (LLM), which is a language model with a large parameter scale. It is designed to understand and generate human language. It is trained with a large amount of text data and can perform a wide range of tasks including text summarization, translation, sentiment analysis, etc. The rapid development of large model technology has brought new opportunities for economic and social development. How to realize the application of large language models is an issue that needs to be studied urgently.
[0003] At present, when modelers build business models for specific business scenarios, they tend to mix some features that do not match the requirements of the business model into the model architecture due to insufficient business knowledge and overly complex business scenarios. This makes the final output business model have poor operating results.
[0004] That is to say, the related art does not provide a method for screening modeling features with high reliability. Summary of the invention
[0005] The purpose of the present disclosure is to provide a causal relationship determination method based on a large language model and related equipment, so as to solve the technical problem that the related art does not provide a method for modeling feature screening with high reliability.
[0006] In a first aspect, the present disclosure provides a causal relationship determination method based on a large language model, the method comprising:
[0007] Step S1: Obtain business information of the target business scenario;
[0008] Step S2: determining a causal relationship between a plurality of characteristic data in the business information to obtain a plurality of first causal data, wherein each first causal data indicates a causal relationship between two characteristic data in the business information;
[0009] Step S3: The multiple first causal data are updated through the large language model to obtain multiple second causal data, wherein each second causal data indicates a causal relationship between two feature data in the business information, and the multiple second causal data are used to construct a business model of the target business scenario, and the multiple first causal data have a first accuracy, and the multiple second causal data have a second accuracy, and the first accuracy is lower than the second accuracy.
[0010] In some embodiments, step S3 includes:
[0011] Step S31: performing causal relationship detection on the plurality of first causal data by using the large language model to obtain a plurality of identification results;
[0012] Step S32: According to the multiple identification results, the multiple first causal data are respectively corrected to obtain multiple second causal data.
[0013] In some embodiments, the identification result indicates the causal relationship detection type to which the feature data pairs in the multiple first causal data belong, and the causal relationship detection type includes at least one of causal relationship error, causal direction error, unknown causal relationship, missing causal relationship and correct causal relationship.
[0014] In one embodiment, step S31 includes:
[0015] Step S311: constructing a first prompt word according to the plurality of characteristic data in the business information and the plurality of first causal data;
[0016] Step S312: performing causal relationship detection on the first prompt word according to the large language model to obtain the multiple identification results.
[0017] In some embodiments, step S32 includes:
[0018] Step S321: determining the operation type corresponding to each feature data pair in the plurality of first causal data according to the causal relationship detection type to which each feature data pair in the plurality of first causal data belongs, wherein the feature data pair includes cause feature data and result feature data;
[0019] Step S322: According to the operation type corresponding to each characteristic data pair in the plurality of first causal data, respectively modify the causal relationship of each characteristic data pair in the plurality of first causal data;
[0020] The operation types include a delete operation for deleting a causal relationship to which a feature data pair belongs, a modify direction operation for updating a causal direction of a causal relationship to which a feature data pair belongs, and a add operation for adding a causal relationship to which a feature data pair belongs.
[0021] In some embodiments, the method further comprises:
[0022] Step S323: displaying the characteristic data pairs corresponding to the plurality of identification results, and determining at least one target type from the operation types corresponding to each characteristic data pair in the plurality of first causal data;
[0023] Step S324: batch-correcting the causal relationships of the feature data pairs in the plurality of first causal data associated with the same target type.
[0024] In some embodiments, the method further comprises:
[0025] Step S4: performing causal interpretation on the plurality of second causal data respectively according to the large language model to obtain feature description sub-data and causal relationship interpretation sub-data of each of the second causal data.
[0026] In some embodiments, the method further comprises:
[0027] Step S51: Acquire a target causal relationship corresponding to a target feature in the plurality of second causal data and / or the plurality of first causal data;
[0028] Step S52: constructing a second prompt word according to the target causal relationship and a plurality of characteristic data in the business information;
[0029] Step S53: performing a causal relationship search on the second prompt word according to the large language model to obtain a key feature, wherein there is a causal relationship between the target feature and the key feature, and the key feature is feature data other than the multiple feature data in the business information;
[0030] Step S54: Based on the key features, the business information is updated.
[0031] In a second aspect, the present disclosure provides a causal relationship determination device based on a large language model, the device comprising:
[0032] An information acquisition module is used to acquire business information of the target business scenario;
[0033] A causal discovery module, used to determine the causal relationship between the plurality of characteristic data in the business information, and obtain a plurality of first causal data, wherein each first causal data indicates the causal relationship between two characteristic data in the business information;
[0034] A causal update module is used to update the multiple first causal data through the large language model to obtain multiple second causal data, wherein each second causal data indicates a causal relationship between two feature data in the business information, and the multiple second causal data are used to construct a business model of the target business scenario, and the multiple first causal data have a first accuracy, and the multiple second causal data have a second accuracy, and the first accuracy is lower than the second accuracy.
[0035] In a third aspect, the present disclosure provides an electronic device comprising: a processor, a memory, and a program stored in the memory and executable on the processor, wherein when the program is executed by the processor, the steps of the causal relationship determination method based on a large language model as described in the first aspect above are implemented.
[0036] In a fourth aspect, the present disclosure provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the causal relationship determination method based on a large language model as described in the first aspect above are implemented.
[0037] In the disclosed embodiment, after obtaining and determining the causal relationship between multiple feature data in the business information of the target business scenario, the causal relationship between the multiple feature data is updated through the large language model to tap the large language model's ability to identify and learn causal relationships, and obtain more accurate multiple second causal data. This can not only provide modeling features with higher reliability for modelers to choose from, assist modelers in quickly screening features that have significant causal relationships with business needs, and thereby improve the model performance of the business model ultimately constructed by the modelers; it can also help modelers understand the business logic of the target business scenario more intuitively and quickly, lower the modeler's business basic threshold, shorten the modeler's business learning time for the target business scenario, and improve the modeling efficiency of the modeler for the target business scenario. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] Figure 1 is a schematic diagram of a causal relationship determination method based on a large language model provided by an embodiment of the present disclosure;
[0039] Figure 2 is a schematic diagram of another causal relationship determination method based on a large language model provided by an embodiment of the present disclosure;
[0040] Figure 3 is a schematic diagram of a causal relationship determination device based on a large language model provided by an embodiment of the present disclosure;
[0041] Figure 4 It is a schematic diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION
[0042] The following will be combined with the drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, rather than all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.
[0043] The following is an explanation of the various names involved in the embodiments of the present disclosure.
[0044] Causal discovery: Discovering causal relationships between variables using observed data and little or no intervention data to determine the relationship between causal and outcome variables.
[0045] Characteristic data: refers to business elements (such as cost, product quality, etc.) that may have a positive / negative impact on business objectives (such as turnover, operating profit, etc.) in the target business scenario.
[0046] The “large language model” mentioned in the present invention refers to a large language model (LLM), which is a language model with a large parameter scale. It is designed to understand and generate human language. It is trained with a large amount of text data and can perform a wide range of tasks including text summarization, translation, sentiment analysis, etc.
[0047] Accuracy: The degree to which the content indicated by the data (such as the first causal data or the second causal data described in the present disclosure) is close to the true content, which can also be understood as the credibility or confidence level of the data.
[0048] Please refer to Figure 1 The present disclosure provides a causal relationship determination method based on a large language model, comprising:
[0049] Step S1: Obtain business information of the target business scenario.
[0050] The target business scenario may be any business scenario applicable to modeling and analysis, such as production forecasting scenarios, weather forecasting scenarios, passenger flow forecasting scenarios, customer churn warning scenarios, and other business scenarios.
[0051] Each business scenario has at least one target feature. For example, the business demand corresponding to the production forecast scenario is the product qualification rate and / or product production rate, and the business demand corresponding to the weather forecast scenario is the credibility of the weather forecast results.
[0052] Among them, the above-mentioned business information can be understood as a set of business data that affects the business needs of the target business scenario. For example: when the target business scenario is a production prediction scenario and the corresponding business need is the product qualification rate, the above-mentioned business information can include data indicating the aging degree of workshop equipment, data indicating the workshop production environment, etc.; for example, for a customer churn warning scenario, the corresponding business information may be a data table, which includes multiple fields, such as "gender", "transaction frequency" and "purchasing power level", etc., which are field information authorized by the user or obtained from an open source data set, and the target feature corresponding to the customer churn warning scenario may be the "customer churn" field.
[0053] Exemplarily, the business information can be obtained locally by uploading files; or the business information provided by a data source can be remotely obtained by opening a remote data interface. For example, the business information can be text data, data tables, or image data, etc., which is not limited in the embodiments of the present invention.
[0054] Step S2: determining the causal relationship between the plurality of characteristic data in the business information to obtain a plurality of first causal data.
[0055] Each first causal data indicates a causal relationship between two characteristic data in the business information.
[0056] As described above, after loading the data set (i.e., business information), the causal discovery algorithm can be run to execute the action of step S2 and determine the causal relationship between multiple feature data in the business information (i.e., multiple fields in the data table), thereby obtaining the multiple first causal data.
[0057] The causal discovery algorithm may be determined by the user based on actual needs or may be pre-set. For example, the causal discovery algorithm may include but is not limited to at least one of PC, GES, LiNGAM, IC, and ANM algorithms.
[0058] Furthermore, after obtaining the plurality of first causal data, the plurality of first causal data may be stored in a relational database, a graph database, a cache or a file system to facilitate subsequent application of the plurality of first causal data.
[0059] For example, taking a data set for a customer churn warning scenario as an example, when the data set contains fields such as gender, age, annual income, purchasing power level, transaction frequency, number of transactions, average transaction amount, income level, occupation, customer churn, etc. (i.e., multiple feature data), multiple first causal data can be expressed in the form of "cause feature data -> result feature data" as follows:
[0060] "Gender->Churn",
[0061] "Purchasing Power Level->Annual Income",
[0062] "Transaction frequency->Average transaction amount",
[0063] "Transaction Frequency->Customer Churn",
[0064] "Number of transactions->Customer churn",
[0065] "Revenue Level -> Customer Churn",
[0066] "income level->gender",
[0067] "Age->Churn",
[0068] "Career->Customer Churn".
[0069] It should be noted that, for a plurality of first causal data, the feature data corresponding to any two first causal data are at least partially different, for example:
[0070] "income level->churn" and "income level->gender";
[0071] or,
[0072] "Transaction frequency -> average transaction amount" and "Revenue level -> customer churn".
[0073] Step S3: updating the plurality of first causal data by using the large language model to obtain a plurality of second causal data.
[0074] Among them, each second causal data indicates a causal relationship between two feature data in the business information, the multiple second causal data are used to construct a business model of the target business scenario, and the multiple first causal data have a first accuracy, and the multiple second causal data have a second accuracy, and the first accuracy is lower than the second accuracy.
[0075] It should be noted that in the present disclosure, a feature set consisting of a plurality of feature data corresponding to a plurality of first causal data is a first set, and a feature set consisting of a plurality of feature data corresponding to a plurality of second causal data is a second set. The first set and the second set may be the same set. For example, in the process of updating a plurality of first causal data based on a large language model, the addition or name modification of the feature data constituting the first causal data is not involved.
[0076] The first set may also be a proper subset of the second set, for example, in the process of updating the plurality of first causal data based on the large language model, new feature data that is not involved in or concerned in the causal discovery process is mined;
[0077] The first set and the second set may also be two different sets with an intersection. For example, in the process of updating multiple first causal data based on the large language model, new feature data that is not involved or paid attention to in the causal discovery process is mined, and the large language model also updates the feature names of the feature data used in the causal discovery process.
[0078] In the disclosed embodiment, after obtaining and determining the causal relationship between multiple feature data in the business information of the target business scenario, the causal relationship between the multiple feature data is updated through the large language model to tap the large language model's ability to identify and learn causal relationships, and obtain more accurate multiple second causal data. This can not only provide modeling features with higher reliability for modelers to choose from, assist modelers in quickly screening features that have significant causal relationships with business needs, and thereby improve the model performance of the business model ultimately constructed by the modelers; it can also help modelers understand the business logic of the target business scenario more intuitively and quickly, lower the modeler's business basic threshold, shorten the modeler's business learning time for the target business scenario, and improve the modeling efficiency of the modeler for the target business scenario.
[0079] Exemplarily, the large language model can be a third-party open source large language model, such as Llama, SAIL, Guanaco, MPT, etc.; it can also be a self-developed large language model. The large language model is a language model with a large parameter scale, which is designed to understand and generate human language. It is trained with a large amount of text data and can perform a wide range of tasks including text summarization, translation, sentiment analysis, etc.
[0080] It should be emphasized that in the embodiment of the present invention, the coding settings of steps S1, S2, S3, etc. are for the convenience of explaining the process of implementing the present invention, and are not used to limit the execution order and execution steps of the method in the embodiment of the present invention. According to different actual conditions, the execution order, increase, and reduction of the steps can be changed.
[0081] In some embodiments, step S3 includes:
[0082] Step S31: performing causal relationship detection on the plurality of first causal data by using the large language model to obtain a plurality of identification results;
[0083] Step S32: According to the multiple identification results, the multiple first causal data are respectively corrected to obtain multiple second causal data.
[0084] In some embodiments, the identification result indicates the causal relationship detection type to which the feature data pairs in the multiple first causal data belong, and the causal relationship detection type includes at least one of causal relationship error, causal direction error, unknown causal relationship, missing causal relationship and correct causal relationship.
[0085] In the above embodiment, by setting causal relationship detection types such as causal relationship error, causal direction error, unknown causal relationship and correct causal relationship, and using a large language model to further analyze the multiple first causal data output by the causal discovery algorithm, in order to identify the causal relationship detection type to which the feature data in the multiple first causal data belong, the user is assisted in correcting the multiple first causal data to obtain more accurate multiple second causal data.
[0086] Among them, causal relationship errors (also called false causality) refer to a type of causal relationship that is obviously inconsistent with reality, for example, "age->gender" is very irrational;
[0087] Causal direction error refers to a type of causal relationship in which there is a reasonable causal relationship between two feature data, but the causal direction is wrong. For example, "purchasing power level->annual income" has a strong causal relationship, but the correct causal direction should be "annual income->purchasing power level".
[0088] Unknown causal relationship (also called uncertain causal relationship) refers to a type of causal relationship where it is impossible to determine whether there is a reasonable causal relationship between two feature data, and / or whether the causal direction between the two feature data is reasonable. For example, "occupation->whether customers have churned" can neither be explained as reasonable nor as unreasonable.
[0089] Correct causal relationship (also called reasonable causal relationship) refers to a type of causal relationship in which there is a reasonable causal relationship between two feature data and the causal direction is reasonable.
[0090] Missing causality (also known as omitted causality) means that although there is a causal relationship between two feature data, it is not reflected in multiple first causal data, that is, a type of causal relationship that has not been identified by the causal discovery algorithm.
[0091] In some embodiments, the identification result may also indicate the identification logic of the feature data for the causal relationship detection type to which it belongs (which can also be understood as an explanation of some causal relationship detection types). For example, the explanation for the causal relationship error "age->gender" is that there should be no causal relationship between age and gender, but a correlation relationship.
[0092] In one embodiment, step S31 includes:
[0093] Step S311: constructing a first prompt word according to the plurality of characteristic data in the business information and the plurality of first causal data;
[0094] Step S312: performing causal relationship detection on the first prompt word according to the large language model to obtain the multiple identification results.
[0095] The prompt word (prompt) described in the present invention refers to model instructions and data in the form of text, tables, pictures, etc. input to the large language model, which is intended to prompt or guide the large language model to give an expected output.
[0096] Exemplarily, the prompt words may include at least one of the following:
[0097] Instructions: Specific tasks or instructions you want the model to perform;
[0098] Context: contains external information or additional contextual information to guide the large language model to respond better;
[0099] Input data: user input or questions;
[0100] Output instructions: Specify the type or format of the output.
[0101] The above-mentioned first prompt word can be understood as a prompt word used to guide the large language model to perform causal relationship detection on the multiple first causal data, and then output multiple identification results.
[0102] In some examples, the field information of a data table (i.e., multiple feature data) can be converted into data field information text, and the causal relationship obtained by causal discovery can be converted into causal relationship information text; then the data field information text and the causal relationship information text are filled into a prompt word template for causal relationship identification to form a prompt word instance for causal relationship identification (i.e., the first prompt word), and then the large language model interface is called to obtain the identification result of the large language model for the existing feature relationship corresponding to the prompt word instance, and finally the identification result is stored.
[0103] For example, take a customer churn warning data set as an example. The data set includes fields such as gender, age, annual income, purchasing power level, transaction frequency, number of transactions, average transaction amount, income level, occupation, customer churn, product type, recommendation algorithm, and promotion activities. The causal relationship between the fields is obtained through the causal discovery algorithm, and the field information and the obtained causal relationship are constructed into a prompt word instance (i.e., the first prompt word), and then the large language model interface is called. The identification result returned by the large language model can be:
[0104] {
[0105] "false causality": ["gender->churn"],
[0106] "Causal direction error": ["purchasing power level->annual income"],
[0107] "Reasonable causation":
[0108] "Transaction frequency->Average transaction amount",
[0109] "Transaction Frequency->Customer Churn",
[0110] "Number of transactions->Customer churn",
[0111] "Revenue Level -> Customer Churn",
[0112] "Income Level->Gender"
[0113] ],
[0114] "Causality not confirmed":
[0115] "Age->Churn",
[0116] "Career->Customer Churn"
[0117] ],
[0118] "Omitted Causation":
[0119] "Purchasing Power Level->Customer Churn",
[0120] "Average transaction amount->customer churn"
[0121] "Product Type->Customer Churn"
[0122] "Recommendation algorithm->Customer churn"
[0123] "Promotions->Customer Churn" ]
[0125] "Explanation":"1. Gender is unlikely to directly lead to customer churn, but is more likely to be the effect of hidden variables. 2. There should be no causal relationship between age and gender, but rather a correlation. 3. There is a reasonable causal relationship between transaction frequency, number of transactions, etc. and customer churn. 4. The relationship between age and occupation and customer churn needs further verification. 5. Key factors such as product type, recommendation algorithm, and promotional activities may be omitted.
[0126] "
[0127] }
[0128] In some embodiments, step S32 includes:
[0129] Step S321: determining the operation type corresponding to each feature data pair in the plurality of first causal data according to the causal relationship detection type to which each feature data pair in the plurality of first causal data belongs, wherein the feature data pair includes cause feature data and result feature data;
[0130] Step S322: According to the operation type corresponding to each characteristic data pair in the plurality of first causal data, respectively modify the causal relationship of each characteristic data pair in the plurality of first causal data;
[0131] The operation types include a delete operation for deleting a causal relationship to which a feature data pair belongs, a modify direction operation for updating a causal direction of a causal relationship to which a feature data pair belongs, and a add operation for adding a causal relationship to which a feature data pair belongs.
[0132] Exemplarily, the step S322 may include:
[0133] When the operation type indicates that the causal relationship of the causal relationship to which the feature data pair belongs is wrong, deleting the causal relationship to which the feature data pair belongs;
[0134] or,
[0135] In the case where the operation type indicates that the causal direction of the causal relationship to which the feature data pair belongs is wrong, reversing the causal relationship to which the feature data pair belongs;
[0136] or,
[0137] When the operation type indicates that the causal relationship to which the feature data pair belongs is a missing causal relationship, a causal relationship to which the feature data pair belongs is added.
[0138] It should be noted that, in actual applications, users can also select cause feature data and result feature data by themselves and form a new causal relationship, for example, forming the new causal relationship based on the omitted causal relationship prompted in the identification results returned by the large language model.
[0139] In some embodiments, the method further comprises:
[0140] Step S323: displaying the characteristic data pairs corresponding to the plurality of identification results, and determining at least one target type from the operation types corresponding to each characteristic data pair in the plurality of first causal data;
[0141] Step S324: batch-correcting the causal relationships of the feature data pairs in the plurality of first causal data associated with the same target type.
[0142] In this embodiment, feature data pairs corresponding to multiple identification results and operation types corresponding to the feature data pairs are displayed to allow the user to determine at least one target type therefrom, and then batch processing is performed to facilitate the user's correction operations on the causal relationships of multiple feature data pairs associated with the same target type, thereby simplifying the user's correction operations on the causal relationships and improving the user's experience. For example, the direction of all or part of the causal relationships that are identified as having incorrect causal directions is modified.
[0143] In addition, for feature relationship pairs with unknown causal relationships, users can retain, delete or modify them based on actual needs, and this disclosure does not limit this.
[0144] In some examples, the operation request parameters input by the user may include:
[0145] A) Operation type (add operation, delete operation, change direction operation, etc.)
[0146] B) Causal relationship form to be modified:
[0147] a) Cause characteristic data, effect characteristic data and causal direction;
[0148] b) or select the causal relationship list obtained by identifying and classifying the causal relationship;
[0149] By parsing the user's operation request parameters, the operation type and the causal relationship form to be modified are determined, and the stored causal relationship is updated (such as adding, deleting, modifying, etc.).
[0150] In some embodiments, the method further comprises:
[0151] Step S4: performing causal interpretation on the plurality of second causal data respectively according to the large language model to obtain feature description sub-data and causal relationship interpretation sub-data of each of the second causal data.
[0152] In this embodiment, a large language model is applied to perform causal interpretation on multiple second causal data to explore the large language model's ability to interpret causal relationships, and correspondingly obtain feature description sub-data and causal relationship interpretation sub-data for each of the second causal data, thereby assisting modelers to understand the business logic of the target business scenario more intuitively and quickly, lowering the modeler's business basic threshold, shortening the modeler's business learning time for the target business scenario, and improving the modeler's modeling efficiency for the target business scenario.
[0153] Among them, the feature description sub-data is used to expand the description of the feature data involved in the corresponding second causal data, and the causal relationship interpretation sub-data is used to explain the causal flow logic between the feature data involved in the corresponding second causal data.
[0154] In some examples, the causal relationship selected by the user (i.e., the second causal data) can be filled into the prompt word template for causal relationship interpretation to form a prompt word instance for causal relationship interpretation; then the large language model interface is called to obtain the interpretation result of the large language model for the prompt word instance and return it to the user.
[0155] For example, if the causal relationship selected by the user is set to "transaction frequency->customer churn", after it is encapsulated into a causal relationship interpretation prompt word instance, the return result obtained by calling the large language model interface may be:
[0156] {
[0157] "Causality": "Transaction frequency -> customer churn",
[0158] "Causality Identification": "Reasonable Causality",
[0159] "Discrimination explanation": "Higher transaction frequency indicates that customers are more active, which will reduce the possibility of customer churn, which is in line with intuition and business logic. As a customer transaction feature, transaction frequency precedes customer churn in time and satisfies the causal order. After controlling for other customer characteristics, the impact of transaction frequency on customer churn remains significant, satisfying independent causality.",
[0160] "Transaction frequency": "The number of transactions a customer makes within a certain period of time, reflecting the customer's trading activity",
[0161] "Customer churn": "the situation where customers stop trading and spending on the platform",
[0162] "Causal interpretation": "The higher the transaction frequency, the more active the customer is, and the less likely it is that customer churn will occur. Conversely, the lower the transaction frequency, the less active the customer is, which is more likely to lead to customer churn. There is a reasonable causal relationship between transaction frequency and customer churn."
[0163] }
[0164] In some embodiments, the method further comprises:
[0165] Step S51: Acquire a target causal relationship corresponding to a target feature in the plurality of second causal data and / or the plurality of first causal data;
[0166] Step S52: constructing a second prompt word according to the target causal relationship and a plurality of characteristic data in the business information;
[0167] Step S53: performing a causal relationship search on the second prompt word according to the large language model to obtain a key feature, wherein there is a causal relationship between the target feature and the key feature, and the key feature is feature data other than the multiple feature data in the business information;
[0168] Step S54: Based on the key features, the business information is updated.
[0169] The target feature is the core feature data that the target business scenario focuses on. For example, for a customer churn warning scenario, the corresponding business information may be a data table, which includes multiple fields, such as "gender", "transaction frequency" and "purchasing ability level", etc., which are authorized by the user or obtained from an open source data set. The core feature data that focuses on is the "customer churn" field.
[0170] In addition to using the large language model to analyze the existing feature list (referring to multiple feature data in business information), the large language model is also used to mine other feature data that are not in the existing feature list but have the potential to affect the target feature, so as to enrich the feature selection used by users for modeling and avoid missing key features due to insufficient understanding of the business.
[0171] Exemplarily, the field information of the data table (referring to multiple feature data in the business information) can be converted into data field information text, and the record information that has a causal relationship with the target variable (referring to the target feature) obtained through causal discovery and causal relationship correction functions can be converted into causal relationship information text; and the data field information text and causal relationship information text can be filled into the key causal relationship recommendation prompt word template to form a key causal relationship recommendation prompt word instance, and then the large language model interface is called to obtain the large language model's potential missing key factors (referring to key features) and the causal relationship between the missing key factors and other fields.
[0172] In some embodiments, the execution process of step S54 may be: adding the key features to the business information obtained in step S1 to update the business information; and after executing the action of step S54, the actions of step S2 and step S3 may be executed again on the updated business information to obtain more comprehensive causal relationship information related to the target features.
[0173] For example, take a customer churn warning data set as an example, which includes gender, age, annual income, purchasing power level, transaction frequency, number of transactions, average transaction amount, income level, occupation, customer churn and other fields; the causal relationship between feature data is obtained through causal discovery algorithm and causal relationship correction. The following example of the return result of calling the large language model interface after constructing the field information text and causal relationship text into a prompt word instance is as follows:
[0174] {
[0175] "Potentially missing key factors":[
[0176] "Deposit balances have fallen for three consecutive months",
[0177] "Trading frequency has declined for three consecutive months"
[0178] ],
[0179] "causation":[
[0180] "Age increases -> deposit balance decreases for three consecutive months",
[0181] "Income level declines -> deposit balances decline for three consecutive months",
[0182] "Occupational changes such as unemployment -> deposit balances decline for three consecutive months",
[0183] "Customer churn->Transaction frequency has decreased for 3 consecutive months",
[0184] "Average transaction amount decreased -> deposit balance decreased for three consecutive months",
[0185] "Decline in number of transactions->Decline in transaction frequency for three consecutive months",
[0186] "Purchasing power declines -> transaction frequency declines for three consecutive months",
[0187] "Age increases -> trading frequency decreases for 3 consecutive months",
[0188] "Income level decreased -> transaction frequency decreased for 3 consecutive months" ]
[0190] }
[0191] In some embodiments, the method further comprises:
[0192] Step S6: determining at least one key modeling feature according to the target feature and the plurality of the second causal data;
[0193] Step S7: constructing the business model according to the at least one key modeling feature.
[0194] In some embodiments, step S6 may also be: determining at least one modeling key feature according to the target feature, a plurality of the second causal data and the key feature.
[0195] The execution process of step S6 may be:
[0196] A causal relationship graph is formed based on the target feature, multiple second causal data, and the feature description sub-data and causal relationship interpretation sub-data of each second causal data, and the causal relationship graph is displayed to the user, thereby assisting the user to quickly determine at least one key modeling feature.
[0197] Through the above settings, when the modeler is not very familiar with the target business scenario, the knowledge capabilities of the large language model can be used to assist the modeler in deeply understanding the business relationship between the feature data, fill the knowledge gap of the modeler, and quickly improve the business level of the modeler. At the same time, it is convenient for users to select high-quality feature data that affects business needs, and ultimately improve the modeling efficiency and modeling effect for the target business scenario.
[0198] For ease of understanding, the following examples are provided:
[0199] like Figure 2 As shown, first access the data set of the business requirements of the associated target business scenario, then run the causal discovery algorithm selected by the user on the data set to obtain the causal relationship between different feature data of the data set, and form a causal graph (the causal graph at this time is the first causal graph) based on this, that is, a network graph formed by connecting multiple causal relationships in series, and then construct a prompt word instance corresponding to the causal relationship correction / causal relationship identification / causal relationship interpretation / key causal relationship recommendation function based on the causal graph, and call the large language model interface corresponding to the prompt word instance, so as to obtain a more accurate causal graph;
[0200] Among them, the causal relationship identification action and the causal relationship correction action can be performed in sequence to update the original first causal graph (the causal graph at this time is the second causal graph), and then based on the second causal graph, the key causal relationship recommendation action and the causal relationship correction action are performed in sequence to update the second causal graph (the causal graph at this time is the third causal graph); finally, based on the third causal graph, the causal relationship identification action, the causal relationship interpretation action and the causal relationship correction action are performed in sequence to obtain the final causal graph.
[0201] See also Figure 3 , Figure 3 is a causal relationship determination device based on a large language model provided by an embodiment of the present disclosure, such as Figure 3 As shown, the causal relationship determination device 300 based on the large language model includes:
[0202] An information acquisition module 301 is used to acquire business information of a target business scenario;
[0203] A causal discovery module 302 is used to determine the causal relationship between the plurality of characteristic data in the business information, and obtain a plurality of first causal data, wherein each first causal data indicates the causal relationship between two characteristic data in the business information;
[0204] The causal update module 303 is used to update the multiple first causal data through the large language model to obtain multiple second causal data, wherein each second causal data indicates the causal relationship between two feature data in the business information, and the multiple second causal data are used to construct a business model of the target business scenario, and the multiple first causal data have a first accuracy, and the multiple second causal data have a second accuracy, and the first accuracy is lower than the second accuracy.
[0205] In some embodiments, the data acquisition module 303 includes:
[0206] a causal relationship detection unit, configured to perform causal relationship detection on the plurality of first causal data by using the large language model to obtain a plurality of identification results;
[0207] The correction unit corrects the plurality of first causal data respectively according to the plurality of identification results to obtain a plurality of the second causal data.
[0208] In some embodiments, the identification result indicates the causal relationship detection type to which the feature data pairs in the multiple first causal data belong, and the causal relationship detection type includes at least one of causal relationship error, causal direction error, unknown causal relationship, missing causal relationship and correct causal relationship.
[0209] In some embodiments, the causal relationship detection unit is specifically used to:
[0210] Constructing a first prompt word according to the plurality of characteristic data in the business information and the plurality of first causal data;
[0211] Perform causal relationship detection on the first prompt word according to the large language model to obtain the multiple identification results.
[0212] In some embodiments, the correction unit is specifically used to:
[0213] Determine, according to the causal relationship detection type to which each feature data pair in the plurality of first causal data belongs, the operation type corresponding to each feature data pair in the plurality of first causal data, wherein the feature data pair includes cause feature data and result feature data;
[0214] According to the operation type corresponding to each characteristic data pair in the plurality of first causal data, respectively modifying the causal relationship of each characteristic data pair in the plurality of first causal data;
[0215] The operation types include a delete operation for deleting a causal relationship to which a feature data pair belongs, a modify direction operation for updating a causal direction of a causal relationship to which a feature data pair belongs, and a add operation for adding a causal relationship to which a feature data pair belongs.
[0216] In some embodiments, the correction unit is further configured to:
[0217] Displaying the characteristic data pairs corresponding to the plurality of identification results, and determining at least one target type among the operation types corresponding to each characteristic data pair in the plurality of first causal data;
[0218] The causal relationships of the feature data pairs in the plurality of first causal data associated with the same target type are corrected in batches.
[0219] In some embodiments, the apparatus 300 further includes:
[0220] The causal interpretation module is used to perform causal interpretation on the plurality of second causal data respectively according to the large language model to obtain feature description sub-data and causal relationship interpretation sub-data of each second causal data.
[0221] In some embodiments, the apparatus 300 further includes a causal relationship search module, wherein the causal relationship search module is specifically configured to:
[0222] Acquire a target causal relationship corresponding to a target feature in the plurality of second causal data and / or the plurality of first causal data;
[0223] constructing a second prompt word according to the target causal relationship and a plurality of characteristic data in the business information;
[0224] A causal relationship search is performed on the second prompt word according to the large language model to obtain a key feature, there is a causal relationship between the target feature and the key feature, and the key feature is feature data other than the multiple feature data in the business information.
[0225] The causal relationship determination device 300 based on a large language model provided in the embodiment of the present disclosure can implement each process in the above-mentioned causal relationship determination method embodiment based on a large language model. To avoid repetition, they will not be described here.
[0226] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device and a readable storage medium.
[0227] Figure 4A schematic block diagram of an example electronic device 400 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.
[0228] like Figure 4 As shown, the device 400 includes a computing unit 401, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 402 or a computer program loaded from a storage unit 408 to a random access memory (RAM) 403. In the RAM 403, various programs and data required for the operation of the device 400 can also be stored. The computing unit 401, the ROM 402, and the RAM 403 are connected to each other via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.
[0229] A number of components in the device 400 are connected to the I / O interface 405, including: an input unit 406, such as a keyboard, a mouse, etc.; an output unit 407, such as various types of displays, speakers, etc.; a storage unit 408, such as a disk, an optical disk, etc.; and a communication unit 409, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 409 allows the device 400 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.
[0230] The computing unit 401 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 401 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSP), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 401 performs the various methods and processes described above, such as a causal relationship determination method based on a large language model. For example, in some embodiments, the causal relationship determination method based on a large language model may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 408. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 400 via the ROM 402 and / or the communication unit 409. When the computer program is loaded into the RAM 403 and executed by the computing unit 401, one or more steps of the causal relationship determination method based on the large language model described above may be executed. Alternatively, in other embodiments, the computing unit 401 may be configured to execute the causal relationship determination method based on a large language model in any other appropriate manner (for example, by means of firmware).
[0231] Various embodiments of the systems and techniques described above herein may be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0232] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.
[0233] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0234] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).
[0235] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.
[0236] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.
[0237] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.
[0238] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.
Claims
1. A causal relationship determination method based on a large language model, characterized in that: The method comprises: Obtain business information of the target business scenario; Determine a causal relationship between a plurality of characteristic data in the business information to obtain a plurality of first causal data, wherein each first causal data indicates a causal relationship between two characteristic data in the business information; The multiple first causal data are updated through the large language model to obtain multiple second causal data, wherein each second causal data indicates a causal relationship between two feature data in the business information, and the multiple second causal data are used to construct a business model of the target business scenario, and the multiple first causal data have a first accuracy, and the multiple second causal data have a second accuracy, and the first accuracy is lower than the second accuracy.
2. The method according to claim 1, characterized in that The updating of the plurality of first causal data by using the large language model to obtain a plurality of second causal data includes: Performing causal relationship detection on the plurality of first causal data by using the large language model to obtain a plurality of identification results; According to the multiple identification results, the multiple first causal data are respectively corrected to obtain multiple second causal data.
3. The method according to claim 2, characterized in that The identification result indicates the causal relationship detection type to which the feature data pairs in the multiple first causal data belong, and the causal relationship detection type includes at least one of causal relationship error, causal direction error, unknown causal relationship, missing causal relationship and correct causal relationship.
4. The method according to claim 2, characterized in that The causal relationship detection is performed on the plurality of first causal data by the large language model to obtain a plurality of identification results, including: Constructing a first prompt word according to the plurality of characteristic data in the business information and the plurality of first causal data; Perform causal relationship detection on the first prompt word according to the large language model to obtain the multiple identification results.
5. The method according to any one of claims 2 to 4, characterized in that: The step of respectively modifying the plurality of first causal data according to the plurality of identification results comprises: Determine, according to the causal relationship detection type to which each feature data pair in the plurality of first causal data belongs, the operation type corresponding to each feature data pair in the plurality of first causal data, wherein the feature data pair includes cause feature data and result feature data; According to the operation type corresponding to each characteristic data pair in the plurality of first causal data, respectively modifying the causal relationship of each characteristic data pair in the plurality of first causal data; The operation types include a delete operation for deleting a causal relationship to which a feature data pair belongs, a modify direction operation for updating a causal direction of a causal relationship to which a feature data pair belongs, and a add operation for adding a causal relationship to which a feature data pair belongs.
6. The method according to claim 5, characterized in that After determining the operation type corresponding to each feature data pair in the plurality of first causal data according to the causal relationship detection type to which each feature data pair in the plurality of first causal data belongs, the method further includes: Displaying the characteristic data pairs corresponding to the plurality of identification results, and determining at least one target type among the operation types corresponding to each characteristic data pair in the plurality of first causal data; The causal relationships of the feature data pairs in the plurality of first causal data associated with the same target type are corrected in batches.
7. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: Causally interpreting the plurality of second causal data respectively according to the large language model to obtain feature description sub-data and causal relationship interpretation sub-data of each of the second causal data.
8. The method according to any one of claims 1 to 6, characterized in that: After the plurality of first causal data are updated by the large language model to obtain a plurality of second causal data, the method further includes: Acquire a target causal relationship corresponding to a target feature in the plurality of second causal data and / or the plurality of first causal data; constructing a second prompt word according to the target causal relationship and a plurality of characteristic data in the business information; Performing a causal relationship search on the second prompt word according to the large language model to obtain a key feature, wherein there is a causal relationship between the target feature and the key feature, and the key feature is feature data other than the multiple feature data in the business information; Based on the key feature, the business information is updated.
9. A causal relationship determination device based on a large language model, characterized in that: The device comprises: An information acquisition module is used to acquire business information of the target business scenario; A causal discovery module, used to determine the causal relationship between the plurality of characteristic data in the business information, and obtain a plurality of first causal data, wherein each first causal data indicates the causal relationship between two characteristic data in the business information; A causal update module is used to update the multiple first causal data through the large language model to obtain multiple second causal data, wherein each second causal data indicates a causal relationship between two feature data in the business information, and the multiple second causal data are used to construct a business model of the target business scenario, and the multiple first causal data have a first accuracy, and the multiple second causal data have a second accuracy, and the first accuracy is lower than the second accuracy.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the causal relationship determination method based on a large language model are performed as described in any one of claims 1 to 8.