Prediction system and control method thereof, and training method for prediction system

Abstract

The present invention relates to a prediction system, a control method thereof, and a training method for the prediction system. More particularly, the present invention relates to a prediction system capable of predicting a valid customer company (or valid customer) in a business-to-business (B2B) and / or business-to-consumer (B2C) business situation, a control method thereof, and a training method for the prediction system.

Description

Prediction system and its control method, and learning method of the prediction system

[0001] The present invention relates to a prediction system, a control method thereof, and a learning method for the prediction system. More specifically, the present invention relates to a prediction system capable of predicting effective customers (or effective clients) in B2B (Business-to-Business) and / or B2C (Business-to-Consumer) sales situations, a control method thereof, and a learning method for the prediction system.

[0002] Recent advancements in artificial intelligence have led to a surge in cases achieving outstanding results across various industries. In particular, advances in machine learning (ML) and deep learning technologies are contributing significantly to the development of AI models that learn patterns from massive amounts of data and support prediction and decision-making.

[0003] Meanwhile, the quantity and quality of training data directly impact the generalization performance of AI models. High-quality data enables models to make more accurate predictions, and integration and preprocessing of diverse data sources can maximize data utility.

[0004] On the other hand, data imbalance can reduce the predictive accuracy of AI models. Most datasets are imbalanced, with some classes significantly more or less represented than others. This can lead to AI models being biased toward classes that frequently appear. For example, in business data, positive outcomes (e.g., purchases) are often relatively rare compared to negative outcomes (e.g., no purchases). This can lead to data imbalance, negatively impacting the learning and prediction performance of AI models.

[0005] Accordingly, efficient learning of artificial intelligence models must be considered, and research on methods to address the imbalanced data problem is actively underway.

[0006] The present invention aims to solve the problem of imbalanced data and provide a prediction system and a control method thereof that can be universally utilized in various industrial fields, as well as a learning method of the prediction system.

[0007] More specifically, the present invention provides a prediction system capable of predicting valid customers and establishing an optimal sales strategy, a control method thereof, and a learning method of the prediction system.

[0008] Furthermore, the present invention provides a method for learning a prediction model that can predict effective customers by analyzing various customer data.

[0009] In order to solve the problem discussed above, a learning method of a prediction system, which is performed cooperatively by a memory and at least one processor according to the present invention, may include a step of specifying a learning data set, a step of configuring a plurality of different sub-data sets using the learning data set, a step of learning a prediction model to be learned for each of the different sub-data sets, a step of obtaining a plurality of learned prediction models each learned with the different sub-data sets based on the learning, a step of inputting input data to be predicted into each of the plurality of learned prediction models, a step of obtaining a plurality of prediction values ​​for the input data from each of the plurality of learned prediction models, and a step of specifying a final prediction value for the input data using the plurality of prediction values.

[0010] In an embodiment, the learning data set is configured to include a plurality of records having values ​​for a plurality of different categories, and in the step of configuring the plurality of different sub-data sets, the plurality of different sub-data sets can be configured based on a value corresponding to a specific category among the plurality of categories.

[0011] In an embodiment, the learning data set includes MQL (Marketing Qualified Lead) data configured to have values ​​for the plurality of different categories, wherein the specific category is a category indicating whether a customer's purchase conversion has occurred, and the value corresponding to the specific category may be configured to have a first value or a second value depending on whether a customer's purchase conversion has occurred.

[0012] In an embodiment, each of the plurality of learned prediction models may be configured to predict a value for the specific category.

[0013] In an embodiment, the method further comprises a step of performing feature engineering on the learning data set,

[0014] In the step of performing the above feature engineering, a derived category may be generated using at least some of the categories among the plurality of categories and values ​​corresponding to the at least some of the categories, and a value corresponding to the generated derived category may be specified.

[0015] In an embodiment, the learning data set may further include the derived category and values ​​corresponding to the derived category.

[0016] In an embodiment, the value corresponding to the specific category is configured to have the first value or the second value, and in the step of configuring the plurality of different sub-data sets, at least a portion of the plurality of records may be included in each of the plurality of different sub-data sets such that a composition ratio of a first record including the first value for the specific category and a second record including the second value for the specific category among the plurality of records satisfies a preset composition ratio criterion.

[0017] In an embodiment, the preset composition ratio criterion may relate to ensuring that, in each of the plurality of different sub-data sets, the number of first records including the first value for the specific category and the number of second records including the second value for the specific category have the same ratio.

[0018] In an embodiment, the number of the different sub-data sets may be determined based on the number of second records including the second value for the specific category and the number of first records including the first value for the specific category among the total number of records included in the learning data set.

[0019] In the embodiment, the method further includes a step of determining the number of different sub-data sets, wherein the determining step may be based on a value obtained by dividing the number of second records including the second value for the specific category by the number of first records including the first value for the specific category.

[0020] In an embodiment, each of the plurality of different sub-data sets may include all first records having the first value for the specific category among the records included in the learning data set, and may include a portion of second records having the second value for the specific category among the records included in the learning data set, in an amount equal to the number of first records included in each of the plurality of different sub-data sets.

[0021] In an embodiment, each of the plurality of different sub-data sets may include the same first record, and each of the plurality of different sub-data sets may include different second records.

[0022] In an embodiment, the learning target prediction model includes a plurality of prediction models based on a GBDT (Gradient Boosting Decision Tree) algorithm, and in the learning step, the plurality of prediction models are each trained for each of the different sub-data sets, and the plurality of learned prediction models trained for each of the different sub-data sets can be obtained.

[0023] In the embodiment, in the learning step, as a result of learning the plurality of prediction models for each of the different sub-data sets, the number of learned prediction models can be obtained as many as the number N of the different sub-data sets multiplied by the number M of the plurality of prediction models.

[0024] In an embodiment, the number of the plurality of prediction values ​​obtained from the plurality of learned prediction models may correspond to a value obtained by multiplying the number N of the different sub-data sets by the number M of the plurality of prediction models.

[0025] In the embodiment, in the step of specifying the final prediction value, soft voting based on the plurality of prediction values ​​may be performed to specify the final prediction value.

[0026] According to the present invention, a method for predicting a valid customer, which is performed by a cooperative memory and a processor, may include the steps of: receiving prediction target customer data that is a target of prediction from a user terminal; inputting the prediction target customer data into each of a plurality of prediction models, each of which is trained with different sub-data sets divided based on the purchasing customer data among a learning data set composed of purchasing customer data and non-purchasing customer data; obtaining, as outputs of each of the plurality of prediction models, a plurality of prediction values ​​indicating a probability that a customer corresponding to the prediction target customer data is a valid customer; using the plurality of prediction values, specifying a final prediction value for the prediction target customer data; and using the specified final prediction value, providing, to the user terminal, information on whether a customer corresponding to the prediction target customer data is a valid customer.

[0027] The system according to the present invention is a prediction system including a memory and at least one processor, wherein the memory and the processor cooperate to form a plurality of different sub-data sets using a learning data set, train a prediction model to be learned for each of the different sub-data sets, and based on the learning, obtain a plurality of learned prediction models each learned with the different sub-data sets, input data to be predicted into each of the plurality of learned prediction models, obtain a plurality of prediction values ​​for the input data from each of the plurality of learned prediction models, and use the plurality of prediction values ​​to specify a final prediction value for the input data.

[0028] A program according to the present invention is a program that is executed by one or more processes in an electronic device and can be stored in a computer-readable recording medium, wherein the program may include instructions for performing the steps of: specifying a learning data set; configuring a plurality of different sub-data sets using the learning data set; training a learning target prediction model for each of the different sub-data sets; obtaining a plurality of learned prediction models each trained with the different sub-data sets based on the learning; inputting input data to be a prediction target into each of the plurality of learned prediction models; obtaining a plurality of prediction values ​​for the input data from each of the plurality of learned prediction models; and specifying a final prediction value for the input data using the plurality of prediction values.

[0029] A learning method of a computerized prediction system according to the present invention, which comprises the steps of: specifying a learning data set configured to include a plurality of records having values ​​for a plurality of different categories; distinguishing each of the plurality of records included in the learning data set based on a value corresponding to a target category among the plurality of categories; configuring a plurality of different sub-data sets based on an index corresponding to each of the distinguished plurality of records; and learning a learning target prediction model for each of the plurality of different sub-data sets.

[0030] In an embodiment, the learning data set includes MQL (Marketing Qualified Lead) data configured to have values ​​for the plurality of different categories, and in the step of configuring the plurality of different sub-data sets, the plurality of different sub-data sets having a preset size can be configured based on an index corresponding to each of the plurality of records distinguished based on a value corresponding to the target category.

[0031] In the embodiment, in the step of distinguishing the plurality of records, in order to form the plurality of different sub-data sets, the plurality of records are distinguished based on a value that each of the plurality of records includes for the target category, the target category is a category indicating whether a customer's purchase conversion has occurred, and the value corresponding to the target category may be configured to have a first value or a second value depending on whether a customer's purchase conversion has occurred.

[0032] In the embodiment, in the step of distinguishing each of the plurality of records, a record including the first value for the target category among the plurality of records may be distinguished as a first record, and a record including the second value for the target category among the plurality of records may be distinguished as a second record.

[0033] In an embodiment, the index corresponding to each of the plurality of separated records may include a first index corresponding to the first record and a second index corresponding to the second record.

[0034] In an embodiment, the method further includes a step of storing the plurality of records classified based on a value corresponding to the target category and an index corresponding to each of the plurality of classified records in a predetermined storage, and in the step of configuring the plurality of different sub-data sets, the plurality of different sub-data sets having the predetermined size can be configured based on the index corresponding to each of the plurality of classified records stored in the predetermined storage.

[0035] In an embodiment, the plurality of separated records include a first record including a first value for the target category and a second record including a second value for the target category, and in the storing step, the first record and the first index corresponding to the first record and the second record and the second index corresponding to the second record are stored in the predetermined storage, respectively, and in the configuring step of the plurality of different sub-data sets, the plurality of different sub-data sets having the predetermined size can be configured based on the first index corresponding to the first record and the second index corresponding to the second record stored in the predetermined storage.

[0036] In the embodiment, in the step of configuring the plurality of different sub-data sets, at least some of the plurality of different records to be included in each of the plurality of different sub-data sets are specified based on an index corresponding to each of the plurality of different records, and the specified at least some of the records are included in each of the plurality of different sub-data sets to configure the plurality of different sub-data sets having the preset size.

[0037] In the embodiment, in the step of configuring the plurality of different sub-data sets, at least some of the plurality of separated records may be included in each of the plurality of different sub-data sets such that the composition ratio of the first record including the first value for the target category and the second record including the second value for the target category among the plurality of separated records satisfies a preset composition ratio criterion.

[0038] In an embodiment, the preset composition ratio criterion may relate to ensuring that, in each of the plurality of different sub-data sets, the number of first records including the first value for the target category and the number of second records including the second value for the target category have the same ratio.

[0039] In an embodiment, the number of the different sub-data sets may be determined based on the number of second records including the second value for the target category and the number of first records including the first value for the target category among the total number of the plurality of separated records, or may be determined based on the number of second indexes corresponding to the second records and the number of first indexes corresponding to the first records among the total number of the plurality of separated records. In an embodiment, the number of the different sub-data sets may further be included, and in the determining step, the number of second records including the second value for the target category may be determined based on a value obtained by dividing the number of first records including the first value for the target category by a value obtained by dividing the number of second indexes corresponding to the second records by the number of first indexes corresponding to the first record.

[0040] In an embodiment, the number of the different sub-data sets is determined based on the number of storage servers in which the different sub-data sets are to be stored, and when the number of the different sub-data sets is determined based on the number of storage servers, the different sub-data sets can be stored in the storage servers.

[0041] In an embodiment, each of the plurality of different sub-data sets may include all of the first records having the first value for the target category among the plurality of separated records, and may include a portion of the second records having the second value for the target category among the plurality of separated records, as many as the number of first records included in each of the plurality of different sub-data sets.

[0042] In an embodiment, each of the plurality of different sub-data sets may include the same first record, and each of the plurality of different sub-data sets may include different second records.

[0043] In an embodiment, based on the learning, the method may further include a step of obtaining a plurality of learned prediction models each learned with the plurality of different sub-data sets, a step of inputting input data to be predicted into each of the plurality of learned prediction models, a step of obtaining a plurality of predicted values ​​for the input data from each of the plurality of learned prediction models, and a step of specifying a final predicted value for the input data using the plurality of predicted values.

[0044] In the embodiment, in the learning step, the plurality of prediction models are each trained for each of the plurality of different sub-data sets, and the plurality of learned prediction models trained for each of the plurality of different sub-data sets can be obtained.

[0045] In the embodiment, in the step of specifying the final prediction value, soft voting may be performed based on the plurality of prediction values ​​obtained from the plurality of learned prediction models to specify the final prediction value.

[0046] A prediction system according to the present invention, comprising a memory configured to store executable instructions, and one or more processors configured to perform operations by executing one or more instructions, specifies a learning data set configured to include a plurality of records having values ​​for a plurality of different categories, and distinguishes the plurality of records included in the learning data set based on a value corresponding to a target category among the plurality of categories, and configures a plurality of different sub-data sets based on an index corresponding to each of the distinguished plurality of records, and trains a learning target prediction model for each of the plurality of different sub-data sets.

[0047] A program according to the present invention is a program that is executed by one or more processes in an electronic device and can be stored in a computer-readable recording medium, and may include instructions for performing the steps of: specifying a learning data set configured to include a plurality of records having values ​​for a plurality of different categories; distinguishing each of the plurality of records included in the learning data set based on a value corresponding to a target category among the plurality of categories; configuring a plurality of different sub-data sets based on an index corresponding to each of the plurality of distinguished records; and learning a learning target prediction model for each of the plurality of different sub-data sets.

[0048] A method for predicting a valid customer, which is performed cooperatively by a memory and at least one processor according to the present invention, may include the steps of: receiving customer data of a customer who is a target of purchase prediction for a specific product; inputting the customer data into at least one prediction model, which is related to the specific product and is trained with a plurality of sub-data sets generated using MQL (Marketing Qualified Lead) data; obtaining a probability value that the customer is a valid customer as an output of the prediction model; and providing a prediction result of whether the customer is a valid customer who will purchase the specific product using the probability value through a service page output to a user terminal.

[0049] In an embodiment, the service page provides product information of the specific product and customer information related to the customer, and the customer information may include purchase probability information of the customer as the prediction result.

[0050] In an embodiment, if there are multiple customers, the service page may include purchase probability information for the specific product for each of the multiple customers.

[0051] In an embodiment, the MQL (Marketing Qualified Lead) data related to the specific product may be composed of purchasing customer data and non-purchasing customer data for the specific product.

[0052] In an embodiment, the plurality of sub-data sets may be generated by dividing the MQL data based on the purchasing customer data.

[0053] In an embodiment, the plurality of sub-data may be configured based on the purchasing customer data so that the purchasing customer data and the non-purchasing customer data satisfy a preset composition ratio criterion.

[0054] In an embodiment, the preset composition ratio criterion may relate to ensuring that the number of purchasing customer data and the number of non-purchasing customer data included in each of the plurality of different sub-data sets have the same ratio.

[0055] In an embodiment, the number of the plurality of different sub-data sets may be determined based on the number of purchasing customer data and the number of non-purchasing customer data among the total number of records included in the MQL data.

[0056] In the embodiment, in the step of inputting the customer data, the customer data is input into each of the plurality of prediction models each trained with the MQL data, and in the step of obtaining the probability value, a plurality of prediction values ​​are obtained from each of the plurality of prediction models, and using the plurality of prediction values, a probability value that the customer is a valid customer can be specified.

[0057] In an embodiment, the plurality of prediction models are configured as prediction models based on a GBDT (Gradient Boosting Decision Tree) algorithm, and the plurality of prediction models can be learned for each different sub-data set.

[0058] In the embodiment, in the step of obtaining the probability value, soft voting can be performed based on the plurality of prediction values ​​obtained from each of the plurality of prediction models.

[0059] In an embodiment, the customer data may include at least one of the customer's name, account, contact information, email address, job title, location information, country of affiliation, and information related to the customer's company.

[0060] In an embodiment, the MQL data may be collected from a source based on at least one of a preset database, web crawling, API, and a pre-connected server.

[0061] The system according to the present invention is a valid customer prediction system including a memory and at least one processor, wherein the memory and the processor cooperate to receive customer data of a customer who is a target of purchase prediction for a specific product, input the customer data into at least one prediction model that is trained with a plurality of sub-data sets generated using MQL (Marketing Qualified Lead) data and is related to the specific product, and obtains a probability value that the customer is a valid customer as an output of the prediction model, and using the probability value, provides a prediction result on whether the customer is a valid customer who will purchase the specific product through a service page output to a user terminal.

[0062] A program according to the present invention is a program that is executed by one or more processes in an electronic device and can be stored in a computer-readable recording medium, and may include instructions for performing the steps of: receiving customer data of a customer who is a target of purchase prediction for a specific product; inputting the customer data into at least one prediction model that is trained with a plurality of sub-data sets generated using MQL (Marketing Qualified Lead) data and is related to the specific product; obtaining a probability value that the customer is a valid customer as an output of the prediction model; and providing a prediction result of whether the customer is a valid customer who will purchase the specific product using the probability value through a service page output to a user terminal.

[0063] As discussed above, the prediction system and its control method according to the present invention, and the learning method of the prediction system, can effectively respond to various business situations by providing a prediction model that has learned various business data.

[0064] Furthermore, the prediction system, its control method, and its learning method according to the present invention can address the imbalance problem of various business data, thereby providing learning from balanced learning data. This allows the prediction model to learn from balanced input data, thereby maintaining stable and high prediction performance even with diverse inputs during actual use.

[0065] Furthermore, the prediction system, its control method, and its learning method according to the present invention can solve the problem of imbalanced data in real-world usage environments by learning balanced business data. In other words, the present invention can enhance the generalization performance of the prediction model, enabling more accurate sales conversion predictions in real-world sales environments, and enabling the efficient allocation of sales resources and the development of optimized sales strategies.

[0066] Furthermore, the prediction system, its control method, and its learning method according to the present invention can provide an environment in which various customer data can be analyzed to develop customized sales strategies tailored to customer characteristics. This allows companies to flexibly respond to diverse customer types and market environments, thereby strengthening long-term customer relationships and significantly improving the performance of various businesses. Furthermore, companies can optimize performance in the global market and develop customized strategies tailored to the specific characteristics of each country. In other words, the present invention can provide crucial insights for corporate strategic decision-making and contribute to improving long-term business performance.

[0067] Furthermore, according to the prediction system and its control method, and the learning method of the prediction system according to the present invention, the entire data set can be equally divided into preset sizes and multiple different sub-data sets can be configured based on index information. This enables the present invention to perform various combination experiments without wasting storage space. In particular, the present invention can effectively alleviate data imbalance problems during learning by configuring sub-data sets to satisfy ratio conditions according to target classes. This can contribute to improving the accuracy and generalization performance of the prediction model.

[0068] Furthermore, the prediction system, its control method, and its learning method according to the present invention allow for the simultaneous consideration of data transmission efficiency and storage space utilization by uniformly configuring the entire data set to a preset size. This enables parallel learning of the prediction model and reduces the overall learning time.

[0069] Figure 1 is a conceptual diagram for explaining a prediction system and its control method according to the present invention.

[0070] Figures 2a and 2b are conceptual diagrams for explaining a prediction system according to the present invention.

[0071] Figure 3 is a flowchart for explaining a learning method of a prediction system according to the present invention.

[0072] Figures 4, 5, 6, 7, 8 and 9 are conceptual diagrams for explaining a learning method of a prediction system according to the present invention.

[0073] Figure 10 is a flowchart for explaining a valid customer prediction method of a prediction system according to the present invention.

[0074] Figure 11 is a conceptual diagram for explaining a valid customer prediction method of a prediction system according to the present invention.

[0075] Figures 12 and 13 are conceptual diagrams illustrating a prediction system according to another embodiment of the present invention.

[0076] FIG. 14a and FIG. 14b are flowcharts for explaining a learning method of a prediction system according to another embodiment of the present invention.

[0077] Figures 15a and 15b are conceptual diagrams illustrating a learning data set according to another embodiment of the present invention.

[0078] FIGS. 16A and 16B are conceptual diagrams illustrating an embodiment of distinguishing a plurality of records included in a learning data set according to another embodiment of the present invention.

[0079] Figure 17 is a conceptual diagram for explaining a learning method of a prediction system according to another embodiment of the present invention.

[0080] Figures 18 and 19 are conceptual diagrams for explaining an embodiment of efficiently processing data by utilizing an index of a data sequence unit in the present invention.

[0081] Hereinafter, embodiments disclosed in this specification will be described in detail with reference to the attached drawings. Regardless of the drawing numbers, identical or similar components will be given the same reference numbers, and redundant descriptions thereof will be omitted. The suffixes "module" and "part" used for components in the following description are assigned or used interchangeably only for the convenience of writing the specification, and do not in themselves have distinct meanings or roles. In addition, when describing the embodiments disclosed in this specification, if it is determined that a specific description of a related known technology may obscure the gist of the embodiments disclosed in this specification, a detailed description thereof will be omitted. In addition, the attached drawings are only intended to facilitate easy understanding of the embodiments disclosed in this specification, and the technical ideas disclosed in this specification are not limited by the attached drawings, and should be understood to include all modifications, equivalents, and substitutes included in the spirit and technical scope of the present invention.

[0082] Terms that include ordinal numbers, such as first, second, etc., may be used to describe various components, but the components are not limited by these terms. These terms are used solely to distinguish one component from another.

[0083] When a component is referred to as being "connected" or "connected" to another component, it should be understood that it may be directly connected or connected to that other component, but that there may be other components intervening. Conversely, when a component is referred to as being "directly connected" or "connected" to another component, it should be understood that there are no other components intervening.

[0084] Singular expressions include plural expressions unless the context clearly indicates otherwise.

[0085] In this application, terms such as “include” or “have” are intended to specify the presence of a feature, number, step, operation, component, part or combination thereof described in the specification, but should be understood not to exclude in advance the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts or combinations thereof.

[0086] Hereinafter, the present invention will be described in more detail with reference to the attached drawings. Fig. 1 is a conceptual diagram for explaining a prediction system and a control method thereof according to the present invention, and Figs. 2a and 2b are conceptual diagrams for explaining a prediction system according to the present invention. Fig. 3 is a flowchart for explaining a learning method of the prediction system according to the present invention, and Figs. 4, 5, 6, 7, 8, and 9 are conceptual diagrams for explaining a learning method of the prediction system according to the present invention. Furthermore, Fig. 10 is a flowchart for explaining a valid customer prediction method of the prediction system according to the present invention, and Fig. 11 is a conceptual diagram for explaining a valid customer prediction method of the prediction system according to the present invention.

[0087] The prediction system, its control method, and its learning method according to the present invention can be usefully utilized in various environments and situations. For example, the prediction system can be useful in predicting potential customers' purchasing probabilities to develop effective marketing strategies, or in predicting market demand for specific products or services to optimize inventory management and production planning.

[0088] In one embodiment, the prediction system according to the present invention can be usefully utilized in a B2B (Business-to-Business) sales situation. Here, B2B can refer to transactions or commercial activities between businesses (or companies). This can refer to a business model in which a specific company provides a product (or merchandise) or service to another company. For example, a software company providing a software solution to another company, or a parts manufacturer supplying parts to another company, can be considered a B2B transaction. In other words, unlike B2C (Business-to-Consumer), which targets consumers, B2B is characterized by focusing on relationships between companies.

[0089] Furthermore, the prediction system, its control method, and its learning method according to the present invention can be applied and utilized effectively in various industries and services. For example, a prediction model utilizing the learning method according to the present invention can be utilized in the medical industry to diagnose rare diseases, in the financial industry to detect fraudulent transactions, or in the security industry to detect and respond to security threats early.

[0090] While the purpose and application of the prediction system in this specification is described as being related to B2B and / or B2C sales, it is not necessarily limited to this. For example, the prediction system according to the present invention can be applied and utilized effectively in various industries, such as healthcare, finance, and security, as discussed above. Furthermore, the term "customer" in the present invention can be used interchangeably with "client company" to convey the same meaning.

[0091] A prediction system and its control method according to the present invention will be briefly described with reference to FIG. 1. As illustrated in FIG. 1, the prediction system (100) according to the present invention may include at least one of a data processing unit (110), a model unit (120), a prediction unit (130), and a control unit (140).

[0092] Although not shown, the prediction system (100) may include one or more processors, which may include one or more general-purpose processors and / or one or more special-purpose processors (e.g., digital signal processors, tensor processing units (TPUs), graphics processing units (GPUs), neural network processing units (NPUs), application-specific integrated circuits, application-specific integrated circuits (ASICs), etc.). The one or more processors may be configured to execute instructions stored (or included) in memory, computer-readable instructions, and / or other instructions described herein. Such an answer generation system and method may enable the memory and at least one processor to cooperate to perform the data processing described below. The processor may perform a series of operations and data processing using data and information stored in the memory.

[0093] Meanwhile, the data processing unit (110) may be configured to collect data from various sources (e.g., database, web crawling, API, server linked to the prediction system (100), external server, etc.) and perform preprocessing on the collected data.

[0094] The data processing unit (110) can collect various data used for training at least one model (121, 122, 123) included in the model unit (120). For example, as illustrated in FIGS. 2A and 2B , the data processing unit (110) can collect MQL (Marketing Qualified Lead) data (210) configured to have values ​​for multiple different categories from various sources.

[0095] Here, MQL data can be understood as information on potential customers (or clients) selected through marketing activities, used to identify potential customers who have shown interest in a product or service or have the potential to purchase it. Marketing qualified lead data may include various elements related to the customer's record and / or behavior of interest in a product or service. For example, various elements may include at least one of the following: customer (or client) information (e.g., customer's name, customer's account (or customer identification number or code), contact information, email address, job title, location information, country of affiliation, affiliated company (or company), etc.), company information to which the customer belongs (e.g., company name, industry, size, etc.), type and / or category of products or services in which the customer has shown interest, customer event records (e.g., customer's website visit history (or number of visits), customer's purchase history, product page views, product inquiries, survey responses, etc.), and information related to the customer's purchase intent (e.g., information regarding expected budget, expected purchase timing, etc.).

[0096] However, the collected data is not necessarily limited to the examples mentioned above. In one embodiment, in addition to the marketing qualification data (210), the data processing unit (110) may collect at least one of product data (e.g., product identification information (or code), product name and description, product price, inventory status, product category, product rating and review, product launch date, product specifications and features, etc.), sales process data (e.g., lead information, sales representative information, sales opportunity information, sales activity records, sales stages, contract information, performance indicators, etc.), and market trend data (e.g., market research reports, competitor information, industry trends, consumer behavior, economic indicators, technology trends, regional (or national) characteristics and regulatory information, etc.). For convenience of explanation, the collected data will be described below by naming them all as a “learning data set (or, learning data, 210)” without distinguishing them separately.

[0097] Meanwhile, the data processing unit (110) can perform preprocessing on the learning data set (210).

[0098] The data processing unit (110) can clean the learning data set (210), process errors or missing values ​​in the learning data set (210), and detect (or identify) and remove outliers or duplicate records (or data). For example, the data processing unit (110) can replace missing values ​​in the learning data set (210) with an average value or delete them, and detect and remove outliers (e.g., outliers) with abnormally large or small values ​​and duplicate records.

[0099] Additionally, if the learning data set (210) includes categorical data (or variables), the data processing unit (110) can convert the categorical data into a numerical form that can be understood by the artificial intelligence model. For example, the data processing unit (110) can convert the categorical data into a multidimensional vector using at least one of one-hot encoding and / or label encoding.

[0100] In addition, the data processing unit (110) can adjust the range of numerical data (or continuous data) so that all variables have the same range. For example, the data processing unit (110) can convert numerical data into data with a mean of 0 and a variance of 1 through normalization (e.g., Z-Score normalization) for numerical data, or can convert continuous data into data between 0 and 1 through scaling (e.g., Min-Max scaling) for continuous data. This can be understood as data processing to prevent the results from being distorted by the size of a specific variable during artificial intelligence model training or to prevent the artificial intelligence model from being biased toward a specific feature.

[0101] Furthermore, the data processing unit (110) can expand (or augment) data that an artificial intelligence model can learn by generating new variables (derived variables) from the learning data set (210) through feature engineering for the learning data set (210) in which existing variables have been preprocessed.

[0102] In this case, the data processing unit (110) can generate derived variables (or derived categories) from the learning data set (210) based on RFM (Recency-Frequency-Monetary) analysis during the feature engineering process.

[0103] RFM analysis is a marketing method used to evaluate and segment customers. It can include recency, frequency, and monetary values. Recency refers to the time elapsed since a customer's last purchase, frequency refers to the number of purchases a customer makes over a given period, and monetary values ​​can refer to the total amount a customer spends over a given period.

[0104] In one embodiment, the data processing unit (110) may extract specific data (or variables, for example, sales representative (ex: “lead_owner”), customer identification information (ex: “customer_idx”), etc.) with high feature importance from the learning data set (210) based on RFM analysis, and generate derived variables for the extracted specific data (for example, a variable indicating the experience level or frequency of the representative (ex: “lead_owner_job”), a variable indicating whether or not the customer has made a repeat purchase (ex: “customer_idx_count”), a variable combining the experience of the sales representative and the frequency of the customer’s return visit (ex: “oppty”), etc.).

[0105] In another embodiment, the data processing unit (110) may separate year and month information using date data (e.g., “lead_date”) included in the learning data set (210) and generate a derived variable (e.g., “lead_date_yearmonth”) that includes the customer’s recent purchasing activity.

[0106] Meanwhile, the data processing unit (110) can configure at least one sub-data set using the learning data set (210).

[0107] For example, a training dataset may be imbalanced, with significantly more or fewer instances of certain values ​​than instances of others. This can lead to AI models being biased toward frequently occurring classes. For example, in MQL data, instances of customer conversions are often less frequent than instances of non-conversions. This can lead to data imbalance, negatively impacting the learning and prediction performance of AI models.

[0108] To address this data imbalance problem, the data processing unit (110) may configure multiple different sub-data sets, each of which has a composition ratio of multiple records containing different values ​​for a specific category among the multiple data (or records) included in the learning data set (210), satisfying a preset composition ratio criterion. More specific details regarding this will be described later.

[0109] Meanwhile, the model unit (120) may be configured to include at least one learning target prediction model. For example, the model unit (120) may include at least one of the first model (121), the second model (122), and the third model (123) that are learning targets.

[0110] The first model (121), also known as the "CatBoost model," may be a model specialized for processing categorical data (or variables or features). The first model (121) can prevent target leakage problems that may occur in categorical data by utilizing a regularization technique (or method) called "Ordered Target Statistics" and / or "Ordered Boosting." Furthermore, the first model (121) can utilize a symmetric tree structure to distribute balanced data at each level of the tree. This first model (121) can prevent overfitting and achieve high predictive performance.

[0111] The second model (122) may also be referred to as a “LightGBM (LGBM) model”, and may be a model that maximizes learning speed and reduces memory usage while maintaining high prediction performance by using the “Gradient-based One-Side Sampling (GOSS)” and / or “Exclusive Feature Bundling (EFB)” methods. Gradient-based One-Side Sampling (GOSS) samples data according to the size of the gradient to reduce the amount of computation, and Exclusive Feature Bundling (EFB) can reduce the number of variables by bundling rare features. In addition, the second model (122) can learn deeply about a specific part of the data by using a leaf-wise tree growth method, and can better identify complex data patterns.

[0112] The third model (123) may also be named an “XGBoost model” and may be a model based on the Gradient Boosting Decision Tree (GBDT) algorithm optimized for high predictive performance and overfitting prevention. The third model (123) may utilize regularization to prevent overfitting and tree pruning to reduce model complexity by removing unnecessary branches. The third model (123) provides flexibility in missing value processing and, through a level-wise tree growth method, can perform broad learning so that various characteristics are well reflected by evenly dividing all nodes.

[0113] As seen above, the first model (121), the second model (122), and the third model (123) may be models based on the Gradient Boosting Decision Tree (GBDT) algorithm, and may divide data and perform learning based on the decision tree.

[0114] However, the models included in the model section (120) of the present invention are not necessarily limited to the models mentioned above, and may further include various models. It goes without saying that the models included in the model section (120) of the present invention may be one or more, and may be varied in various ways, depending on the case.

[0115] Meanwhile, each of the models (121, 122, 123) included in the model unit (120) may be input with a plurality of sub-data sets (221, 222, 223) generated from the data processing unit (110). In this case, each of the plurality of models (121, 122, 123) may receive a plurality of sub-data sets (221, 222, 223) as input and perform learning on each of the plurality of sub-data sets (221, 222, 223).

[0116] Specifically, the first model (121), the second model (122), and the third model (123) independently perform learning for each of the multiple sub-data sets (221, 222, 223), and when learning of each model (121, 122, 123) is completed, multiple learned prediction models can be obtained.

[0117] Here, “multiple learned prediction models” may mean learned prediction models corresponding to the number of “N” multiplied by the number of “M” multiple prediction models (121, 122, 123) multiplied by the number of “N” multiple different sub-data sets, each of which is learned by learning multiple prediction models (121, 122, 123) for each of multiple sub-data sets (221, 222, 223).

[0118] That is, when the training of each of the multiple prediction models (121, 122, 123) for each of the N different sub-data sets is completed, each of the multiple prediction models (121, 122, 123) may include multiple learned prediction models trained for each of the N sub-data sets. In the present invention, the prediction model may also be called a “Binary Classification model” or a BalancedTreeMarketer model.

[0119] Meanwhile, the prediction unit (130) can be configured to specify a final prediction result (or final prediction value) using the output values ​​of at least one learned prediction model (or N learned models).

[0120] Specifically, the prediction unit (130) can perform soft voting based on multiple prediction values ​​output from each of multiple learned prediction models to determine the final prediction value.

[0121] In one embodiment, the prediction unit (130) calculates (or produces) an averaged probability (e.g., “Sales Conversion Probability”, 230) by averaging a plurality of predicted values ​​(or predicted probabilities) independently predicted by a plurality of learned prediction models based on soft voting, and can specify a final predicted value (e.g., “Sales Conversion Predict”, or customer conversion, 240) based on the calculated probability (230).

[0122] Here, soft voting can be understood as one of the ensemble techniques, a technique that determines a final prediction by combining (or averaging) the results (or probabilities) independently predicted by multiple artificial intelligence models. That is, the prediction unit (130) can determine the final prediction result (or predicted value) by combining the results (or predicted values) output from each of the multiple learned prediction models.

[0123] Additionally, the averaged probability is a result of synthesizing the predicted values ​​output by trained prediction models, and can be understood as representing the likelihood of a customer purchasing a product or service (purchase conversion likelihood) as a probability value. For example, the prediction unit (130) can express the probability value as a value between 0 and 1. In this case, a value of 0.7 may indicate that a customer has a 70% chance of purchasing the product.

[0124] Furthermore, the final prediction value (240) is the final extracted prediction result, which can be understood as a binary classification representing whether or not a customer will purchase a product or service. For example, the prediction unit (130) can express "purchased (1)" if the customer is predicted to purchase the product, and "not purchased (0)" if the customer is predicted not to purchase the product.

[0125] That is, the prediction unit (130) can compare the averaged probability (230) with a preset threshold value, and if the averaged probability (230) exceeds the preset threshold value, specify the final prediction value (240) as “purchased (1)”, and if it does not exceed the threshold value, specify the final prediction value as “not purchased (0)”. More specific details about this will be described later.

[0126] Meanwhile, the control unit (140) may perform a role of controlling the overall operation of the present invention and the prediction system (100). The control unit (140) may process signals, data, information, etc. input or output through the components discussed above, or perform a series of data processing to provide or process appropriate information and functions to the user.

[0127] In one embodiment, the control unit (140) may provide a service page (1000) to the user terminal (10). The service page (1000) may provide a list (or a customer list) of at least one company that interacts (e.g., conducts transactions, collaborates, etc.) with a specific company. In this case, the control unit (140) may provide information on the purchase probability of each customer company of a specific product (e.g., “PuriCare Object Collection Water Purifier”) sold by a specific company, predicted by the prediction system (100), in one area of ​​the service page (1000).

[0128] Meanwhile, the present invention aims to address the problem of imbalanced data and provide a prediction system, its control method, and a learning method for the prediction system that can be applied universally across various industries. More specifically, the present invention aims to provide a prediction system capable of predicting effective customers by analyzing various customer data. Below, we will examine the learning method for the prediction system (or prediction model) in more detail.

[0129] First, in the present invention, a process of specifying a learning data set can be performed (S310, see FIG. 3).

[0130] The control unit (140) can specify a learning data set to be used for learning the learning target prediction model.

[0131] In this case, the criteria for specifying the learning data set may vary. The control unit (140) may specify the learning data set to be used for learning the target prediction model based on various criteria.

[0132] In one embodiment, the control unit (140) may collect (or receive) a data set from at least one of various sources (e.g., a database (DB), web crawling, API, a server linked to the prediction system (100), an external server, etc.) and specify the collected data set as a learning data set to be used for learning the prediction model to be learned.

[0133] In another embodiment, the control unit (140) may specify a data set stored in at least one of various repositories (e.g., storage, memory, database (DB), etc.) as a learning data set to be used for learning the target prediction model.

[0134] A learning data set may include various types of data. For example, as illustrated in FIG. 4, the learning data set (410) may include at least one of MQL data, product data, sales process data, and market trend data. The data included in the learning data set may be in the form of at least one of numerical data, categorical data, and text data. However, the format of the data included in the learning data set is not necessarily limited to the examples mentioned, and it is obvious that the data may include other types of data.

[0135] Such a learning data set (410) may be configured to include a plurality of records having values ​​for a plurality of different categories.

[0136] Here, a record represents at least one data unit, and a record may contain multiple categories of data values ​​(i.e., multiple fields or properties). In a database, this can also be called a "row." For example, in an Excel sheet, each row represents a record, and each column within a row can represent data values ​​for various categories within the record.

[0137] That is, each piece of data included in a data set or a data unit containing multiple categories of data values ​​can be called a “record” or “sample.”

[0138] The learning data set (410) may include MQL data configured to have values ​​for multiple different categories. Furthermore, in the present invention, categories may also be referred to as "categories," "features," "variables," or "elements."

[0139] Before examining the preprocessing process for the learning data set (410), let us first examine the multiple categories included in the learning data set (410) and the values ​​for the multiple categories, together with FIG. 5.

[0140] The first category (e.g., “ID”, 501) is an arbitrary value that uniquely identifies each data entry, and its primary purpose may be to calculate an f1 score by comparing it with the 39th category (e.g., “is_converted, 539). The first category (501) can be used to measure accuracy by matching each predicted result with the actual result, and the first category (501) can also be used to evaluate model performance.

[0141] The second category (e.g., "bant_submit", 502) is a variation of the BANT (Budget, Authority, Need, Timeline) framework, which can be used to assess MQL quality. Here, "Budget" refers to the customer's budget, indicating the funds available for allocation to projects or purchases. "Authority (Title)" refers to the customer's position / rank, indicating whether the customer has decision-making authority. Additionally, "Needs" can refer to the customer's specific needs, problems, or goals that the product or service should address. "Timeline" can refer to the customer's desired delivery date.

[0142] The third category (e.g., "customer_country", 503) represents the customer's nationality, and its value can be expressed as " / region / country (e.g., Asia / Korea)". This third category (503) can provide valuable information for regional sales strategies, localized service offerings, and approaches based on legal and cultural understanding. Furthermore, this category can be utilized to develop strategies that take into account time differences, language barriers, and cultural differences that may arise in international business relationships.

[0143] The fourth category (ex: “customer_country.1”, 504) can mean a region or country, such as the corporate region of the company in charge.

[0144] The fifth category (e.g., “business_unit”, 505) refers to the business unit within the company that corresponds to the product or service requested in the MQL, and can be divided into multiple categories (e.g., (ID, AS, IT, Solution, CM)). These categories are important for understanding the nature of the lead and assigning the appropriate sales team or specialist, and can be used for performance analysis, resource allocation, and strategy development for each business unit.

[0145] The sixth category (ex: “com_reg_ver_win_rate”, 506) is a weight calculated by calculating the opportunity rate based on a specific business area (Vertical Level 1), a specific business unit, or a region, and can be used to predict the possibility of future success based on past success rates.

[0146] The seventh category (e.g., "customer_idx", 507) stores the customer company name and the number of times the company submitted data, which can indirectly indicate the customer company's level of engagement or interest. A high value indicates that the company frequently inquires or interacts with you, potentially indicating a high level of interest or purchase intent. For example, the seventh category (507) can be used for customer segmentation, prioritization, and developing customized marketing strategies.

[0147] The 8th category (ex: “customer_type”, 508) is data that classifies the customer’s occupation and can be useful for developing target marketing or customized sales strategies.

[0148] The 9th category (ex: “enterprise”, 509) indicates the size of the customer company, which can be divided into enterprises and small and medium-sized businesses (SMB).

[0149] The 10th category (e.g., “historical_existing_cnt”, 510) can refer to the number of times that the customer (or company) has successfully converted sales in the past. The 10th category (510) can be useful for assessing customer loyalty or the likelihood of repeat purchases. A high value can be understood as indicating a strong business relationship with the customer and a high possibility of future transactions.

[0150] The eleventh category (e.g., "id_strategic_ver", 511) may include a weighted value indicating the strategic importance of a combination of a specific business unit (BU) and a specific business area (Vertical Level 1). The eleventh category (511) can be utilized to optimize resource allocation by reflecting the company's strategic priorities and to increase focus in specific business areas.

[0151] Similar to Category 11 (511), Category 12 (e.g., “it_strategic_ver”, 512) may include weighted values ​​indicating the strategic importance of a combination of a specific Business Unit and a specific Business Area (Vertical Level 1). This weighting is for a specific Business Unit (e.g., IT Business Unit), enabling efficient technical staffing and planning.

[0152] The 13th category (e.g., “idit_strategic_ver”, 513) may refer to a composite indicator that integrates the 11th category (511) and the 12th category (512). If at least one of the 11th category (511) and / or the 12th category (512) has a value of 1, the 13th category (513) may be assigned a weighting of 1. The 13th category (513) provides an integrated strategic importance encompassing the ID and IT domains and may be utilized as a consideration factor when determining enterprise-wide resource allocation.

[0153] The 14th category (e.g., "customer_job", 514) can represent categorical data representing occupational groups. This 14th category (514) allows for communication methods that take into account occupational characteristics and allows for customer grouping based on occupation.

[0154] Category 15 (e.g., "lead_desc_length", 515) could refer to the total length of the customer's lead description text. This could indirectly indicate the customer's level of interest or engagement, or reflect the complexity of their needs or issues.

[0155] The 16th category (e.g., "inquiry_type", 516) can contain information categorizing the customer's inquiry type. For example, the 16th category (516) can be divided into multiple categories (e.g., 71) including product information inquiries, purchase consultations, and quotation requests. This can help identify the customer's purchasing stage and serve as a key element in developing marketing strategies. Furthermore, the 16th category (516) can be used to assign appropriate departments or personnel based on the inquiry type, thereby assisting in sales conversion.

[0156] The 17th category (e.g., "product_category", 517) may represent the parent category of the requested product. For example, the 17th category (517) may be divided into multiple categories (e.g., 357) including tablets, TVs, washing machines, refrigerators, etc. This allows for the development of marketing strategies focused on customer-desired categories.

[0157] The 18th category (e.g., "product_subcategory", 518) can further categorize the requested product's subcategories. For example, the 18th category (518) can be divided into multiple (e.g., 330) subcategories, such as OLED, QLED, and 8K TVs, allowing for a more detailed product classification system. This allows for the identification of precise customer needs and more targeted marketing.

[0158] Category 19 (e.g., "product_modelname", 519) could refer to the model name of a specific product requested by the customer. For example, since the customer provides very specific information, we can accurately understand their interests. Based on this, we can create customized proposals and develop personalized sales approaches. This can ultimately increase customer satisfaction and sales conversion rates.

[0159] The 20th category (e.g., "customer_position", 520) can indicate the customer's position within the company. This can help determine the customer's level of authority in making purchasing decisions. Furthermore, the 20th category (520) can be a key factor in developing differentiated sales and marketing strategies based on position.

[0160] The 21st category (e.g., "response_corporate", 521) can be a string data type that indicates the name of the company's legal entity responsible for customer inquiries or transactions. This category (521) can play a crucial role in corporate structures with multiple subsidiaries. By identifying which legal entity is primarily involved in customer response or sales processes, this category can clarify responsibilities across internal organizations and maintain consistency in customer management. Furthermore, this category can provide insights necessary for analyzing the performance of each legal entity, optimizing resource allocation, and developing company-wide sales strategies.

[0161] Category 22 (e.g., "expected_timeline", 522) can indicate the deadline for processing a task requested by the customer. This can be utilized as an important indicator in predictive models, as a customer's specific schedule may signal a strong purchase intent. Furthermore, the urgency of Category 22 (522) can be used to estimate the likelihood and speed of a deal. For example, a short deadline may imply quick decision-making and a high conversion rate, while a long deadline may indicate a larger deal or a complex decision-making process. Effectively utilizing Category 22 (522) can help optimize the sales team's resource allocation and develop a customized customer approach strategy. In other words, Category 22 (522) can be a factor contributing to increased B2B sales conversion rates.

[0162] Category 23 (e.g., "ver_cus", 523) can quantify the impact of a combination of a specific business area and customer type on sales conversion in B2B sales. A weight of 1 can be assigned to a category that belongs to a specific business area and also has an end-user customer type. This allows for assessing the likelihood of success for sales targeting direct end users in a specific business area. Category 23 (523) reflects the importance of customer segmentation in B2B sales strategies and can help identify business areas where an end-user-centric approach may be more effective.

[0163] The 24th category (e.g., "ver_pro", 524) can be used to weight combinations of specific business areas (Vertical Level 1) and product types (Product Category). This category (524) can be used to determine whether a specific product type has a higher sales conversion rate in a specific business area. A combination weighted by 1 may indicate that the product type is competitive or in high demand in that business area. This category (524) can be used to identify product groups that should be focused on for each business area and to develop customized sales strategies.

[0164] Category 25 (e.g., "ver_win_rate_x", 525) can be a composite weighted category that simultaneously considers the relative importance and success rate of each vertical. It is calculated by multiplying the percentage of total leads accounted for by the vertical's sales conversion success rate within that vertical. Category 25 (525) allows for a more balanced evaluation by considering not only the success rate but also the overall weight of the vertical. This allows for a more accurate assessment of the true importance of each vertical when allocating sales resources and developing strategies.

[0165] Category 26 (e.g., "ver_win_ratio_per_bu", 526) can represent the sales conversion success rate for each business unit (or division) within a specific business area. This can demonstrate how effectively each business unit is performing sales in a specific vertical. Category 26 (526) can be used to identify which specific business units are performing the best within each vertical, which can be utilized to optimally share processes and optimize resource allocation within the organization. Furthermore, based on Category 26 (526), ​​customized sales strategies can be developed that leverage the strengths of each business unit.

[0166] Category 27 (e.g., "business_area", 527) can represent a client's main business area. Category 27 (527) can play a key role in predicting B2B sales conversion rates. By understanding a client's business area through Category 27 (527), you can develop a customized approach tailored to that industry. Furthermore, Category 27 (527) can be used to analyze past success patterns in specific business areas to optimize sales strategies for new clients in similar industries. This can lead to efficient allocation of sales resources and improved conversion rates.

[0167] Category 28 (e.g., "business_subarea", 528) can be a more specific category that categorizes a client's business areas. Category 28 (528) can help you more accurately identify a client's specific needs and requirements. Utilizing Category 28 (528) in a predictive model can enable highly segmented market access. Based on Category 28 (528), you can develop more sophisticated sales strategies and increase conversion rates.

[0168] The 29th category (e.g., "lead_owner", 529) can represent the name of the sales representative responsible for each opportunity. This category (529) can play a crucial role in analyzing individual and team performance in predictive models. Furthermore, this category can be used to understand the impact of a specific sales representative's sales skills, experience, or industry expertise on conversion rates. Furthermore, this category can be used to develop optimal lead allocation strategies and analyze collaboration patterns among team members to improve overall sales performance.

[0169] The 30th category (e.g., "lead_date", 530) can indicate the date a sales opportunity (lead) was first created. The 30th category (530) can be crucial for considering temporal factors in predictive models. Furthermore, the 30th category (530) can be used to analyze the time it takes from lead generation to actual deal closure, seasonal trends, and performance changes over a specific period. Furthermore, the 30th category (530) can be used to understand the impact of a lead's recency on conversion rates, enabling the development of timely and effective follow-up strategies. This can optimize the sales cycle and increase conversion rates.

[0170] The 31st category (e.g., "lead_from_channel", 531) may represent the marketing channel from which sales opportunity information was collected. This 31st category (531) can be crucial for evaluating the effectiveness of each marketing channel in a predictive model. By analyzing the quality and conversion rate of leads acquired through a specific channel based on this 31st category (531), the most effective marketing channel can be identified. Furthermore, based on this 31st category (531), marketing budget allocation can be optimized and customized sales strategies can be developed for each channel. Consequently, lead quality can be improved and overall sales conversion rates can be increased.

[0171] The 32nd category (e.g., "event_name", 532) may be a category that indicates the name of a specific marketing event where a sales activity occurred. This 32nd category (532) can be crucial for evaluating the effectiveness of each marketing event in a predictive model. By analyzing the quality and conversion rate of leads generated through a specific event based on this 32nd category (532), the most successful event types can be identified. Furthermore, based on this 32nd category (532), future marketing event planning and resource allocation can be optimized, and customized follow-up sales strategies tailored to the characteristics of each event can be developed. Consequently, event ROI can be improved and overall sales conversion rates can be increased.

[0172] The 33rd category (e.g., "prefer_ver_count", 533) can represent the distribution ratio of conversion cases within a specific business unit across a specific business area. This category (533) can be crucial for identifying the strengths of each business unit in a predictive model. By analyzing which verticals a specific business unit demonstrates high success rates based on this category, the most effective target market for each business unit can be identified. This allows for the development of specialized strategies for each business unit. Consequently, by maximizing the strengths of each business unit, the overall sales conversion rate can be improved.

[0173] The 34th category (e.g., "prefer_ver_mean", 534) is calculated based on similar criteria as the 33rd category (533), but can represent a percentage of profit values ​​instead of a simple sample count. The 34th category (534) is crucial for identifying areas of profitability strength for each business unit in the prediction model. By analyzing which verticals generate high profits for a specific business unit based on the 34th category (534), strategies can be developed that consider actual profit contributions, rather than simply the number of successful cases. This allows for focused sales activities on high-profit verticals and improved overall operating profitability.

[0174] Category 35 (e.g., "transfer_agreement", 535) can indicate whether a customer has consented to the export of their lead information overseas. Category 35 (535) can be a valuable indicator in predictive models for assessing a customer's openness and willingness to collaborate globally. Customers who consent to information export are more likely to be interested in a broader range of services or global solutions. Based on Category 35 (535), you can make customized offers for products or services that require international collaboration and utilize it to develop global sales strategies.

[0175] Category 36 (e.g., "ver_win_rate_mean_upper", 536) can be a category that represents a value of 1 if the average of each vertical is exceeded, and 0 otherwise. Category 36 (536) can be important for evaluating relative performance within each vertical in a predictive model. By analyzing the characteristics of cases that perform above average based on Category 36 (536), key elements of successful sales strategies can be identified. This allows best practices to be applied to other cases to improve overall sales performance.

[0176] Category 37 (e.g., "expected_budget", 537) can represent a customer's desired budget range. Category 37 (537) can be a key indicator for assessing a customer's purchasing intent and project scale in a predictive model. Based on Category 37 (537), appropriate products or services can be suggested based on budget size, and customized solutions can be developed that meet the customer's financial expectations. Furthermore, Category 37 (537) can be used to identify optimal target segments by analyzing conversion rates by budget range, optimizing resource allocation and improving overall sales performance. Notably, Category 37 (537) can be a category responsible for "Money" when applying a traditional RFM model.

[0177] Category 38 (e.g., "lead_description", 538) can contain customer-specific requirements. Category 38 (538) can play a key role in identifying specific customer needs and interests in predictive models. By analyzing Category 38 (538) using text mining and natural language processing (NLP), potential customer needs and preferences can be identified. Based on Category 38 (538), customized proposals can be created and personalized sales approaches developed. This can ultimately lead to increased customer satisfaction and improved sales conversion rates.

[0178] The 39th category (e.g., "is_converted", 539) is a core category representing the final outcome of a sales activity. It can represent sales success (or failure) as a binary value (e.g., 1: success, 0: failure). This can be the target category (or specific category) that we ultimately aim to predict in the predictive model. Based on the 39th category (539), we can analyze the impact of various categories and identify the characteristics of successful sales cases. Furthermore, the 39th category (539) can be used to evaluate the predictive accuracy of the predictive model and facilitate continuous model improvement and optimization. Consequently, by accurately predicting the 39th category (539), we can support efficient allocation of sales resources and strategic decision-making, thereby improving overall B2B sales performance.

[0179] The 40th category (e.g., “len_expected_timeline”, 540) may be a derived category generated during the preprocessing of the 22nd category (522). Based on the 40th category (540), the data consistency problem of the 22nd category (522) can be resolved.

[0180] Category 41 (e.g., "countrycoinside", 541) can be a derived category indicating whether the customer's nationality and the regional information (continent) based on the company's legal entity name are the same. Based on Category 41 (541), sales strategies can be developed that take regional characteristics into account.

[0181] Category 42 (e.g., "lead_owner_job", 542) may be a derived category created from Category 29 (529) to quantify the experience and proficiency of salespeople in a B2B sales environment. The frequency with which salespeople appear in the data set is counted, with higher frequency indicating greater sales caseloads. Based on Category 42 (542), experienced representatives can be assigned to important leads or complex cases, optimizing resource allocation. This, in turn, can improve customer satisfaction and sales conversion rates.

[0182] Category 43 (e.g., "customer_idx_count", 543) can be an important indicator (or derived category) of customer loyalty and purchase intent. The number of appearances for each customer in Category 7 (507) was counted, with a high count indicating frequent customer inquiries for transactions. This indicates ongoing interest in the product or service and can reflect the strength of potential purchase intent. Category 43 (543) can be used to identify key targets for building long-term business relationships and indicate a high likelihood of future purchases of the company's various products.

[0183] The 44th category (e.g., "oppty", 544) can be a derived category designed to predict sales conversion rates in a B2B sales environment. This category (544) extends the frequency concept of the traditional RFM model, combining the salesperson's experience (e.g., "lead_owner_job") with the customer's return frequency (e.g., "customer_idx_count"). By quantifying and calculating the synergy between experienced salespeople and loyal customers, this category can enable more accurate sales performance predictions that go beyond simple transaction frequency and consider the qualitative aspects of sales relationships.

[0184] The 45th category (e.g., “vertical_level”, 545) can be used to identify and weight strategically important verticals within each business area. The 45th category (545) can be a derived category created by analyzing existing weighting variables, such as the 11th category (511), the 12th category (512), and the 23rd category (523). In certain industries, data that is not weighted by these weighting variables can be considered less strategically important verticals within that industry. Based on this logic, the 45th category (545) can filter out vertical data that is not strategically important and assign additional weight to data corresponding to important verticals. This can effectively identify the most promising verticals within each business area and develop customized sales strategies accordingly, contributing to improved overall business performance.

[0185] Category 46 (e.g., “weight_expected_timeline”, 546) can be a derived category used to predict the progress of customer transactions, a key indicator in the B2B sales process. The original data for Category 46 (546) included email addresses and consultation content unrelated to the actual timeline. However, considering the nature of B2B business, where the absence of a clear timeline agreement reduces the likelihood of actual transactions, Category 46 (546) was improved. Specifically, a weighting method was applied to data containing words indicating dates or periods. This approach assigns greater importance to data likely to contain actual timeline information, enabling more accurate predictions of sales conversion potential. This approach can enhance the efficiency of the B2B sales process and contribute to the development of more accurate sales strategies.

[0186] The 47th category (e.g., "qcut", 547) is a method for dividing numerical data into intervals based on quantiles. Traditional RFM models use qcut to divide data into a specific number of groups, ensuring that each group contains an equal number of data points. This ensures that each group reflects the unique characteristics of the data points. The appropriate number of groups was determined by visualizing the importance of variables. The eight derived categories created using qcut were created by partitioning various numerical data into multiple groups with equal frequency. The partitioning ensures that the number of data points in each group is approximately equal. This is a methodology frequently used in traditional RFM models. This approach minimizes the influence of extreme values ​​and allows for effective comparison of characteristics between groups. Based on the results of visualized analysis of variable importance, each group was divided into an appropriate number of groups. This method allows for more clearly revealing the unique characteristics of each group and, while preserving the characteristics of continuous variables, leverages the advantages of categorical data, making it flexible for various analysis techniques.

[0187] The 48th category (e.g., “lead_date_yearmonth”, 548) can be a time-based variable (derived category) created by combining the year and month of the customer lead generation time. The creation process of the 48th category (548) is as follows. The 30th category (530) was grouped into various time units such as month, year, half-year, and quarter, and then analyzed. Among the various time units, the form combining year and month showed the highest correlation and was selected. This can reflect a company’s business cycle. The yearly element takes into account the fact that a company’s product lineup or strategy changes depending on the year, and the monthly element reflects the tendency for a customer’s purchasing cycle or budget execution pattern to be concentrated in a specific month. The 48th category (548) can more accurately capture customer behavior patterns over time and provide useful insights for establishing marketing strategies for each period.

[0188] The 49th category (e.g., “second_event”, 549) may be a derived category created to independently utilize important information extracted from the existing 32nd category (532). The 32nd category (532) may have a structure such as “(business_unit)(second_event)(lead_from_channel)(date).” In this structure, all elements except “second_event” already existed as individual variables. However, “second_event” was the only one not expressed as an independent variable. The “event_name” variable is composed of four elements, and because of the various values ​​of each element, it has the characteristic of being highly distributed overall. This can make it difficult to find meaningful patterns during data analysis or modeling. Therefore, by extracting “second_event” as a separate variable, important information can be utilized more effectively. This can contribute to more accurately reflecting the characteristics of the data and increasing the accuracy of the analysis.

[0189] As discussed above, the learning data set (410) may include MQL data configured to have values ​​for multiple different categories (501 to 550) and multiple categories (501 to 550).

[0190] The 50th category (e.g., "is_fresh", 550) can be a derived category created to improve customer segmentation accuracy. This 50th category (550) can categorize customers into types such as completely new customers, customers who have previously inquired but not yet completed a transaction, and customers with past transaction experience. This segmentation can provide valuable insights into sales strategy development, as each customer type requires a different approach and has different chances of success. In particular, the second type of customer may have different needs and expectations than completely new customers, so categorizing them separately can aid in effective customer management.

[0191] Meanwhile, the control unit (140) can perform preprocessing on the learning data set (410) (see FIG. 4).

[0192] First, the control unit (140) can purify the learning data set (410), process errors or missing values ​​in the learning data set (410), and detect and remove outliers or duplicate records.

[0193] In one embodiment, the control unit (140) can replace missing values ​​in the learning data set (410) with an average value or delete them, and identify and remove outliers and duplicate records.

[0194] And, when categorical data is included in the learning data set (410), the control unit (140) can convert the categorical data into numeric data that can be understood by the prediction model.

[0195] In one embodiment, the control unit (140) may use at least one of one-hot encoding and / or label encoding to convert a specific category (e.g., “is_converted”, 539) into numeric data that a prediction model can understand (e.g., “1” for purchase, “0” for non-purchase).

[0196] Furthermore, the control unit (140) can adjust the range of the numeric data and / or continuous data when the learning data set (410) includes at least one of numeric data and / or continuous data.

[0197] In one embodiment, the control unit (140) can convert numeric data into data with a mean of 0 and a variance of 1 through Z-Score normalization for numeric data, or can convert continuous data into data between 0 and 1 through Min-Max scaling for continuous data.

[0198] Meanwhile, the control unit (140) can create a new category (or variable or data) from the learning data set (410) through a feature engineering process for the learning data set (410).

[0199] Specifically, the control unit (140) may perform feature engineering on a learning data set (410) whose existing categories have been preprocessed (e.g., refined, normalized, scaled, etc.) to generate derived categories using at least some of the categories and values ​​corresponding to at least some of the categories among the plurality of categories included in the learning data set (410).

[0200] Here, “creating a derivative category” can be understood as extracting additional information (or meaning) from an existing category (or original category) or creating a new category (derived category).

[0201] First, the control unit (140) can generate a derived variable for at least one category among a plurality of categories by considering the domain (see FIG. 4).

[0202] Specifically, the control unit (140) can generate a derived category using at least one category among multiple categories and a value corresponding to said at least one category based on specific domain knowledge (or an analysis technique specialized for a specific domain). In this case, the control unit (140) can understand which categories are important and which combinations are meaningful through specific domain knowledge.

[0203] In one embodiment, as illustrated in FIG. 5, the control unit (140) may generate a derived category (e.g., “lead_date_yearmonth”, 548) using an existing category (e.g., “lead_date”, 530) and a value corresponding to the existing category (e.g., “2024-08-09”) based on expert knowledge of a specific domain (e.g., a marketing domain). The derived category (548) may be understood as a category utilized to analyze lead data at a specific point in time.

[0204] And, the control unit (140) can specify a value corresponding to a derived category based on the fact that a derived category is generated from an existing category. For example, the control unit (140) can specify a value (ex: “2024-08”) corresponding to the derived category (ex: “lead_date_yearmonth”, 548) based on the fact that a derived category is generated from an existing category (ex: “lead_date”, 530) and a value (ex: “2024-08-09”) corresponding to the existing category.

[0205] In addition, as discussed above, the present invention can generate derived categories from a learning data set (410) based on RFM (Recency-Frequency-Monetary) analysis.

[0206] More specifically, the control unit (140) can extract at least one category with high feature importance and a value corresponding to at least one category from the learning data set (410) based on RFM analysis, and generate a derived category using the extracted category and the value corresponding to the extracted category.

[0207] In one embodiment, as illustrated in FIG. 5, the control unit (140) may extract a seventh category (ex: “customer_idx”, 507) having high feature importance and a value corresponding to the seventh category (507) (ex: “CompanyA-1”), a 29th category (ex: “lead_owner”, 529) and a value corresponding to the 29th category (529) (ex: “John Doe”) from the learning data set (410) based on RFM analysis, and may generate a derived category using each of the extracted categories (507, 529) and the value corresponding to each of the extracted categories. In this case, at least one derived category can be created among the 42nd category (e.g., “lead_owner_job”, 542) that indicates the experience level or frequency of the representative, the 43rd category (e.g., “customer_idx_count”, 543) that indicates whether or not the customer makes a repeat purchase, and the 44th category (e.g., “oppty”, 544) that combines the sales representative’s experience and the customer’s return visit frequency.

[0208] And, the control unit (140) can specify a value corresponding to (or corresponding to) a derived category. For example, the control unit (140) can specify a value corresponding to the 42nd category (ex: “lead_owner_job”, 542) (ex: “25”), a value corresponding to the 43rd category (ex: “customer_idx_count”, 543) (ex: “10”), and a value corresponding to the 44th category (ex: “oppty”, 544) (ex: “0.85”).

[0209] Through this, the learning data set (410) may further include derived categories and values ​​corresponding to the derived categories generated through the derived variable generation process (or feature engineering).

[0210] In this way, the present invention can improve the performance of a prediction model by generating new derived variables from existing data, thereby allowing the prediction model to learn meaningful patterns.

[0211] Meanwhile, in the present invention, a process of constructing multiple different sub-data sets using a learning data set can be performed (S320, see FIG. 3).

[0212] The control unit (140) can configure multiple different sub-data sets using the learning data set (410).

[0213] In the present invention, “configuring a plurality of different sub-data sets” can be understood as configuring each of the plurality of different sub-data sets using at least some of the plurality of records such that the composition ratio of each record containing different values ​​for a specific category among the plurality of records satisfies a preset standard.

[0214] The control unit (140) can configure multiple different sub-data sets based on a value corresponding to a specific category among multiple categories (501 to 550).

[0215] To this end, the control unit (140) can specify a specific category that serves as a criterion for composing different sub-data sets among a plurality of categories (501 to 550).

[0216] Here, a specific category may be a category indicating whether a customer's purchase conversion has occurred. The control unit (140) may specify a "39th category (e.g., "is_converted", 539)" among multiple categories (501 to 550), which corresponds to a category indicating whether a customer's purchase conversion has occurred. For convenience of explanation, the specified "39th category (539)" will be referred to as a "specific category (539)" in the following description.

[0217] As discussed above, a specific category (539) is a category that represents the final result of a sales activity, and can express whether a sales goal is achieved (e.g., whether a sales goal is achieved, such as signing a contract and / or purchasing a product) as a binary value (e.g., success is “1”, failure is “0”).

[0218] In this case, a specific category (539) can be configured to have different values ​​depending on whether a customer conversion has occurred.

[0219] Here, the different values ​​may include a first value and a second value. More specifically, the first value may be a value corresponding to a customer's purchase conversion, and the second value may be a value corresponding to a customer's purchase conversion not occurring.

[0220] That is, the value corresponding to a specific category (539) can be configured to have a first value and a second value depending on whether a customer's purchase conversion has occurred.

[0221] Furthermore, a specific category (539) may correspond to a "target category" that the learning target prediction model seeks to predict. For example, the present invention analyzes the influence of multiple categories (e.g., 501 to 538 and 540 to 550, etc.) based on a specific category (539), thereby identifying the characteristics of successful sales cases.

[0222] However, in the present invention, the specific category is not necessarily limited to the "39th category (539)." For example, the specific category and the values ​​corresponding to the specific category may vary depending on the purpose or use of the prediction system (100), and multiple specific categories may be specified, not just one.

[0223] In one embodiment, let us assume that the 43rd category (e.g., “customer_idx_count”, 543) representing customer loyalty and purchase intent is specified as a specific category. In this case, the first value and the second value corresponding to the 39th category (539) and the first value and the second value corresponding to the 43rd category (543) specified as a specific category may be different from each other. The first value corresponding to the 43rd category (543) may be a value corresponding to a case where the customer has a high purchase intent (e.g., “1” for high purchase intent), and the second value may be a value corresponding to a case where the customer has a low purchase intent (e.g., “0” for low purchase intent).

[0224] Meanwhile, the control unit (140) can analyze the learning data set (410) to form a plurality of different sub-data sets.

[0225] Here, “analyzing the learning data set (410)” can be understood as understanding multiple records (or data) included in the learning data set (410) and determining (or analyzing) what value each record has based on the understanding result.

[0226] As we have seen above, a record can exist containing data values ​​corresponding to each category.

[0227] Specifically, the control unit (140) can analyze a plurality of records included in the learning data set (410) based on a specific category (539), and, based on the analysis result, classify a first record including a first value for the specific category (539) among the plurality of records, and a second record including a second value for the specific category (539).

[0228] In one embodiment, let us assume that there are a total of “59299” data included in the learning data set (410). Based on the analysis results for the learning data set (410), the control unit (140) can classify, among the multiple records included in the learning data set (410), a record (e.g., “4850 records”) including a first value for a specific category (539) as a first record, and a record (e.g., “54449 records”) including a second value for a specific category (539) as a second record.

[0229] Next, the control unit (140) can calculate (or produce) the ratio of the first record and the second record included in the learning data set (410) based on the classified first record and the second record.

[0230] More specifically, the control unit (140) can specify the number of classified first records and second records, respectively, and calculate (or produce) the ratio of the first records and second records included in the learning data set (410) based on the specified number.

[0231] In one embodiment, the control unit (140) can specify the number of classified first records as “4850” and the number of classified second records as “54449”, and based on the specified numbers of first and second records, calculate the ratio of the first records (e.g., 8.18%) and the ratio of the second records (e.g., “91.82%). In this case, the total ratio of the first and second records can be understood as “1:11”.

[0232] Furthermore, the control unit (140) can determine the number of different sub-data sets to be included in each of the first and second records based on the ratio (or coefficient) of the specified first and second records.

[0233] Here, the number of different sub-data sets can be determined based on the number of second records including the second value for a specific category (539) and the number of first records including the first value for a specific category (539) among the total number of records included in the learning data set (410).

[0234] More specifically, the number of different sub-data sets can be determined based on the number of second records containing a second value for a particular category (539) divided by the number of first records containing a first value for a particular category (539).

[0235] The control unit (140) can determine the number of different sub-data sets based on the value obtained by dividing the number of second records including the second value for a specific category (539) included in the learning data set (410) by the number of first records including the first value for the specific category (539). For example, as illustrated in FIG. 6, among the total number of records (ex: “59299”) included in the learning data set (600), assume that the number of first records including the first value for the specific category (539) is “4850” and the number of second records including the second value for the specific category (539) is “54449.” The number of different sub-data sets (601 to 611) can be determined as “11” based on the value obtained by dividing the number of second records (e.g., “54449”) containing the second value for a specific category (539) by the number of first records (e.g., “4850”) containing the first value for a specific category (539).

[0236] Meanwhile, the control unit (140) can include at least some of the plurality of records in each of the plurality of different sub-data sets (601 to 611) so that the composition ratio of the first record and the second record included in the learning data set (600) satisfies a preset composition ratio criterion.

[0237] Here, the preset composition ratio criterion may relate to ensuring that, in each of the plurality of different sub-data sets (601 to 611), the number of first records containing a first value for a specific category (539) and the number of second records containing a second value for the specific category (539) have the same ratio.

[0238] That is, the control unit (140) can configure a plurality of different sub-data sets (601 to 611) having the same ratio (e.g., balanced) of the number of first records and second records containing different values ​​for a specific category (539).

[0239] First, the control unit (140) can include a first record containing a first value for a specific category (539) in each of a plurality of different sub-data sets (601 to 611).

[0240] At this time, the control unit (140) can include the first record in each of a plurality of different sub-data sets (601 to 611) while maintaining the original number of first records containing the first value for a specific category (539).

[0241] More specifically, the control unit (140) may include the first record in each of a plurality of different sub-data sets (601 to 611) while maintaining the original number (e.g., “4850”) of the first records having the first value for the specific category (539) so that all of the records included in the learning data set (600) having the first value for the specific category (539) are included.

[0242] In this case, it can be understood that each of the multiple different sub-data sets (601 to 611) contains the same first record.

[0243] Next, the control unit (140) can include a portion of a second record including a second value for a specific category (539) among the records included in the learning data set (600) in each of a plurality of different sub-data sets (601 to 611).

[0244] Here, the number of second records included in each of the plurality of different sub-data sets (601 to 611) can be determined based on the number of first records included in each of the plurality of different sub-data sets (601 to 611).

[0245] The control unit (140) can include a portion of the second record in each of the plurality of different sub-data sets (601 to 611) so that the number of second records is equal to the number of first records included in each of the plurality of different sub-data sets (601 to 611).

[0246] In this case, for each of the plurality of different sub-data sets (601 to 611), different second records are extracted in the number of first records included in each of the plurality of different sub-data sets (601 to 611), and the extracted different second records can be included in each of the plurality of different sub-data sets (601 to 611).

[0247] In one embodiment, in the process of extracting a portion of the second record, the control unit (140) may extract each of the different second records a number of times corresponding to the number of the plurality of sub-data sets (601 to 611) (e.g., “11”). The number of each of the different second records may correspond to the number of first records included in each of the plurality of different sub-data sets (601 to 611), and the control unit (140) may include each of the different second records in each of the plurality of different sub-data sets (601 to 611).

[0248] That is, each of the multiple different sub-data sets (601 to 611) may include as many different second records as there are first records.

[0249] However, while the above-described embodiment describes the process of configuring (or determining) "11" different sub-data sets, the number of sub-data sets in the present invention is not necessarily limited to this. The number of sub-data sets may vary depending on the total number of records included in the learning data set, or the ratio (or number) of the first and second records.

[0250] In one embodiment, let us assume that the total number of records included in the learning data set is “60,000”, and among them, the number of first records including the first value for a specific category (539) is “8,000”, and the number of second records including the second value for the specific category (539) is “52,000”. The control unit (140) may determine the number of different sub-data sets to be “7” based on a value obtained by dividing the number of second records including the second value for the specific category (539) (e.g., “52,000”) by the number of first records including the first value for the specific category (539) (e.g., “8,000”).

[0251] In another embodiment, let us assume that the total number of records included in the learning data set is “50,000”, and among them, the number of first records including the first value for a specific category (539) is “3,000”, and the number of second records including the second value for the specific category (539) is “47,000”. The control unit (140) may determine the number of different sub-data sets to be “16” based on a value obtained by dividing the number of second records including the second value for the specific category (539) (e.g., “47,000”) by the number of first records including the first value for the specific category (539) (e.g., “3,000”).

[0252] In this way, the present invention can construct different sub-data sets, each with a first record and a second record containing different values ​​and having the same ratio, and each sub-data set can be independently used for model training. This allows the present invention to address data imbalance issues, prevent the model from overfitting to specific classes, and improve the model's predictive performance.

[0253] Meanwhile, in the present invention, a process of training a learning target prediction model for each different sub-data set can be performed (S330, see FIG. 3).

[0254] Furthermore, in the present invention, a process of obtaining multiple learned prediction models, each learned using different sub-data sets, based on learning can be performed (S340, see FIG. 3).

[0255] As discussed above, the present invention may include at least one prediction model that is the subject of learning. For example, the prediction system (100) may include at least one of the first model (121), second model (122), and third model (123) that are the subjects of learning.

[0256] In this case, the learning target prediction model may be a prediction model based on the Gradient Boosting Decision Tree (GBDT) algorithm. However, the learning method according to the present invention is not necessarily limited to prediction models based on the GBDT algorithm, and can be applied to a variety of models.

[0257] As illustrated in FIGS. 6 and 7, the control unit (140) can process a plurality of different sub-data sets (601 to 611) as inputs to each of the plurality of prediction models (121, 122, 123) in order to independently train each of the plurality of prediction models (121, 122, 123).

[0258] Specifically, the control unit (140) can train a plurality of prediction models (121, 122, 123) for each of a plurality of different sub-data sets (601 to 611). In this case, each of the plurality of prediction models (121, 122, 123) can receive a plurality of different sub-data sets (601 to 611) as input and perform training on each of the plurality of different sub-data sets (601 to 611).

[0259] In one embodiment, each of the first model (121), the second model (122), and the third model (123) can independently perform learning on each of a plurality of different sub-data sets (601 to 611).

[0260] The control unit (140) trains a plurality of prediction models for each of a plurality of different sub-data sets (601 to 611), and when the training of the plurality of prediction models (121, 122, 123) is completed, a plurality (e.g., N) of trained prediction models trained with each of the plurality of different sub-data sets (601 to 611) can be obtained (see FIG. 4).

[0261] The control unit (140) can obtain a plurality of learned prediction models (e.g., “33”), which is the product of the number N (e.g., “11”) of different sub-data sets (601 to 611) and the number M (e.g., “3”) of multiple prediction models (121, 122, 123), by learning each of the plurality of prediction models (121, 122, 123) for each different sub-data set (601 to 611).

[0262] First, when a plurality of different sub-data sets (601 to 611) are input to the first model (121), the first model (121) can perform learning for each of the plurality of different sub-data sets (601 to 611). In this case, the control unit (140) can obtain a plurality of learned prediction models (e.g., 11) each learned with the plurality of different sub-data sets (601 to 611) as learning results for each of the plurality of different sub-data sets (601 to 611) of the first model (121). For example, as illustrated in FIGS. 7 and 8, the control unit (140) can train the first model (121) for each of the first sub-data set (ex: “DS 1 (Balanced Data Set 1)”, 601), the second sub-data set (ex: “DS 2 (Balanced Data Set 2)”, 602) to the Nth sub-data set (ex: “DS 2 (Balanced Data Set 2)”, or the 11th sub-data set, 611), thereby obtaining a plurality of trained prediction models (121a, 121b, 121c) trained with the first sub-data set (601), the second sub-data set (602) to the Nth sub-data set (611), respectively.

[0263] In addition, when a plurality of different sub-data sets (601 to 611) are input to the second model (122), the second model (122) can perform learning for each of the plurality of different sub-data sets (601 to 611). In this case, the control unit (140) can obtain a plurality of learned prediction models (e.g., 11) each learned with the plurality of different sub-data sets (601 to 611) as learning results for each of the plurality of different sub-data sets (601 to 611) of the second model (122). For example, the control unit (140) can train the second model (122) for each of the first sub-data set (ex: “DS 1 (Balanced Data Set 1)”, 601), the second sub-data set (ex: “DS 2 (Balanced Data Set 2)”, 602) to the Nth sub-data set (ex: “DS 2 (Balanced Data Set 2)”, or the 11th sub-data set, 611), thereby obtaining a plurality of trained prediction models (122a, 122b, 122c) trained with the first sub-data set (601), the second sub-data set (602) to the Nth sub-data set (611), respectively.

[0264] Furthermore, when a plurality of different sub-data sets (601 to 611) are input to the third model (123), the third model (123) can perform learning for each of the plurality of different sub-data sets (601 to 611). In this case, the control unit (140) can obtain a plurality of learned prediction models (e.g., 11) each learned with the plurality of different sub-data sets (601 to 611) as learning results for each of the plurality of different sub-data sets (601 to 611) of the third model (123). For example, the control unit (140) can train the third model (123) for each of the first sub-data set (ex: “DS 1 (Balanced Data Set 1)”, 601), the second sub-data set (ex: “DS 2 (Balanced Data Set 2)”, 602) to the Nth sub-data set (ex: “DS 2 (Balanced Data Set 2)”, or the 11th sub-data set, 611), thereby obtaining a plurality of trained prediction models (123a, 123b, 123c) trained with the first sub-data set (601), the second sub-data set (602) to the Nth sub-data set (611), respectively.

[0265] That is, when the training of each of the multiple prediction models (121, 122, 123) for each of the N different sub-data sets is completed, each of the multiple prediction models (121, 122, 123) may include multiple learned prediction models trained for each of the N sub-data sets. In this case, it can be understood that the multiple learned prediction models are included as many times as the product of the number N of different sub-data sets and the number M of the multiple prediction models.

[0266] Through the process discussed above, the control unit (140) can obtain a plurality of learned prediction models (e.g., 33) equal to the number of “11” different sub-data sets (601 to 611) multiplied by the number of “3” multiple prediction models (121, 122, 123).

[0267] However, the number of multiple learned prediction models obtained may vary depending on the number of sub-data sets (N) and the number of prediction models (M).

[0268] In one embodiment, assume that the number of different sub-data sets is "20" and the number of multiple prediction models is "2." In this case, the number of learned prediction models obtained may be "40."

[0269] In another embodiment, assume the number of different sub-data sets is "10" and the number of multiple prediction models is "5." In this case, the number of learned prediction models obtained may be "50."

[0270] In this way, the present invention maximizes data diversity and improves model generalization performance by independently training each model for each different sub-dataset. In other words, the process described above can contribute to reducing model overfitting and improving generalization performance.

[0271] Meanwhile, in the present invention, a process of inputting input data to be predicted into each of a plurality of learned prediction models can be performed (S350, see FIG. 3).

[0272] Furthermore, in the present invention, a process of obtaining multiple prediction values ​​for input data from each of multiple learned prediction models can be performed (S360, see FIG. 6).

[0273] In this case, the input data inputted into the learned model may exist in various forms depending on the purpose or use of the prediction system (100). In the present invention, the purpose of utilizing the prediction system (100) is related to the field of "marketing and / or business," and therefore, the explanation will be given on the premise that input data related to the field of "marketing and / or business" is input.

[0274] As illustrated in FIGS. 8 and 9, the control unit (140) can process at least one input data (ex: “Input Data”, 810) as input to each of a plurality of learned prediction models (121a, 121b, 121c, 122a, 122b, 122c, 123a, 123b, 123c). Here, the input data (810) may include, for example, i) categorical data representing the customer's occupation (ex: “customer_job”, 811), ii) a variable representing the marketing channel from which sales opportunity information was collected (ex: “lead_from_channel”, 812), iii) text data containing requirements (or needs) or interests directly written by the customer (ex: “lead_description”, 813), iv) text data representing the customer's interest or participation (ex: “lead_desc_length”, 814), v) a variable representing the profit ratio generated from a specific vertical (ex: “prefer_ver_mean”, 815), vi) a variable representing the upper category of a product requested by the customer (ex: “product_category”, 816), vii) a variable representing the subcategory of a product requested by the customer (ex: “product_subcategory”, 817), viii) a variable representing the model name of a specific product requested by the customer (ex: “product_modelname”, 818) may include at least one of the following.

[0275] However, the information contained in the input data is not limited to the examples mentioned above and may include various other data. For example, the input data may further include customer MQL data and / or customer lead data. As another example, the input data may further include data related to the various categories (501 to 550) discussed above (see Figure 5).

[0276] The control unit (140) can obtain multiple prediction values ​​for input data from each of multiple learned prediction models (121a, 121b, 121c, 122a, 122b, 122c, 123a, 123b, 123c).

[0277] More specifically, the control unit (140) can obtain a plurality of prediction values ​​output from each of a plurality of learned prediction models (121a, 121b, 121c) obtained through learning of the first model (121), a plurality of learned prediction models (122a, 122b, 122c) obtained through learning of the second model (122), and a plurality of learned prediction models (123a, 123b, 123c) obtained through learning of the third model (123).

[0278] In one embodiment, when input data (810) is input to each of a plurality of learned prediction models (121a, 121b, 121c) obtained through learning of the first model (121), as illustrated in FIGS. 8 and 9, each of the plurality of learned prediction models (121a, 121b, 121c) can output a prediction value for the input data (810). In this case, the prediction model (or first model, 121a) learned with the first sub-data set (601) can output a first prediction value (901), the prediction model (121b) learned with the second sub-data set (602) can output a second prediction value (902), and the prediction model (121c) learned with the Nth sub-data set (611) can output an Nth prediction value (903).

[0279] In another embodiment, when input data (810) is input to each of a plurality of learned prediction models (122a, 122b, 122c) obtained through learning of the second model (122), each of the plurality of learned prediction models (122a, 122b, 122c) can output a prediction value for the input data (810). In this case, the prediction model (or second model, 122a) learned with the first sub-data set (601) can output a first prediction value (911), the prediction model (122b) learned with the second sub-data set (602) can output a second prediction value (912), and the prediction model (122c) learned with the Nth sub-data set (611) can output an Nth prediction value (913).

[0280] In another embodiment, when input data (810) is input to each of a plurality of learned prediction models (123a, 123b, 123c) obtained through learning the third model (123), each of the plurality of learned prediction models (123a, 123b, 123c) can output a prediction value for the input data (810). In this case, the prediction model (or third model, 123a) learned with the first sub-data set (601) can output a first prediction value (921), the prediction model (123b) learned with the second sub-data set (602) can output a second prediction value (922), and the prediction model (123c) learned with the Nth sub-data set (611) can output an Nth prediction value (923).

[0281] In this case, the number of multiple prediction values ​​(901, 902, 903, 911, 912, 913, 921, 922, 923) obtained from multiple learned prediction models (121a, 121b, 121c, 122a, 122b, 122c, 123a, 123b, 123c) may correspond to a value obtained by multiplying the number N of different sub-data sets by the number M of multiple prediction models. For example, the control unit (140) may obtain multiple prediction values ​​in the number corresponding to the product of “11” of the number of different sub-data sets (601 to 611) and “3” of the number of multiple prediction models (121, 122, 123) (ex: “33”).

[0282] Meanwhile, in the present invention, a process of specifying a final prediction value for input data can be performed using multiple prediction values ​​(S370, see FIG. 3).

[0283] The control unit (140) can use the output of at least one learned prediction model to specify a final prediction value for the input data (810).

[0284] Each of the plurality of learned prediction models (121a, 121b, 121c, 122a, 122b, 122c, 123a, 123b, 123c) discussed above can be configured to predict a value for a specific category (539). For example, each of the plurality of learned prediction models (121a, 121b, 121c, 122a, 122b, 122c, 123a, 123b, 123c) can predict whether a customer will make a purchase when input data is entered.

[0285] Specifically, the control unit (140) can specify a final prediction value for the input data (810) by using a plurality of prediction values ​​(901, 902, 903, 911, 912, 913, 921, 922, 923) obtained from each of a plurality of learned prediction models (121a, 121b, 121c, 122a, 122b, 122c, 123a, 123b, 123c).

[0286] First, the control unit (140) can perform soft voting based on multiple prediction values ​​(901, 902, 903, 911, 912, 913, 921, 922, 923) to specify the final prediction value (see FIG. 6).

[0287] Here, soft voting can be understood as one of the ensemble techniques, a technique that determines the final prediction by averaging the results (or classes) independently predicted by multiple artificial intelligence models.

[0288] The control unit (140) can calculate (or produce) an averaged probability (or purchase conversion probability, sales conversion probability, final prediction probability, etc.) by performing an average on a plurality of predicted values ​​(901, 902, 903, 911, 912, 913, 921, 922, 923) based on soft voting.

[0289] Here, the averaged probability is a result of synthesizing multiple prediction values ​​output by multiple learned prediction models, and can be understood as representing the likelihood of a customer purchasing a product or service (purchase conversion likelihood) as a probability value. For example, the control unit (140) can express the probability value as a value between 0 and 1. In this case, a value of 0.7 may indicate that the customer has a 70% chance of purchasing the product.

[0290] Furthermore, the control unit (140) can specify the final predicted value (or sales conversion, purchase conversion, customer conversion, etc.) based on the averaged probability.

[0291] Here, the final predicted value is the final extracted prediction result, which can be understood as a binary classification representing whether or not the customer will purchase the product or service. For example, the control unit (140) can express "purchased (1)" if the customer is predicted to purchase the product, and "not purchased (0)" if the customer is predicted not to purchase the product.

[0292] In this case, the control unit (140) can compare the averaged probability with a preset threshold value, and if the averaged probability satisfies (or exceeds) the preset threshold value, specify the final predicted value as “purchased (1)”, and if it does not satisfy (or exceed) the threshold value, specify the final predicted value as “not purchased (0)”.

[0293] The control unit can specify the final predicted value for the input data (810) based on the mathematical formula (930) shown in FIG. 9.

[0294] At this time, let's assume that the sales conversion probability is calculated as "0.7 (70%)" and the preset threshold value is set to "0.65 (65%) or more." The control unit (140) can determine whether the averaged probability (e.g., "70%") satisfies the preset threshold value (e.g., "65% or more").

[0295] In one embodiment, as a result of the judgment, the control unit (140) can specify the final predicted value (ex: “sales conversion predicted”, 940) as “purchased (1)” based on the averaged probability (ex: “70%”) satisfying a preset threshold (ex: “65% or more”).

[0296] In another embodiment, let's assume that the averaged probability is calculated as "0.6 (60%)". As a result of the judgment, the control unit (140) can specify the final predicted value (ex: "Sales conversion prediction", 940) as "Do not purchase (0)" based on the fact that the averaged probability (ex: "60%") does not satisfy the preset threshold value (ex: "65% or more").

[0297] In this way, by combining the output values ​​of multiple models, the present invention can offset the prediction errors of individual models and improve overall prediction accuracy. This allows for more accurate predictions of customer conversion likelihood, thereby enhancing the effectiveness of marketing and sales strategies.

[0298] That is, in the present invention, by averaging the prediction results of multiple learned models to produce a final prediction value, uncertainty that may arise when relying on a single model can be reduced, and the characteristics of each model can be utilized to the maximum extent to provide an optimized prediction result.

[0299] Meanwhile, in the inference stage, the valid customer prediction method of the prediction system according to the present invention, as illustrated in FIG. 10, includes the steps of receiving (or inputting) prediction target customer data that is the target of prediction from a user terminal (S1010), inputting the prediction target customer data into each of a plurality of prediction models that are each trained with different sub-data sets divided based on purchasing customer data among a learning data set composed of purchasing customer data and non-purchasing customer data (S1020), obtaining a plurality of prediction values ​​representing the probability that a customer corresponding to the prediction target customer data is a valid customer as outputs of each of the plurality of prediction models (S1030), using the plurality of prediction values ​​to specify a final prediction value for the prediction target customer data (S1040), and using the specified final prediction value to provide information to the user terminal as to whether the customer corresponding to the prediction target customer data is a valid customer (S1050), thereby making it possible to predict whether a customer related to the customer data input by the user will purchase the company's product or service.

[0300] Here, a valid customer (or valid customer) can mean a customer who has a clear demand for a specific product or service of a specific company and is likely to purchase it.

[0301] In one embodiment, as illustrated in FIG. 11, when the control unit (140) receives prediction target customer data that is the target of prediction from the user terminal (10), the control unit (140) can input the prediction target customer data into each of a plurality of prediction models that are each trained with different sub-data sets divided based on the purchasing customer data among the learning data sets composed of purchasing customer data and non-purchasing customer data.

[0302] In addition, the control unit (140) obtains a plurality of prediction values ​​representing the probability that a customer corresponding to the target customer data for prediction is a valid customer as an output of each of the plurality of prediction models, and can use the plurality of prediction values ​​to specify a final prediction value for the target customer data for prediction.

[0303] Furthermore, the control unit (140) may provide information to the user terminal (10) as to whether the customer corresponding to the target customer data to be predicted is a valid customer using a specified final prediction value. For example, as illustrated in FIG. 11, the control unit (140) may provide prediction results (1021, 1022, 1023) as to whether the customer (or customer company, U1, U2, U3) related to the customer data (1020) input by the user will purchase a specific product (ex: “PuriCare Object Collection Water Purifier”) of a specific company through a service page (1000) output to the user terminal (10).

[0304] In this case, the first customer (U1) can be understood as having a very high purchase conversion rate for a specific product (1010) with a purchase probability of “80%,” and the third customer (U3) can be understood as having a low purchase conversion rate for a specific product (1010) with a purchase probability of “30%.”

[0305] Meanwhile, the present invention can equally divide the entire data set into preset sizes, construct multiple distinct sub-data sets based on index information, and train a model using the multiple distinct sub-data sets constructed based on the index. Below, a more detailed description will be given with reference to the attached drawings. Figures 12 and 13 are conceptual diagrams illustrating a prediction system according to another embodiment of the present invention.

[0306] FIGS. 14A and 14B are flowcharts for explaining a learning method of a prediction system according to another embodiment of the present invention. FIGS. 15A and 15B are conceptual diagrams for explaining a learning data set according to another embodiment of the present invention. FIGS. 16A and 16B are conceptual diagrams for explaining an embodiment of distinguishing a plurality of records included in a learning data set according to another embodiment of the present invention. FIG. 17 is a conceptual diagram for explaining a learning method of a prediction system according to another embodiment of the present invention. Furthermore, FIGS. 18 and 19 are conceptual diagrams for explaining an embodiment of efficiently processing data by utilizing an index of a data sequence unit in the present invention.

[0307] Meanwhile, as illustrated in FIG. 12, a prediction system (100) according to another embodiment of the present invention may include at least one of an input unit (110), an output unit (120), a communication unit (130), a storage unit (140), a data collection unit (150), a data processing unit (160), a model unit (170), a prediction unit (180), and a control unit (190).

[0308] Although not shown, the prediction system (100) according to the present invention may include one or more processors, which may include one or more general-purpose processors and / or one or more special-purpose processors (e.g., a digital signal processor, a tensor processing unit (TPU), a graphics processing unit (GPU), a neural network processing unit (NPU), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a quantum processing unit (or quantum processor, QPU), etc.). The one or more processors may be configured to execute instructions stored (or included) in the storage unit (140), computer-readable instructions, and / or other instructions described herein. The prediction system and the control method thereof according to the present invention may allow the memory and at least one processor to cooperate to perform data processing as described below. The processor may perform a series of operations and data processing using data and information stored in the memory. At this time, the memory may be a component of the storage unit (140).

[0309] In addition, the prediction system (100) according to the present invention can perform data processing and calculation processes using quantum gates, quantum entanglement, and quantum superposition states, taking into account implementation in a quantum computer environment. For example, the present invention can perform parallel operations based on qubits, and such quantum operations can operate complementarily with existing classical computers.

[0310] These quantum computers may include qubit-based parallel computing and high-speed data processing devices leveraging quantum entanglement, enabling hardware-based computational optimization using FPGAs and ASICs. Furthermore, quantum computers can utilize quantum processors capable of qubit-based parallel computing, and data processing efficiency can be improved through hybrid architectures with existing classical computers.

[0311] Meanwhile, the input unit (110) may be configured as a means for data input and may be configured in various types. For example, the input unit (110) may be configured to receive user input. The input unit (110) may be configured to receive user input from a user terminal (10). Here, “receiving input” may mean receiving an input signal (or selection signal) corresponding to the user’s input based on the input being made by the user through the input unit configuration provided in the user terminal (10).

[0312] Here, the user terminal (10) may include at least one of a mobile phone, a smart phone, a notebook computer, a laptop computer, a slate PC, a tablet PC, an ultrabook, a desktop computer, a digital broadcasting terminal, a personal digital assistant (PDA), a portable multimedia player (PMP), a navigation device, and a wearable device (e.g., a smartwatch, a smart glass, a head mounted display (HMD)).

[0313] In addition, in the present invention, the input unit (110) does not necessarily mean a hardware means, and can be understood as a passage for receiving input from a user.

[0314] The input unit (110) may also be referred to as a user interface module. The input unit (110) may include a touch screen, computer mouse, keyboard, keypad, touch pad, trackball, joystick, voice recognition module, or other similar device. However, the present invention does not limit the type of input unit (110).

[0315] Here, the user input may include documents, text, images (or videos), voice, etc. In this case, the prediction system (100) may further include a module that converts voice into text.

[0316] Next, the output unit (120) can output information through an output unit configuration (e.g., a display unit, a touch screen, a speaker, etc.) provided in a user terminal (10) linked with a prediction system (100) according to the present invention. For example, the output unit (120) can output a page (or service page, 1000) linked with the prediction system (100) according to the present invention to the display unit of the user terminal (10). In addition, the output unit (120) does not necessarily mean a hardware means, and can be understood as a passage for outputting results to the user.

[0317] Next, the communication unit (130) may be connected to a user terminal (10), a server (e.g., a server linked to the system (100), a central server, an external server, etc.), a device, and at least one network through a wireless or wired network, and may be configured to receive or transmit overall data and information necessary for the operation of the prediction system (100) according to the present invention.

[0318] The communication unit (130) can support various communication methods depending on the communication standards of the communicating device.

[0319] For example, the communication unit (130) may be configured to communicate with a communication target using at least one of WLAN (Wireless LAN), Wi-Fi (Wireless-Fidelity), Wi-Fi (Wireless Fidelity) Direct, DLNA (Digital Living Network Alliance), WiBro (Wireless Broadband), WiMAX (World Interoperability for Microwave Access), HSDPA (High Speed ​​Downlink Packet Access), HSUPA (High Speed ​​Uplink Packet Access), LTE (Long Term Evolution), LTE-A (Long Term Evolution-Advanced), 5G (5th Generation Mobile Telecommunication), Bluetooth™, RFID (Radio Frequency Identification), Infrared Data Association (IrDA), UWB (Ultra-Wideband), ZigBee, NFC (Near Field Communication), Wi-Fi Direct, and Wireless USB (Wireless Universal Serial Bus) technologies.

[0320] Next, the storage unit (140, or memory) serves to store various data related to the present invention, and may include one or more non-transitory computer-readable storage media that can be read and / or accessed by at least one of the one or more processors.

[0321] The one or more computer-readable storage media may include volatile and / or non-volatile storage components, such as optical, magnetic, organic, or other memory or disk storage devices. In some examples, the storage (140) may be implemented using a single physical device (e.g., a single optical, magnetic, organic, or other memory or disk storage device), while in other examples, the storage (140) may be implemented using two or more physical devices.

[0322] The storage unit (140) may include computer-readable instructions and additional data. The storage unit (140) may include storage necessary to perform at least some of the methods, scenarios, and techniques described herein and / or at least some of the functions of the devices and networks.

[0323] Furthermore, at least a portion of the storage unit (140) may be a cloud storage or a cloud server. The storage unit (140) may store at least a portion of data corresponding to user input received from the input unit (110) and a training data set (or training data).

[0324] That is, it can be understood that the storage unit (140) is sufficient as a space in which information necessary for the operation of the prediction system (100) according to the present invention is stored, and there are no restrictions on the physical space.

[0325] Furthermore, the storage unit (140) may store a computer program including computer program instructions. Furthermore, the storage unit (140) may store a computer program including computer program instructions that control the operation of the system (100) or control the operation of the control unit (180) when loaded onto the processor of the system (100).

[0326] Next, the data collection unit (150) may be configured to perform the role of collecting data required for the answer generation system (100) according to the present invention from various sources (e.g., database (DB), website, API, server linked to the prediction system (100), central server, external server, cloud storage, user terminal (10), etc.).

[0327] The data collection unit (150) can collect various data used for training at least one model (171, 172, 173) included in the model unit (170). For example, as illustrated in FIG. 13, the data collection unit (150) can collect a training data set (200) to be used for training the at least one model (171, 172, 173) through various sources. In this case, the training data set (200) can be configured to include MQL (Marketing Qualified Lead) data configured to have values ​​for a plurality of different categories.

[0328] Here, MQL data can be understood as information on potential customers (or clients) selected through marketing activities, used to identify potential customers who have shown interest in a product or service or have the potential to purchase it. Marketing qualified lead data may include various elements related to the customer's record and / or behavior of interest in a product or service. For example, various elements may include at least one of the following: customer (or client) information (e.g., customer's name, customer's account (or customer identification number or code), contact information, email address, job title, location information, country of affiliation, affiliated company (or company), etc.), company information to which the customer belongs (e.g., company name, industry, size, etc.), type and / or category of products or services in which the customer has shown interest, customer event records (e.g., customer's website visit history (or number of visits), customer's purchase history, product page views, product inquiries, survey responses, etc.), and information related to the customer's purchase intent (e.g., information regarding expected budget, expected purchase timing, etc.).

[0329] However, the collected data is not necessarily limited to the examples mentioned above. In one embodiment, in addition to MQL data, the data processing unit (160) may collect at least one of product data (e.g., product identification information (or code), product name and description, product price, inventory status, product category, product rating and review, product launch date, product specifications and features, etc.), sales process data (e.g., lead information, sales representative information, sales opportunity information, sales activity records, sales stages, contract information, performance indicators, etc.), and market trend data (e.g., market research reports, competitor information, industry trends, consumer behavior, economic indicators, technology trends, regional (or national) characteristics and regulatory information, etc.). For convenience of explanation, the collected data will be described below by naming them all as “learning data sets (or learning data)” without distinguishing them separately.

[0330] Next, the data processing unit (160) may be configured to perform preprocessing on the data collected from the data collection unit (150). The data processing unit (160) may perform preprocessing on the learning data set (200).

[0331] The data processing unit (160) can clean the learning data set (200), process errors or missing values ​​in the learning data set (200), and detect (or identify) and remove outliers or duplicate records (or data). For example, the data processing unit (160) can replace missing values ​​in the learning data set (200) with an average value or delete them, and detect and remove outliers (e.g., outliers) with abnormally large or small values ​​and duplicate records.

[0332] Additionally, if the learning data set (200) includes categorical data (or variables), the data processing unit (160) can convert the categorical data into a numerical form that can be understood by the artificial intelligence model. For example, the data processing unit (160) can convert the categorical data into a multidimensional vector using at least one of one-hot encoding and / or label encoding.

[0333] In addition, the data processing unit (160) can adjust the range of numerical data (or continuous data) so that all variables have the same range. For example, the data processing unit (160) can convert numerical data into data with a mean of 0 and a variance of 1 through normalization (e.g., Z-Score normalization) for numerical data, or can convert continuous data into data between 0 and 1 through scaling (e.g., Min-Max scaling) for continuous data. This can be understood as data processing to prevent the results from being distorted by the size of a specific variable during AI model learning or to prevent the AI ​​model from being biased toward a specific feature.

[0334] Furthermore, the data processing unit (160) can expand (or augment) data that an artificial intelligence model can learn by creating new variables (derived variables) from the learning data set through feature engineering for the learning data set in which existing variables have been preprocessed.

[0335] In this case, the data processing unit (160) can generate derived variables (or derived categories) from the learning data set (200) based on RFM (Recency-Frequency-Monetary) analysis during the feature engineering process.

[0336] RFM analysis is a marketing method used to evaluate and segment customers. It can include recency, frequency, and monetary values. Recency refers to the time elapsed since a customer's last purchase, frequency refers to the number of purchases a customer makes over a given period, and monetary values ​​can refer to the total amount a customer spends over a given period.

[0337] In one embodiment, the data processing unit (160) may extract specific data (or variables, for example, sales representative (ex: “lead_owner”), customer identification information (ex: “customer_idx”), etc.) with high feature importance from the learning data set (200) based on RFM analysis, and generate derived variables for the extracted specific data (for example, a variable indicating the experience level or frequency of the representative (ex: “lead_owner_job”), a variable indicating whether or not the customer has made a repeat purchase (ex: “customer_idx_count”), a variable combining the experience of the sales representative and the frequency of the customer’s return visit (ex: “oppty”), etc.).

[0338] In another embodiment, the data processing unit (160) may separate year and month information using date data (ex: “lead_date”) included in the learning data set (200) and generate a derived variable (ex: “lead_date_yearmonth”) that includes the customer’s recent purchasing activity.

[0339] Meanwhile, the data processing unit (160) may assign an index to each of the plurality of records (or data) included in the learning data set (200). Here, the index may refer to identification information (or identifier, reference value, etc.) assigned to uniquely identify or reference each of the plurality of records included in the learning data set. It may be configured to include identification information in the form of a unique identifier (ID) generated based on the order, position, unique key, etc. of the corresponding record, or a numeric value, character value, or hash value.

[0340] Such an index may be assigned to each of the plurality of records included in the learning data set (200) once the collection of the learning data set (200) is completed. Alternatively, once the classification of each of the plurality of records according to the target category is completed, the index may be assigned to correspond to each classified record. In this case, the index may be utilized (or used, utilized, etc.) to distinguish each record or construct a sub-data set. That is, the index may be matched with a record and stored in a predetermined storage (e.g., a storage unit (140) or a memory, etc.), and may be usefully used in performing record reconstruction, model learning, or data set construction for evaluation in a subsequent processing step.

[0341] Meanwhile, the data processing unit (160) can configure at least one sub-data set using the learning data set (200).

[0342] For example, a training dataset may be imbalanced, with significantly more or fewer instances of certain values ​​than instances of others. This can lead to AI models being biased toward frequently occurring classes. For example, in MQL data, instances of customer conversions are often less frequent than instances of non-conversions. This can lead to data imbalance, negatively impacting the learning and prediction performance of AI models.

[0343] To address this data imbalance problem, the data processing unit (160) may configure multiple different sub-data sets, each of which has a composition ratio of multiple records containing different values ​​for a target category (or specific category) among the multiple data (or records) included in the learning data set (200), satisfying a preset composition ratio criterion. More specific details regarding this will be described later.

[0344] Meanwhile, the model unit (170) may be configured to include at least one learning target prediction model. For example, the model unit (170) may include at least one of the first model (171), the second model (172), and the third model (173) that are learning targets.

[0345] The first model (171), also known as the "CatBoost model," may be a model specialized for processing categorical data (or variables or features). The first model (171) can prevent target leakage problems that may occur in categorical data by utilizing a regularization technique (or method) called "Ordered Target Statistics" and / or "Ordered Boosting." Furthermore, the first model (171) can utilize a symmetric tree structure to distribute balanced data at each level of the tree. This first model (171) can prevent overfitting and achieve high predictive performance.

[0346] The second model (172) may also be referred to as a “LightGBM (LGBM) model” and may be a model that maximizes learning speed and reduces memory usage while maintaining high prediction performance by using the “Gradient-based One-Side Sampling (GOSS)” and / or “Exclusive Feature Bundling (EFB)” methods. Gradient-based One-Side Sampling (GOSS) reduces the amount of computation by sampling data according to the size of the gradient, and Exclusive Feature Bundling (EFB) can reduce the number of variables by bundling rare features. In addition, the second model (172) can learn deeply about a specific part of the data by using a leaf-wise tree growth method and can better identify complex data patterns.

[0347] The third model (173) may also be named an “XGBoost model” and may be a model based on the Gradient Boosting Decision Tree (GBDT) algorithm optimized for high predictive performance and overfitting prevention. The third model (173) may utilize regularization to prevent overfitting and tree pruning to reduce model complexity by removing unnecessary branches. The third model (173) provides flexibility in missing value processing and, through a level-wise tree growth method, can perform broad learning so that various characteristics are well reflected by evenly dividing all nodes.

[0348] As seen above, the first model (171), the second model (172), and the third model (173) may be models based on the Gradient Boosting Decision Tree (GBDT) algorithm, and may divide data and perform learning based on the decision tree.

[0349] However, the models included in the model section (170) of the present invention are not necessarily limited to the models mentioned above, and may further include various models. In the present invention, the number of models included in the model section (170) may be one or more depending on the case, and the number of models included in the model section (170) may vary.

[0350] Meanwhile, a plurality of different sub-data sets (211, 212, 213) generated from the data processing unit (160) can be input to each of the models (171, 172, 173) included in the model unit (170). In this case, the plurality of models (171 172, 173) can each receive a plurality of different sub-data sets (211, 212, 213) as input and perform learning on each of the plurality of different sub-data sets (211, 212, 213).

[0351] Specifically, the first model (171), the second model (172), and the third model (173) independently perform learning for each of a plurality of different sub-data sets (211, 212, 213), and when learning of each model (171 172, 173) is completed, a plurality of learned prediction models can be obtained.

[0352] Here, the term “multiple learned prediction models” may include learned prediction models corresponding to the number of “N” multiplied by the number of “M” multiplied by ...

[0353] That is, when the training of each of the multiple prediction models (171 172, 173) for each of the N different sub-data sets is completed, each of the multiple prediction models ((171 172, 173)) may include multiple learned prediction models trained for each of the N sub-data sets. In the present invention, the prediction model may also be named a “Binary Classification model” or a BalancedTreeMarketer model.

[0354] Meanwhile, the prediction unit (180) can be configured to specify a final prediction result (or final prediction value) using the output value of at least one learned prediction model (or N learned models).

[0355] Specifically, the prediction unit (180) can perform soft voting based on multiple prediction values ​​output from each of multiple learned prediction models to determine the final prediction value.

[0356] In one embodiment, the prediction unit (180) calculates (or produces) an averaged probability (or sales conversion probability, 220) by averaging a plurality of prediction values ​​(or prediction probabilities) independently predicted by a plurality of learned prediction models based on soft voting, and can specify a final prediction value (or sales conversion prediction, or customer conversion, etc., 230) based on the calculated sales conversion probability (220).

[0357] Here, soft voting can be understood as one of the ensemble techniques, a technique that determines a final prediction by combining (or averaging) the results (or probabilities) independently predicted by multiple artificial intelligence models. That is, the prediction unit (180) can determine the final prediction result (or predicted value) by combining the results (or predicted values) output from each of the multiple learned prediction models.

[0358] Additionally, the averaged probability is a result of synthesizing the predicted values ​​output by trained prediction models, and can be understood as representing the likelihood of a customer purchasing a product or service (purchase conversion likelihood) as a probability value. For example, the prediction unit (130) can express the probability value as a value between 0 and 1. In this case, a value of 0.7 may indicate that a customer has a 70% chance of purchasing the product.

[0359] Furthermore, the final prediction value (230) is the final extracted prediction result, which can be understood as a binary classification representing whether or not a customer will purchase a product or service. For example, the prediction unit (180) can express "purchased (1)" if the customer is predicted to purchase the product, and "not purchased (0)" if the customer is predicted not to purchase the product.

[0360] That is, the prediction unit (180) can compare the sales conversion probability (220) with a preset threshold value, and if the sales conversion probability (220) exceeds the preset threshold value, specify the final prediction value (230) as “purchased (1)”, and if it does not exceed the preset threshold value, specify the final prediction value as “not purchased (0)”. More specific details about this will be described later.

[0361] Next, the control unit (190) may perform a role of controlling the overall operation of the prediction system (100) related to the present invention. The control unit (190) may process signals, data, information, etc. input or output through the components of the prediction system (100) discussed above, or perform a series of data processing to provide or process appropriate information and functions to the user. The control unit (190) may be physically implemented by the processor discussed above.

[0362] In one embodiment, the control unit (190) may provide a service page (1000) to the user terminal (10). The service page (1000) may provide a list (or customer list, 1020) of at least one company that interacts (e.g., conducts transactions, collaborates, etc.) with a specific company. In this case, the control unit (190) may provide information on the purchase probability of each customer company of a specific product (e.g., “PuriCare Object Collection Water Purifier”) sold by a specific company, predicted by the prediction system (100), in one area of ​​the service page (1000).

[0363] Meanwhile, the present invention aims to address the problem of imbalanced data and provide a prediction system, its control method, and a learning method for the prediction system that can be applied universally across various industries. More specifically, the present invention aims to provide a prediction system capable of predicting effective customers by analyzing various customer data. Below, we will examine the learning method for the prediction system (or prediction model) in more detail.

[0364] In the present invention, a process of specifying a learning data set configured to include a plurality of records having values ​​for a plurality of different categories may be performed (S310, see FIG. 14a).

[0365] The control unit (190) can specify a learning data set (400) to be used for learning the learning target prediction model (S401, see FIG. 4).

[0366] In this case, the criteria (or method, method, etc.) for specifying the learning data set (400) may vary. The control unit (190) may specify the learning data set (400) to be used for learning the target prediction model based on various criteria.

[0367] In one embodiment, the control unit (190) may collect (or receive) a data set from at least one of various sources (e.g., a database (DB), a website, an API, a server linked to the prediction system (100), a central server, an external server, and a cloud storage), and may specify the collected data set as a learning data set (400) to be used for learning the prediction model to be learned.

[0368] In another embodiment, the control unit (190) may specify a data set stored in at least one of various storages (e.g., storage unit (140, or memory), storage server, etc.) as a learning data set (400) to be used for learning the target prediction model.

[0369] The learning data set (400) may include various types of data. For example, the learning data set (400) may include at least one of MQL data, product data, sales process data, and market trend data. The data included in the learning data set may be in the form of at least one of numerical data, categorical data, and text data. However, the form of data included in the learning data set is not necessarily limited to the examples mentioned, and it is obvious that the data may include other types of data.

[0370] Such a learning data set (400) may be configured to include a plurality of records having values ​​for a plurality of different categories.

[0371] Here, a record represents at least one data unit, and a record may contain multiple categories of data values ​​(i.e., multiple fields or properties). In a database, this can also be called a "row." For example, in an Excel sheet, each row represents a record, and each column within a row can represent data values ​​for various categories within the record.

[0372] That is, each piece of data included in a data set or a data unit containing multiple categories of data values ​​can be called a “record” or “sample.”

[0373] The learning data set (400) may include MQL data configured to have values ​​for multiple different categories. Furthermore, in the present invention, categories may also be referred to as "categories," "features," "variables," or "elements."

[0374] Before examining the preprocessing process for the learning data set (400), let us first look at the multiple categories (401 to 450) included in the learning data set (400) and the values ​​for the multiple categories (401 to 450) with reference to FIGS. 15a and 15b.

[0375] The first category (e.g., “ID”, 401) is an arbitrary value that uniquely identifies each data entry, and its primary purpose may be to calculate an f1 score by comparing it with the 39th category (e.g., “is_converted, 439). The first category (401) can be used to measure accuracy by matching each predicted result with the actual result, and the first category (401) can also be used to evaluate model performance.

[0376] The second category (e.g., "bant_submit", 402) is a variation of the BANT (Budget, Authority, Need, Timeline) framework, which can be used to assess MQL quality. Here, "Budget" refers to the customer's budget, indicating the funds available for allocation to projects or purchases. "Authority (Title)" refers to the customer's position / rank, indicating whether the person has decision-making authority. Additionally, "Needs" can refer to the customer's specific needs, problems, or goals that the product or service should address. "Timeline" can refer to the customer's desired delivery date.

[0377] The third category (e.g., "customer_country", 403) represents the customer's nationality, and its value can be expressed as " / region / country (e.g., Asia / Korea)". This third category (403) can provide valuable information for regional sales strategies, localized service offerings, and approaches based on legal and cultural understanding. Furthermore, this third category (403) can be utilized to develop strategies that take into account time differences, language barriers, and cultural differences that may arise in international business relationships.

[0378] The fourth category (ex: “customer_country.1”, 404) can mean a region or country, such as the corporate region of the company in charge.

[0379] The fifth category (e.g., “business_unit”, 405) refers to the business unit within the company that corresponds to the product or service requested in the MQL, and can be divided into multiple categories (e.g., (ID, AS, IT, Solution, CM)). These categories are important for understanding the nature of the lead and assigning the appropriate sales team or specialist, and can be used for performance analysis, resource allocation, and strategy development for each business unit.

[0380] The sixth category (ex: “com_reg_ver_win_rate”, 406) is a weight calculated by calculating the opportunity rate based on a specific business area (Vertical Level 1), a specific business unit, or a region, and can be used to predict the possibility of future success based on past success rates.

[0381] The seventh category (e.g., "customer_idx", 407) stores the customer company name and the number of times the company submitted data, which can indirectly indicate the customer company's level of engagement or interest. A high value indicates that the company frequently inquires or interacts with you, potentially indicating a high level of interest or purchase intent. For example, the seventh category (407) can be used for customer segmentation, prioritization, and developing customized marketing strategies.

[0382] The 8th category (ex: “customer_type”, 408) is data that classifies the customer’s occupation and can be useful for developing target marketing or customized sales strategies.

[0383] The 9th category (ex: “enterprise”, 409) indicates the size of the customer company, which can be divided into enterprises and small and medium-sized businesses (SMB).

[0384] The 10th category (e.g., “historical_existing_cnt”, 410) can refer to the number of times that the customer (or company) has successfully converted sales in the past. The 10th category (410) can be useful for assessing customer loyalty or the likelihood of repeat purchases. A high value can be understood as indicating a strong business relationship with the customer and a high possibility of future transactions.

[0385] The eleventh category (e.g., "id_strategic_ver", 411) may include a weighted value indicating the strategic importance of a combination of a specific business unit (BU) and a specific business area (Vertical Level 1). The eleventh category (411) can be utilized to optimize resource allocation by reflecting the company's strategic priorities and to increase focus in specific business areas.

[0386] Similar to Category 11 (411), Category 12 (e.g., “it_strategic_ver”, 412) may include weighted values ​​indicating the strategic importance of a combination of a specific Business Unit (BU) and a specific Business Area (Vertical Level 1). This weighting is for a specific Business Unit (e.g., IT Business Unit), enabling efficient technical staffing and planning.

[0387] The 13th category (e.g., “idit_strategic_ver”, 413) may refer to a composite indicator that integrates the 11th category (411) and the 12th category (412). If at least one of the 11th category (411) and / or the 12th category (412) has a value of 1, the 13th category (413) may be assigned a weighting of 1. The 13th category (413) provides an integrated strategic importance encompassing the ID and IT domains and may be utilized as a consideration factor when determining enterprise-wide resource allocation.

[0388] The 14th category (e.g., "customer_job", 414) can represent categorical data representing occupational groups. This 14th category (414) allows for the adoption of communication methods that consider occupational characteristics and allows for the grouping of customers based on occupation.

[0389] Category 15 (e.g., "lead_desc_length", 415) could refer to the total length of the customer's lead description text. This could indirectly indicate the customer's level of interest or engagement, or reflect the complexity of their needs or issues.

[0390] The 16th category (e.g., "inquiry_type", 416) can contain information categorizing the customer's inquiry type. For example, the 16th category (416) can be divided into multiple categories (e.g., 71) including product information inquiries, purchase consultations, and quotation requests. This can help identify the customer's purchasing stage and serve as a key element in developing marketing strategies. Furthermore, the 16th category (416) can assist in sales conversion by assigning appropriate departments or personnel based on the inquiry type.

[0391] The 17th category (e.g., "product_category", 417) may represent the parent category of the requested product. For example, the 17th category (417) may be divided into multiple categories (e.g., 357) including tablets, TVs, washing machines, refrigerators, etc. This allows for the development of marketing strategies focused on customer-desired categories.

[0392] The 18th category (e.g., "product_subcategory", 418) can further categorize the requested product's subcategories. For example, the 18th category (418) can be divided into multiple (e.g., 330) subcategories, such as OLED, QLED, and 8K TVs, allowing for a more detailed product classification system. This allows for the identification of precise customer needs and more targeted marketing.

[0393] Category 19 (e.g., "product_modelname", 419) could refer to the specific product model name requested by the customer. For example, since the customer provides very specific information, it allows us to accurately understand their interests. Based on this, we can create customized proposals and develop personalized sales approaches. This can ultimately increase customer satisfaction and sales conversion rates.

[0394] The 20th category (e.g., "customer_position", 420) can indicate the customer's position within the company. This can help determine the customer's level of authority in making purchasing decisions. Furthermore, the 20th category (420) can be a crucial factor in developing differentiated sales and marketing strategies based on position.

[0395] The 21st category (e.g., "response_corporate", 421) can be a string data type that indicates the name of the company's legal entity responsible for customer inquiries or transactions. This category (421) can play a crucial role in corporate structures with multiple subsidiaries. By identifying which legal entity is primarily involved in customer response or sales processes, this category can clarify responsibilities across internal organizations and maintain consistency in customer management. Furthermore, this category can provide insights necessary for analyzing the performance of each legal entity, optimizing resource allocation, and developing company-wide sales strategies.

[0396] Category 22 (e.g., "expected_timeline", 422) can indicate the deadline requested by the customer. This can be utilized as an important indicator in predictive models, as a customer's specific schedule may signal a strong purchase intent. Furthermore, the urgency of Category 22 (422) can be used to estimate the likelihood and speed of a deal. For example, a short deadline may imply quick decision-making and a high conversion rate, while a long deadline may indicate a larger deal or a complex decision-making process. Effectively utilizing Category 22 (422) can help optimize the sales team's resource allocation and develop a customized customer approach strategy. In other words, Category 22 (422) can be a factor contributing to increased B2B sales conversion rates.

[0397] Category 23 (e.g., "ver_cus", 423) can quantify the impact of a combination of a specific business area and customer type on sales conversion in B2B sales. A weight of 1 can be assigned to a category that belongs to a specific business area and also has an end-user customer type. This allows for assessing the likelihood of success for sales targeting direct end users in a specific business area. Category 23 (423) reflects the importance of customer segmentation in B2B sales strategies and can help identify business areas where an end-user-centric approach may be more effective.

[0398] The 24th category (e.g., "ver_pro", 424) can be used to weight combinations of specific business areas (Vertical Level 1) and product types (Product Category). This category (424) can be used to determine whether a specific product type has a higher sales conversion rate in a specific business area. A combination weighted by 1 may indicate that the product type is competitive or in high demand in that business area. This category (424) can be used to identify product groups that should be focused on for each business area and to develop customized sales strategies.

[0399] Category 25 (e.g., "ver_win_rate_x", 425) can be a composite weighted category that simultaneously considers the relative importance and success rate of each vertical. It is calculated by multiplying the percentage of total leads accounted for by the vertical's sales conversion success rate within that vertical. Category 25 (425) allows for a more balanced evaluation by considering not only the success rate but also the overall weight of the vertical. This allows for a more accurate assessment of the true importance of each vertical when allocating sales resources and developing strategies.

[0400] Category 26 (e.g., "ver_win_ratio_per_bu", 426) can represent the sales conversion success rate for each business unit (or division) within a specific business area. This can demonstrate how effectively each business unit is performing sales in a specific vertical. Category 26 (426) can be used to identify which specific business units are performing best within each vertical, which can be utilized to optimize process sharing and resource allocation within the organization. Furthermore, based on Category 26 (426), customized sales strategies can be developed that leverage the strengths of each business unit.

[0401] Category 27 (e.g., "business_area", 427) can represent a client's main business area. Category 27 (427) can play a key role in predicting B2B sales conversion rates. By understanding a client's business area through Category 27 (427), you can develop a customized approach tailored to that industry. Furthermore, Category 27 (427) can be used to analyze past success patterns in specific business areas to optimize sales strategies for new clients in similar industries. This can lead to efficient allocation of sales resources and improved conversion rates.

[0402] Category 28 (e.g., "business_subarea", 428) can be a more specific category that categorizes a client's business areas. Category 28 (428) can help you more accurately identify a client's specific needs and requirements. Utilizing Category 28 (428) in a predictive model can enable highly segmented market access. Based on Category 28 (428), you can develop more sophisticated sales strategies and increase conversion rates.

[0403] The 29th category (e.g., "lead_owner", 429) can represent the name of the sales representative responsible for each opportunity. This category (429) can play a crucial role in analyzing individual and team performance in predictive models. Furthermore, this category can be used to understand the impact of a specific sales representative's sales skills, experience, or industry expertise on conversion rates. Furthermore, this category can be used to develop optimal lead allocation strategies and analyze collaboration patterns among team members to improve overall sales performance.

[0404] The 30th category (e.g., "lead_date", 430) can indicate the date a sales opportunity (lead) was first created. The 30th category (430) can be crucial for considering temporal factors in predictive models. Furthermore, the 30th category (430) can be used to analyze the time it takes from lead generation to actual deal closure, seasonal trends, and performance changes over a specific period. Furthermore, the 30th category (430) can be used to understand the impact of a lead's recency on conversion rates, enabling the development of timely and effective follow-up strategies. This can optimize the sales cycle and increase conversion rates.

[0405] The 31st category (e.g., "lead_from_channel", 431) may represent the marketing channel from which sales opportunity information was collected. This 31st category (431) can be crucial for evaluating the effectiveness of each marketing channel in a predictive model. By analyzing the quality and conversion rate of leads acquired through a specific channel based on this 31st category (431), the most effective marketing channel can be identified. Furthermore, based on this 31st category (431), marketing budget allocation can be optimized and customized sales strategies can be developed for each channel. Consequently, lead quality can be improved and overall sales conversion rates can be increased.

[0406] The 32nd category (e.g., "event_name", 432) may be a category that indicates the name of a specific marketing event where a sales activity occurred. This 32nd category (432) can be crucial for evaluating the effectiveness of each marketing event in a predictive model. By analyzing the quality and conversion rate of leads generated through a specific event based on this 32nd category (432), the most successful event types can be identified. Furthermore, based on this 32nd category (432), future marketing event planning and resource allocation can be optimized, and customized follow-up sales strategies tailored to the characteristics of each event can be developed. Consequently, event ROI can be improved and overall sales conversion rates can be increased.

[0407] The 33rd category (e.g., "prefer_ver_count", 433) can represent the distribution ratio of cases converted from a specific business unit across a specific business area. This category (433) can be crucial for identifying the strengths of each business unit in a predictive model. By analyzing which verticals a specific business unit demonstrates high success rates based on this category, the most effective target market for each business unit can be identified. This allows for the development of specialized strategies for each business unit. Consequently, by maximizing the strengths of each business unit, the overall sales conversion rate can be improved.

[0408] The 34th category (e.g., "prefer_ver_mean", 434) is calculated based on similar criteria as the 33rd category (433), but can represent a percentage of profit values ​​instead of a simple sample count. The 34th category (434) is crucial for identifying areas of profitability strength for each business unit in a predictive model. By analyzing which verticals generate high profits for a specific business unit based on the 34th category (434), strategies can be developed that consider actual profit contributions, rather than simply the number of successful cases. This allows for focused sales activities on high-profit verticals and improved overall operating profitability.

[0409] Category 35 (e.g., "transfer_agreement", 435) can indicate whether a customer has consented to the export of their lead information overseas. Category 35 (435) can be a valuable indicator in predictive models for assessing a customer's openness and willingness to collaborate globally. Customers who consent to information export are more likely to be interested in a broader range of services or global solutions. Based on Category 35 (435), customized proposals for products or services requiring international collaboration can be made and utilized in developing global sales strategies.

[0410] The 36th category (e.g., "ver_win_rate_mean_upper", 436) can be expressed as 1 if the average of each vertical is exceeded, and 0 otherwise. The 36th category (436) can be crucial for evaluating relative performance within each vertical in a predictive model. By analyzing the characteristics of cases that perform above average based on the 36th category (436), key elements of successful sales strategies can be identified. This allows best practices to be applied to other cases to improve overall sales performance.

[0411] Category 37 (e.g., "expected_budget", 437) can represent a customer's desired budget range. Category 37 (437) can be a key indicator for assessing a customer's purchasing intent and project scale in a predictive model. Based on Category 37 (437), appropriate products or services can be suggested based on budget size, and customized solutions can be developed that meet the customer's financial expectations. Furthermore, Category 37 (437) can be used to identify optimal target segments by analyzing conversion rates by budget range, optimizing resource allocation and improving overall sales performance. Notably, Category 37 (437) can be the category responsible for "Money" when applying a traditional RFM model.

[0412] Category 38 (e.g., "lead_description", 438) can contain customer-specific requirements. Category 38 (438) can play a key role in identifying specific customer needs and interests in predictive models. By analyzing Category 38 (438) using text mining and natural language processing (NLP), potential customer needs and preferences can be identified. Based on Category 38 (438), customized proposals can be created and personalized sales approaches developed. This can ultimately lead to increased customer satisfaction and improved sales conversion rates.

[0413] The 39th category (e.g., "is_converted", 439) is a key category representing the final outcome of a sales activity. It can represent sales success (or failure) as a binary value (e.g., 1: success, 0: failure). This can be the target category (or specific category) that we ultimately aim to predict in the predictive model. Based on the 39th category (439), we can analyze the impact of various categories and identify the characteristics of successful sales cases. Furthermore, the 39th category (439) can be used to evaluate the predictive accuracy of the predictive model and facilitate continuous model improvement and optimization. Consequently, by accurately predicting the 39th category (439), we can support efficient allocation of sales resources and strategic decision-making, thereby improving overall B2B sales performance.

[0414] The 40th category (e.g., “len_expected_timeline”, 440) may be a derived category generated during the preprocessing of the 22nd category (422). Based on the 40th category (440), the data consistency problem of the 22nd category (422) can be resolved.

[0415] Category 41 (e.g., "countrycoinside", 441) can be a derived category indicating whether the customer's nationality and the regional information (continent) based on the company's legal entity name are the same. Based on Category 41 (441), sales strategies can be developed that take regional characteristics into account.

[0416] Category 42 (e.g., "lead_owner_job", 442) may be a derived category created from Category 29 (429) to quantify the experience and proficiency of salespeople in a B2B sales environment. The frequency with which salespeople appear in the data set is counted, with higher frequency indicating greater sales caseloads. Based on Category 42 (442), experienced representatives can be assigned to important leads or complex cases, optimizing resource allocation. This can ultimately improve customer satisfaction and sales conversion rates.

[0417] Category 43 (e.g., "customer_idx_count", 443) can be an important indicator (or derived category) of customer loyalty and purchase intent. The number of appearances for each customer in Category 7 (407) was counted, with a high count indicating a customer's frequent inquiries for transactions. This indicates ongoing interest in the product or service and can reflect the strength of their potential purchase intent. Category 43 (443) can be used to identify key targets for building long-term business relationships and indicate a high likelihood of future purchases of the company's various products.

[0418] The 44th category (e.g., "oppty", 444) can be a derived category designed to predict sales conversion rates in a B2B sales environment. This category (444) extends the frequency concept of the traditional RFM model, combining the experience of a salesperson (e.g., "lead_owner_job") with the frequency of customer revisits (e.g., "customer_idx_count"). By quantifying and calculating the synergy between experienced salespeople and loyal customers, this category can enable more accurate sales performance predictions that go beyond simple transaction frequency and consider the qualitative aspects of sales relationships.

[0419] The 45th category (e.g., “vertical_level”, 445) can be used to identify and weight strategically important verticals within each business area. The 45th category (445) can be a derived category created by analyzing existing weighting variables, such as the 11th category (411), the 12th category (412), and the 23rd category (423). In certain industries, data that is not weighted by these weighting variables can be considered less strategically important verticals within that industry. Based on this logic, the 45th category (445) can filter out vertical data that is not strategically important and assign additional weight to data corresponding to important verticals. This can effectively identify the most promising verticals within each business area and develop customized sales strategies accordingly, contributing to improved overall business performance.

[0420] Category 46 (e.g., “weight_expected_timeline”, 446) can be a derived category used to predict the progress of customer transactions, serving as a key indicator in the B2B sales process. The original data for Category 46 (446) included email addresses and consultation content unrelated to the actual timeline. However, considering the nature of B2B business, where the absence of a clear timeline agreement reduces the likelihood of actual transactions, Category 46 (446) was improved. Specifically, a weighting method was applied to data containing words indicating dates or periods. This approach assigns greater importance to data likely to contain actual timeline information, enabling more accurate predictions of sales conversion potential. This approach can enhance the efficiency of the B2B sales process and contribute to the development of more accurate sales strategies.

[0421] The 47th category (e.g., "qcut", 447) is a method of dividing numerical data into intervals based on quantiles. Traditional RFM models use qcut to divide data into a specific number of groups, ensuring that each group contains an equal number of data points. This ensures that each group reflects the unique characteristics of the data points. The appropriate number of groups was determined by visualizing the importance of variables. The eight derived categories created using qcut were created by partitioning various numerical data into multiple groups with equal frequency. The partitioning ensures that the number of data points in each group is approximately equal. This is a methodology frequently used in traditional RFM models. This approach minimizes the influence of extreme values ​​and allows for effective comparison of characteristics between groups. Based on the results of visualized analysis of variable importance, each group was divided into an appropriate number of groups. This method allows for more clearly revealing the unique characteristics of each group and, while preserving the characteristics of continuous variables, leverages the advantages of categorical data, making it flexible for various analysis techniques.

[0422] The 48th category (e.g., “lead_date_yearmonth”, 448) can be a time-based variable (derived category) created by combining the year and month of the customer lead generation time. The process of creating the 48th category (448) is as follows. The 30th category (430) was grouped into various time units such as month, year, half-year, and quarter, and then analyzed. Among the various time units, the form combining year and month showed the highest correlation and was selected. This can reflect a company’s business cycle. The yearly element takes into account the fact that a company’s product lineup or strategy changes depending on the year, and the monthly element reflects the tendency for a customer’s purchasing cycle or budget execution pattern to be concentrated in a specific month. The 48th category (448) can more accurately capture customer behavior patterns over time and provide useful insights for establishing marketing strategies for each period.

[0423] The 49th category (e.g., “second_event”, 449) may be a derived category created to independently utilize important information extracted from the existing 32nd category (432). The 32nd category (432) may have a structure such as “(business_unit)(second_event)(lead_from_channel)(date).” In this structure, all elements except “second_event” already existed as individual variables. However, “second_event” was the only one not expressed as an independent variable. The “event_name” variable is composed of a combination of four elements, and as a result of the various values ​​of each element, it has the characteristic of being highly distributed overall. This can make it difficult to find meaningful patterns during data analysis or modeling. Therefore, by extracting “second_event” as a separate variable, important information can be utilized more effectively. This can contribute to more accurately reflecting the characteristics of the data and increasing the accuracy of the analysis.

[0424] The 50th category (e.g., "is_fresh", 450) can be a derived category created to improve customer segmentation accuracy. The 50th category (450) can categorize customers into types such as completely new customers, customers who have previously inquired but not yet completed a transaction, and customers with past transaction experience. This segmentation can provide valuable insights into sales strategy development, as each customer type requires a different approach and has different chances of success. In particular, the second type of customer may have different needs and expectations than completely new customers, so categorizing them separately can aid in effective customer management.

[0425] As discussed above, the learning data set (400) may include MQL data configured to have values ​​for multiple different categories (401 to 450) and multiple categories (401 to 450).

[0426] Meanwhile, the control unit (190) can perform preprocessing on the learning data set (400).

[0427] First, the control unit (190) can purify the learning data set (400), process errors or missing values ​​in the learning data set (400), and detect and remove outliers or duplicate records.

[0428] In one embodiment, the control unit (190) can replace missing values ​​in the learning data set (400) with an average value or delete them, and identify and remove outliers and duplicate records.

[0429] And, when categorical data is included in the learning data set (400), the control unit (190) can convert the categorical data into numeric data that can be understood by the prediction model.

[0430] In one embodiment, the control unit (190) may use at least one of one-hot encoding and / or label encoding to convert a specific category (e.g., “is_converted”, 439) into numeric data that a prediction model can understand (e.g., “1” for purchase, “0” for non-purchase).

[0431] Furthermore, the control unit (190) can adjust the range of the numeric data and / or continuous data when the learning data set (400) includes at least one of numeric data and / or continuous data.

[0432] In one embodiment, the control unit (190) can convert numeric data into data with a mean of 0 and a variance of 1 through Z-Score normalization for numeric data, or can convert continuous data into data between 0 and 1 through Min-Max scaling for continuous data.

[0433] Meanwhile, the control unit (190) can create a new category (or variable or data) from the learning data set (400) through a feature engineering process for the learning data set (400).

[0434] Specifically, the control unit (190) performs feature engineering on a learning data set (400) whose existing categories have been preprocessed (e.g., refined, normalized, scaled, etc.), so as to generate derived categories using at least some of the categories and values ​​corresponding to at least some of the categories among the plurality of categories included in the learning data set (400).

[0435] Here, “creating a derivative category” can be understood as extracting additional information (or meaning) from an existing category (or original category) or creating a new category (derived category).

[0436] First, the control unit (190) can generate a derived variable for at least one category among a plurality of categories by considering the domain (see FIG. 4).

[0437] Specifically, the control unit (190) can generate a derived category using at least one category among multiple categories and a value corresponding to said at least one category based on specific domain knowledge (or an analysis technique specialized for a specific domain). In this case, the control unit (190) can understand which categories are important and which combinations are meaningful through specific domain knowledge.

[0438] In one embodiment, as illustrated in FIGS. 15A and 15B , the control unit (190) may generate a derived category (e.g., “lead_date_yearmonth”, 448) using an existing category (e.g., “lead_date”, 430) and a value corresponding to the existing category (e.g., “2024-08-09”) based on expert knowledge of a specific domain (e.g., a marketing domain). The derived category (448) may be understood as a category utilized to analyze lead data at a specific point in time.

[0439] And, the control unit (190) can specify a value corresponding to a derived category based on the fact that a derived category is generated from an existing category. For example, the control unit (190) can specify a value (ex: “2024-08”) corresponding to the derived category (ex: “lead_date_yearmonth”, 448) based on the fact that a derived category (ex: “lead_date”, 430) is generated from an existing category and a value (ex: “2024-08-09”) corresponding to the existing category.

[0440] In addition, as discussed above, the present invention can generate derived categories from a learning data set (400) based on RFM (Recency-Frequency-Monetary) analysis.

[0441] More specifically, the control unit (190) can extract at least one category with high feature importance and a value corresponding to at least one category from the learning data set (400) based on RFM analysis, and generate a derived category using the extracted category and the value corresponding to the extracted category.

[0442] In one embodiment, as illustrated in FIGS. 15A and 15B, the control unit (190) may extract a seventh category (ex: “customer_idx”, 407) having high feature importance and a value corresponding to the seventh category (407) (ex: “CompanyA-1”), a 29th category (ex: “lead_owner”, 429) and a value corresponding to the 29th category (429) (ex: “John Doe”), based on RFM analysis, from the learning data set (400), and may generate a derived category using each of the extracted categories (407, 429) and the value corresponding to each of the extracted categories. In this case, at least one derived category can be created among a 42nd category indicating the experience level or frequency of the representative (e.g., “lead_owner_job”, 442), a 43rd category indicating whether or not the customer makes a repeat purchase (e.g., “customer_idx_count”, 443), and a 44th category combining the sales representative’s experience and the customer’s return visit frequency (e.g., “oppty”, 444).

[0443] And, the control unit (190) can specify a value corresponding to (or corresponding to) a derived category. For example, the control unit (190) can specify a value corresponding to the 42nd category (ex: “lead_owner_job”, 442) (ex: “25”), a value corresponding to the 43rd category (ex: “customer_idx_count”, 443) (ex: “10”), and a value corresponding to the 44th category (ex: “oppty”, 444) (ex: “0.85”).

[0444] Through this, the learning data set (400) may further include derived categories and values ​​corresponding to the derived categories generated through the derived variable generation process (or feature engineering).

[0445] In this way, the present invention can improve the performance of a prediction model by generating new derived variables from existing data, thereby allowing the prediction model to learn meaningful patterns.

[0446] Meanwhile, in the present invention, a process of distinguishing multiple records included in a learning data set based on a value corresponding to a target category among multiple categories may be performed (S320, see FIG. 14a).

[0447] The control unit (190) can distinguish multiple records included in the learning data set (400) based on a value (Value) corresponding to a target category among multiple categories (401 to 450).

[0448] To this end, the control unit (190) can specify a target category that serves as a criterion for distinguishing a plurality of records included in the learning data set (400) among a plurality of categories (401 to 450).

[0449] Here, the target category may be a category indicating whether a customer has converted to a purchase. The control unit (190) may specify the "39th category (e.g., "is_converted", 439)", which corresponds to a category indicating whether a customer has converted to a purchase, among multiple categories (401 to 450), as the target category. For convenience of explanation, the specified "39th category (439)" will be referred to as the "target category (439)" below. In the present invention, the target category may also be referred to as a "specific category."

[0450] As discussed above, the target category (439) is a category that represents the final result of sales activities, and can express whether sales are successful (e.g., when sales goals such as contract signing and / or product purchase are achieved) as a binary value (e.g., success is “1”, failure is “0”).

[0451] In this case, the target category (439) can be configured to have different values ​​depending on whether a customer purchase conversion has occurred.

[0452] Here, the different values ​​may include a first value and a second value. More specifically, the first value may be a value corresponding to a customer's purchase conversion (e.g., "Ture"), and the second value may be a value corresponding to a customer's purchase conversion (e.g., "False").

[0453] That is, the value corresponding to the target category (439) can be configured to have a first value and a second value depending on whether a customer purchase conversion has occurred.

[0454] Furthermore, the target category (439) may correspond to the "target category" that the learning target prediction model seeks to predict. For example, the present invention can analyze the influence of multiple categories based on the target category (439) and identify the characteristics of successful sales cases.

[0455] However, in the present invention, the target category is not necessarily limited to the 39th category (439). For example, the target category and the values ​​corresponding to the target category may vary depending on the purpose or use of the prediction system (100), and multiple target categories may be specified, not just one.

[0456] In one embodiment, let us assume that the 43rd category (e.g., “customer_idx_count”, 443) representing customer loyalty and purchase intent is specified as the target category. In this case, the first value and the second value corresponding to the 39th category (439) and the first value and the second value corresponding to the 43rd category (443) specified as the target category may be different from each other. The first value corresponding to the 43rd category (443) may be a value corresponding to a case where the customer has a high purchase intent (e.g., “1” for high purchase intent), and the second value may be a value corresponding to a case where the customer has a low purchase intent (e.g., “0” for low purchase intent).

[0457] Furthermore, in the present invention, each of the plurality of records may be assigned (or mapped, matched, set, assigned, created, included, etc.) an index corresponding to each of the plurality of records. For example, as illustrated in FIG. 16A, let us assume that a total of 59,299 records are included in the learning data set (400). When the learning data set (400) is collected, the control unit (190, or data processing unit (160)) may assign an index (e.g., index_row_1, index_row_2, index_row_3, index_row_4, index_row_5, index_row_6, index_row_7, index_row_8, index_row_9, index_row_10, etc.) to each of the plurality of records included in the learning data set (400). Information about these plurality of records and the index corresponding to each of the plurality of records may be stored in a predetermined storage (e.g., storage unit (140) or memory). Additionally, it may be understood that the learning data set (400) includes a plurality of records and an index corresponding to each of the plurality of records, and this learning data set (400) may also be stored and exist in a predetermined storage.

[0458] Meanwhile, the control unit (190) may distinguish (or classify) each of the plurality of records included in the learning data set (400) based on the values ​​each of the records includes for the target category (439) in order to configure multiple different sub-data sets. In this case, the process of distinguishing the plurality of records in the present invention may also be understood as being performed by the data processing unit (160). However, for the convenience of explanation, the following description will be made on the assumption that the process of distinguishing the plurality of records is performed by the control unit (190).

[0459] The control unit (190) can analyze the learning data set (400) to distinguish each of the multiple records included in the learning data set (400).

[0460] Here, “analyzing the learning data set (400)” can be understood as understanding a plurality of records (or data) included in the learning data set (400) and, based on the understanding result, determining (or analyzing) what value each record has for the target category (439).

[0461] As discussed above, records may contain data values ​​corresponding to each category. The control unit (190) may classify multiple records included in the learning data set (400) into different classes (or labels, groups, types, etc.) based on the values ​​each record has for the target category (439).

[0462] Specifically, the control unit (190) can analyze a plurality of records included in the learning data set (400) based on the target category (439), and, based on the analysis result, classify a first record including a first value for the target category (539) and a second record including a second value for the target category (439) among the plurality of records.

[0463] For example, as illustrated in FIG. 16A, let us assume that the total number of records included in the learning data set (400) is “59299.” Based on the analysis results for the learning data set (400), the control unit (190) can distinguish, among the multiple records included in the learning data set (400), a record (ex: “4850 records”) that includes a first value (ex: “Ture”) for the target category (439) as the first record. In addition, the control unit (190) can distinguish, as the second record, a record (ex: “54449 records”) that includes a second value (ex: “False”) for the target category (439).

[0464] In this case, the index corresponding to each of the plurality of separated records may include a first index corresponding to each of the plurality of first records (e.g., index_row_1, index_row_3, index_row_5, index_row_7, index_row_9, index_row_A, etc.) and a second index corresponding to each of the plurality of second records (e.g., index_row_2, index_row_4, index_row_6, index_row_9, index_row_11, index_row_B, etc.).

[0465] Meanwhile, in the present invention, the index assigned to each of the plurality of records may also be assigned after the classification of each of the plurality of records is completed according to the target category (439).

[0466] For example, as illustrated in FIG. 16b, let us assume that the distinction between a first record (e.g., “4850 records”) including a first value (e.g., “Ture”) for a target category (439) and a second record (e.g., “54449 records”) including a second value (e.g., “False”) for the target category (439) among a plurality of records included in a learning data set (400) has been completed. The control unit (190) may assign a preset different index to each of the plurality of classified records. Here, the preset different index may include at least one of a first index assigned to a record having a first value for the target category (439) and a second index assigned to a record having a second value for the target category (439).

[0467] Accordingly, the control unit (190) may assign a first index (e.g., true_index_row_1, true_index_row_2, true_index_row_3, true_index_row_4, true_index_row_5, true_index_row_N, etc.) to each of a plurality of first records having a first value for the target category (439), and may assign a second index (e.g., false_index_row_1, false_index_row_2, false_index_row_3, false_index_row_4, false_index_row_5, false_index_row_N, etc.) to each of a plurality of second records having a second value for the target category (439). In this case, the first index may be configured in the format of “true_index_row_...”, and the second index may be configured in the format of “false_index_row_...”. This can also be understood as the identification information for the first index assigned to correspond to the first record and the second index assigned to correspond to the second record being different. However, the format in which the indexes are structured is not necessarily limited to the examples mentioned above, and it is obvious that it can be varied in various ways by the prediction system (100) or the administrator (or user) of the prediction system (100).

[0468] In this way, once the collection of the training data set is complete, an index corresponding to each record can be assigned to each record included in the training data set. Alternatively, once the classification of each record according to the target category is complete, an index can be assigned to correspond to each classified record. However, the present invention does not limit the order in which the indexes are assigned to each record.

[0469] Furthermore, the control unit (190) may store a plurality of records classified based on values ​​corresponding to the target category (439) and an index corresponding to each of the plurality of classified records in a predetermined storage (e.g., storage unit (140) or memory, etc.). As an example, the control unit (190) may group the plurality of classified records and store them in a predetermined storage, or store them in a predetermined storage in a list format.

[0470] As discussed above, the plurality of distinct records may include a first record containing a first value for the target category (439) and a second record containing a second value for the target category (439).

[0471] The control unit (190) can match a first record and a first index corresponding to the first record and store the matched record in a predetermined storage. In addition, the control unit (190) can match a second record and a second index corresponding to the second record and store the matched record in a predetermined storage.

[0472] In one embodiment, the control unit (190) may group (or list) a plurality of first records and a first index corresponding to each of the plurality of first records and store them in a predetermined storage. In the present invention, a group including a plurality of first records and a first index matching each of the plurality of first records may also be named as a “first record group (or first group)” or a “first record list (or first list)” or a “first index group” or a “first index list”.

[0473] In another embodiment, the control unit (190) may group (or list) a plurality of second records and second indexes corresponding to each of the plurality of second records and store them in a predetermined storage. In the present invention, a group including a plurality of second records and a second index matching each of the plurality of second records may also be named a “second record group (or second group)” or a “second record list (or second list)” or a “second index group” or a “second index list”.

[0474] Meanwhile, in the present invention, a process of configuring multiple different sub-data sets can be performed based on an index corresponding to each of a plurality of separated records (S330, see FIG. 14a).

[0475] When the control unit (190) completes the classification of multiple records included in the learning data set (400) (S401), it can configure multiple different sub-data sets using the classified multiple records (S403, see FIG. 14b).

[0476] In the present invention, “configuring a plurality of different sub-data sets” can be understood as configuring each of the plurality of different sub-data sets using at least some of the plurality of records such that the composition ratio of each record containing different values ​​for the target category among the plurality of records satisfies a preset composition ratio criterion (or preset composition criterion).

[0477] The control unit (190) can configure a plurality of different sub-data sets based on a value corresponding to a target category (439) among a plurality of categories (401 to 450). More specifically, the control unit (190) can configure a plurality of different sub-data sets having a preset size based on an index corresponding to each of a plurality of records classified based on a value corresponding to the target category (439). For example, the preset size can be set to a size between 512 KB and 2 MB, taking into account both data transmission efficiency and storage space utilization. However, the preset size can be variously changed by the prediction system (100) or an administrator (or user) of the prediction system (100).

[0478] As discussed above, a predetermined storage may store a plurality of distinct records and an index corresponding to each of the plurality of distinct records. The control unit (190) may configure a plurality of different sub-data sets having a preset size based on the indexes corresponding to each of the plurality of distinct records stored in the predetermined storage. For example, the control unit (190) may configure a plurality of different sub-data sets having a preset size based on a first index corresponding to a first record and a second index corresponding to a second record stored in the predetermined storage.

[0479] In this regard, the control unit (190) can specify at least some of the plurality of separated records to be included in each of the plurality of different sub-data sets based on an index corresponding to each of the plurality of separated records, and can include at least some of the specified records in each of the plurality of different sub-data sets to form a plurality of different sub-data sets having a preset size.

[0480] Here, specifying at least a portion of the distinct plurality of records to be included in each of the plurality of different sub-data sets based on the indexes can be understood as specifying at least a portion of the first records and the second records to be included in each of the plurality of different sub-data sets based on the indexes corresponding to each of the first records and the second records. That is, it can be understood as a method of selecting at least a portion of the first records and the second records only as much as is necessary to form the plurality of different sub-data sets by utilizing the indexes corresponding to each of the first records and the second records. For example, the first index corresponding to the first record and the second index corresponding to the second record can be utilized to calculate the ratio of the first records and the second records, determine the number of sub-data sets, or determine the number of the first records and the second records to be included in each of the plurality of different sub-data sets.

[0481] First, the control unit (190) can calculate (or produce) the ratio of the first record and the second record included in the learning data set (400) (or stored in a predetermined storage) based on the distinguished first record and the second record. Alternatively, the control unit (190) can calculate the ratio of the first record and the second record stored in the predetermined storage based on the first index corresponding to each of the distinguished plurality of first records and the second index corresponding to each of the plurality of second records.

[0482] The control unit (190) can specify the number of the first and second records, respectively, and calculate the ratio of the first and second records based on the specified number.

[0483] In one embodiment, the control unit (190) may specify the number of the first records as “4850” and the number of the second records as “54449”, and, based on the specified numbers of the first and second records, may calculate the ratio of the first records (e.g., 8.18%) and the ratio of the second records (e.g., “91.82%). In this case, the total ratio of the first and second records may be understood to be “1:11”.

[0484] Furthermore, the control unit (190) can determine the number of different sub-data sets in which the first record and the second record are to be included, respectively, based on the ratio (or coefficient) of the specified first record and the second record.

[0485] Here, the number of different sub-data sets may be determined based on the number of second records containing the second value for the target category (439) and the number of first records containing the first value for the target category (439) among the total number of the plurality of separated records. Alternatively, the number of second indices corresponding to the second records and the number of first indices corresponding to the first records may be determined based on the total number of the plurality of separated records.

[0486] More specifically, the control unit (190) may determine the number of different sub-data sets based on a value obtained by dividing the number of second records containing the second value for the target category (439) by the number of first records containing the first value for the target category (439). Alternatively, the control unit (190) may determine the number of different sub-data sets based on a value obtained by dividing the number of corresponding second indexes of the second records by the number of corresponding first indexes of the first record.

[0487] For example, as illustrated in FIG. 17, among the total number (e.g., “59299”) of multiple records (first records and second records) included in the learning data set (600) (or stored in a predetermined storage), let’s assume that the number of first records including the first value for the target category (439) is determined as “4850” and the number of second records including the second value for the target category (439) is determined as “54449”. The control unit (190) may determine the number of different sub-data sets (601 to 611) as “11” based on a value obtained by dividing the number of second records (or second indexes corresponding to the second records) (e.g., “54449”) by the number of first records (or first indexes corresponding to the first record) (e.g., “4850”).

[0488] In this way, the present invention utilizes the index assigned to each record, enabling the flexible construction of sub-data sets of various combinations without physically dividing the entire dataset. In other words, the present invention can flexibly construct sub-data sets of various combinations without record duplication by referencing the index corresponding to each record.

[0489] Meanwhile, the control unit (190) can include at least some of the divided plurality of records in each of the plurality of different sub-data sets (601 to 611) so that the composition ratio of the first record and the second record satisfies a preset composition ratio standard.

[0490] Here, the preset composition ratio criterion may relate to ensuring that, in each of the plurality of different sub-data sets (601 to 611), the number of first records containing a first value for the target category (439) and the number of second records containing a second value for the target category (439) have the same ratio.

[0491] That is, the control unit (190) can configure a plurality of different sub-data sets (601 to 611) in which the number of first records and second records containing different values ​​for the target category (439) is in the same ratio (e.g., balanced). This configuration method can also be understood as an equal division method that evenly divides (or splits) data into each of the plurality of different sub-data sets (601 to 611).

[0492] First, the control unit (190) can include a first record including a first value for a target category (439) among a plurality of separated records in each of a plurality of different sub-data sets (601 to 611).

[0493] At this time, the control unit (190) can include the first record in each of a plurality of different sub-data sets (601 to 611) while maintaining the original number of first records containing the first value for the target category (439).

[0494] More specifically, the control unit (190) may include the first record in each of the plurality of different sub-data sets (601 to 611) while maintaining the original number (e.g., “4850”) of the first records having the first value for the target category (439) so that all of the first records having the first value for the target category (439) among the plurality of separated records are included.

[0495] In this case, it can be understood that each of the multiple different sub-data sets (601 to 611) contains the same first record.

[0496] Next, the control unit (190) can include a portion of a second record containing a second value for a target category (439) among the plurality of separated records in each of a plurality of different sub-data sets (601 to 611).

[0497] Here, the number of second records included in each of the plurality of different sub-data sets (601 to 611) can be determined based on the number of first records included in each of the plurality of different sub-data sets (601 to 611).

[0498] The control unit (190) can include a portion of the second record in each of the plurality of different sub-data sets (601 to 611) so that the number of second records is equal to the number of first records included in each of the plurality of different sub-data sets (601 to 611).

[0499] In this case, for each of the plurality of different sub-data sets (601 to 611), different second records are extracted in the number of first records included in each of the plurality of different sub-data sets (601 to 611), and the extracted different second records can be included in each of the plurality of different sub-data sets (601 to 611).

[0500] In one embodiment, in the process of extracting a portion of the second record, the control unit (190) may extract each of the different second records a number of times corresponding to the number of the plurality of sub-data sets (601 to 611) (e.g., “11”). The number of each of the different second records may correspond to the number of first records included in each of the plurality of different sub-data sets (601 to 611), and the control unit (190) may include each of the different second records in each of the plurality of different sub-data sets (601 to 611).

[0501] That is, each of the multiple different sub-data sets (601 to 611) may include as many different second records as there are first records.

[0502] However, while the above-described embodiment describes the process of configuring (or determining) "11" different sub-data sets, the number of sub-data sets in the present invention is not necessarily limited to this. The number of sub-data sets may vary depending on the total number of records included in the learning data set, or the ratio (or number) of the first and second records.

[0503] In one embodiment, let us assume that the total number of records included in the learning data set is “60,000”, and among them, the number of first records including the first value for the target category (439) is “8,000”, and the number of second records including the second value for the target category (439) is “52,000”. The control unit (190) may determine the number of different sub-data sets to be “7” based on a value obtained by dividing the number of second records including the second value for the target category (439) (e.g., “52,000”) by the number of first records including the first value for the target category (439) (e.g., “8,000”).

[0504] In another embodiment, let us assume that the total number of records included in the learning data set is “50,000”, and among them, the number of first records including the first value for the target category (439) is “3,000”, and the number of second records including the second value for the target category (439) is “47,000”. The control unit (190) may determine the number of different sub-data sets to be “16” based on a value obtained by dividing the number of second records including the second value for the target category (439) (e.g., “47,000”) by the number of first records including the first value for the target category (439) (e.g., “3,000”).

[0505] In this way, the present invention can construct different sub-data sets, each with a first record and a second record containing different values ​​and having the same ratio, and each sub-data set can be independently used for model training. This allows the present invention to address data imbalance issues, prevent the model from overfitting to specific classes, and improve the model's predictive performance.

[0506] Meanwhile, in the present invention, a learning target prediction model may be trained for each of a plurality of different sub-data sets (S330), and based on the learning, a process of obtaining a plurality of learned prediction models, each trained for a plurality of different sub-data sets, may be performed (S340, see FIG. 14a).

[0507] As discussed above, the present invention may include at least one prediction model that is the subject of learning. For example, the prediction system (100) may include at least one of the first model (171), second model (172), and third model (173) that are the subjects of learning.

[0508] In this case, the learning target prediction model may be a prediction model based on the Gradient Boosting Decision Tree (GBDT) algorithm. However, the learning method according to the present invention is not necessarily limited to prediction models based on the GBDT algorithm, and can be applied to a variety of models.

[0509] As illustrated in FIG. 17, the control unit (190) can process a plurality of different sub-data sets (601 to 611) as inputs to each of the plurality of learning target prediction models (171, 172, 123) in order to independently train each of the plurality of prediction models (171, 172, 173) (S405, see FIG. 14b).

[0510] Specifically, the control unit (190) can train a plurality of prediction models (171, 172, 173) for each of a plurality of different sub-data sets (601 to 611). In this case, each of the plurality of prediction models (171, 172, 173) can receive a plurality of different sub-data sets (601 to 611) as input and perform training on each of the plurality of different sub-data sets (601 to 611).

[0511] In one embodiment, each of the first model (171), the second model (172), and the third model (173) can independently perform learning on each of a plurality of different sub-data sets (601 to 611).

[0512] The control unit (190) trains a plurality of prediction models for each of a plurality of different sub-data sets (601 to 611), and when the training of the plurality of prediction models (171, 172, 173) is completed, a plurality (e.g., N) of trained prediction models trained with each of the plurality of different sub-data sets (601 to 611) can be obtained (S407, see FIG. 14b).

[0513] The control unit (190) can obtain a plurality of learned prediction models (e.g., “33”), which is the product of the number N (e.g., “11”) of different sub-data sets (601 to 611) and the number M (e.g., “3”) of multiple prediction models (171, 172, 173), by learning a plurality of prediction models (171, 172, 173) for each different sub-data set (601 to 611).

[0514] First, when a plurality of different sub-data sets (601 to 611) are input to the first model (171), the first model (171) can perform learning for each of the plurality of different sub-data sets (601 to 611). In this case, the control unit (190) can obtain a plurality (e.g., 11) learned prediction models, each learned with the plurality of different sub-data sets (601 to 611), as learning results for each of the plurality of different sub-data sets (601 to 611) of the first model (171). For example, the control unit (190) can train the first model (171) for each of the first sub-data set (601), the second sub-data set (602) to the Nth sub-data set (or the 11th sub-data set, 611), thereby obtaining a plurality of trained prediction models (171a, 171b, 171c) trained with the first sub-data set (601), the second sub-data set (602) to the Nth sub-data set (611), respectively.

[0515] In addition, when a plurality of different sub-data sets (601 to 611) are input to the second model (172), the second model (172) can perform learning for each of the plurality of different sub-data sets (601 to 611). In this case, the control unit (190) can obtain a plurality (e.g., 11) learned prediction models, each learned with the plurality of different sub-data sets (601 to 611), as learning results for each of the plurality of different sub-data sets (601 to 611) of the second model (172). For example, the control unit (190) can train the second model (172) for each of the first sub-data set (601), the second sub-data set (602) to the Nth sub-data set (or the 11th sub-data set, 611), thereby obtaining a plurality of trained prediction models (172a, 172b, 172c) trained with the first sub-data set (601), the second sub-data set (602) to the Nth sub-data set (611), respectively.

[0516] Furthermore, when a plurality of different sub-data sets (601 to 611) are input to the third model (173), the third model (173) can perform learning for each of the plurality of different sub-data sets (601 to 611). In this case, the control unit (190) can obtain a plurality (e.g., 11) learned prediction models, each learned with the plurality of different sub-data sets (601 to 611), as learning results for each of the plurality of different sub-data sets (601 to 611) of the third model (173). For example, the control unit (190) can train the third model (173) for each of the first sub-data set (601), the second sub-data set (602) to the Nth sub-data set (or the 11th sub-data set, 611), thereby obtaining a plurality of trained prediction models (173a, 173b, 173c) trained with the first sub-data set (601), the second sub-data set (602) to the Nth sub-data set (611), respectively.

[0517] That is, when the training of each of the multiple prediction models (171, 172, 173) for each of the N different sub-data sets is completed, each of the multiple prediction models (171, 172, 173) may include multiple learned prediction models trained for each of the N sub-data sets. In this case, it can be understood that the multiple learned prediction models are included as many times as the product of the number N of different sub-data sets and the number M of the multiple prediction models.

[0518] Through the process discussed above, the control unit (190) can obtain a plurality of learned prediction models (e.g., 33) equal to the product of the number of different sub-data sets (601 to 611), “11,” and the number of multiple prediction models (171, 172, 173), “3.”

[0519] However, the number of multiple learned prediction models obtained may vary depending on the number of sub-data sets (N) and the number of prediction models (M).

[0520] In one embodiment, assume that the number of different sub-data sets is "20" and the number of multiple prediction models is "2." In this case, the number of learned prediction models obtained may be "40."

[0521] In another embodiment, assume the number of different sub-data sets is "10" and the number of multiple prediction models is "5." In this case, the number of learned prediction models obtained may be "50."

[0522] In this way, the present invention maximizes data diversity and improves model generalization performance by independently training each model for each different sub-dataset. In other words, the process described above can contribute to reducing model overfitting and improving generalization performance.

[0523] Meanwhile, in the present invention, input data to be predicted is input into each of a plurality of learned prediction models, and a plurality of predicted values ​​for the input data can be obtained from each of the plurality of learned prediction models.

[0524] In this case, the input data inputted into the learned model may exist in various forms depending on the purpose or use of the prediction system (100). In the present invention, the purpose of utilizing the prediction system (100) is related to the field of "marketing and / or business," and therefore, the explanation will be given on the premise that input data related to the field of "marketing and / or business" is input.

[0525] The control unit (190) can process at least one input data as input to each of a plurality of learned prediction models (171a, 171b, 171c, 172a, 172b, 172c, 173a, 173b, 173c). Here, the input data may include at least one of, for example, i) categorical data representing the customer's occupation (e.g., “customer_job”), ii) a variable representing a marketing channel from which sales opportunity information is collected (e.g., “lead_from_channel”), iii) text data containing requirements (or needs) or interests directly written by the customer (e.g., “lead_description”), iv) text data representing the customer's interest or participation (e.g., “lead_desc_length”), v) a variable representing a profit ratio generated from a specific vertical (e.g., “prefer_ver_mean”), vi) a variable representing a parent category of a product requested by the customer (e.g., “product_category”), vii) a variable representing a subcategory of a product requested by the customer (e.g., “product_subcategory”), and viii) a variable representing a model name of a specific product requested by the customer (e.g., “product_modelname”).

[0526] However, the information contained in the input data is not limited to the examples mentioned above and may include various other data. For example, the input data may further include customer MQL data and / or customer lead data. As another example, the input data may further include data related to the various categories (401 to 450) discussed above (see Figure 5).

[0527] The control unit (190) can obtain multiple prediction values ​​for input data from each of multiple learned prediction models (171a, 171b, 171c, 172a, 172b, 172c, 173a, 173b, 173c).

[0528] More specifically, the control unit (190) can obtain a plurality of prediction values ​​output from each of a plurality of learned prediction models (171a, 171b, 171c) obtained through learning of the first model (171), a plurality of learned prediction models (172a, 172b, 172c) obtained through learning of the second model (172), and a plurality of learned prediction models (173a, 173b, 173c) obtained through learning of the third model (173).

[0529] In one embodiment, as illustrated in FIG. 17, when input data is input to each of a plurality of learned prediction models (171a, 171b, 171c) obtained through learning of a first model (171), each of the plurality of learned prediction models (171a, 171b, 171c) can output a prediction value for the input data. In this case, the prediction model (171a) learned with the first sub-data set (601) can output a first prediction value (621a), the prediction model (171b) learned with the second sub-data set (602) can output a second prediction value (621b), and the prediction model (171c) learned with the Nth sub-data set (611) can output an Nth prediction value (621c).

[0530] In another embodiment, when input data is input to each of a plurality of learned prediction models (172a, 172b, 172c) obtained through learning of the second model (172), each of the plurality of learned prediction models (172a, 172b, 172c) can output a prediction value for the input data. In this case, the prediction model (172a) learned with the first sub-data set (601) can output a first prediction value (622a), the prediction model (172b) learned with the second sub-data set (602) can output a second prediction value (622b), and the prediction model (172c) learned with the Nth sub-data set (611) can output an Nth prediction value (622c).

[0531] In another embodiment, when input data is input to each of a plurality of learned prediction models (173a, 173b, 173c) obtained through learning the third model (173), each of the plurality of learned prediction models (173a, 173b, 173c) can output a prediction value for the input data. In this case, the prediction model (173a) learned with the first sub-data set (601) can output a first prediction value (623a), the prediction model (173b) learned with the second sub-data set (602) can output a second prediction value (623b), and the prediction model (173c) learned with the Nth sub-data set (611) can output an Nth prediction value (623c).

[0532] In this case, the number of multiple prediction values ​​(621a, 621b, 621c, 622a, 622b, 622c, 623a, 623b, 623c) obtained from multiple learned prediction models (171a, 171b, 171c, 172a, 172b, 172c, 173a, 173b, 173c) may correspond to a value obtained by multiplying the number N of different sub-data sets by the number M of multiple prediction models. For example, the control unit (190) may obtain multiple prediction values ​​in the number corresponding to the product of “11” of the number of different sub-data sets (601 to 611) and “3” of the number of multiple prediction models (171, 172, 173) (ex: “33”).

[0533] Meanwhile, in the present invention, a process of specifying a final prediction value for input data can be performed using multiple prediction values ​​(S370, see FIG. 3).

[0534] The control unit (190) can use the output of at least one learned prediction model to specify a final prediction value for the input data (810).

[0535] Each of the plurality of learned prediction models (171a, 171b, 171c, 172a, 172b, 172c, 173a, 173b, 173c) discussed above can be configured to predict a value for a target category (439). For example, each of the plurality of learned prediction models (171a, 171b, 171c, 172a, 172b, 172c, 173a, 173b, 173c) can predict whether a customer will make a purchase when input data is entered.

[0536] Specifically, the control unit (190) can specify a final prediction value for the input data (810) using a plurality of prediction values ​​(621a, 621b, 621c, 622a, 622b, 622c, 623a, 623b, 623c) obtained from each of a plurality of learned prediction models (171a, 171b, 171c, 172a, 172b, 172c, 173a, 173b, 173c).

[0537] First, the control unit (190) can perform soft voting based on a plurality of prediction values ​​(621a, 621b, 621c, 622a, 622b, 622c, 623a, 623b, 623c) to specify the final prediction value.

[0538] Here, soft voting can be understood as one of the ensemble techniques, a technique that determines the final prediction by averaging the results (or classes) independently predicted by multiple artificial intelligence models.

[0539] The control unit (190, or prediction unit (180)) can perform averaging on a plurality of prediction values ​​(621a, 621b, 621c, 622a, 622b, 622c, 623a, 623b, 623c) based on soft voting, and calculate (or produce) an averaged probability (or purchase conversion probability, sales conversion probability, final prediction probability, etc.).

[0540] Here, the averaged probability is a result of synthesizing multiple prediction values ​​output by multiple learned prediction models, and can be understood as representing the likelihood of a customer purchasing a product or service (purchase conversion likelihood) as a probability value. For example, the control unit (190) can express the probability value as a value between 0 and 1. In this case, a value of 0.7 may indicate that the customer has a 70% chance of purchasing the product.

[0541] Furthermore, the control unit (190) can specify the final predicted value (or sales conversion, purchase conversion, customer conversion, etc.) based on the averaged probability (S409, see FIG. 14b).

[0542] Here, the final predicted value is the final extracted prediction result, which can be understood as a binary classification representing whether or not the customer will purchase the product or service. For example, the control unit (190) can express "purchased (1)" if the customer is predicted to purchase the product, and "not purchased (0)" if the customer is predicted not to purchase the product.

[0543] In this case, the control unit (190) can compare the averaged probability with a preset threshold value, and if the averaged probability satisfies (or exceeds) the preset threshold value, specify the final predicted value as “purchased (1)”, and if it does not satisfy (or exceed) the threshold value, specify the final predicted value as “not purchased (0)”.

[0544] For example, let's assume that the sales conversion probability is calculated as "0.7 (70%)" and the preset threshold value is set to "0.65 (65%) or more." The control unit (190) can determine whether the averaged probability (e.g., "70%") satisfies the preset threshold value (e.g., "65% or more").

[0545] In one embodiment, as a result of the judgment, the control unit (190) may specify the final prediction value (i.e., sales conversion prediction, 630) as “purchased (1)” based on the averaged probability (e.g., “70%”) satisfying a preset threshold (e.g., “65% or more”).

[0546] In another embodiment, let's assume that the averaged probability is calculated as "0.6 (60%)." As a result of the judgment, the control unit (190) can specify the final predicted value (i.e., sales conversion prediction, 630) as "not purchased (0)" based on the fact that the averaged probability (e.g., "60%") does not satisfy a preset threshold (e.g., "65% or more").

[0547] In this way, by combining the output values ​​of multiple models, the present invention can offset the prediction errors of individual models and improve overall prediction accuracy. This allows for more accurate predictions of customer conversion likelihood, thereby enhancing the effectiveness of marketing and sales strategies.

[0548] That is, in the present invention, by averaging the prediction results of multiple learned models to produce a final prediction value, uncertainty that may arise when relying on a single model can be reduced, and the characteristics of each model can be utilized to the maximum extent to provide an optimized prediction result.

[0549] Meanwhile, the multiple different sub-data sets discussed above can be stored in a specific storage and utilized in various situations.

[0550] The prediction system (100) stores a plurality of different sub-data sets configured through an equal division method in a predetermined storage, and can usefully utilize the plurality of different sub-data sets stored in the predetermined storage in various situations.

[0551] In one embodiment, when the prediction system (100) determines that additional learning (or fine-tuning) is required for a learned prediction model (or a plurality of learned prediction models), the prediction system (100) may utilize an index corresponding to a record included in each of a plurality of different sub-data sets to select (or specify) only as many records as are required for additional learning of the learned prediction model. Let us assume that 1000 first records and 500 second records are required for additional learning of the learned prediction model. In this case, as illustrated in FIG. 18, the prediction system (100) may utilize an index (e.g., true_index_row...) corresponding to a first record included in at least one of the plurality of different sub-data sets (701, 702, 703) and an index (e.g., false_index_row...) corresponding to a second record to select each of the first records and the second records as many as are required for additional learning of the learned prediction model. And, the prediction system (100) can use the selected first record and second record for additional learning of the learned prediction model.

[0552] In another embodiment, when the prediction system (100) determines that evaluation (or verification) of a learned prediction model (or a plurality of learned prediction models) is necessary, the prediction system (100) may utilize an index corresponding to a record included in each of a plurality of different sub-data sets to select only as many records as are necessary for evaluating the learned prediction model. Let us assume that 3,000 first records and 3,000 second records are necessary for evaluating the learned prediction model. In this case, the prediction system (100) may utilize an index (e.g., true_index_row...) corresponding to a first record included in at least one of the plurality of different sub-data sets (701, 702, 703) and an index (e.g., false_index_row...) corresponding to a second record to select as many first records and second records as are necessary for evaluating the learned prediction model. In addition, the prediction system (100) may utilize the selected first records and second records for evaluating the learned prediction model.

[0553] Meanwhile, as discussed above, multiple records identified from the learning data set may be stored in a predetermined storage in the form of groups and / or lists, respectively. For example, as illustrated in FIG. 19, a first record group (810) including a first record and a first index corresponding to the first record, and a second record group (820) including a second record and a second index corresponding to the second record may be stored and exist.

[0554] In one embodiment, when the prediction system (100) determines that additional learning for a learned prediction model (or multiple learned prediction models) is necessary, the prediction system (100) can utilize multiple different record groups (810, 820) to select only as many records as are necessary for additional learning of the learned prediction model. Let us assume that 1000 first records and 1000 second records are necessary for additional learning of the learned prediction model. In this case, the prediction system (100) can utilize an index (e.g., true_index_row...) corresponding to a first record included in the first record group (810) and an index (e.g., false_index_row...) corresponding to a second record included in the second record group (820) to select as many first records and second records as are necessary for additional learning of the learned prediction model. In addition, the prediction system (100) can utilize the selected first records and second records for additional learning of the learned prediction model.

[0555] In another embodiment, when the prediction system (100) determines that evaluation (or verification) of a learned prediction model (or a plurality of learned prediction models) is necessary, the prediction system (100) may utilize a plurality of different record groups (810, 820) to select only as many records as are necessary for evaluating the learned prediction model. Let us assume that 3000 first records and 3000 second records are necessary for evaluating the learned prediction model. In this case, the prediction system (100) may utilize an index (e.g., true_index_row...) corresponding to a first record included in the first record group (810) and an index (e.g., false_index_row...) corresponding to a second record included in the second record group (820) to select as many first records and second records as are necessary for evaluating the learned prediction model. In addition, the prediction system (100) may utilize the selected first records and second records for evaluating the learned prediction model.

[0556] In another embodiment, when the prediction system (100) needs to configure multiple sub-data sets for training a target prediction model, it can configure multiple different sub-data sets required for training the target prediction model using multiple different record groups (810, 820). Since the method for configuring multiple different sub-data sets has been described above, a detailed description thereof will be omitted to avoid duplication.

[0557] Meanwhile, the prediction system (100) according to the present invention can operate in a cluster environment. Here, the cluster environment may include a computing environment in which multiple servers (nodes) are configured to operate as a single system. It is generally used in high-performance computing (HPC), large-scale data processing (Hadoop, Spark), cloud storage systems (e.g., Ceph, HDFS), etc. In other words, the cluster environment is a method of configuring multiple servers (or devices, computers, etc.) as a single aggregate and operating them as a single system, and is used for the purposes of high performance, high availability, load distribution, etc.

[0558] As discussed above, the present invention can configure multiple sub-data sets by applying an even partitioning method that evenly divides the entire data set (or the entire data file) into physical pieces of a certain size, i.e., data blocks and / or sub-data sets. Each data block (or sub-data set) is set to have a preset size (e.g., between 512 KB and 2 MB), which may take into account both data transfer efficiency and storage space utilization. In other words, even partitioning can be a key basis for determining parallel read performance in subsequent steps, and the present invention can flexibly select only the necessary amount of data by utilizing an index of data sequence units. In this case, the index can be used as metadata indicating the order or location information of the evenly divided data blocks and / or sub-data sets. In other words, when the entire data file is divided into multiple blocks, each block is assigned a unique number (e.g., block #0, block #1, block #N-1, etc.), and it can be a logical identifier that allows for selecting or combining only the necessary blocks based on the number.

[0559] In this regard, when the prediction system (100) according to the present invention operates in a cluster environment, it may include at least one (N) storage server. In this case, the prediction system may divide the entire data file into a number corresponding to the N storage servers, so that each storage server (200) may distribute the data evenly. Here, including N storage servers means the number of storage servers constituting the cluster, and it can be understood that there are N storage servers that play a role in storing data within one system (cluster), i.e., nodes that can store data. That is, in the present invention, data is divided evenly and stored in each storage server (200), and each storage server (200) can read (or search) data in parallel when necessary.

[0560] Accordingly, in the present invention, the number of sub-data sets may be determined based on the number of storage servers in which different sub-data sets are stored. For example, if the prediction system (100) includes 11 storage servers, the number of sub-data sets may be determined as 11, which is the number corresponding to the storage servers. For another example, if the prediction system (100) includes 20 storage servers, the number of sub-data sets may be determined as 20, which is the number corresponding to the storage servers.

[0561] When the number of different sub-data sets is determined based on the number of storage servers, the prediction system (100) can configure a number of different sub-data sets corresponding to the number of storage servers and store the configured multiple different sub-data sets in the storage servers. For example, let's assume that there are 11 storage servers included in the prediction system (100). The prediction system (100) can configure a number of different sub-data sets corresponding to the 11 storage servers and store the configured multiple different sub-data sets in each of the 11 storage servers. That is, the prediction system (100) can configure a number of sub-data sets corresponding to the number of storage servers and store the configured multiple sub-data sets in each of the storage servers (200).

[0562] Meanwhile, in the inference stage, the method for predicting a valid customer of the prediction system according to the present invention comprises the steps of receiving (or inputting) prediction target customer data that is the subject of prediction from a user terminal, inputting the prediction target customer data into each of a plurality of prediction models that are each trained as different sub-data sets divided (or evenly divided) based on the purchasing customer data among a learning data set composed of purchasing customer data and non-purchasing customer data, obtaining a plurality of prediction values ​​representing the probability that a customer corresponding to the prediction target customer data is a valid customer as an output of each of the plurality of prediction models, specifying a final prediction value for the prediction target customer data using the plurality of prediction values, and providing information to the user terminal as to whether the customer corresponding to the prediction target customer data is a valid customer using the specified final prediction value, thereby making it possible to predict whether a customer related to the customer data input by the user will purchase the company's product or service.

[0563] Here, a valid customer (or valid customer) can mean a customer who has a clear demand for a specific product or service of a specific company and is likely to purchase it.

[0564] In one embodiment, when the control unit (190) receives prediction target customer data that is the target of prediction from the user terminal (10), the control unit (190) can input the prediction target customer data into each of a plurality of prediction models that are each trained with different sub-data sets divided based on the purchasing customer data among the learning data sets composed of purchasing customer data and non-purchasing customer data.

[0565] In addition, the control unit (190) obtains a plurality of prediction values ​​representing the probability that a customer corresponding to the target customer data for prediction is a valid customer as an output of each of the plurality of prediction models, and can use the plurality of prediction values ​​to specify a final prediction value for the target customer data for prediction.

[0566] Furthermore, the control unit (190) may provide information to the user terminal (10) as to whether the customer corresponding to the target customer data to be predicted is a valid customer using a specified final prediction value. For example, as illustrated in FIG. 12, the control unit (190) may provide prediction results (1021, 1022, 1023) as to whether the customer (or customer company, U1, U2, U3) related to the customer data (1020) input by the user will purchase a specific product (ex: “PuriCare Object Collection Water Purifier”) of a specific company through a service page (1000) output to the user terminal (10).

[0567] In this case, the first customer (U1) can be understood as having a very high purchase conversion rate for a specific product (1010) with a purchase probability of “80%,” and the third customer (U3) can be understood as having a low purchase conversion rate for a specific product (1010) with a purchase probability of “30%.”

[0568] As discussed above, the prediction system and its control method according to the present invention, and the learning method of the prediction system, can effectively respond to various business situations by providing a prediction model that has learned various business data.

[0569] Furthermore, the prediction system, its control method, and its learning method according to the present invention can address the imbalance problem of various business data, thereby providing learning from balanced learning data. This allows the prediction model to learn from balanced input data, thereby maintaining stable and high prediction performance even in diverse inputs (e.g., without being biased toward specific data and in various situations) during actual use.

[0570] Furthermore, the prediction system, its control method, and its learning method according to the present invention can solve the problem of imbalanced data in real-world usage environments by learning balanced business data. In other words, the present invention can enhance the generalization performance of the prediction model, enabling more accurate sales conversion predictions in real-world sales environments, and enabling the efficient allocation of sales resources and the development of optimized sales strategies.

[0571] Furthermore, the prediction system, its control method, and its learning method according to the present invention can provide an environment in which various customer data can be analyzed to develop customized sales strategies tailored to customer characteristics. This allows companies to flexibly respond to diverse customer types and market environments, thereby strengthening long-term customer relationships and significantly improving the performance of various businesses. Furthermore, companies can optimize performance in the global market and develop customized strategies tailored to the specific characteristics of each country. In other words, the present invention can provide crucial insights for corporate strategic decision-making and contribute to improving long-term business performance.

[0572] Furthermore, according to the prediction system and its control method, and the learning method of the prediction system according to the present invention, the entire data set can be equally divided into preset sizes and multiple different sub-data sets can be configured based on index information. This enables the present invention to perform various combination experiments without wasting storage space. In particular, the present invention can effectively alleviate data imbalance problems during learning by configuring sub-data sets to satisfy ratio conditions according to target classes. This can contribute to improving the accuracy and generalization performance of the prediction model.

[0573] Furthermore, the prediction system, its control method, and its learning method according to the present invention allow for the simultaneous consideration of data transmission efficiency and storage space utilization by uniformly configuring the entire data set to a preset size. This enables parallel learning of the prediction model and reduces the overall learning time.

[0574] Meanwhile, the present invention discussed above can be implemented as a program that is executed by one or more processes on a computer and can be stored on a medium (or recording medium) that can be read by the computer.

[0575] Furthermore, the present invention discussed above can be implemented as computer-readable code or instructions on a program-recorded medium. In other words, the present invention can be provided in the form of a program.

[0576] Meanwhile, computer-readable media include all types of recording devices that store data that can be read by a computer system. Examples of computer-readable media include hard disk drives (HDDs), solid-state disk drives (SSDs), silicon disk drives (SDDs), ROMs, RAMs, CD-ROMs, magnetic tapes, floppy disks, and optical data storage devices.

[0577] Furthermore, the computer-readable medium may include a storage device and may be a server or cloud storage device accessible via communication. In this case, the computer may download the program according to the present invention from the server or cloud storage device via wired or wireless communication.

[0578] Furthermore, in the present invention, the computer described above is an electronic device equipped with a processor, i.e., a CPU (Central Processing Unit), and there is no particular limitation on its type.

[0579] Meanwhile, the detailed description above should not be construed as limiting in any respect and should be considered illustrative. The scope of the present invention should be determined by a reasonable interpretation of the appended claims, and all modifications within the equivalent scope of the present invention are intended to be included within the scope of the present invention.

Claims

1. In a method in which memory and at least one processor are performed cooperatively, A step of specifying a training data set configured to include values ​​for multiple different categories; A step of using the above learning data set to configure a plurality of different sub-data sets based on a specific value of the specific category so that different values ​​corresponding to a specific category among the plurality of categories satisfy a preset composition ratio criterion; A step of training a target prediction model for each of the above different sub-data sets; Based on the above learning, a step of obtaining multiple learned prediction models, each learned with a different sub-data set; A step of inputting input data to be predicted into each of the plurality of learned prediction models; A step of obtaining a plurality of prediction values ​​for the input data from each of the plurality of learned prediction models; and A learning method of a prediction system, characterized in that it includes a step of specifying a final prediction value for the input data using the plurality of prediction values.

2. In paragraph 1, A learning method for a prediction system, characterized in that the above learning data set is configured to include a plurality of records having values ​​for the plurality of different categories.

3. In paragraph 2, The above learning data set includes MQL (Marketing Qualified Lead) data configured to have values ​​for the above different multiple categories, The above specific category is a category that indicates whether a customer's purchase conversion has occurred. A learning method of a prediction system, characterized in that the values ​​corresponding to the above specific category are configured to have a first value or a second value depending on whether a customer's purchase conversion has occurred.

4. In paragraph 2, Each of the above multiple learned prediction models, A learning method of a prediction system, characterized in that it is configured to predict values ​​for the above specific category.

5. In paragraph 2, Further comprising a step of performing feature engineering on the above learning data set, In the step of performing the above feature engineering, Generating a derived category using at least some of the above categories and values ​​corresponding to at least some of the above categories, A learning method of a prediction system characterized by specifying a value corresponding to the above-mentioned generated derivative category.

6. In paragraph 5, A learning method for a prediction system, characterized in that the learning data set further includes the derived category and a value corresponding to the derived category.

7. In paragraph 2, The values ​​corresponding to the above specific category are configured to have a first value or a second value, In the step of configuring the above multiple different sub-data sets, A learning method of a prediction system, characterized in that at least a portion of the plurality of records is included in each of the plurality of different sub-data sets so that the composition ratio of the first record including the first value for the specific category among the plurality of records and the second record including the second value for the specific category satisfies a preset composition ratio criterion.

8. In paragraph 7, The above-mentioned preset composition ratio criteria are: For each of the above multiple different sub-data sets, A learning method of a prediction system, characterized in that the number of first records including the first value for the specific category and the number of second records including the second value for the specific category have the same ratio.

9. In paragraph 8, The number of different sub-data sets above is A learning method of a prediction system, characterized in that the learning method is determined based on the number of second records including the second value for the specific category among the total number of records included in the learning data set, and the number of first records including the first value for the specific category.

10. In paragraph 9, Further comprising the step of determining the number of different sub-data sets, In the above decision-making step, A learning method of a prediction system, characterized in that the number of second records including the second value for the specific category is determined based on a value obtained by dividing the number of first records including the first value for the specific category.

11. In paragraph 8, In each of the above multiple different sub-data sets, Among the records included in the above learning data set, all first records having the first value for the specific category are included, A learning method of a prediction system, characterized in that, among the records included in the learning data set, a portion of second records having the second value for the specific category is included in the number of first records included in each of the plurality of different sub-data sets.

12. In paragraph 11, Each of the above multiple different sub-data sets contains the same first record, A learning method of a prediction system, characterized in that each of the plurality of different sub-data sets includes different second records.

13. In paragraph 1, The above learning target prediction model includes multiple prediction models based on the GBDT (Gradient Boosting Decision Tree) algorithm, In the above learning step, Each of the above multiple prediction models is trained for each of the above different sub-data sets, A learning method of a prediction system characterized in that it obtains a plurality of learned prediction models each learned with the different sub-data sets.

14. In paragraph 13, In the above learning step, A learning method of a prediction system, characterized in that, as a result of learning each of the plurality of prediction models for each of the different sub-data sets, a number of the plurality of learned prediction models is obtained by multiplying the number N of the different sub-data sets by the number M of the plurality of prediction models.

15. In paragraph 14, A learning method of a prediction system, characterized in that the number of the plurality of predicted values ​​obtained from the plurality of learned prediction models corresponds to a value obtained by multiplying the number N of the different sub-data sets by the number M of the plurality of predicted models.

16. In paragraph 15, In the step of specifying the final predicted value, A learning method of a prediction system characterized in that soft voting is performed based on the plurality of prediction values ​​to specify the final prediction value.

17. In a method for predicting an effective customer, the method is performed cooperatively by a memory and at least one processor, A step of receiving prediction target customer data that is the target of prediction from a user terminal; A step of inputting the above prediction target customer data into each of a plurality of prediction models, each of which is trained as a different sub-data set divided based on a specific value of a specific category so that different values ​​corresponding to a specific category among the plurality of categories satisfy a preset composition ratio criterion, using a learning data set configured to include values ​​for a plurality of different categories; A step of obtaining a plurality of prediction values ​​representing the probability that a customer corresponding to the prediction target customer data is a valid customer as an output of each of the plurality of prediction models; A step of specifying a final prediction value for the prediction target customer data using the plurality of prediction values; and A valid customer prediction method comprising a step of providing information to the user terminal as to whether a customer corresponding to the prediction target customer data is a valid customer using the specified final prediction value.

18. In the prediction system, The system comprises a memory and at least one processor, The above memory and the above processor cooperate to: Using a learning data set configured to include values ​​for multiple different categories, multiple different sub-data sets are configured based on specific values ​​of a specific category so that different values ​​corresponding to a specific category among the multiple categories satisfy a preset composition ratio criterion, Train the target prediction model for each of the above different sub-data sets, Based on the above learning, multiple learned prediction models are obtained, each learned with a different sub-data set, Input data to be predicted is input into each of the plurality of learned prediction models, From each of the plurality of learned prediction models, obtain a plurality of prediction values ​​for the input data, A prediction system characterized in that a final prediction value for the input data is specified using the plurality of prediction values.

19. A program stored on a computer-readable medium and executed by one or more processes on an electronic device, The above program is, A step of using a learning data set configured to include values ​​for a plurality of different categories, and configuring a plurality of different sub-data sets based on a specific value of a specific category so that different values ​​corresponding to a specific category among the plurality of categories satisfy a preset composition ratio criterion; A step of training a target prediction model for each of the above different sub-data sets; Based on the above learning, a step of obtaining multiple learned prediction models, each learned with a different sub-data set; A step of inputting input data to be predicted into each of the plurality of learned prediction models; A step of obtaining a plurality of prediction values ​​for the input data from each of the plurality of learned prediction models; and A program stored on a computer-readable recording medium, characterized in that it includes commands for performing a step of specifying a final prediction value for the input data using the plurality of prediction values.

Images