Method, computing device, and computer program for artificial intelligence-based advertisement efficiency prediction

Abstract

Provided are a method, a computing device, and a computer program for artificial intelligence-based advertisement efficiency prediction. An artificial intelligence-based advertisement efficiency prediction method according to various embodiments of the present disclosure is a method performed by a computing device, and comprises the steps of: generating an advertisement efficiency prediction model by using advertisement data as learning data; and predicting the advertisement efficiency of a target advertisement by analyzing advertisement data of the target advertisement by using the generated advertisement efficiency prediction model.

Description

AI-based advertising efficiency prediction method, computing device and computer program

[0001] Various embodiments of the present disclosure relate to an artificial intelligence-based advertising efficiency prediction method, a computing device, and a computer program.

[0002] In the era of digital marketing, advertising plays a pivotal role in corporate growth and revenue generation. In particular, the execution of advertisements through online platforms constitutes a significant portion of a company's marketing strategy, and the accurate prediction of advertising efficiency serves as an essential element for the optimal allocation of marketing budgets and the maximization of ROI (Return on Investment). Effective prediction of advertising efficiency enables companies to utilize their limited marketing budgets most efficiently and provides a crucial basis for formulating marketing strategies by assessing the performance of advertising campaigns in advance.

[0003] However, predicting advertising efficiency is a highly complex task due to a combination of variables, including the diversity of advertising media, changes in target customer bases, and the dynamic nature of the market environment. Factors influencing advertising efficiency encompass not only quantitative indicators such as advertising costs, execution timing, targeting information, and ad creative characteristics, but also qualitative elements like changes in market trends, competitor advertising activities, and shifts in consumer purchasing patterns. Existing simple analytical methods face limitations in accurately identifying and predicting the interactions of these multidimensional variables.

[0004] Conventional methods for predicting advertising efficiency have primarily relied on statistical analysis based on past advertising execution data. This approach estimates future advertising efficiency by utilizing historical performance data and employing statistical techniques such as regression or time-series analysis. Additionally, it was common practice to predict advertising campaign performance by combining qualitative judgments based on the experience and intuition of advertising experts. However, these traditional approaches fail to adequately reflect the complex and non-linear nature of the advertising ecosystem and exhibit limitations in prediction accuracy within a rapidly changing digital environment.

[0005] In particular, conventional methods suffer from the problem of failing to effectively handle the time-series characteristics of advertising data and the complex correlations between various variables. Advertising efficiency is subject to a lag effect where previous performance influences current performance, and accurately modeling this temporal dependence is key to improving prediction performance. However, existing simple regression or time-series models fail to sufficiently learn these complex temporal patterns, resulting in reduced prediction accuracy.

[0006] Furthermore, another critical issue in predicting advertising efficiency is balancing model training time with prediction performance. While high-performance models generally require more training time, real-world advertising environments demand rapid prediction results for quick decision-making. Existing approaches have faced difficulties in building practical advertising efficiency prediction systems because they perform biased optimization that considers only prediction accuracy or prioritizes training efficiency.

[0007] To overcome these limitations, research on predicting advertising efficiency using machine learning and deep learning technologies has recently been actively conducted. However, most studies rely on a single model or fail to comprehensively consider prediction performance and computational efficiency when selecting a model. Therefore, there is a growing need for a methodology to select an optimal model that systematically compares various models and simultaneously considers prediction accuracy and learning efficiency.

[0008] The aforementioned background technology is one that the inventor possessed or acquired in the process of deriving the content of the present disclosure, and it cannot be considered as prior art disclosed to the general public prior to the filing of this application.

[0009] The problem that the present disclosure aims to solve is to provide an AI-based method, apparatus, and program for predicting advertising efficiency that can significantly improve the accuracy and practicality of predicting advertising efficiency compared to existing simple statistical analysis methods, by generating multiple AI models using advertising data as training data and selecting an optimal model by comprehensively considering prediction accuracy and learning efficiency, for the purpose of resolving the aforementioned conventional problems.

[0010] The problems that this disclosure aims to solve are not limited to those mentioned above, and other unmentioned problems will be clearly understood by a person skilled in the art from the description below.

[0011] An artificial intelligence-based advertising efficiency prediction method according to one embodiment of the present disclosure for solving the above-described problem may include, in a method performed by a computing device, a step of generating an advertising efficiency prediction model using advertising data as training data and a step of predicting the advertising efficiency of a target advertisement by analyzing the advertising data of a target advertisement using the generated advertising efficiency prediction model.

[0012] In various embodiments, the step of generating the advertising efficiency prediction model may include the step of extracting a plurality of features from a plurality of advertising data for a plurality of advertisements, the step of generating a plurality of models by training each of different types of models using at least one of the extracted plurality of features, and the step of selecting one of the generated plurality of models as the advertising efficiency prediction model by considering the prediction error and the training time.

[0013] In various embodiments, the advertising data may include first advertising data including at least one of advertising cost, parking information, inflow path, and holiday information, and second advertising data including an advertising KPI (Key Performance Indicator) and a lag variable of the advertising KPI, and the step of generating the plurality of models may include generating a regression-based comparison model using a first feature extracted from the first advertising data, preprocessing the extracted first feature and generating a neural network-based basic neural network model using the preprocessed first feature, and generating a plurality of neural network-based extended neural network models using the preprocessed first feature and a second feature extracted from the second advertising data.

[0014] In various embodiments, the step of generating the plurality of extended neural network models may include: changing the type and number of input variables by adding at least one second feature among the plurality of second features extracted from the second advertising data to the input variables used for training the generated basic neural network model; adding a network structure for time series processing to the generated basic neural network model; and adjusting the learning parameters of the generated basic neural network model—the learning parameters may include at least one of a batch size, a loss function, and a learning rate.

[0015] In various embodiments, the advertising data may include first advertising data including at least one of advertising cost, parking information, inflow path, and holiday information, and second advertising data including an advertising KPI (Key Performance Indicator) and a lag variable of the advertising KPI, and the step of generating the plurality of models may include the step of generating a tree-based basic tree model using a first feature extracted from the first advertising data and a second feature extracted from the second advertising data, and the step of generating a plurality of modified tree models by adjusting the learning parameters of the generated basic tree model.

[0016] In various embodiments, the step of generating the plurality of variant tree models may include the step of generating the plurality of variant tree models having different performance characteristics by changing or adjusting learning parameters including at least one of a learning rate, a boosting type, a loss function, early stopping criteria, and a number of learning iterations (num round) for the generated base tree model.

[0017] In various embodiments, the step of selecting the advertising efficiency prediction model may include the step of calculating a prediction error between the advertising efficiency predicted through the generated plurality of models and the actual advertising efficiency, the step of quantifying the training time of the generated plurality of models, and the step of selecting the model among the generated plurality of models that has the minimum sum of the quantified training time and the calculated prediction error as the advertising efficiency prediction model, and the calculated prediction error may include at least one of RMSE (Root Mean Square Error) and MAE (Mean Absolute Error).

[0018] In various embodiments, the step of calculating the prediction error may include the step of calculating a first prediction error for the entire data set, the step of calculating a second prediction error for a partial data set in which the target variable exceeds a predetermined value among the entire data set, and the step of assigning a weight of a predetermined size to the calculated second prediction error.

[0019] A computing device for performing an artificial intelligence-based advertising efficiency prediction method according to another embodiment of the present disclosure for solving the above-described problem comprises a memory for storing one or more instructions and a processor for executing the one or more instructions stored in the memory, and the processor may perform a method including the step of generating an advertising efficiency prediction model using advertising data as training data by executing the one or more instructions, and the step of predicting the advertising efficiency of a target advertisement by analyzing the advertising data of a target advertisement using the generated advertising efficiency prediction model.

[0020] A computer program according to another embodiment of the present disclosure for solving the above-described problem may be combined with a computing device, which is hardware, and may be stored on a non-transient recording medium readable by the computing device so as to enable a method comprising the steps of generating an advertising efficiency prediction model using advertising data as training data and predicting the advertising efficiency of a target advertisement by analyzing the advertising data of the target advertisement using the generated advertising efficiency prediction model.

[0021] Other specific details of the present disclosure are included in the detailed description and drawings.

[0022] According to various embodiments of the present disclosure, by generating a plurality of artificial intelligence models using advertising data as training data and selecting an optimal model by comprehensively considering prediction accuracy and learning efficiency, there is an advantage in that the accuracy and practicality of predicting advertising efficiency can be significantly improved compared to existing simple statistical analysis methods.

[0023] In addition, by comparing neural network-based models and tree-based models step-by-step and quantitatively evaluating prediction errors through RMSE and MAE, it is possible to systematically select the optimal prediction model according to various advertising data characteristics, which has the advantage of effectively responding to the diversity of the advertising operation environment.

[0024] The effects of the present disclosure are not limited to those mentioned above, and other unmentioned effects will be clearly understood by a person skilled in the art from the description below.

[0025] The following drawings attached to this specification illustrate preferred embodiments of the present disclosure and serve to further enhance understanding of the technical concept of the present disclosure together with the detailed description of the invention; therefore, the present disclosure should not be interpreted as being limited only to the matters described in such drawings.

[0026] FIG. 1 is a drawing illustrating an artificial intelligence-based advertising efficiency prediction system according to one embodiment of the present disclosure.

[0027] FIG. 2 is a diagram illustrating the hardware configuration of a computing device that performs an artificial intelligence-based advertising efficiency prediction method according to another embodiment of the present disclosure.

[0028] FIG. 3 is a flowchart of an artificial intelligence-based advertising efficiency prediction method according to another embodiment of the present disclosure.

[0029] FIG. 4 is a flowchart of a method for generating a plurality of neural network-based models in various embodiments.

[0030] FIG. 5 is a flowchart of a method for generating a plurality of tree-based models in various embodiments.

[0031] FIG. 6 is a flowchart of a method for calculating a prediction error between predicted advertising efficiency and actual advertising efficiency in various embodiments.

[0032] FIGS. 7 and 8 are diagrams illustrating, in various embodiments, the process of generating and selecting an advertising efficiency prediction model.

[0033] Figures 9 to 12 show experimental data of an advertising efficiency prediction model.

[0034] The advantages and features of the present disclosure and the methods for achieving them will become clear by referring to the embodiments described below in detail together with the accompanying drawings. However, the present disclosure is not limited to the embodiments disclosed below but may be implemented in various different forms. These embodiments are provided merely to ensure that the disclosure is complete and to fully inform those skilled in the art of the scope of the present disclosure, and the present disclosure is defined only by the scope of the claims.

[0035] The terms used herein are for describing the embodiments and are not intended to limit the disclosure. In this specification, the singular form includes the plural form unless specifically stated otherwise in the text. As used herein, "comprises" and / or "comprising" do not exclude the presence or addition of one or more other components in addition to the components mentioned.

[0036] Throughout this specification, the same reference numerals refer to the same components, and "and / or" includes each of the mentioned components and all combinations of one or more thereof. Although terms such as "first," "second," etc., are used to describe various components, they are not limited by these terms. These terms are used merely to distinguish one component from another. Accordingly, the first component mentioned below may be the second component within the technical scope of this disclosure.

[0037] As used herein, the terms “part” or “module” refer to hardware components such as software, FPGAs, or ASICs, and the “part” or “module” performs certain roles. However, the “part” or “module” is not limited to software or hardware. The “part” or “module” may be configured to reside in an addressable storage medium or configured to run on one or more processors. Thus, by example, the “part” or “module” includes components such as software components, object-oriented software components, class components, and task components, as well as processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functions provided within the components and “parts” or “modules” may be combined into a smaller number of components and “parts” or “modules,” or further separated into additional components and “parts” or “modules.”

[0038] Spatially relative terms such as "below," "beneath," "lower," "above," and "upper" may be used to facilitate the description of the relationship between one component and other components as illustrated in the drawings. Spatially relative terms should be understood as encompassing different orientations of components during use or operation, in addition to the orientations depicted in the drawings. For example, if a component depicted in a drawing is inverted, a component described as "below" or "beneath" of another component may be placed "above" of that component. Therefore, the exemplary term "below" may encompass both the lower and upper directions. Components may also be oriented in other directions, and accordingly, spatially relative terms may be interpreted according to the orientation.

[0039] Expressions such as "first," "second," or "first," "second" as used in this specification are used to distinguish one object from another when referring to a plurality of objects of the same kind, unless otherwise indicated by the context, and do not limit the order or importance of said objects.

[0040] Expressions used herein such as “A, B, and C,” “A, B, or C,” “A, B, and / or C,” or “at least one of A, B, and C,” “at least one of A, B, or C,” “at least one of A, B, and / or C,” “at least one selected from A, B, and C,” “at least one selected from A, B, or C,” “at least one selected from A, B, and / or C,” etc., may mean each of the listed items or all possible combinations of the listed items. For example, “at least one selected from A and B” may refer to (1) A, (2) at least one of A, (3) B, (4) at least one of B, (5) at least one of A and at least one of B, (6) at least one of A and B, (7) at least one of B and A, and (8) all of A and B.

[0041] As used herein, the expression “based on” is used to describe one or more factors affecting an act or action of a decision or judgment described in the phrase or sentence containing such expression, and such expression does not exclude additional factors affecting said act or action of a decision or judgment.

[0042] As used in this specification, the expression that a certain component (e.g., a first component) is "connected" or "connected" to another component (e.g., a second component) may mean that the said certain component is not only directly connected or connected to the said other component, but is also connected or connected through a new other component (e.g., a third component).

[0043] As used herein, the expression "configured to" may have meanings such as "set to," "capable of," "modified to," "made to," or "capable of." Such expression is not limited to the meaning of "specifically designed in hardware," and, for example, a processor configured to perform a specific operation may mean a generic-purpose processor capable of performing that specific operation by executing software.

[0044] Unless otherwise defined, all terms used herein (including technical and scientific terms) may be used in a meaning commonly understood by those skilled in the art to which this disclosure pertains. Additionally, terms defined in commonly used dictionaries are not to be interpreted ideally or excessively unless explicitly and specifically defined otherwise.

[0045] In this specification, the term "computer" refers to any type of hardware device comprising at least one processor, and may be understood to include software configurations operating on said hardware device according to the embodiments. For example, the term "computer" may be understood to include smartphones, tablet PCs, desktops, laptops, and user clients and applications running on each of these devices, but is not limited thereto.

[0046] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the attached drawings.

[0047] Each step described in this specification is described as being performed by a computer, but the subject of each step is not limited thereto, and depending on the embodiment, at least some of each step may be performed on different devices.

[0048]

[0049] FIG. 1 is a diagram illustrating an artificial intelligence-based advertising efficiency prediction system according to one embodiment of the present disclosure.

[0050] Referring to FIG. 1, an artificial intelligence-based advertising efficiency prediction system according to one embodiment of the present disclosure may include a computing device (100), a user terminal (200), an external server (300), and a network (400).

[0051] Here, the artificial intelligence-based advertising efficiency prediction system illustrated in FIG. 1 is according to one embodiment, and its components are not limited to the embodiment illustrated in FIG. 1 and may be added, changed, or deleted as needed.

[0052] In one embodiment, the computing device (100) can perform an artificial intelligence-based advertising efficiency prediction method.

[0053] In one embodiment, the computing device (100) can generate an advertising efficiency prediction model using advertising data as training data and predict the advertising efficiency of a target advertisement by analyzing the advertising data of a target advertisement using the generated advertising efficiency prediction model.

[0054] Here, advertising data (first advertising data and second advertising data) may refer to data including various information related to advertising execution, advertising performance, characteristics of advertising target users, and attributes of advertising content.

[0055] For example, the first advertising data may include basic advertising execution and targeting information such as advertising cost, week number, weekday, channel, operating system, channel type, holiday, genre, and gender.

[0056] For example, the second advertising data may include various advertising performance indicators as Key Performance Indicators (KPIs), such as impressions, clicks, installs, sign-ups (e.g., number of member sign-ups acquired through advertising), purchases (e.g., number of actual purchases made after an ad click or ad impression), revenue (e.g., total revenue generated through advertising), first purchases, first reads, and D1 actives (e.g., number of users who reconnected one day after installation), as well as time lag variables for these KPIs. For example, lag variables ranging from 1 to 14 days may be included, such as the number of installs one day ago (Install_lag1), the number of installs two days ago (Install_lag2), ..., the number of installs 14 days ago (Install_lag14). However, it is not limited to this and can be modified in various ways that are obvious to a person of ordinary technical skill.

[0057] Here, targeted advertising refers to advertising that is the target for predicting advertising efficiency, and may be advertising performed in various ways (e.g., online banner ads, specific app installation ads, social media-based ads, etc.) to achieve specific goals such as attracting new users, promoting repurchase, or increasing app installations.

[0058] Here, advertising efficiency refers to an indicator that quantitatively represents the effects of advertising execution, and may be advertising performance metrics such as the number of app installations via ads, installation rate, click-through rate (CTR), conversion rate (CVR), return on ad spend (ROAS), or return on investment (ROI). Furthermore, the conversion rate (CVR) refers to the percentage of users who actually performed the action intended by the advertiser after encountering the ad; for instance, it may be the percentage of users who clicked the ad and subsequently achieved a specific conversion, such as a purchase, sign-up, or app installation.

[0059] Here, an advertising efficiency prediction model (e.g., a neural network) consists of one or more network functions, and one or more network functions may consist of a set of interconnected computational units that can generally be referred to as 'nodes'. These 'nodes' may also be referred to as 'neurons'. One or more network functions are composed of at least one node. The nodes (or neurons) constituting one or more network functions may be interconnected by one or more 'links'.

[0060] In an advertising efficiency prediction model, one or more nodes connected via links can form a relative relationship between input and output nodes. The concepts of input and output nodes are relative; any node in an output node relationship with respect to one node may be in an input node relationship with respect to another node, and vice versa. As previously mentioned, the input node versus output node relationship can be generated based on links. One or more output nodes may be connected to a single input node via links, and vice versa.

[0061] In a relationship between input and output nodes connected via a single link, the value of the output node can be determined based on data input into the input node. Here, the link interconnecting the input and output nodes may have a weight. The weight may be variable and may be varied by a user or an algorithm to perform the desired function of the advertising efficiency prediction model. For example, if one or more input nodes are interconnected to a single output node by respective links, the output node value may be determined based on the values ​​input into the input nodes connected to the output node and the weight set on the link corresponding to each input node.

[0062] As described above, the advertising efficiency prediction model forms input-output node relationships within the demand forecasting model by interconnecting one or more nodes through one or more links. The characteristics of the advertising efficiency prediction model can be determined based on the number of nodes and links within the model, the relationships between the nodes and links, and the weight values ​​assigned to each link. For example, if two advertising efficiency prediction models exist with the same number of nodes and links but different weight values ​​between the links, the two models may be recognized as different from each other.

[0063] Some of the nodes constituting the advertising efficiency prediction model may form a layer based on their distances from the initial input node. For example, a set of nodes with a distance of n from the initial input node may form n layers. The distance from the initial input node can be defined by the minimum number of links that must be traversed to reach that node from the initial input node. However, this definition of a layer is arbitrary for illustrative purposes, and the degree of a layer within the advertising efficiency prediction model may be defined in a way different from that described above. For example, the layer of nodes may be defined by their distance from the final output node.

[0064] The initial input node may refer to one or more nodes within the advertising efficiency prediction model to which data is directly input without passing through links in relation to other nodes. Alternatively, within the advertising efficiency prediction model network, it may refer to nodes that do not have other input nodes connected by links in relation to nodes based on links. Similarly, the final output node may refer to one or more nodes within the advertising efficiency prediction model that do not have output nodes in relation to other nodes. Additionally, the hidden node may refer to nodes constituting the advertising efficiency prediction model that are neither the initial input node nor the final output node. An advertising efficiency prediction model according to one embodiment of the present disclosure may have more nodes in the input layer than nodes in the hidden layer that are close to the output layer, and may be an advertising efficiency prediction model in which the number of nodes decreases as one progresses from the input layer to the hidden layer.

[0065] An advertising efficiency prediction model may include one or more hidden layers. The hidden nodes of a hidden layer can take the output of the previous layer and the output of neighboring hidden nodes as input. The number of hidden nodes for each hidden layer may be the same or different. The number of nodes in the input layer may be determined based on the number of data fields in the input data and may be the same or different from the number of hidden nodes. The input data fed into the input layer can be processed by the hidden nodes of the hidden layer and output by the fully connected layer (FCL), which is the output layer.

[0066] In various embodiments, the advertising efficiency prediction model may be a deep learning model.

[0067] A deep learning model (e.g., a deep neural network (DNN)) can refer to an artificial intelligence model that includes multiple hidden layers in addition to input and output layers. Using a deep neural network, one can identify the latent structures of data. That is, one can identify the latent structures of photos, text, videos, voice, and music (e.g., what objects are in a photo, what the content and emotions of the text are, what the content and emotions of the voice are, etc.).

[0068] Deep neural networks may include, but are not limited to, convolutional neural networks (CNN), recurrent neural networks (RNN), autoencoders, Generative Adversarial Networks (GAN), restricted Boltzmann machines (RBM), deep belief networks (DBN), Q networks, U networks, and Siamese networks.

[0069] In various embodiments, the network function may include an autoencoder. Here, the autoencoder may be a type of artificial neural network for outputting output data similar to the input data.

[0070] An autoencoder may include at least one hidden layer, and an odd number of hidden layers may be placed between the input and output layers. The number of nodes in each layer may be reduced from the number of nodes in the input layer to an intermediate layer called the bottleneck layer (encoding), and then expanded symmetrically from the bottleneck layer to the output layer (symmetrical to the input layer). The nodes of the dimensionality reduction layer and the dimensionality restoration layer may or may not be symmetrical. Additionally, the autoencoder can perform non-linear dimensionality reduction. The number of nodes in the input and output layers may correspond to the number of sensors remaining after the preprocessing of the input data. In the autoencoder structure, the number of nodes in the hidden layers included in the encoder may have a structure where it decreases as it moves away from the input layer. Since the number of nodes in the bottleneck layer (the layer with the fewest nodes located between the encoder and the decoder) may not transmit a sufficient amount of information if it is too small, it may be maintained at a certain number or higher (e.g., more than half the number of nodes in the input layer).

[0071] In various embodiments, the computing device (100) may be connected to a user terminal (200) via a network (400), and may obtain advertising data and advertising data of a target advertisement to predict advertising efficiency from the user terminal (200), and may predict advertising efficiency by analyzing the advertising data of the target advertisement through an advertising efficiency prediction model, and may provide information regarding the predicted advertising efficiency to the user terminal (200).

[0072] Here, the user terminal (200) may refer to any form of entity(s) in a system having a mechanism for communicating with a computing device (100). For example, such a user terminal (200) may include a PC (personal computer), a notebook, a mobile terminal, a smartphone, a tablet PC, and a wearable device, and may include any type of terminal capable of connecting to a wired or wireless network. Additionally, the user terminal (200) may include any computing device implemented by at least one of an agent, an API (Application Programming Interface), and a plug-in. Additionally, the user terminal (200) may include an application source and / or a client application.

[0073] Additionally, the network (400) may refer to a connection structure capable of exchanging information between each node, such as multiple terminals and servers. For example, the network (400) may include a Local Area Network (LAN), a Wide Area Network (WAN), the World Wide Web (WWW), a wired / wireless data network, a telephone network, a wired / wireless television network, a Controller Area Network (CAN), and Ethernet.

[0074] Wireless data communication networks may include, but are not limited to, 3G, 4G, 5G, 3GPP (3rd Generation Partnership Project), 5GPP (5th Generation Partnership Project), LTE (Long Term Evolution), WIMAX (World Interoperability for Microwave Access), Wi-Fi, Internet, LAN (Local Area Network), Wireless LAN (Wireless Local Area Network), WAN (Wide Area Network), PAN (Personal Area Network), RF (Radio Frequency), Bluetooth network, NFC (Near-Field Communication) network, satellite broadcasting network, analog broadcasting network, DMB (Digital Multimedia Broadcasting) network, etc.

[0075] In one embodiment, an external server (300) may be connected to a computing device (100) via a network (400) and may store and manage various information and data (e.g., advertising data for various advertisements and advertising data for target advertisements, etc.) required for the computing device (100) to perform an artificial intelligence-based advertising efficiency prediction method, or may collect, store, and manage various information and data (e.g., model performance evaluation, advertising prediction results, etc.) derived as the computing device (100) performs an artificial intelligence-based advertising efficiency prediction method. For example, the external server (300) may be a storage server separately provided outside the computing device (100), but is not limited thereto. Hereinafter, with reference to FIG. 2, the hardware configuration of a computing device (100) that performs an artificial intelligence-based advertising efficiency prediction method will be described.

[0076]

[0077] FIG. 2 is a diagram illustrating the hardware configuration of a computing device that performs an artificial intelligence-based advertising efficiency prediction method according to another embodiment of the present disclosure.

[0078] Referring to FIG. 2, a computing device (100) according to another embodiment of the present disclosure may include one or more processors (110), a memory (120) for loading a computer program (151) executed by the processor (110), a bus (130), a communication interface (140), and a storage (150) for storing the computer program (151). Here, FIG. 2 illustrates only the components related to the embodiments of the present disclosure. Accordingly, a person skilled in the art to which the present disclosure pertains will understand that other general-purpose components may be included in addition to the components illustrated in FIG. 2.

[0079] The processor (110) controls the overall operation of each component of the computing device (100). The processor (110) may be configured to include a CPU (Central Processing Unit), an MPU (Micro Processor Unit), an MCU (Micro Controller Unit), a GPU (Graphic Processing Unit), or any form of processor well known in the art of the present disclosure.

[0080] Additionally, the processor (110) may perform operations for at least one application or program for executing the method according to the embodiments of the present disclosure, and the computing device (100) may have one or more processors.

[0081] In various embodiments, the processor (110) may further include Random Access Memory (RAM) (not shown) and Read-Only Memory (ROM) (not shown) for temporarily and / or permanently storing signals (or data) processed within the processor (110). Additionally, the processor (110) may be implemented in the form of a System on Chip (SoC) comprising at least one of a graphics processing unit, RAM, and ROM.

[0082] Memory (120) stores various data, instructions and / or information. Memory (120) may load a computer program (151) from storage (150) to execute a method / operation according to various embodiments of the present disclosure. When a computer program (151) is loaded into memory (120), the processor (110) may perform the method / operation by executing one or more instructions constituting the computer program (151). Memory (120) may be implemented as a volatile memory such as RAM, but the technical scope of the present disclosure is not limited thereto.

[0083] The bus (130) provides communication functions between components of the computing device (100). The bus (130) can be implemented as various types of buses, such as an address bus, a data bus, and a control bus.

[0084] The communication interface (140) supports wired and wireless internet communication of the computing device (100). Additionally, the communication interface (140) may support various communication methods other than internet communication. To this end, the communication interface (140) may be configured to include a communication module well known in the art of the present disclosure. In some embodiments, the communication interface (140) may be omitted.

[0085] Storage (150) can store a computer program (151) non-temporarily. When performing an AI-based advertising review compliance judgment process through a computing device (100), storage (150) can store various information necessary to provide the AI-based advertising review compliance judgment process.

[0086] The storage (150) may be configured to include non-volatile memory such as ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), flash memory, a hard disk, a removable disk, or any form of computer-readable recording medium well known in the art to which the present disclosure belongs.

[0087] A computer program (151) may include one or more instructions that cause a processor (110) to perform a method / operation according to various embodiments of the present disclosure when loaded into memory (120). That is, the processor (110) may perform the method / operation according to various embodiments of the present disclosure by executing the one or more instructions.

[0088] In one embodiment, the computer program (151) may include one or more instructions for performing an artificial intelligence-based advertising efficiency prediction method, which includes the step of generating an advertising efficiency prediction model using advertising data as training data and the step of predicting the advertising efficiency of a target advertisement by analyzing the advertising data of a target advertisement using the generated advertising efficiency prediction model.

[0089] The steps of the method or algorithm described in connection with the embodiments of the present disclosure may be implemented directly in hardware, implemented as a software module executed by hardware, or implemented by a combination thereof. The software module may reside in RAM (Random Access Memory), ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), Flash Memory, a hard disk, a removable disk, a CD-ROM, or any form of computer-readable recording medium well known in the art to which the present disclosure belongs.

[0090] The components of the present disclosure may be implemented as a program (or application) and stored on a medium to be executed in combination with a computer, which is hardware. The components of the present disclosure may be executed as software programming or software elements, and similarly, embodiments may be implemented in programming or scripting languages ​​such as C, C++, Java, assembler, etc., including various algorithms implemented as a combination of data structures, processes, routines, or other programming configurations. Functional aspects may be implemented as algorithms executed on one or more processors. Hereinafter, with reference to FIGS. 3 through 8, an artificial intelligence-based advertising efficiency prediction method performed by a computing device (100) will be described.

[0091]

[0092] FIG. 3 is a flowchart of an artificial intelligence-based advertising efficiency prediction method according to another embodiment of the present disclosure.

[0093] Referring to FIG. 3, in step S110, the computing device (100) can extract multiple features from multiple advertising data for multiple advertisements.

[0094] Here, advertising data may include first advertising data and second advertising data, which may refer to data including various information related to advertising execution, advertising performance, characteristics of advertising target users, and attributes of advertising content.

[0095] More specifically, the first advertising data may include basic information related to advertising execution and targeting, such as advertising costs, week information, day of the week information, traffic source, operating system, channel type, holiday information, genre, gender, etc. Here, advertising costs refer to the total costs incurred for advertising execution during a specific period, week information indicates the annual week number at the time of advertising execution (e.g., week 42), and day of the week information refers to the day of the week on which the advertisement was executed. The traffic source refers to the platform or medium to which the user was exposed to the advertisement, and may include, for example, Facebook ads, Google AdWords, Instagram ads, etc. The operating system refers to the device operating system of the target user (e.g., Android, iOS), and the holiday information may be binary data indicating whether the date of advertising execution was a holiday.

[0096] Second advertising data may include Key Performance Indicators (KPIs) representing advertising performance and time lag variables (Lag variables) of these KPIs. Here, advertising KPIs refer to advertising performance indicators such as impressions, clicks, installations, sign-ups, purchases, revenue, first purchases, first views, and D1 activations (e.g., the number of users who reconnected one day after installation). Impressions represent the number of times an advertisement was displayed to a user, clicks represent the number of times a user clicked on an advertisement, and installations represent the number of times an app was installed through an advertisement. Sign-ups represent the number of sign-ups acquired through an advertisement, purchases represent the number of actual purchases made after an ad click or ad display, and revenue represents the total revenue generated by the advertisement. First purchases represent the number of first purchases made by new users, first views represent the number of users who consumed content for the first time within the app, and D1 activations represent the number of users who reconnected to the app one day after installation.

[0097] Time lag variables (Lag variables) are intended to reflect the lag effect affecting advertising efficiency. For example, they may include lag variables such as the number of installations prior to a reference point (e.g., Installations X days ago (Install_lagX)), clicks (e.g., Clicks X days ago (Click_lagX)), advertising costs (e.g., Cost_lagX)), and impressions (e.g., Impression_lagX)). By considering the time delay between the execution of an advertisement and the occurrence of actual user behavior (e.g., installation, purchase), these variables play a crucial role in modeling the cumulative effect of advertising and patterns of performance change over time.

[0098] In various embodiments, the computing device (100) can extract a first feature from the first advertising data and extract a second feature from the second advertising data. Here, feature extraction refers to the process of converting raw advertising data into a form that a machine learning model can learn.

[0099] In the first feature extraction process, the computing device (100) can perform one-hot encoding or embedding transformation on categorical data, and can perform preprocessing such as normalization or standardization on continuous data. For example, in the case of inflow path data, embedding processing can be performed to convert categorical data in the form of strings into numeric vectors, and in the case of advertising cost data, the distribution of the data can be normalized through log transformation (e.g., log1pCost).

[0100] In the second feature extraction process, the computing device (100) can process advertising KPI data and their time delay variables by considering time series characteristics. For example, in the case of installation count data, by extracting not only the installation count value at the current point in time but also delay variables for installation counts in the past prior to the reference point in time, the model can learn temporal patterns and cumulative effects.

[0101] Additionally, the computing device (100) can improve computational efficiency by analyzing the correlation between the extracted features to select features of high importance or by applying a dimensionality reduction technique. At this time, the importance of each feature can be determined according to its impact on advertising efficiency prediction, and by selecting a feature with high representativeness among the features with high correlation, the complexity of the model can be reduced while maintaining prediction performance.

[0102] In step S120, the computing device (100) can generate multiple models by training each of different types of models using at least one of the extracted multiple features.

[0103] Step S120 is a step in which the computing device (100) systematically generates various types of advertising efficiency prediction models by utilizing multiple features extracted in Step S110. This step provides a basis for selecting a model optimized for various advertising data environments and prediction requirements by generating multiple models with different algorithmic characteristics without relying on a single model.

[0104] In various embodiments, the computing device (100) can generate a plurality of models through the following methods.

[0105] First, multiple models can be generated using a neural network-based model generation method, starting from a basic fully connected network and progressively generating an extended neural network model that includes a Long Short-Term Memory (LSTM) structure with time series processing capabilities.

[0106] Next, using a tree-based model generation method, a basic tree model is created by training each tree model with ad data as input variables based on gradient boosting algorithms such as CatBoost, XGBoost, and LightGBM, and then multiple modified tree models are generated by adjusting the training parameters.

[0107] Here, each model type has unique advantages that allow it to effectively learn different characteristics of advertising data. Neural network-based models excel at learning complex non-linear relationships, and in particular, extended neural network models including LSTM structures can effectively model temporal lag effects and time series patterns that affect advertising efficiency.

[0108] On the other hand, tree-based models can effectively process advertising data containing a mix of categorical and numerical variables, and have the advantage of providing relatively fast learning speeds and interpretability.

[0109] The multiple models generated in this stage are differentiated from one another in terms of input feature configuration, network structure complexity, and learning parameter settings, and thus may exhibit different prediction performance and learning efficiency even for the same advertising data. This diversity serves as the basis for selecting the optimal model in the subsequent stage S130, which comprehensively considers prediction accuracy and training time.

[0110] In addition, the computing device (100) applies various learning strategies, such as hyperparameter optimization, application of normalization techniques, and early termination conditions, during the learning process of each model so that each model can achieve optimal performance. This enables fair performance comparison between models and provides various options to find a balance point between prediction accuracy and computational efficiency required in an actual advertising operation environment. Below, with reference to FIGS. 4 and FIGS. 5, a method for generating multiple models will be described in detail.

[0111]

[0112] FIG. 4 is a flowchart of a method for generating a plurality of neural network-based models in various embodiments.

[0113] Referring to FIG. 4, in step S210, the computing device (100) can generate a regression-based comparison model using a first feature extracted from the first advertisement data.

[0114] Here, the first feature may refer to basic advertising execution and targeting information extracted from the first advertising data by the computing device (100), and specifically may include advertising costs, parking information, day of the week information, entry path, operating system, channel type, holiday information, genre, gender, etc. These first features are basic variables representing the basic conditions for advertising execution and the characteristics of target users, and may be essential elements that serve as the starting point for predicting advertising efficiency.

[0115] For example, advertising costs may be a continuous variable representing the total budget allocated to advertising execution during a specific period, and week information may be a numeric variable representing which week of the year the advertising execution takes place in. The traffic source is a categorical variable representing the platform or medium where the advertisement was placed, and may have values ​​such as 'Facebook', 'Google', 'Instagram', for example, and holiday information is a binary variable representing whether the advertising execution day is a holiday or a weekday, and may have a value of 0 or 1.

[0116] The comparison model generated by the computing device (100) may be a reference model functioning as a baseline model in the present disclosure, and may be a Power model based on traditional statistical regression analysis techniques. This model is a non-neural network-based model of traditional regression that does not include a neural network structure, and the computing device (100) may be trained by taking a first feature in its raw form as input without applying advanced data preprocessing techniques such as embedding transformation or normalization.

[0117] Specifically, the computing device (100) may use a subsampling method for categorical variables, such as inflow path and week information, when creating a comparison model. Subsampling is a basic categorical data processing method that does not use advanced embedding techniques, and the computing device (100) may apply a traditional method of simply mapping each category to a unique integer value or converting it into a dummy variable. Unlike embedding techniques used in neural network-based models, this may have limitations in that it cannot learn semantic relationships or similarities between categories.

[0118] The comparison model generated by the computing device (100) can represent the most basic approach to predicting advertising efficiency and can model advertising costs and basic temporal elements (e.g., week, day of the week), media information, etc., as simple linear or polynomial regression relationships. Although this model may not fully reflect complex non-linear relationships, time-series patterns, or interaction effects between variables, it has the advantage of being easy to interpret and fast to calculate.

[0119] The comparison model generated by the computing device (100) can be used as a reference point when evaluating the performance of various improvement models generated in subsequent stages. That is, when evaluating the prediction accuracy and learning efficiency of basic neural network models, extended neural network models, or tree-based models, the computing device (100) can quantitatively evaluate the effectiveness of each model by measuring the degree of improvement relative to the performance of the comparison model as a percentage. Through this, the computing device (100) can objectively verify the actual performance improvement effect of complex model structures or advanced preprocessing techniques, and can be used as a standard to evaluate the return on investment (ROI) of performance improvement relative to the increase in model complexity.

[0120] In step S220, the computing device (100) can preprocess the extracted first feature and generate a neural network-based basic neural network model using the preprocessed first feature.

[0121] Here, the preprocessing process of the first feature performed by the computing device (100) may refer to the process of converting raw advertising data into a form that a neural network model can effectively learn. For example, the computing device (100) may convert the advertising cost, which is a continuous variable, by performing a log transformation (e.g., log1pCost). This log transformation takes into account the characteristic that advertising cost data generally has a wide range of values ​​and exhibits a skewed distribution, and the computing device (100) can normalize the distribution of the data and improve the learning stability of the model through the log transformation.

[0122] In addition, as another example, the computing device (100) can perform embedding processing on categorical variables such as inflow path and parking information.

[0123] Here, embedding processing is a technique for converting categorical data from a high-dimensional vector space into a low-dimensional continuous vector, and the computing device (100) can learn semantic similarity and relationships between categories through this. For example, inflow paths of social media series such as 'Facebook' and 'Instagram' can be placed close to each other in the embedding space, which can enable a neural network model to better learn the efficiency patterns of advertisements with similar media characteristics.

[0124] For example, the basic neural network model generated by the computing device (100) may refer to a neural network model composed of a fully connected layer (FC), which may serve as the basis for an extended neural network model to be generated in a subsequent step. The basic neural network model may have a typical feedforward neural network structure composed of an input layer, one or more hidden layers, and an output layer, and the computing device (100) may train the model to solve a regression problem for predicting advertising efficiency using a preprocessed first feature as input.

[0125] Here, the computing device (100) can input preprocessed first features to the input layer of the basic neural network model, and may include, for example, a log-transformed advertising cost (e.g., log1pCost), an embedded inflow path vector (e.g., Channel (w / Embedding)), an embedded week information vector (e.g., WeekNum (w / Embedding)), etc. The hidden layer can learn non-linear relationships between these input features, and the output layer can finally output a predicted value for an advertising efficiency indicator (e.g., number of installations).

[0126] The basic neural network model serves as a standard model for generating extended neural network models, and the computing device (100) can generate various extended neural network models based on this basic neural network model by adding input variables, changing the network structure, or adjusting learning parameters. That is, the extended neural network models described below may refer to models that have improved performance through additional features or structural improvements based on the structure of the basic neural network model and learned weights.

[0127] In various embodiments, the computing device (100) may optimize weights using a backpropagation algorithm when training a basic neural network model, and may use Mean Absolute Error (MAE) or Root Mean Square Error (RMSE) as the loss function. Additionally, the computing device (100) may apply normalization techniques such as dropout and batch normalization to prevent overfitting.

[0128] In step S230, the computing device (100) can generate a plurality of neural network-based extended neural network models using the preprocessed first feature and the second feature extracted from the second advertisement data.

[0129] The extended neural network model generated by the computing device (100) may refer to a neural network model that improves performance by expanding input variables, improving the network structure, or optimizing learning parameters based on a basic neural network model. The extended neural network model may be designed to improve advertising efficiency prediction performance through additional features or structural improvements, using the structure of the basic neural network model and learned weights as reference points.

[0130] In various embodiments, the computing device (100) may apply the following main methods in generating an extended neural network model.

[0131] First, the computing device (100) can change the type and number of input variables by adding at least one second feature among a plurality of second features extracted from second advertising data to the input variables used for training the generated basic neural network model. For example, the computing device (100) can additionally input exposure delay variables (e.g., Impression_lag1 to Impression_lag7) for a predetermined period (e.g., 7 days) in addition to the preprocessed first features (e.g., log-transformed advertising cost, embedded inflow path, embedded week information) used in the basic neural network model, thereby modeling the impact of past advertising exposure effects over a predetermined period on current advertising efficiency. Additionally, the computing device (100) can expand the range of delay variables to enable learning longer-term temporal patterns.

[0132] Next, the computing device (100) can add a network structure for time series processing to the generated basic neural network model. Here, the network structure for time series processing may refer to a Long Short-Term Memory (LSTM) layer, and the computing device (100) can create an extended neural network model of an FC+LSTM structure by adding an LSTM layer to an existing fully connected layer (FC layer) structure. Since the LSTM structure has the characteristic of being able to effectively learn the long-term dependency of time series data, the computing device (100) can more accurately model the lag effect and temporal pattern affecting advertising efficiency through this.

[0133] Additionally, the computing device (100) can adjust the learning parameters of the generated basic neural network model. Here, the learning parameters may include at least one of a batch size, a loss function, and a learning rate. For example, if the computing device (100) reduces the batch size from 256 to 64, the learning process may become somewhat unstable, but effects such as improved memory efficiency, improved generalization performance, and prevention of overfitting can be expected. A smaller batch size reduces the GPU memory usage of the computing device (100), allowing more model parameters to be loaded into memory, and enables the model to adapt to various data patterns through more frequent weight updates.

[0134] For example, if the computing device (100) changes the loss function from MAE to SmoothL1Loss, it may be possible to respond specifically to outliers that may occur in the ad efficiency prediction data. The ad efficiency prediction data may contain outliers such as extreme ad spending in a specific campaign or a sudden surge in clicks, and by using SmoothL1Loss, the computing device (100) can react less sensitively to these outliers to maintain the stability of the model, while reacting more sensitively to general error ranges to achieve faster learning and accuracy improvement. That is, the computing device (100) can apply a loss function that can improve prediction performance while reflecting outliers and non-linearity, which are characteristics of the data.

[0135] For example, when the computing device (100) adjusts the learning rate, a dual effect of fast initial convergence and stable fine-tuning can be obtained. The computing device (100) may apply a learning rate decay or scheduler technique that applies a large learning rate at the beginning of learning to converge quickly, and then gradually lowers the learning rate to stably search for the vicinity of the optimal point. In addition, the computing device (100) can improve the generalization performance of the model through appropriate learning rate adjustment, reduce overfitting, and increase the probability of reaching a solution with good generalization performance among various local minima. In particular, since advertising efficiency data contains a complex combination of time series, outliers, and non-linearity, learning instability may occur if the computing device (100) applies only a fixed learning rate; however, by dynamically adjusting the learning rate, it can flexibly respond to outliers or data variability. In addition, the computing device (100) can avoid the inefficiency of learning for a long time with an unnecessarily small learning rate and maintain an optimal convergence speed to ensure training time and resource efficiency.

[0136] In this way, the computing device (100) can generate various extended neural network models based on the basic neural network model, and each extended neural network model can learn various characteristics and patterns of advertising data by having different input variable configurations, network structures, or learning settings. For example, the computing device (100) can generate an FC+LSTM model including a 7-day lag variable and an FC+LSTM model including a 14-day lag variable, respectively, to compare and evaluate the impact of the temporal range on model performance.

[0137]

[0138] FIG. 5 is a flowchart of a method for generating a plurality of tree-based models in various embodiments.

[0139] Referring to FIG. 5, in step S310, the computing device (100) can generate a tree-based basic tree model using a first feature extracted from the first advertising data and a second feature extracted from the second advertising data.

[0140] In one embodiment, the computing device (100) can generate a tree-based basic tree model by directly combining a first feature extracted from first advertising data and a second feature extracted from second advertising data. Here, the tree-based model may refer to a machine learning algorithm based on a decision tree structure and may operate by learning prediction rules while hierarchically partitioning data. The tree-based model can effectively process advertising data containing a mixture of categorical and numerical variables and may have the advantage of intuitively interpreting the importance of each variable.

[0141] For example, the tree-based basic tree model generated by the computing device (100) may be based on a gradient boosting algorithm, and as a specific example, at least one of the algorithms CatBoost, XGBoost, and LightGBM may be used. Here, CatBoost is specialized in processing categorical variables, so the computing device (100) can effectively process categorical advertising data such as entry paths or genres, and XGBoost has an excellent balance of accuracy and speed, so the computing device (100) can simultaneously achieve fast learning and high prediction performance. In addition, LightGBM has high memory efficiency, so the computing device (100) can effectively process large amounts of advertising data.

[0142] Here, the basic tree model generated by the computing device (100) may refer to a basic model that serves as a standard for a plurality of variant tree models to be generated in a subsequent step. The basic tree model may be an initial model based on a specific algorithm (e.g., CatBoost, XGBoost, or LightGBM) but trained using standard learning parameters. For example, the computing device (100) may generate a basic tree model using basic settings such as a learning rate of 0.1, 1000 iterations, and 20 early termination conditions, which can serve to verify the basic performance of the algorithm and provide a reference point for subsequent parameter tuning.

[0143] In various embodiments, the computing device (100) can generate a tree-based basic tree model by taking as input the training data of a model selected as the optimal model among a comparison model, a basic neural network model, and a plurality of extended neural network models, considering training time and prediction error. This approach may have the advantage that the computing device (100) can reduce unnecessary training time of the tree model and reduce the number of unnecessary models by utilizing an already verified optimal feature combination.

[0144] For example, if the computing device (100) optimally selects an FC+LSTM model that includes a lag variable of a predetermined period (e.g., 14 days) through a performance comparison of neural network-based models, the computing device (100) can train a tree-based model by utilizing the feature combinations used in the model (e.g., log1pCost, Channel (w / Embedding), WeekNum (w / Embedding), Impression_lag1~14, Click_lag1~14, Install_lag1~14, etc.) as they are. Through this, the computing device (100) can reduce trial and error in feature selection, maximize the performance of the tree model based on already verified effective feature combinations, and improve the efficiency of the entire model development process.

[0145] In step S320, the computing device (100) can generate a plurality of variant tree models by adjusting the learning parameters of the generated basic tree model.

[0146] Specifically, the computing device (100) can generate a plurality of variant tree models having different performance characteristics by changing or adjusting a learning parameter including at least one of a learning rate, a boosting method, a loss function, an early termination condition, and the number of learning iterations for a generated basic tree model.

[0147] More specifically, in one embodiment, the computing device (100) can generate a plurality of modified tree models by adjusting the learning rate of the basic tree model. Here, the learning rate is a hyperparameter that controls the degree of weight updates during the model training process. If the learning rate is high, fast training convergence is possible, but overfitting or unstable training may occur, and if the learning rate is low, stable training is possible, but there is a risk of the training time taking a long time or converging to a local optimum. For example, the computing device (100) can generate modified tree models having different learning characteristics by changing the learning rate of the basic tree model to 0.05, 0.15, 0.2, etc., based on the basic learning rate of 0.1.

[0148] Additionally, in one embodiment, the computing device (100) can generate a plurality of variant tree models by changing the boosting method of the base tree model. Here, the boosting method refers to a methodology for building a strong learner by sequentially combining weak learners in a tree-based model, and each boosting method has different prediction performance and learning characteristics. For example, the computing device (100) can generate variant tree models having different ensemble characteristics by using the Gradient Boosting Decision Tree (GBDT) method as the base, but changing it to the Random Forest method or other ensemble methodologies.

[0149] In addition, in one embodiment, the computing device (100) can generate a plurality of modified tree models by changing the loss function of the basic tree model. Here, the loss function is a function that measures the error between the model's predicted value and the actual value, and different loss functions may be suitable depending on the characteristics of the advertising efficiency prediction problem. For example, the computing device (100) can generate modified tree models that reflect different error characteristics by using the RMSE loss function, which is the basic loss function of the basic tree model, as a reference, but changing it to a Tweedie loss function, a Huber loss function, or a Quantile loss function. In particular, the Tweedie loss function is known to show excellent performance in datasets containing many zero values, such as advertising data, so it can be effectively utilized to generate modified tree models specialized for advertising efficiency prediction.

[0150] Additionally, in one embodiment, the computing device (100) can generate a plurality of variant tree models by adjusting the early termination condition of the basic tree model. Here, the early termination condition refers to a condition in which training is stopped when performance improvement on the validation data is not observed more than a certain number of times during the model training process, and this serves to prevent overfitting and shorten the training time. For example, the computing device (100) can generate variant tree models having different training stability and training time by changing the basic early termination condition of the basic tree model, which is 20 consecutive times no performance improvement, to 10 times, 30 times, 50 times, etc.

[0151] Additionally, in one embodiment, the computing device (100) can generate a plurality of variant tree models by adjusting the number of training iterations of the basic tree model. Here, the number of training iterations refers to the total number of boosting rounds performed in the boosting algorithm, which has a direct effect on the complexity of the model and the training time. For example, the computing device (100) can generate variant tree models with different model complexity and prediction accuracy by changing the number of training iterations of the basic tree model from 1,000 to 500, 1,500, 2,000, etc.

[0152] In various embodiments, the computing device (100) can generate variant tree models having more diverse characteristics by combinatorially adjusting the learning parameters. For example, a first variant tree model can be generated for a CatBoost-based basic tree model with a learning rate of 0.1, an RMSE loss function, an early termination condition of 20 times, and a training iteration count of 1000 times, or a second variant tree model can be generated for an XGBoost-based basic tree model with a learning rate of 0.1, a Tweedie loss function, an early termination condition of 20 times, and a training iteration count of 1000 times, or a third variant tree model can be generated for a LightGBM-based basic tree model with a learning rate of 0.1, a Tweedie loss function, an early termination condition of 20 times, and a training iteration count of 1000 times.

[0153] Here, each variant tree model uses the same first and second features as input data, but may exhibit different prediction performance and learning efficiency due to different learning parameter settings. Through this, the computing device (100) can establish a foundation for systematically exploring and selecting optimized tree-based models according to the characteristics of the advertising data and the prediction goal.

[0154] In addition, in various embodiments, at step S120, the computing device (100) may generate a plurality of models including neural network-based models generated at steps S210 to S230 of FIG. 3 and tree-based models generated at steps S310 to S320 of FIG. 4.

[0155] In one embodiment, the computing device (100) may generate only neural network-based models according to steps S210 to S230 as a plurality of models, and in another embodiment, may generate only tree-based models according to steps S310 to S320 as a plurality of models.

[0156] In another embodiment, the computing device (100) may generate a plurality of models including both neural network-based models according to steps S210 to S230 and tree-based models according to steps S310 to S320. At this time, the basic tree model generated in step S310 may be generated using a first feature extracted from the first advertising data and a second feature extracted from the second advertising data, as described above, or may be trained using input variables as training data from the optimal model among the neural network-based models generated through steps S210 to S230 that has the minimum training time and prediction error. Through this, the computing device (100) can systematically compare the performance of the plurality of models and select the model most suitable for predicting advertising efficiency.

[0157] In various embodiments, the computing device (100) may operate two models by selecting one of the neural network-based models according to steps S210 to S230 as the first advertising efficiency prediction model and one of the tree-based models according to steps S310 to S320 as the second advertising efficiency prediction model, and finally, may operate one of the first advertising efficiency prediction model and the second advertising efficiency prediction model.

[0158]

[0159] Again, referring to FIG. 3, in step S130, the computing device (100) can select one of the generated multiple models as an advertising efficiency prediction model by considering the prediction error and the learning time.

[0160] Step S130 is a process in which the computing device (100) selects the final model most suitable for predicting advertising efficiency from among a plurality of models generated in Step S120, and determines the optimal model that can be practically utilized in an actual advertising operation environment by comprehensively evaluating prediction performance and learning efficiency, rather than simply considering prediction accuracy.

[0161] In one embodiment, the computing device (100) can evaluate the prediction performance for the same target variable for each of the plurality of models generated in step S120 and measure the training time taken during the training process of each model. Here, the prediction performance evaluation is a process of quantifying the difference between the advertising efficiency value predicted by each model and the actual advertising efficiency value through error indicators such as RMSE and MAE, and the training time measurement is a process of recording the actual time taken during the training process of each model in seconds.

[0162] In addition, in one embodiment, the computing device (100) may select a model by calculating a comprehensive evaluation index that considers the balance between the two elements, rather than simply considering the prediction error and the learning time individually. For example, the computing device (100) may normalize the prediction error and the learning time of each model, then apply weights reflecting the importance of prediction accuracy and the importance of learning efficiency to calculate a comprehensive score, and select the model with the highest comprehensive score as the advertising efficiency prediction model. Through this, the computing device (100) can select a practical model that requires a reasonable learning time while maintaining high prediction accuracy, thereby supporting rapid decision-making in an actual advertising operation environment. This will be explained in more detail below with reference to FIG. 6.

[0163]

[0164] FIG. 6 is a flowchart of a method for calculating a prediction error between predicted advertising efficiency and actual advertising efficiency in various embodiments.

[0165] Referring to FIG. 6, in step S410, the computing device (100) can calculate a prediction error between the predicted advertising efficiency and the actual advertising efficiency through a plurality of generated models. Here, the predicted advertising efficiency is an advertising KPI prediction value derived by each of the plurality of models for the same verification dataset, such as the target variable (e.g., number of installations), and may be a predicted value such as the number of app installations, clicks, or impressions, and the actual advertising efficiency may be a measured value such as the number of app installations, clicks, or impressions observed through the actual execution of the verification dataset. Additionally, the calculated prediction error is an indicator that quantifies the difference between the predicted value and the actual value, and may include at least one of RMSE and MAE.

[0166] In one embodiment, the computing device (100) may calculate a first prediction error for the entire data set, calculate a second prediction error for a partial data set in which the target variable exceeds a predetermined value among the entire data set, and assign a weight of a predetermined size to the calculated second prediction error. Here, the entire data set refers to all advertising data used for learning and verification, and the partial data set refers only to meaningful advertising performance data in which the target variable, the advertising efficiency indicator, has a value greater than 0 among the entire data set. In the present disclosure, the second prediction error is separately calculated for a partial data set in which the target variable (Y, e.g., Install) has a predetermined value (e.g., Y > 0), a weight of a predetermined size (e.g., 1.5 times or 2.0 times) is assigned to the calculated second prediction error, and the reliability can be improved by quantifying the prediction error by summing the weighted second prediction error and the first prediction error. This may be a method adopted to place greater importance on the prediction accuracy of data where actual advertising performance occurred, as performing many predictions when Y=0 increases the prediction accuracy but cannot yield meaningful prediction results.

[0167] In step S420, the computing device (100) can quantify the training time of multiple generated models. Specifically, the computing device (100) can measure the actual training time taken during the training process of each model and convert it into a normalized value to enable comparison between models. For example, if the training time of the basic neural network model is 2 minutes and 33 seconds, it can be quantified as 153 seconds; if the training time of the extended neural network model is 1 minute and 3.6 seconds, it can be quantified as 63.6 seconds; and if the training time of the CatBoost model is 2.8 seconds, it can be quantified as 2.8 seconds, thereby establishing a standard for quantitatively comparing the training efficiency of each model. Additionally, the computing device (100) can convert the training time into a normalized value between 0 and 1 by dividing the training time by the maximum training time, or adjust it so that the difference in training time is mitigated through log transformation and balanced with the prediction error.

[0168] In step S430, the computing device (100) may select the model among the generated multiple models that minimizes the sum of the numerical learning time and the calculated prediction error as the advertising efficiency prediction model. Specifically, the computing device (100) may assign a predetermined weight to each model for the normalized prediction error and the normalized learning time, calculate a total score by summing them, and select the model with the lowest total score as the optimal model. For example, the total score may be calculated using a formula such as α × normalized_RMSE + β × normalized_learning_time (where α and β are weights reflecting the importance of prediction accuracy and learning efficiency, respectively), and the computing device (100) may finally select the model that minimizes this total score as the model most suitable for predicting advertising efficiency.

[0169]

[0170] The computing device (100) is created as described above and can predict the advertising efficiency of the target advertisement by analyzing the advertising data of the target advertisement using a selected advertising efficiency prediction model. Specifically, when advertising data for a new target advertisement is input into the computing device (100), the corresponding advertising data is input into the selected advertising efficiency prediction model to perform a prediction operation, and as a result, advertising efficiency indicators such as the expected number of installations, expected click-through rate, and expected conversion rate of the target advertisement can be derived. For example, if target advertisement data is input where the advertising cost for a specific mobile game advertisement is 1 million won, the annual week (week information) at the time of advertising execution is the 42nd week, the inflow source is Facebook, and the target gender is male, the selected advertising efficiency prediction model can predict and provide specific figures such as the expected number of app installations under those conditions being 1,200, the expected click-through rate being 2.3%, and the expected conversion rate being 1.8%.

[0171]

[0172] FIGS. 7 and 8 are diagrams illustrating, in various embodiments, the process of generating and selecting an advertising efficiency prediction model.

[0173] Referring to FIG. 7, the computing device (100) of the present disclosure can generate a basic comparison model by extracting features from advertising data, then generate a basic neural network model through feature preprocessing, then generate a plurality of extended neural network models by adding a Lag variable, and evaluate the prediction performance and learning efficiency of the entire plurality of models to select the optimal model.

[0174] Additionally, with reference to FIG. 8, the computing device (100) of the present disclosure can extract features from advertising data, train a basic tree model based on algorithms such as CatBoost, XGBoost, and LightGBM, generate a plurality of modified tree models by adjusting learning parameters, and evaluate the prediction performance and learning efficiency of the entire plurality of models to select the optimal model.

[0175] In addition, as described above, the multiple models generated in FIG. 7 and the multiple models generated in FIG. 8 can be integrated to evaluate the prediction performance and learning efficiency of the entire set of multiple models and to select the optimal model.

[0176] In addition, as described above, a basic tree model of Fig. 8 can be trained based on the input features of the model selected in Fig. 7, and a plurality of modified tree models can be generated by adjusting the learning parameters. Through this, the prediction performance and learning efficiency of the entire plurality of models, including the generated basic tree model, the plurality of modified tree models, and all models generated in Fig. 7, can be evaluated to select the optimal model.

[0177]

[0178] Figures 9 to 12 show experimental data of an advertising efficiency prediction model.

[0179] Referring to FIGS. 9 and 10, it can be seen that a comparison model, a basic neural network model, and a plurality of extended neural network models generated by the computing device (100) of the present disclosure measured MAE, RMSE, MAE(Y>0), RMSE(Y>0), and training time for a target variable called 'Install', and that the values ​​of the comparison model were converted into percentages (%) to facilitate comparison of each measurement value.

[0180] More specifically, in Figure 9, Baseline (Power model) is a comparison model that uses only basic ad costs, channel information, and week information as features, recording an MAE of 37.09 and an RMSE of 99.379. This was set as the reference value (0%), and the degree of performance improvement of other models was expressed as a percentage. The FC model is a basic neural network model with feature embeddings applied, and the MAE improved by 13.4% and the RMSE by 20.27%. The FC+LSTM model is an extended neural network model that added a 7-day lag variable and showed additional performance improvement. In particular, as shown in Figure 10, the extended neural network models including a 14-day lag variable achieved even more significant performance improvement, and in the case of model [5], the MAE improved by 29.9% and the RMSE by 36.18%, showing the best prediction performance, but it exhibited a relatively long training time.

[0181] Next, referring to FIGS. 11 and 12, model [7] was selected as the optimal neural network model by comprehensively considering the prediction error and training time in FIGS. 9 and 10. Based on the features of model [7], tree models based on CatBoost, XGBoost, and LightGBM algorithms were trained, and the MAE, RMSE, MAE(Y>0), RMSE(Y>0), and training time of each tree model were measured to determine the optimal model among all models. As a result of the experiment, the CatBoost model achieved an MAE improvement of 29.59% with a very fast training time of 2.8 seconds, the XGBoost model recorded an MAE improvement of 39.57% with the fastest training time of 0.5 seconds, and the LightGBM model showed the best prediction performance with an MAE improvement of 42.03% and an RMSE improvement of 43.74% with a training time of 1 minute 58.6 seconds. Through these experimental data, it can be confirmed that the computing device (100) can systematically select the model most suitable for a specific advertising operation environment by balancing prediction accuracy and learning efficiency.

[0182]

[0183] The aforementioned artificial intelligence-based advertising efficiency prediction method has been described with reference to the flowchart illustrated in the drawings. For the sake of simplicity, the artificial intelligence-based advertising efficiency prediction method has been illustrated and described using a series of blocks; however, the present disclosure is not limited to the order of said blocks, and some blocks may be executed in a different order or simultaneously than those illustrated and described in this specification. Furthermore, new blocks not described in this specification and drawings may be added, or some blocks may be deleted or modified.

[0184]

[0185] The method described above may be provided as a computer program stored on a computer-readable recording medium for execution on a computer. The medium may continuously store a program executable by a computer, or temporarily store it for execution or download. Additionally, the medium may be various recording or storage means in the form of a single or multiple hardware components combined, and may not be limited to a medium directly connected to a computer system but may exist distributed over a network. Examples of media may include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and media configured to store program instructions, including ROM, RAM, and flash memory. Furthermore, other examples of media may include recording or storage media managed by app stores that distribute applications or sites and servers that supply or distribute various other software.

[0186] The methods, operations, or techniques of the present disclosure may be implemented by various means. For example, these techniques may be implemented in hardware, firmware, software, or a combination thereof. Those skilled in the art will understand that the various exemplary logical blocks, modules, circuits, and algorithmic steps described in connection with the disclosure herein may be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate such interchangeability between hardware and software, various exemplary components, blocks, modules, circuits, and steps have been generally described above in terms of their functional aspects. Whether such functions are implemented in hardware or in software depends on the design requirements imposed on the specific application and the overall system. Those skilled in the art may implement the functions described in various ways for each specific application, but such implementations should not be construed as departing from the scope of the present disclosure.

[0187] In a hardware implementation, the processing units used to perform the techniques may be implemented in one or more ASICs, DSPs, digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, electronic devices, other electronic units designed to perform the functions described in this disclosure, computers, or a combination thereof.

[0188] Accordingly, the various exemplary logic blocks, modules, and circuits described in connection with the present disclosure may be implemented or performed by any combination of general-purpose processors, DSPs, ASICs, FPGAs or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or those designed to perform the functions described herein. A general-purpose processor may be a microprocessor, but alternatively, the processor may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, for example, a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors coupled with a DSP core, or any other combination of configurations.

[0189] In firmware and / or software implementations, techniques may be implemented as instructions stored on a computer-readable medium such as random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable PROM (EEPROM), flash memory, compact disc (CD), magnetic or optical data storage devices, etc. The instructions may be executable by one or more processors, and the processor(s) may be enabled to perform specific aspects of the functions described in this disclosure.

[0190] When implemented in software, the techniques described above may be stored on a computer-readable medium as one or more instructions or code, or transmitted through a computer-readable medium. Computer-readable media include both computer storage media and communication media, including any medium that facilitates the transmission of a computer program from one place to another. Storage media may be any available media accessible by a computer. As a non-limiting example, such computer-readable media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium accessible by a computer that can be used to transfer or store desired program code in the form of instructions or data structures. Additionally, any connection is appropriately referred to as a computer-readable medium.

[0191] For example, if software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair cable, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, coaxial cable, fiber optic cable, twisted pair cable, digital subscriber line, or wireless technologies such as infrared, radio, and microwave are included within the definition of a medium. As used herein, disks and discs include CDs, laser discs, optical discs, DVDs (digital versatile discs), floppy disks, and Blu-ray discs, wherein disks usually play data magnetically, whereas discs play data optically using a laser. The above combinations should also be included within the scope of computer-readable media.

[0192] The software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, a hard disk, a removable disk, a CD-ROM, or any other known form of storage medium. An exemplary storage medium may be connected to a processor so that the processor can read information from the storage medium or write information to the storage medium. Alternatively, the storage medium may be integrated into the processor. The processor and the storage medium may exist within an ASIC. The ASIC may exist within a user terminal. Alternatively, the processor and the storage medium may exist as separate components within the user terminal.

[0193] Although the embodiments described above have been described as utilizing aspects of the subject matter disclosed herein in one or more standalone computer systems, the present disclosure is not limited thereto and may be implemented in conjunction with any computing environment, such as a network or a distributed computing environment. Furthermore, aspects of the subject matter in the present disclosure may be implemented in a plurality of processing chips or devices, and storage may be similarly affected across a plurality of devices. Such devices may include PCs, network servers, and portable devices.

[0194] Although the present disclosure has been described in relation to some embodiments, various modifications and changes may be made without departing from the scope of the present disclosure as understood by a person skilled in the art to which the invention of the present disclosure pertains. Furthermore, such modifications and changes should be considered to fall within the scope of the claims appended to this specification.

[0195] The relevant details have been described in the best mode for carrying out the invention as described above.

Claims

1. In a method performed by a computing device, A step of generating an advertising efficiency prediction model using advertising data as training data; and A step of predicting the advertising efficiency of the target advertisement by analyzing the advertising data of the target advertisement using the advertising efficiency prediction model generated above; including, AI-based advertising efficiency prediction method.

2. In Paragraph 1, The step of generating the above-mentioned advertising efficiency prediction model is, A step of extracting multiple features from multiple advertising data for multiple advertisements; A step of generating a plurality of models by training each of different types of models using at least one feature among the plurality of extracted features; and A step of selecting one of the plurality of models generated above as an advertising efficiency prediction model by considering the prediction error and training time; including, AI-based advertising efficiency prediction method.

3. In Paragraph 2, The above advertising data is, It includes first advertising data including at least one of advertising costs, parking information, traffic source and holiday information, and second advertising data including an advertising KPI (Key Performance Indicator) and a Lag variable of the advertising KPI. The step of generating the above plurality of models is, A step of generating a regression-based comparison model using a first feature extracted from the first advertising data; A step of preprocessing the first extracted feature and generating a neural network-based basic neural network model using the preprocessed first feature; and A step of generating a plurality of neural network-based extended neural network models using the above-mentioned preprocessed first feature and the second feature extracted from the above-mentioned second advertisement data; including, AI-based advertising efficiency prediction method.

4. In Paragraph 3, The step of generating the above-mentioned multiple extended neural network models is, A step of changing the type and number of input variables by adding at least one second feature among a plurality of second features extracted from the second advertisement data to the input variables used for training the basic neural network model generated above; A step of adding a network structure for time series processing to the above-generated basic neural network model; and A step of adjusting the learning parameters of the above-generated basic neural network model—the learning parameters include at least one of a batch size, a loss function, and a learning rate—; including, AI-based advertising efficiency prediction method.

5. In Paragraph 2, The above advertising data is, It includes first advertising data including at least one of advertising costs, parking information, traffic source and holiday information, and second advertising data including an advertising KPI (Key Performance Indicator) and a Lag variable of the advertising KPI. The step of generating the above plurality of models is, A step of generating a tree-based basic tree model using a first feature extracted from the first advertising data and a second feature extracted from the second advertising data; and A step of generating a plurality of modified tree models by adjusting the learning parameters of the basic tree model generated above; including, AI-based advertising efficiency prediction method.

6. In Paragraph 5, The step of generating the above plurality of variation tree models is, A step of generating a plurality of modified tree models having different performance characteristics by changing or adjusting learning parameters including at least one of a learning rate, a boosting type, a loss function, early stopping criteria, and a number of training iterations (num round) for the above-mentioned basic tree model; including, AI-based advertising efficiency prediction method.

7. In Paragraph 2, The step of selecting the above-mentioned advertising efficiency prediction model is, A step of calculating a prediction error between the predicted advertising efficiency and the actual advertising efficiency through the plurality of models generated above; A step of quantifying the training time of the plurality of models generated above; and A step of selecting, among the plurality of models generated above, the model that minimizes the sum of the quantified training time and the calculated prediction error as the advertising efficiency prediction model; Includes, The prediction error calculated above is, including at least one of RMSE (Root Mean Square Error) and MAE (Mean Absolute Error), AI-based advertising efficiency prediction method.

8. In Paragraph 7, The step of calculating the above prediction error is, A step of calculating a first prediction error for the entire data set; A step of calculating a second prediction error for a partial data set among the entire data set above in which the target variable exceeds a predetermined value; and A step of assigning a weight of a predetermined size to the second prediction error calculated above; including, AI-based advertising efficiency prediction method.

9. Memory for storing one or more instructions; and It includes a processor that executes one or more instructions stored in the memory, The above processor executes the above one or more instructions, A computing device that performs the method of claim 1.

10. A computer program stored on a non-transient recording medium readable from a computing device, combined with a computing device which is hardware, so as to be able to perform the method of claim 1.

Images