Method and system for digital marketing advertisement based on ai analysis of eyes and gaze of user of smart device equipped with camera

Abstract

The present invention provides a technology for precisely identifying the interest of an Internet user and dynamically adjusting an advertisement display according to the interest. That is, the present invention proposes a method and a system for advertisement utilizing AI gaze analysis, the method including the steps of: collecting image or video data including a user face through a user terminal (including a camera) to which an Internet advertisement is exposed so that the collected image or video data can be processed by the AI analysis tool installed in the user terminal or an advertisement server; extracting information related to the user's gaze from the collected image or video data by an analysis tool, and determining, on the basis of the user's gaze information, whether the user's gaze overlaps an advertisement display area in which the advertisement is posted; and changing an exposure method of the advertisement in the user terminal when the user's gaze is determined to be oriented toward the advertisement display area. Furthermore, the present invention also proposes a method capable of maximizing advertisement profits on the basis of analysis of a customer's gaze at a digital advertisement.

Description

Digital marketing advertising method and system based on AI analysis of the eyes and gaze of camera-equipped smart device users.

[0001] The present invention relates to a digital marketing advertising method, computer recording medium, and system based on AI analysis of the eyes and gaze of a camera-equipped smart device user, which can maximize the effectiveness of digital advertising or Internet advertising by analyzing the gaze of a camera-equipped smart device user using artificial intelligence (hereinafter referred to as "AI") and utilizing the analysis results in real time for digital marketing.

[0002] Driven by advancements in wired and wireless networks and computer technology, the digital advertising market is projected to reach a global size of US$700 trillion (i.e., United States Dollars; USD) by 2023 alone. While there may be various definitions and classifications of digital advertising, broadly speaking, internet digital advertising utilizing internet technology operates in three main ways: Cost Per Impression (CPI), Cost Per Click (CPC), and Cost Per Action (CPA).

[0003] CPI is a method where Internet portal sites, such as large Internet portal sites with a large number of users or visitors (end users, clients), charge an advertising fee of, for example, 0.5 to 5 Korean Won (KRW) per exposure to an Internet advertisement. The advertiser may sign an advertising contract for a certain number of exposures, or the advertisement will be guaranteed to be displayed on the portal site for a certain period of time regardless of the number of exposures (a fixed-term advertising method is also called CPT (Cost Per Time). For example, in some cases, an agreement is made to pay a certain amount of advertising fee per 1,000 exposures. Of course, if the Internet portal site uses another Internet advertising agency, the advertising agency's commission may be added, but it is common for large Internet portal sites to offer Internet advertising plans directly to advertisers.

[0004] Under the CPC model, advertisers don't pay for impressions alone, but only when their online ads are clicked. For keyword advertising using the CPC model, for example, clicks can range from 100 won to several thousand won (in Korean Won), and some even operate a bidding system for specific keywords. The higher the advertiser's bid, the higher the priority (e.g., in the search bar, in the ad's position on the internet) it will appear when that keyword is searched. However, this method suffers from persistent problems, such as searches or clicks by machines (e.g., search bots), meaningless, continuous clicks by users, or intentional click fraud intended to defraud competitors. While these can lead to wasted advertising dollars for advertisers, various technological solutions are being proposed to address click fraud.

[0005] CPA is the most advantageous method for advertisers. It operates by collecting advertising fees only when an advertiser achieves a specific goal, such as a user accessing the advertiser's website through an online advertisement and completing membership registration, applying for a credit card, making an actual purchase, or participating in an event. Advertisers typically enter into affiliate marketing agreements with multiple website operators and only pay when the ad generates results. However, if the affiliate marketing strategy is unsuccessful, the planned advertising budget may not be fully utilized. Conversely, if a user cancels a purchase, the advertiser may spend only on advertising fees without achieving any results.

[0006] While CPI, CPC, CPA, and combinations of these Internet advertising methods may seem to differ significantly in terms of their criteria for collecting advertising fees, they all measure advertising costs based on whether the advertiser's online advertisements capture user attention. However, CPI assumes that simply displaying an advertisement somewhere on a website has already captured the attention of Internet users, while CPC assumes that clicks can be used as an indicator of successful advertising. CPA is an Internet advertising technique that recognizes user attention only when it produces a tangible advertising effect.

[0007] However, the various existing Internet advertising techniques fail to provide a clear answer as to how to increase user interest, or if so, what criteria should be used to achieve this. A prime example is targeted advertising, a conventional technology.

[0008] Targeted advertising refers to an advertising strategy that links potential consumer trait information to the product or service that the advertiser wishes to promote or market. For example, if consumer Z, who accessed search engine site X, visits a specific website Y, the operator of the search engine site X can access consumer Z's website visit history and IP (Internet Protocol) address. Therefore, the operator of the search engine site X can increase advertising effectiveness by targeting the advertisements requested from advertiser Y to consumer Z. If consumer Z has visited advertiser Y's website for whatever reason, providing Y's advertisement to that consumer Z can increase the advertising effect. For example, it can be assumed that potential consumer Z is more likely to revisit Y's website and purchase the product after viewing the advertisement. Therefore, targeted advertising is becoming a very important tool, especially in online advertising. In the above example, the operator of the search engine site X can be viewed as acting as an advertising agency, and Y becomes the advertiser who pays X a certain advertising fee and requests advertising with the expectation that X will effectively conduct targeted advertising.

[0009] Of course, targeted advertising isn't limited to search engines. Any platform that serves a large audience, including IPTV, smartphone applications (or apps), and social media platforms, can serve as a targeted advertising platform. Because IPTV can track consumer viewing history, app developers can understand the characteristics of smartphone users who use their apps, and social media platforms can track users' activities, such as social posts and social networking. Therefore, advertisers can utilize IPTV, apps, and social media as advertising platforms, much like placing ads in magazines or newspapers. In short, targeted advertising stems from the expectation that selectively advertising ads likely to attract users will increase their interest.

[0010] Despite these efforts, the current targeted advertising system faces several challenges. These can be broadly categorized into two main issues: those facing the companies that place ads (i.e., advertisers) and those facing internet users or consumers. For advertisers, the primary issue is the low conversion rate (CVR). This rate represents the percentage of internet users who convert to action, such as clicking or making a purchase, after seeing a specific ad. CVRs are often in the 2-10% range. Higher rates, for example, often indicate high cost-per-click (CPC) bids on major internet portals. On the other hand, consumers often perceive internet advertising as a form of public nuisance. Regardless of the display method or cost-per-click (CPA) method—whether pop-ups, banners, or affiliate marketing—ads can be perceived by consumers as cyber pollution, no different from spam.

[0011] As discussed above, the key to Internet advertising lies in the ability to reasonably gauge user interest in the ad. To summarize, (a) CPI advertising assumes a certain level of user interest simply by being exposed online. (b) CPC advertising requires confirmation of a user's click-through behavior to determine whether the ad has captured the attention of an online user. (c) CPA advertising determines interest based on whether the ad has achieved performance based on criteria set by the advertiser, such as actual purchases or event participation. (d) Targeted advertising is an advertising strategy that aims to increase conversion rates by targeting a group of Internet users who are likely to be interested in the advertised content, rather than targeting an unspecified number of people.

[0012] Another factor to consider from an advertiser's perspective is profit margins. For example, a 2016 US study found that if a seller with a 30% margin on regular price offers a 10% discount, their actual profit margins will decrease by more than 50%. In other words, while a 1-2% discount may seem insignificant to buyers, promotional marketing can have a significant impact on profits for sellers. If sales performance remains poor despite promotional marketing, both advertisers and sellers will inevitably suffer from increased advertising costs and reduced profits due to discounts.

[0013] In short, there is a need for a standard that can reasonably determine whether or not consumers are interested in advertising, and furthermore, a method to determine the boundary between the level of promotional benefits that must be advertised to attract consumers' attention while maximizing advertiser profits.

[0014] As previously discussed, advertisements that fail to capture the attention of Internet users can be perceived as cyber pollution. On the other hand, advertisements that capture their attention can not only provide useful online information to users, but also contribute to improved advertising efficiency by helping advertisers generate sales through their advertising spend. However, questions remain as to whether conventional Internet advertising techniques can effectively measure user interest.

[0015] Conventional CPI advertising techniques assume that simply displaying an Internet advertisement on a website has already attracted the user's attention. However, they fail to discern whether the advertisement displayed to Internet users is useful information or just one of the countless uninteresting pieces of information in cyberspace.

[0016] While cost-per-click (CPC) advertising, typically keyword search advertising, may deliver excellent conversion rates for some advertisers who rank highly for specific keywords, it can be difficult for the majority of later-ranked advertisers to achieve the desired results relative to their advertising costs. For example, when a search engine searches for a specific keyword, users must browse multiple Internet pages or wait a long time for the ad to appear. In a situation where the term "cyber pollution" has emerged, Internet users are more likely to browse a few representative ads and then engage in other web activities than to actively seek out later-ranked keyword search ads.

[0017] CPA advertising, also known as affiliate marketing, involves placing separate ad space on websites or even within game apps, requiring users to complete actions, such as participating in events, set by the advertiser in exchange for benefits like in-game currency. In these cases, it can be difficult to determine whether users are truly interested in the in-game currency provided for meeting affiliate marketing conditions, or whether they are interested in the advertisement itself and are participating in the affiliate marketing.

[0018] In the case of online targeted advertising, if the information used for customer targeting is excessive, consumers may feel repulsed by the targeted advertising rather than interested in it, and may feel that their privacy has been violated.

[0019] Moreover, none of the CPI, CPC, CPA, or targeted advertising methods offer a clear answer to how to optimize advertiser profits. This means advertisers are left with no information on how much promotion they need to offer internet users for successful digital marketing. Ultimately, advertisers are the sellers, and issues like margins are business-related issues, not ones that ad agencies or portal sites can address on their behalf.

[0020] The present invention recognizes the above-mentioned problems as challenges to be addressed, but its primary focus lies in the fact that none of the various Internet advertising techniques mentioned above attempt to precisely assess user interest in Internet advertisements and elicit greater engagement from those who have already shown interest. The invention stems from the idea that if a method could accurately assess Internet users' interests and dynamically adjust advertisement display accordingly, thereby eliciting greater attention only from those who have shown interest in Internet advertisements (thereby enhancing advertising effectiveness), all or at least some of the aforementioned problems could be resolved.

[0021] The second core technical challenge recognized by the present invention is to find a solution to the question of how much of an in-ad benefit should be provided to encourage users to make a purchase, thereby increasing conversion rates and maximizing corporate profits. Accordingly, the present invention proposes a method to improve advertiser profits by more precisely tracking user responses to digital advertisements that have already captured the attention of Internet users but have not yet converted to any purchase.

[0022] Additionally, the technical challenges addressed by the present invention are not limited to Internet advertising on traditional PCs (Personal Computers). Rather, the present invention focuses on advertising targeted at users of so-called "smart devices"—IT (Information Technology) or ICT (Information Communications Technology) devices, such as laptops, smartphones, and smart pads equipped with cameras. Considering the recent advancements in smart devices' high computing speeds, ample storage capacity, and the network environments that support them, it is believed that the aforementioned advertising techniques' challenges can be technically addressed to some extent.

[0023] The present invention is designed to solve all or part of the problems described above, and to solve the first technical problem, the present invention presents a technology for precisely identifying the interest of Internet users and dynamically adjusting the advertisement display or exposure method accordingly.

[0024] That is, according to a first aspect of the present invention, a method for advertising by transmitting Internet advertisements from an advertisement server to a user terminal equipped with a camera, the method comprising: a first step of receiving the Internet advertisement from an advertiser terminal; a second step of transmitting the received Internet advertisement to the user terminal so that the Internet advertisement is posted and displayed in an advertisement display area of ​​a viewing screen of the user terminal; a third step of collecting image or video data including a user's face through the user terminal and the camera so that it can be processed by an AI analysis tool installed in the user terminal or the advertisement server; a fourth step of extracting information related to a user's gaze from the collected image or video data by the AI ​​analysis tool and determining, based on the user's gaze information, whether the advertisement display area where the Internet advertisement is posted overlaps with the user's gaze; and a fifth step of changing an exposure method of the Internet advertisement in the user terminal when it is determined that the user's gaze is directed toward the advertisement display area.

[0025] In the case of the advertising method according to the second aspect of the present invention, the changing exposure method is characterized by including a method of increasing the size of the part where the Internet advertisement is posted and exposed or a method of changing the position on the viewing screen where the Internet advertisement is posted and exposed.

[0026] In the advertising method according to the third aspect of the present invention, the AI ​​analysis tool includes a complex computer vision AI analysis tool capable of processing images, videos, or both, and in the fourth step, the criterion for determining whether the advertisement display area and the user's gaze overlap may be composed of at least one or a combination of more of (a) whether the measured focus fixation duration exceeds a predetermined threshold, (b) whether the advertisement display area exists on a scanpath, and (c) whether the display area exists within a convex hull created by the scanpath.

[0027] In the advertising method according to the fourth aspect of the present invention, the AI ​​analysis tool may include a step of generating a user gaze map regarding the user gaze information on the viewing screen and determining whether the advertisement display area and the user gaze overlap based on the map.

[0028] A fifth aspect of the present invention relates to a system implementing an advertising method according to the first aspect described above. The fifth aspect of the present invention is an advertising server system for transmitting Internet advertisements to a user terminal equipped with a camera for advertising, comprising: a main server for receiving the Internet advertisements from an advertiser terminal and transmitting the received Internet advertisements to the user terminal so that the Internet advertisements are posted and displayed in an advertisement display area of ​​a viewing screen of the user terminal; and a database for collecting, classifying, and storing image or video data including a user's face through the user terminal and the camera so that it can be processed by an AI analysis tool installed in the main server, wherein information related to a user's gaze is extracted from the collected image or video data by the AI ​​analysis tool, and based on the user's gaze information, the main server determines whether the advertisement display area where the Internet advertisements are posted overlaps with the user's gaze, and when it is determined that the user's gaze is directed toward the advertisement display area, the main server changes the display method of the Internet advertisements in the user terminal.

[0029] Next, a sixth aspect of the present invention focuses on the second technical problem of the present invention described above, and a digital advertising method using AI gaze analysis is presented, characterized in that it includes a first step in which an advertising server receives an Internet advertisement from an advertiser terminal; a second step in which the received Internet advertisement is transmitted to a user terminal equipped with a camera so that the Internet advertisement is posted and displayed in a predetermined advertisement display area of ​​a display screen of the user terminal; a third step in which image or video data including a user's face is collected through the user terminal and the camera so that it can be processed by an AI analysis tool installed in the user terminal or the advertising server; a fourth step in which the AI ​​analysis tool extracts information related to a user's gaze from the collected image or video data and determines whether the posted and displayed Internet advertisement attracts the user's attention based on the user's gaze information according to predetermined judgment criteria; and a fifth step in which, if the posted and displayed Internet advertisement is determined to attract the user's attention, a level of benefits included in the Internet advertisement is adjusted upward in real time.

[0030] A seventh aspect of the present invention relates to a system implementing the sixth aspect described above, comprising: a main server for receiving Internet advertisements from an advertiser terminal, transmitting the received Internet advertisements to a user terminal equipped with a camera, and posting and displaying the Internet advertisements in a predetermined advertisement display area of ​​a display screen of the user terminal; and a database for collecting and storing image or video data including a user's face through the user terminal and the camera, wherein the main server executes an AI analysis tool installed in the main server through interaction with the database, and the AI ​​analysis tool extracts information related to a user's gaze from the collected image or video data, and determines whether the posted and displayed Internet advertisement attracts the user's attention based on the user's gaze information according to predetermined judgment criteria, and when it is determined that the posted and displayed Internet advertisement attracts the user's attention, an operation is performed for upwardly adjusting the level of benefits included in the Internet advertisement in real time.

[0031] Furthermore, the features of the second to fourth aspects described above in the present invention can be designed and changed as a system or method so as to be applied to the fifth to seventh aspects, and it is also possible to implement the technical features according to all aspects of the present invention as a program and store it in a computer-readable recording medium.

[0032] First, according to the first to fifth aspects of the present invention described above, the present invention has as its common core the aim of precisely identifying the interests of Internet users and dynamically adjusting advertisement display accordingly.

[0033] That is, recently, smart devices such as smartphones are equipped with cameras. The present invention proposes a technology that interprets the user's gaze on an advertisement as the user's interest in the advertisement by determining which part of the smartphone screen an Internet user visiting a website with, for example, the user is looking at and whether the area the gaze is on is an area where an advertisement is displayed. In short, the present invention presents a method that can reasonably and technically solve the user interest analysis, which is the core of Internet advertising, by incorporating AI technology that tracks eye movement (Eye Tracking) and determines the user's gaze (Gaze Detection) into Internet advertising.

[0034] Since the present invention directly identifies the user's gaze as the user's interest, it is possible to determine more precisely whether the user is interested in the advertisement compared to the conventional CPI advertising method, and thus, it is possible to calculate more reasonable advertising costs.

[0035] Comparing the present invention to the CPC method, the present invention differs significantly from the CPC method in that it analyzes user interest from the very beginning, before the user clicks. Specifically, the present invention proposes a method for modifying the display method of an online advertisement, such as enlarging or repositioning it, if the advertisement is determined to be of interest to the user. This can be understood as a technical solution for encouraging users to take actions such as clicking, or in other words, for gaining greater user interest.

[0036] However, this invention does not propose the idea of ​​creating a separate advertising page for affiliate marketing, as with CPA advertising. By introducing a technology that determines which advertisements users are interested in based on AI analysis, the invention presents a distinct advertising technique from existing affiliate marketing or CPA advertising. Comparing this invention to targeted advertising, it can be viewed as an AI advertising technology applicable to any internet user using a smart device, eliminating the need to pre-collect information on internet users' characteristics and select target advertisers.

[0037] In particular, the present invention proposes the use of a complex computer vision AI analysis tool capable of processing images, videos, or both, to analyze the interests of Internet users. As previously mentioned, the present invention focuses on Internet advertising targeting users of smart devices equipped with cameras. That is, the present invention utilizes a camera installed in a smart device to analyze which part of the screen the user is currently viewing is fixated on, thereby precisely identifying the user's interest. Thus, the present invention provides technical measures to ensure that advertisers can accept the user interest measurement method according to the present invention as reasonable. In particular, the present invention presents a reasonable user interest determination method that incorporates AI technology, including criteria for determining user interest, such as the duration of the user's gaze fixation, whether the advertisement was exposed within the scan path of the user's gaze, and whether the Internet advertisement was present within the space created by the scan path.

[0038] Furthermore, the present invention dynamically changes the exposure method of Internet advertisements based on user gaze analyzed by AI, and by using a gaze map generated in real time or previously generated for each web page, it is possible to analyze where on the screen of a smartphone, for example, the space for placing advertisements would be most ideal when viewed in light of this gaze map.

[0039] If AI gaze analysis according to the present invention is performed in real time on the ad server, the ad display method can be adjusted in real time to suit the situation, which is expected to maximize advertising efficiency. Specifically, if the present invention changes the display method of an existing Internet advertisement to attract more user attention and clicks, the successful click history can be stored in real time and fed back to the advertiser's terminal, allowing both the advertiser and the advertising agency to establish an advertising fee policy that satisfies both.

[0040] In short, the first to fifth aspects of the present invention filter out advertisements that do not attract the attention of Internet users and advertisements that attract the attention of Internet users using AI, and thereby enable dynamic advertisement exposure that can increase the advertising effect of advertisements that can be useful Internet information to users, thereby not only improving the efficiency of advertising cost execution from the advertiser's perspective, but also leading to the effect of making users recognize advertisements presented according to the present invention as useful information.

[0041] Next, according to the 6th and 7th aspects of the present invention described above, the present invention proposes a digital advertising method that selects digital advertisements that attract the attention of Internet users using AI gaze recognition technology, and observes at what level of promotion Internet users actively respond to the advertisements by gradually changing the benefit conditions of the advertisements in real time.

[0042] The effects of aspects 6 and 7 of the present invention are as follows. That is, by observing the incremental benefits of the advertisement currently being viewed by the user in real time, the ad server can determine the level of benefits required to convert the user from simply viewing the advertisement to making a purchase. This accumulated data is expected to be of great help to advertisers in determining the level of benefits necessary to elicit a response from potential consumers. Thus, AI gaze analysis according to the present invention can be performed in real time by the ad server, and in this case, by changing the benefit conditions offered by the currently displayed advertisement in real time, the user's behavior can be more effectively converted.

[0043] FIG. 1 is a diagram illustrating an Internet advertising system using AI gaze analysis according to one embodiment of the present invention.

[0044] FIG. 2 is a diagram showing the configuration of AI software that can be used for AI gaze analysis according to one embodiment of the present invention.

[0045] FIG. 3 is a diagram for explaining the operation method of computer vision AI software according to one embodiment of the present invention.

[0046] FIGS. 4 and 5 are diagrams for exemplarily explaining image data and related gaze data analysis information analyzed in real time by computer vision AI software according to one embodiment of the present invention.

[0047] FIG. 6 is a diagram showing an example of analyzing a user's gaze using AI and synthesizing it with a specific website in the form of a scanpath (gaze movement path) map based on saccade (instantaneous movement of the eye) according to one embodiment of the present invention.

[0048] FIG. 7 is a diagram showing an example of a heat map created by analyzing a user's gaze using AI and based on the period of time during which the eye's focus is fixed (a moment when the eye is relatively fixed) according to one embodiment of the present invention.

[0049] FIG. 8 is a diagram illustrating an example of changing the exposure method for advertisements that a user is interested in after completing AI analysis of the user's gaze according to one embodiment of the present invention.

[0050] FIG. 9 is a diagram illustrating another example of changing the exposure method for advertisements that a user is interested in after completing AI analysis of the user's gaze according to one embodiment of the present invention.

[0051] FIG. 10 is a diagram showing an example of upwardly adjusting a benefit proposal (Offering) to a user in real time based on an AI analysis result according to one embodiment of the present invention.

[0052] Hereinafter, an exemplary embodiment of the present invention will be described with reference to the attached drawings.

[0053] FIG. 1 is a diagram illustrating an Internet advertising system (500) using AI gaze analysis according to one embodiment of the present invention. Referring to FIG. 1, the Internet advertising system (500, "AI advertising system") using AI gaze analysis according to one embodiment of the present invention is composed of an AI advertising server and a database (100). As will be described later in FIG. 8, the database (110) is linked with the server (100), and therefore may be collectively referred to as the AI ​​advertising server and database (100). As will also be described later in FIG. 2, the AI ​​software (200) according to the present invention can also be viewed as being integrated and installed in the AI ​​advertising server and database (100), as illustrated in FIG. 1. In order for the AI ​​advertising server and database (100) to operate as an advertising agency, interaction with an advertiser terminal (300) is required, and interaction with a client or consumer terminal (400, for example, a smartphone of an Internet user who will be exposed to advertisements; a client terminal) via a wired or wireless network is also required.

[0054] Meanwhile, with regard to the camera (410), further explanation will be given with reference to FIG. 8, which will be described later. Recently, the camera (410, regardless of front or rear camera) of a smartphone (400) includes, for example, a wide-angle camera (411) capable of high-quality shooting of 12 MP (megapixels), a telephoto camera (412), and an ultra-wide camera (413). Furthermore, for example, other small devices such as a camera flash (414), a LiDAR (Light Detection and Ranging) scanner (not shown), and a microphone (not shown) for noise cancellation of background sound when shooting an image are often attached around these cameras (411, 412, 413). In the present invention, for the sake of convenience, various types of cameras (411, 412, 413) or, for example, an infrared camera attachable to a smart device, will be collectively referred to as a "camera (410)."

[0055] Referring back to FIG. 1, in step S1, the AI ​​advertising server and database (100) receives an online advertising request for Internet advertisements from an advertiser terminal (300) based on a predetermined advertising fee contract. The advertising exposure method, such as the posting period, posting location, and posting method of the Internet advertisement, is usually determined in advance in this advertising fee contract. However, the present invention may require a system setting that allows the AI ​​advertising server and database (100) to dynamically control the advertising exposure method, preferably in real time. In addition, as in the case of FIG. 10 described below, the present invention requires the AI ​​advertising server and database (100) to be able to increase the level of promotional benefits presented by the advertisement all at once or in stages. Therefore, it may be necessary to consult in advance with the advertiser (300) so that the AI ​​advertising server and database (100) has the authority to change the promotional benefits in real time.

[0056] For reference, in the case where the AI ​​advertising server and database (100) executes and processes AI software (200) in real time, for example, it is particularly desirable to improve the computing speed and computing performance in hardware by combining a central processing unit (CPU), a graphic processing unit (GPU), field programmable gate arrays (FPGA), and various accelerator cards in the AI ​​advertising server and database (100), in order to process complex AI operations.

[0057] In step S2, the AI ​​advertising server and database (100) transmits Internet advertisements to the client terminal (400) based on the request of the advertiser (300). Referring to FIGS. 8 and 9, the Internet advertisement "Dress Sale! Shop Here!" transmitted in step S2 will be placed in a specific advertising area (421) within the screen (420) of the smartphone (400). Referring to FIG. 10, the digital banner advertisement (920) "10% Coupon!" transmitted in step S2 will be placed in another specific advertising area (422) within the screen (420) of the smartphone (400).

[0058] In step S3 of FIG. 1, the AI ​​advertising server and database (100) preferably receives images or video data captured by the camera (410) of the client terminal (400) in real time. Of course, in order to increase the effectiveness of advertising from the client terminal (400), consent for personal information regarding camera capture and external transmission of the data must be obtained in advance. The image or video data transmitted in step S3 is preferably facial data including at least the user's eyes. While AI can filter out images containing objects or backgrounds other than the user, all or at least a portion of facial data including the eyes is required to analyze the user's gaze information according to the present invention.

[0059] In step S3 of FIG. 1, in addition to the user's facial information, the AI ​​advertising server and database (100) may receive and collect various client data information. For example, if the AI ​​advertising server and database (100) is a search engine operator, it may receive information such as the website history or content viewing history accessed by the client terminal (400) with the user's consent to provide personal information. In this way, by collecting and storing tracking information (hereinafter "client tracking information") regarding the user's app usage or web activities occurring on the client terminal (400), the AI ​​advertising server and database (100) can compile more accurate advertising performance and analyze whether or not an Internet advertisement was clicked due to a certain user action. The client tracking information collected and stored in this way may also be subject to analysis by the AI ​​software (200) described later in FIG. 2.

[0060] In step S4, AI gaze analysis, which will be explained in more detail in FIGS. 2 and 3, is performed. The physical subject of the analysis is the AI ​​advertising server and database (100), as illustrated in FIG. 1. However, the AI ​​software (200) may be installed on another external server (not shown) that can be connected to the AI ​​advertising server and database (100) via a wired or wireless network to execute step S4, or the design may be changed to execute AI gaze analysis in a state where it is installed with the user's consent on a client terminal (400) and only the results are transmitted to the AI ​​advertising server and database (100).

[0061] Next, in step S5, the AI ​​advertising server and database (100) determines and executes an advertising exposure method that can attract more attention of the client terminal (400) user based on the user gaze analysis results obtained in step S4, with the consent of the advertiser.

[0062] Here, an advertisement exposure method that can attract more attention of the user is further described with reference to, for example, FIGS. 8 and 9. If, as a result of the user gaze analysis by the AI ​​software (200), it is determined that the user is looking at the advertisement area (421) where the Internet advertisement "Dress Sale! Shop Here!" described above is located, (a) in order to attract more attention from the user, an Internet advertisement (900) having the same advertisement content produced in a larger size can be transmitted to the client terminal (400), preferably in real time. Of course, the newly produced Internet advertisement (900) can be placed in the advertisement area (421) where the original Internet advertisement was located, as in FIG. 8, or can be placed in another appropriate location (not shown), particularly in a location where it is convenient for the user to click by hand, by utilizing the user gaze map (Gaze Map, see FIGS. 6 and 7) information described later. In addition, the AI ​​advertisement server and database (100) can additionally place an image of a person's hand (910) clicking something in the advertisement area (421), as illustrated in FIG. 9, to (b) make it easier for the user to click on the Internet advertisement "Dress Sale! Shop Here!" that the user is looking at. That is, if the user has already shown interest in an advertisement, by adding an additional means (910, for example, an image of a person's finger clicking or an Animated GIF file) to induce a clicking action, as illustrated in FIG. 9, the user is naturally induced to click on the advertisement "Dress Sale! Shop Here!" It should be noted that in the case of the present invention, since a means (for example, 910) to further attract the user's interest is provided when the AI ​​determines that the user has already expressed interest, the user is more likely to find the advertisement method of interacting with his or her gaze fun rather than feeling uncomfortable, and thus, to actually click.

[0063] Finally, in step S6, the AI ​​advertising server and database (100) reports (or feedbacks) the results obtained through steps S1 to S5 (which may include the aforementioned client tracking information, AI analysis results, and advertising performance or statistical data based on the results, depending on whether consent was given to providing personal information) to the advertiser terminal (300). Based on the report, the advertiser terminal (300) can receive assistance in establishing future advertising strategies, such as creating new Internet advertisements or developing new marketing strategies, and then return to step S1 so that the advertiser terminal (300) can request Internet advertisements from the AI ​​advertising server and database (100) again. In addition, as will be described later, step S6 includes a step of transmitting to the advertiser terminal (300) the results of whether a new advertisement (i.e., drawing symbol 930) that has enhanced the benefits of a promotion, as illustrated in FIG. 10, actually induced a user's purchase.

[0064] FIG. 2 is a diagram showing the configuration of AI software (200) that can be used for AI gaze analysis according to one embodiment of the present invention.

[0065] AI software (200) typically has to go through several processes, such as (a) problem definition, (b) data acquisition and preparation, (c) model development and training, (d) model evaluation and refinement, (e) deployment of AI in an actual product, and (f) execution of machine learning operations, repeatedly, either periodically or aperiodically. Since these processes are not completely independent of each other but are interconnected, as described above, the AI ​​analysis according to the present invention does not need to be limited to being performed physically in only one place. Rather, it may be desirable for an external server (not shown) and an AI advertising server and database (100) to be linked to efficiently assist the AI ​​operation process. In addition, as already described, according to another embodiment of the present invention, at least a part of the AI ​​software (200) can be configured to be run on a client terminal (400).

[0066] Continuing with reference to FIG. 2, the AI ​​software (200) according to one embodiment of the present invention may include a generative AI tool (210). The generative AI tool (210) receives training data and, based on the patterns and structures of the input training data, generates similar text, images, or media. In the present invention, for example, it is possible to produce a means (e.g., 910) that can attract more attention from users, as shown in FIG. 9, in various formats such as images, texts, and videos, and this production process may be assisted by the generative AI tool (210). In this case, the generative AI tool (210) may be particularly important to advertisers, because, as described above, the means (e.g., 910) that can attract more attention from users is ultimately a means for increasing conversion rates, i.e., CVR. On the other hand, the generative AI tool (210) may become particularly important to advertisers, for example, as illustrated in FIG. 10 , which will be described later. This is because not only can a new advertisement (930, FIG. 10 ) produced using the generative AI tool (210) serve as an important indicator of how much promotional benefits must be increased before users' purchasing behaviors are converted, but it can also directly impact the advertiser's (300) margin.

[0067] Furthermore, the present invention proposes a method of adopting a large language model (211, Large Language Model; LLM) as a sub-model for training AI software (200) with a large number of sample languages ​​related to marketing or Internet advertisements in order to improve the performance of a generative AI tool (210). In addition, since recent Internet advertisements are often made in videos produced by others, it is also desirable to equip the generative AI tool (210) with a multimodal foundation model (212, Multimodal Foundation Model; MFM) capable of simultaneous learning of text, images, and videos.

[0068] The AI ​​software (200) of FIG. 2 may include a machine learning (ML) tool (220). For example, numerous selfie camera data stored on the user's smartphone (400) may be utilized as important machine learning training data for the AI ​​software (200) to analyze the user's gaze. Furthermore, by collecting the user's facial data, head posture data, and gaze data in various situations, such as when the user is wearing glasses or a mask, or when the user is viewing the smartphone at an angle different from the usual angle, the accuracy of the user's gaze analysis may be increased.

[0069] A machine learning tool (220) according to one embodiment of the present invention may adopt a deep learning model (221) that divides given data into multiple layers and analyzes them, and may also adopt a supervised learning model (222) that implements learning modeling so that the user's gaze analysis results can lead to output values ​​such as interest inducing means in the form of drawing symbols 900 or 910 in the attached FIGS. 8 and 9. In other words, rather than having the machine do all the inference, it is a method of driving machine learning by inputting in advance an expected value of what kind of result is desirable. Of course, it is also possible to adopt an unsupervised learning model (223) that does not provide the above learning label data to the training data, but rather improves the accuracy of user gaze recognition by having the AI ​​software (200) itself imitate the training data on a neural network and correct trial and error.

[0070] Additionally, it is preferable that the AI ​​software (200) according to one embodiment of the present invention include a natural language processing (NLP) tool (230). While it is true that data related to the user's face and eyes are most important in the gaze analysis of the present invention, this is to enable filtering out text or background engraved on the user's clothing during the gaze analysis.

[0071] The natural language processing tool (230) preferably includes a Natural Language Understanding (NLU) model (231) that enables a machine to interpret a given sentence using lexicon, parser, and grammar rules. A Natural Language Generation (NLG) model (232), which sometimes enables things that are not grammatically correct or non-linguistic to be expressed in language, may be particularly useful in the AI ​​analysis (i.e., step S4) according to the present invention and the aforementioned step S5. Note that since the AI ​​advertising system (500) according to the present invention is not intended for a specific country, the translation function of the AI ​​software (200) according to the present invention may be quite important, although it may seem unrelated to facial recognition. For example, if a traveler traveling from Korea to France is browsing a smartphone (400) in front of a specific French museum, images or text information regarding local French museum promotions reflected in the smartphone's (400) camera (410) can be subject to AI analysis. Similarly, if, for example, the new advertisement (930) of FIG. 10 includes a foreign language, a natural language processing tool (230) could be usefully utilized.

[0072] Referring back to Figure 2, the AI ​​software (200) according to the present invention must include a computer vision tool (240). The present invention utilizes a technique for analyzing user gaze and estimating user interest in Internet advertisements based on the results. Therefore, the computer vision tool (240) is essential for user gaze analysis.

[0073] First, it is preferable that the computer vision tool (240) use an object detection model (241) that extracts only the objects necessary for gaze analysis from images or videos of a person's pupils, nose, mouth, hair, glasses, hands, and other objects in the background. For example, in an offline store, there is a case where a user compares the on-site product price with the online price while looking at the screen of a smartphone (400). In this case, in order to accurately recognize the user's gaze reflected in the camera (410) of the smartphone (400), it may be necessary to not only recognize the user's face, but also distinguish objects in the background other than the user.

[0074] The Scene Understanding model (242) is also one of the AI ​​models that can be adopted in the computer vision tool (240). The Scene Understanding model (242) performs AI analysis on which objects in an image or video should be treated as more important, and which objects have a certain degree of importance or priority over other objects. Even a user's face, which is merely a collection of pixels from the machine's perspective, can be segmented into various objects such as eyes, nose, mouth, pupils, irises, and eyelids by applying the Scene Understanding model (242), and for the purpose of gaze analysis, calculations on which objects should be treated as important become possible.

[0075] The Face Detection and Recognition model (243) of FIG. 2 is a technology that basically searches for and identifies human faces in digital images or videos. It is an AI technology utilized in social media, photo organizing apps, facial recognition security access control, and even criminal investigations, and can infer the age (e.g., pupillary reflex darkens with age) and gender, as well as emotions, of the subject of analysis from facial expressions and appearance. That is, the user's facial data collected by the camera (410) according to the present invention becomes the subject of analysis by the above face detection and recognition model (243), and this is combined with the user gaze analysis results described below to enable more precise analysis of the user's interest in advertisements. For example, the AI ​​advertisement server and database (100) can detect changes in the user's facial expression after viewing an advertisement and provide feedback to the advertiser (300) as to whether the advertisement needs to be modified.

[0076] Meanwhile, the Eye and Gaze Tracking model (244) of FIG. 2 is a core AI component in the present invention. The Eye and Gaze Tracking model (244) can be broadly divided into two sub-fields. One is to determine the position of the eye (Eye Localization), and the other is to determine the direction of the eye's gaze (Gaze Estimation). For reference, when analyzing eyes using AI, the "eye" mainly refers to the pupil (including both the dark pupil and the bright pupil) and the iris, and in addition to pixel data on the pupil and iris, image or video information related to the iris reflection, limbus, pupil contour, and eyelid is also utilized for eye analysis. In other words, eye localization focuses on accurately determining the presence and location of a human eye within a given image or video, while gaze estimation focuses on interpreting the location of the eye for each frame of an image or video and determining the person's current gaze and its movement direction in three-dimensional space.

[0077] Due to the technical differences described above, it may be difficult to treat the eye tracking model and the gaze tracking model identically in principle. However, from the perspective of eye oculography, it is possible to combine the two models and refer to them as an integrated eye and gaze tracking model (244), as illustrated in FIG. 2. For the purposes of the present invention, the integrated term "eye and gaze tracking model (244)" is deemed sufficient.

[0078] For reference, during the gaze tracking phase, AI can also reference information about the position and posture of a human head. Furthermore, even within the eye area, color can slightly vary depending on the surrounding environment, and research is being conducted on the impact of the surrounding environment on the color of the eyes within an image. Therefore, the present invention proposes a comprehensive analysis of information about the human face, facial expressions, body and head posture, glasses (especially necessary for correcting distortion of eye pixel data caused by thick magnifying glasses), and the background screen within the camera, rather than simply performing AI analysis of the eyes. As continually emphasized, since the present invention determines whether a user is interested in Internet advertisements based on AI gaze analysis, it will be possible to more accurately determine the user's interest through various combinations of the models illustrated in Figure 2.

[0079] It can be easily understood that the computer vision tool (240) of FIG. 2 includes a motion analysis model (245) in light of the aforementioned aspects. For example, since an eyelid can sometimes appear as a straight line or as an oval in a single image depending on the position of a person's head, defining a parameter such as "the shape of the eyelid must be a certain shape" may actually cause errors in accurate gaze analysis. The motion analysis model (245) is a technique for determining what motion was performed in a captured image based on two or more continuous image sequences created by a camera or high-speed camera that captures the image. Such motion analysis data may be necessary for precise calculation of the eye and gaze tracking model (244).

[0080] Finally, the computer vision tool (240) may additionally include a text recognition model (246), typically an optical character recognition (OCR) model (246). If a user image captured by the camera (410) of a smartphone (400) includes a large amount of text as a background, this may provide some assistance in AI analysis as a supplementary means to the scene understanding model (242) mentioned above.

[0081] Before moving on to Figure 3, the AI ​​software (200) according to the present invention is a so-called "Chat GPT" that is recently popular. TM "I would like to clearly point out that this is different from the above. As shown in FIG. 2, the AI ​​software (200) for user gaze analysis according to the present invention may include a generative AI tool (210), but this does not mean that the AI ​​advertising technique according to the present invention is completely unrelated to a system similar to Chat GPT, for example, where the user asks something to the AI ​​in an AI dialogue window and Chat GPT provides a result. Since the present invention does not include a component such as Ghat GPT, where the user thinks about the optimal question to ask the AI ​​to obtain the desired result, the AI ​​gaze analysis technology according to the present invention or the technology for changing the advertisement exposure method accordingly is unrelated to Chat GPT.

[0082] FIG. 3 is an exemplary drawing for explaining the operation method of AI software (200) equipped with a computer vision tool (240) according to one embodiment of the present invention.

[0083] Referring to FIG. 3, image or video data captured by a camera (410) of a smartphone (400) is provided as the initial input value of a computer vision tool (240) through step SS1. The image or video data is input to an eye tracking unit (240a) in step SS2 and to an eye detection unit (240d) in step SS4. In step SS3, the eye tracking unit (240a) identifies pixel data related to the pupil and iris in the image or video, and collects pixel data related to corneal reflection, limbus, pupil outline, and eyelid, thereby executing an eye localization operation to accurately determine the presence and location of a human eye in a given image or video. The eye location information obtained in step SS3 may be transmitted to the eye detection unit (240d) together with the camera data transmission in step SS4. The eye detection unit (240d) can cyclically correct the image or video data continuously input from the camera (410) in real time by feeding back the initial (and subsequent) eye position detection results recognized by the eye detection unit (240d) to the eye tracking unit (240a) in step SS5.

[0084] Meanwhile, the head posture determination unit (240b) analyzes information about the head posture among the images or videos of the camera (410) (step SS7), and sends the analysis result to the gaze determination unit (240c) in step SS8. The eye tracking unit (240a) also transmits data related to the position of the eyes to the gaze determination unit (240c) in step SS6. The gaze determination unit (240c) performs a gaze analysis operation to determine where the user's eyes are directed in a three-dimensional space by combining, for example, the eye position information received from the eye tracking unit (240a) and the information about the head posture received from the head posture determination unit (240b). The final analysis results of the gaze determination unit (240c) and the eye detection unit (240d) (i.e., gaze coordinates of the gaze determination unit (240c) and eye position coordinates of the eye detection unit (240d)) are each transmitted to the application unit (100a) existing in the AI ​​advertisement server and database (100), and then the application unit (100a) executes step S4 of FIG. 1. That is, the application unit (100a) is responsible for organizing, classifying, and transmitting the user gaze analysis results to the AI ​​advertisement server and database (100) just before the AI ​​advertisement server and database (100) determines and executes an advertisement exposure plan that can attract more attention of the client terminal (400) user in step S5. Furthermore, the application unit (100a) determines whether a new advertisement (e.g., drawing symbol 930, Fig. 10) should be generated based on the AI ​​software (200)'s judgment result regarding whether the user is looking at an existing advertisement (e.g., drawing symbol 920, Fig. 10). In order to objectively determine whether the user is looking at an existing advertisement (e.g., drawing symbol 920, Fig. 10), it is desirable to establish mathematical criteria, such as those in Table 1 described below.

[0085] FIGS. 4 and 5 are diagrams for exemplarily explaining image data and related gaze data analysis information analyzed in real time by computer vision AI software according to one embodiment of the present invention. In FIGS. 4 and 5, the drawing symbol 600 represents a camera image (Video) screen (600) including a user recognized in real time by, for example, a camera (410) of a smartphone (400). The application unit (100a) of FIG. 3 utilizes the gaze analysis results as shown in FIGS. 4 and 5.

[0086] Referring to FIGS. 4 and 5 together, a camera image screen (600) is virtually overlapped with an area (610) for recognizing a user's face. In this virtual face recognition area (610), virtual dots called key marks (611) are created throughout the face image as shown in FIGS. 4 and 5, and these key marks (611) also move in real time according to the movement of the user's face or eyes. Thereafter, a preset calculation program is used to calculate the pitch (angle when nodding the head up and down) and yaw (angle when turning the head left and right, also called dori dori movement) values ​​of the target person. For reference, the pitch is measured from -90 degrees to +90 degrees based on an imaginary horizontal line passing through both ears, and the yaw can be measured from -180 degrees to +180 degrees based on an imaginary vertical line passing through the crown of the head when the head is held straight. These pitch and yaw values ​​also change in real time according to the person's movement, and can be overlapped on the camera image (600) in the form of drawing symbol 620 as shown in FIGS. 4 and 5.

[0087] In FIGS. 4 and 5, it can be seen that the virtually overlapped arrow (630) and the also virtually displayed point (640) change as the pitch and yaw values ​​change. The arrow (630) indicates the direction of the user's gaze, and the point (640) indicates the point within the smart device (400) that the user is looking at. Accordingly, if the user turns his / her head in a different direction without looking at the screen (420) of the smart phone (400) at all, the point (640) that the user is looking at may not appear.

[0088] Next, FIG. 6 is a drawing showing an example of a screen (700) that analyzes a user's gaze using AI according to one embodiment of the present invention and synthesizes it with a specific website in the form of a scanpath (gaze movement path) map based on saccade (instantaneous movement of the eye). That is, while FIGS. 4 and 5 illustrate an image (600) captured by a camera, FIG. 6 can be viewed as a gaze map that tracks how a user's gaze moved on a screen (420, see, for example, FIG. 8) of a smartphone (400) based on the gaze analysis results as in FIGS. 4 and 5. In short, the screen (700) of FIG. 6 shows a first example of a gaze map.

[0089] Referring to FIG. 6, it can be seen that the user did not scan the area where the Internet advertisement (710) was located with his or her eyes at all. That is, according to the gaze map (600) of FIG. 6, the user's eyes started from P1 and repeated saccadic eye movements, looked only up to points P2, P3, ..., P13, and did not look at the webpage any further. In this case, it is assumed that a marketing method that attempts to forcibly draw the user's attention to the Internet advertisement (710), such as forcibly opening a pop-up window, does not have much meaning in the present invention. This is because an Internet advertisement (710) that fails to attract the user's attention may be perceived as cyber pollution.

[0090] FIG. 7 is a diagram illustrating an example of a heat map (800) created by analyzing a user's gaze using AI and based on the duration of fixation (a moment when the eyes are relatively fixed) according to one embodiment of the present invention. The term "heat" is used because, like a thermal imaging camera, the longer a user gazes, the stronger the color is expressed.

[0091] In short, the screen (800) of FIG. 7 can be seen as a second example of a gaze map produced to improve advertising efficiency according to the present invention. In the case of FIG. 7, the darker the concentration (i.e., heat index) is, the longer the user gazed at it. For example, assuming that the areas with relatively high heat indices in FIG. 7 are areas H1 and H2, it can be confirmed that area H2 overlaps with the Internet advertisement (810). In other words, it can be seen that the Internet advertisement (810) of FIG. 7 has attracted the user's attention, and in step S4 of FIG. 1, the AI ​​advertisement server and database (100) will prepare to move on to step S5.

[0092] Of course, for example, in FIG. 7, the level of gaze concentration (or heat index) acquired by an Internet advertisement (810) can be used as a threshold for determining whether or not a user is interested. Even in the example of FIG. 6, whether or not a user is considered interested simply because an Internet advertisement (710) is present on the scan path may be a matter of system settings rather than an absolute interest determination criterion.

[0093] The following table presents variables related to eye measurement that can be used by AI software (200) including a user eye and gaze tracking model (244) according to the present invention. Rather than all of these variables being used as criteria for determining whether an advertisement has attracted the user's attention, thresholds can be set by combining one or more of these variables.

[0094] <Example of eye measurement variable values> Variables related to eye tracking Unit of variable Number of saccades (N) (N: natural number) Average size of saccades Pixel (Pixel) Length of scan path Pixel Duration of scan path Msec (1 / 1000 second) Convex hull Pixel 2 (Size of the area created by connecting the outermost vertices of the scan path) Spatial Density % (Ratio of the total screen area to the area where the eye fixation occurred during the scan path) Fixation period Msec (1 / 1000 second) Number of fixations N (N: natural number) Pitch and Yaw Degree (angle)

[0095] For example, in Table 1 above, when the main server (not shown) of the AI ​​advertising server and database (100) determines whether the user's gaze overlaps with an advertisement display area (e.g., symbol 421 of FIG. 8, described later) on which an Internet advertisement (e.g., a Dress Sale! Shop Here! advertisement of FIG. 8, described later) is posted, the determination criteria may be set as a combination of at least one or more of (a) whether the measured focus fixation duration exceeds a predetermined threshold, (b) whether the advertisement display area exists on the scanpath, and (c) whether the display area exists within the convex hull created by the scanpath. For reference, the convex hull refers to the size of the area created by connecting the outermost vertices of the scanpath, and in the case of FIG. 6, it would be a polygon connected in the order of P1, P13, P4, P5, P8, P2, and P3, for example. The "Shop Now! For Sale!" advertisement (710) in the example illustrated in FIG. 6 can be considered not to have attracted user attention even according to criterion (c) among the three exemplary criteria described above. Even in the case where the AI ​​advertisement server and database (100) track whether the user's gaze overlaps with the advertisement display area (422, FIG. 10) where the Internet advertisement "10% Coupon" (920) illustrated in FIG. 10 is posted, one or more of Table 1 and the three criteria described above may be applied.

[0096] FIG. 8 is a diagram illustrating an example of changing the exposure method for advertisements that a user is interested in after completing AI analysis of the user's gaze according to one embodiment of the present invention. As described in FIG. 1, in step S3, client tracking information and image or video data collected by the camera (410) are transmitted to and stored in the AI ​​advertising server and database (100). In the case of FIG. 8, the advertisement area (421) is already displaying the advertisement "Dress Sale! Shop Here!" banner that has received an advertisement request from an advertiser (300), and analysis of the user's gaze looking at the display screen (420) of the smartphone (400) is preferably performed in real time on the AI ​​advertising server and database (100).

[0097] If the AI ​​analysis result determines that the advertisement satisfies the threshold of user interest among the variables presented in Table 1, for example, the AI ​​advertisement server and database (100) proceeds to step S5, and, based on the user gaze analysis result performed in step S4, determines an advertisement exposure method that can attract more attention of the client terminal (400) user, and preferably executes the method in real time. For example, in the case of FIG. 8, if the user gazes at the "Dress Sale! Shop Here!" banner for a certain period of time, the AI ​​advertisement server and database (100) immediately sends a command to increase the size of the "Dress Sale! Shop Here!" banner advertisement in step S5 to take action to attract more attention of the user.

[0098] FIG. 9 is a diagram illustrating another example of changing the exposure method for an advertisement that a user is interested in after completing AI analysis of the user's gaze according to one embodiment of the present invention. The difference from FIG. 8 is that the AI ​​advertising server and database (100) executes step S5 so that a finger icon (910) meaning "Click here" is displayed on the "Dress Sale! Shop Here!" banner advertisement displayed in the advertising area (421). Meanwhile, the database (110) commonly illustrated in FIGS. 8 and 9 classifies and stores image or video data collected by the camera (410) in the AI ​​advertising server and database (100), and the billing part (120) is, for example, a server for cost settlement. In the case where a user click is successfully made through a change in the advertising exposure method, such as drawing symbol 900 or 910 according to the present invention, the details are aggregated and statistics are provided to the advertiser (300), and additional advertising fees can be charged for successful marketing portions based on the use of the AI ​​function.

[0099] Finally, Figure 10 illustrates an example of upgrading a benefit offer to a user in real time based on AI analysis results, according to one embodiment of the present invention. As illustrated in Figure 10, the user is currently viewing a "10% Coupon" advertisement (920). Note that this conclusion is derived after the AI ​​analysis, which determines that the predetermined criteria exemplified in Table 1 above have been met, is completed in real time after Step S3.

[0100] Afterwards, the AI ​​advertising server and database (100) transmits a new advertisement (930) in real time and displays it on the screen (420) of the client terminal (400). At this time, for example, it would be desirable to use an AI model related to language processing mentioned above to provide a text as a condition, such as "Thank you for looking at this coupon now! As a token of our appreciation, we will provide you with a 20% discount coupon for 1 minute", to help the user understand why the coupon benefit level was suddenly adjusted upward.

[0101] For reference, the creation of a new advertisement (930) in FIG. 10 can be handled by the generative AI tool (210) as described above, or an advertisement previously received and stored from the advertiser's terminal (300) can be used as the new advertisement (930). In either case, since upward adjustment of the advertisement's benefits can be a sensitive matter related to advertising costs and margins, prior approval from the advertiser (300) must be obtained.

[0102] If a purchase conversion occurs when a new advertisement (930) is presented, an additional advertising fee similar to a success fee can be charged and transmitted along with client tracking information in step S6 of the aforementioned FIG. 1, and the work related to such fees can be handled by the billing server (120).

[0103] So far, with reference to FIGS. 1 to 10, we have examined in detail the advertising method, system, and related AI program based on AI gaze analysis according to the present invention. As described above, the present invention proposes to use an AI analysis tool (200, 240) that is a complex computer vision capable of processing images, videos, or both, in particular, in analyzing the interests of Internet users. As previously mentioned, the present invention focuses on Internet advertisements (e.g., 710, 810) targeting users of a smart device (400) equipped with a camera (410). That is, the present invention uses a camera equipped in a smart device to analyze which part of the screen (420) that the user is currently viewing is fixated on, and thereby precisely confirms the user's interest, thereby providing technical measures so that the advertiser (300) can also accept the method of measuring the user's (400's) interest according to the present invention as reasonable. In particular, it is noteworthy that by objectifying and quantifying the criteria for determining the presence of user interest based on the results of gaze analysis and introducing a threshold value, the reliability of advertising effectiveness can be highly recognized by advertisers (300), and the advertising exposure method has been made more accessible to consumers (400) by targeting only useful information.

[0104] Furthermore, the present invention dynamically changes the exposure method of Internet advertisements, for example, as shown in FIG. 8 or FIG. 9, based on the user's gaze analyzed by AI, and uses the gaze map shown in FIG. 6 or FIG. 7 as a criterion for determining such dynamic change, thereby enabling analysis of where on the screen of a smartphone the space for placing an advertisement would be most ideal when viewed in light of the gaze map as shown in FIG. 6 or FIG. 7. This analysis is expected to be included in the feedback provided to the advertiser (300) in step S6, thereby providing the advertiser (300) with better insight into subsequent advertisements.

[0105] In addition, if the AI ​​gaze analysis according to the present invention is performed in real time in the AI ​​advertisement server and database (100), the advertisement exposure method can be changed in real time according to the situation, which is expected to maximize advertisement efficiency. In particular, since the present invention determines the user's interest based on the user's web surfing that changes from moment to moment and the resulting pupil movement, it may be more desirable to configure a series of actions from step S3 to S5 of FIG. 1 or the entire process illustrated in FIG. 1 to be performed in real time.

[0106] As described above, the present invention uses AI to filter out advertisements that do not attract the attention of Internet users and advertisements that do, and thereby enables dynamic advertisement exposure that can increase the advertising effect of advertisements that can provide useful Internet information to users, thereby improving the efficiency of advertising cost execution from the advertiser's perspective, and from the user's perspective, advertisements presented according to the present invention can be recognized as useful information.

[0107] On the other hand, in the case of the present invention, since not only the exposure method of Internet advertisements is changed, but also the promotional benefits of the advertisements themselves are adjusted in real time as exemplified in FIG. 10, it is expected that step S6 can provide the advertiser (300) with marketing insights that cannot be provided by other existing advertising techniques. That is, in the example of FIG. 10, if it is determined that a 20% discount rate will bring about a better sales increase effect even if it reduces the margin, it may be possible to return to step S1 and designate a new advertisement (930) instead of the existing advertisement (920) as the advertisement to be commissioned.

[0108] In addition, the new advertisement (930) with enhanced promotional benefits according to the present invention may be provided to all consumers viewing the existing advertisement (920), or may be provided on a trial basis to only some consumers depending on the restrictions set by the advertiser. Because the purpose of the present invention is to provide the most reasonable marketing insight to the advertiser (300), although not illustrated in FIG. 10, the process of the AI ​​advertisement server and database (100) determining whether or not the new advertisement (930) can be transmitted may be performed before step S5 is executed.

[0109] As described above, the present invention not only changes the way advertisements are displayed, but also utilizes AI to filter out ads that fail to capture the attention of Internet users from those that do. This, in turn, increases the advertising benefits of advertisements that provide valuable online information to users, actively encouraging conversions such as clicks or purchases. This will allow advertisers to gauge the level of promotional benefits necessary to drive users to make a purchase.

[0110] The drawings and detailed description of the present invention are not intended to limit the scope of the present invention. Anyone skilled in the art will be able to make various design changes without departing from the technical spirit of the present invention by referring to the attached drawings and detailed description. For example, in the attached drawings of the present invention, the AI ​​software (200) is depicted as performing computational operations within the AI ​​advertising server and database (100). However, as described above, all or part of the functions of the AI ​​software (200) may be installed in the client terminal (400) with the customer's consent or may receive support from an external AI server (200).

Claims

1. A method for advertising by sending Internet advertisements from an advertisement server to a user terminal equipped with a camera, A first step of receiving the above Internet advertisement from an advertiser's terminal; A second step of transmitting the received Internet advertisement to the user terminal so that the Internet advertisement is posted and displayed in the advertisement display area of ​​the viewing screen of the user terminal; A third step of collecting image or video data including the user's face through the user terminal and the camera so that it can be processed by an AI analysis tool installed on the user terminal or the advertising server; A fourth step of extracting information related to the user's gaze from the collected image or video data by the AI ​​analysis tool, and determining whether the user's gaze overlaps with the advertisement display area where the Internet advertisement is posted based on the user's gaze information; and An Internet advertising method using AI gaze analysis, characterized in that it includes a fifth step of changing the display method of the Internet advertisement within the user terminal when it is determined that the user's gaze is directed toward the advertisement display area.

2. In paragraph 1, An Internet advertising method using AI gaze analysis, characterized in that the above-mentioned changing exposure method includes a method of increasing the size of the part where the Internet advertisement is posted and exposed or a method of changing the location on the viewing screen where the Internet advertisement is posted and exposed.

3. In paragraph 1, The above AI analysis tool includes a complex computer vision AI analysis tool that can process images, videos, or both. An Internet advertising method using AI gaze analysis, characterized in that the criteria for determining whether the advertisement display area and the user's gaze overlap in the fourth step are comprised of at least one or a combination of more than one of (a) whether the measured focus fixation period exceeds a predetermined threshold, (b) whether the advertisement display area exists on the scan pass, and (c) whether the display area exists within a convex hull created by the scan pass.

4. In paragraph 1, An Internet advertising method using AI gaze analysis, characterized in that the AI ​​analysis tool comprises a step of generating a user gaze map regarding the user gaze information on the viewing screen and determining whether the advertisement display area and the user gaze overlap based on the map.

5. In an advertising server system that sends Internet advertisements to a user terminal equipped with a camera, A main server that receives the Internet advertisement from the advertiser's terminal and transmits the received Internet advertisement to the user's terminal so that the Internet advertisement is posted and displayed in the advertisement display area of ​​the viewing screen of the user's terminal; and A database is included that collects, classifies and stores image or video data including the user's face through the user terminal and the camera so that it can be processed by an AI analysis tool installed on the main server. By the AI ​​analysis tool, information related to the user's gaze is extracted from the collected image or video data, and based on the user's gaze information, the main server determines whether the user's gaze overlaps with the advertisement display area where the Internet advertisement is posted. An Internet advertising system using AI gaze analysis, characterized in that the main server changes the display method of the Internet advertisement within the user terminal when it is determined that the user's gaze is directed toward the advertisement display area.

6. The first step is for the advertising server to receive Internet advertisements from the advertiser's terminal; A second step of transmitting the received Internet advertisement to a user terminal equipped with a camera and displaying and exposing the Internet advertisement in a predetermined advertisement exposure area of ​​the display screen of the user terminal; A third step of collecting image or video data including the user's face through the user terminal and the camera so that it can be processed by an AI analysis tool installed on the user terminal or the advertising server; A fourth step of extracting information related to the user's gaze from the collected image or video data by the AI ​​analysis tool and determining whether the posted and exposed Internet advertisement attracts the user's attention based on the user's gaze information according to predetermined judgment criteria; and A digital advertising method using AI gaze analysis, characterized in that it includes a fifth step of adjusting the level of benefits included in the Internet advertisement in real time when it is determined that the posted and exposed Internet advertisement attracts the user's attention.

7. A main server that receives Internet advertisements from an advertiser's terminal, transmits the received Internet advertisements to a user terminal equipped with a camera, and posts and displays the Internet advertisements in a designated advertisement exposure area on the display screen of the user terminal; and A database that collects and stores image or video data including the user's face through the user terminal and the camera, A digital advertising system using AI gaze analysis, characterized in that the main server executes an AI analysis tool installed in the main server by interaction with the database, the AI ​​analysis tool extracts information related to a user's gaze from the collected image or video data, determines whether the posted and exposed Internet advertisement attracts the user's attention based on the user's gaze information according to predetermined judgment criteria, and if it is determined that the posted and exposed Internet advertisement attracts the user's attention, performs an operation of upwardly adjusting the level of benefits included in the Internet advertisement in real time.

Images