System for providing user-customized advertisement content and operating method thereof

Abstract

According to various embodiments of the present invention, an electronic device for displaying a user-customized advertisement image comprises a memory, a display, a camera, a communication module, and a processor. The memory stores instructions for instructing the processor to perform the operations below when the processor operates. The operations may comprise the steps of: identifying a user from an image captured by using a camera; checking, using a processor, user characteristic information of the user identified from the captured image and gaze information comprising a gaze position and a gaze direction; transmitting the user characteristic information and the gaze information to a cloud server; receiving, from the cloud server, a customized advertisement image generated in response to the user characteristic information; displaying the customized advertisement image through the display in place of the advertisement image; receiving, from the cloud server, image change information generated in response to the gaze information; and changing and displaying at least a part of the customized advertisement image according to the image change information.

Description

User-customized advertising content delivery system and method of operation thereof

[0001] The present invention relates to digital advertising technology, and more particularly to a system and method for providing customized advertising content by analyzing user characteristics (age, gender, gaze direction, etc.) in real time.

[0002] Existing digital advertising technology operated by providing fixed advertising content identically to multiple users. This conventional approach fails to provide a personalized user experience and hinders user immersion. In particular, advertising effectiveness was limited because it could not reflect users' interests or characteristics in real time.

[0003] With the advancement of digital advertising technology, advertising media incorporating cutting-edge technologies such as digital signage have emerged. Digital signage is a technology that provides information using displays such as LED, LCD, and OLED, enabling the delivery of dynamic and visually rich content that goes beyond static text or images. However, existing digital signage systems still lack user personalization and fail to provide customized advertising content based on user interests and characteristics.

[0004] In addition, conventional technology faces the problem of excessive costs and time required for advertising production. Advanced technology-based advertisements, such as 3D anamorphic ads used to promote specific products or services, require high production costs and time, making them accessible only to large advertisers. Due to these limitations, small and medium-sized advertisers struggle to produce and deliver personalized, high-quality advertisements.

[0005] While there have been attempts to personalize advertisements using user data, they have primarily relied on basic data analysis and failed to reflect real-time analysis or user interactions. Consequently, there is a growing need for a system that combines real-time data analysis with the generation of customized content.

[0006] Accordingly, the present invention has been devised to solve the aforementioned problems and provides a system and method for increasing the efficiency of advertising by analyzing user characteristics and gaze information in real time and generating and providing customized advertising content based thereon.

[0007] An electronic device for displaying a user-customized advertising video according to various embodiments of the present invention comprises: a memory; a display; a camera; a communication module; and a processor, wherein the memory stores instructions that cause the processor to perform the following operations when the processor is operating, the operations may include: identifying a user from an image captured using a camera; confirming, using the processor, user characteristic information of the user identified from the captured image and gaze information including the position and direction of the gaze of the gaze; transmitting the user characteristic information and the gaze information to the cloud server; receiving a customized advertising video generated in response to the user characteristic information from the cloud server; displaying the customized advertising video through the display instead of the advertising video; receiving video change information generated in response to the gaze information from the cloud server; and changing and displaying at least a portion of the customized advertising video according to the video change information.

[0008] According to one embodiment, the step of changing and displaying at least a portion of the customized advertisement video according to the image change information may include: measuring the distance between the user's pupils and the head angle included in the captured image; determining the user's gaze area based on the measured distance between the user's pupils, the head angle, and gaze information; and changing and displaying the display for at least one object present in the user's gaze area among the customized advertisement video.

[0009] According to one embodiment, the operations may include: a step of determining whether at least one object existing in the viewing area is a two-dimensional object; a step of converting the two-dimensional object into a three-dimensional object; a step of creating a skeletal structure corresponding to the created three-dimensional object; and a step of displaying the three-dimensional object combined with the skeletal structure through a display.

[0010] According to one embodiment, the user characteristic information includes at least one of the user's age and gender, and the customized advertising video can be generated by the cloud server: determining an advertising storyboard based on a pre-set advertising plan based on the identified user characteristic information, and generating the customized advertising content through a generative AI based on the determined advertising storyboard.

[0011] According to one embodiment, the cloud server can: calculate the content optimization degree of the customized advertisement video based on the user response to the customized advertisement video, and if the calculated optimization degree is less than a preset threshold, select a second customized advertisement video in which parameters related to the user response can be adjusted, and transmit the second customized advertisement video to the electronic device.

[0012] According to one embodiment, the above content optimization level can be determined based on user attention lead time, which indicates the time it takes for the content to attract user attention; user engagement rate, which indicates the ratio of time spent interacting with the content; content gaze time, which indicates the average time the user looks at the content with focus; and the frequency of the user's gaze deviation from the content.

[0013] According to one embodiment, the content optimization degree is calculated by the following mathematical formula, and

[0014]

[0015] L is the user attention lead time, T is the user engagement rate, V is the content gaze time, and D is the gaze break frequency.

[0016] According to one embodiment, the user focus lead time is calculated using the user's gaze information as the time from the start of content playback until the user's gaze is fixed on the display, the user engagement rate is calculated as the ratio of the time from when the content is displayed on the display until a reaction by the user occurs to the content playback time, the content gaze time is calculated using the user's gaze information as the ratio of the time accumulated in frame units for the user's fixed gaze on the content to the total content playback time, and the gaze departure frequency can be calculated using the gaze vector generated based on the gaze information and the position of the display as the ratio of the time the gaze vector leaves the display to the content playback time.

[0017] According to the present invention, advertising effectiveness can be maximized by providing advertisements tailored to the interests and characteristics of users. In addition, according to the present invention, advertising production costs and time can be reduced by utilizing AI-based analysis and content generation technologies.

[0018] FIG. 1 illustrates a user-customized advertising content provision system according to an embodiment of the present invention.

[0019] FIG. 2 is a block diagram of an electronic device according to one embodiment of the present invention.

[0020] FIG. 3 is an exemplary flowchart of an algorithm in which an electronic device according to one embodiment of the present invention operates.

[0021] Figure 4 is a diagram illustrating the user's gaze position and gaze direction in an advertising device.

[0022] FIG. 5 illustrates an example of generating a three-dimensional image of a product according to one embodiment of the present disclosure.

[0023] FIG. 6 is a diagram illustrating a rigging operation for a static object according to one embodiment of the present disclosure.

[0024] The advantages and features of the present invention and the methods for achieving them will become clear by referring to the embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below but may be implemented in various different forms. These embodiments are provided merely to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the invention, and the present invention is defined only by the scope of the claims. Accordingly, in some embodiments, well-known process steps, well-known device structures, and well-known techniques are not specifically described to avoid the present invention being interpreted ambiguously. Throughout the specification, like reference numerals refer to like components.

[0025] In this specification, terms such as first, second, third, etc., may be used to describe various components, but these components are not limited by these terms. The terms are used for the purpose of distinguishing one component from other components. For example, without departing from the scope of the present invention, a first component may be named a second or third component, etc. Since the present invention is capable of various modifications and may have various embodiments, specific embodiments are illustrated in the drawings and described in detail.

[0026] However, this is not intended to limit the invention to specific embodiments and should be understood to include all modifications, equivalents, and substitutions that fall within the spirit and scope of the invention. Similar reference numerals have been used for similar components in the description of each figure.

[0027] When it is stated that one component is "connected" or "connected" to another component, it should be understood that while it may be directly connected or connected to that other component, there may also be other components in between. On the other hand, when it is stated that one component is "directly connected" or "directly connected" to another component, it should be understood that there are no other components in between.

[0028] The terms used in this application are used merely to describe specific embodiments and are not intended to limit the invention. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, terms such as "comprising" or "having" are intended to specify the presence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0029] Hereinafter, embodiments of the present invention will be described with reference to the attached drawings. Identical reference numerals in each drawing indicate identical components. In describing the present invention, specific descriptions regarding related known functions or configurations are omitted to avoid obscuring the essence of the invention.

[0030] FIG. 1 illustrates a user-customized advertising content provision system according to an embodiment of the present invention.

[0031] A user-customized advertising content provision system according to one embodiment of the present invention may include an advertising device (100) and a cloud server (200).

[0032] The advertising device (100) can capture an image of the surroundings, identify a user included in the captured image, obtain characteristic information of the identified user, and track the gaze position and gaze direction of the identified user. The advertising device (100) can transmit the characteristic information of the identified user to a cloud server (200), receive advertising content provided by the cloud server (200), and display it on a display unit equipped in the advertising device (100).

[0033] The cloud server (200) may refer to a server that provides advertising content for an advertiser who wishes to advertise through the advertising device (100). The cloud server (200) receives user characteristic information obtained from the advertising device (100), determines advertising content to display to the user based on the user characteristic information, converts advertising content according to the user's gaze position and gaze direction to create user-customized advertising content, and transmits user-customized advertising content to the advertising device (100).

[0034] The cloud server (200) can convert a 2D image of a product to be advertised into a 3D image through advertising content using a deep learning model, and can provide an interactive advertising experience to a user viewing the advertising content by providing advertising content that includes a 3D product image rendered according to the direction of gaze of a user looking at the advertising device (100). The cloud server (200) can generate a 3D image of the product according to the direction of gaze of the user by combining the product's texture and a textureless structural model.

[0035] The electronic device (100) and the cloud server (200) can be implemented in an edge computing manner that provides real-time data analysis, processing, and resource management functions. That is, the cloud server (200) may be a network device that partially performs the role of a server by being located near the electronic device (100), which is the data generation point, as an edge node of edge computing, and the electronic device (100) may operate as an edge device.

[0036] Figure 1 illustrates a case where the electronic device (100) and the cloud server (200) are separated, but is not limited thereto, and some or all of the functions performed by the electronic device (100) can be implemented to be performed by the cloud server (200).

[0037] For example, instead of the electronic device (100) analyzing the captured image, the electronic device (100) transmits only the image of the surroundings to the cloud server (200), and the cloud server (200) identifies the user included in the captured image, obtains characteristic information of the identified user, and can track the gaze position and gaze direction of the identified user.

[0038] Similarly, functions performed on a cloud server (200) can be implemented to be performed on an electronic device (100). For example, the electronic device (100) can independently determine customized advertising content to be displayed to a user based on user characteristic information without a separate process of transmitting and receiving data with a cloud server (200), and can convert the determined customized advertising content to generate and display user-customized advertising content to be displayed to the user. Additionally, the electronic device (100) can convert a 2D image of a product to be advertised through advertising content into a 3D image through a deep learning model, and output advertising content including a 3D product image rendered according to the user's gaze direction through a display.

[0039] The advertising device (100) may be an outdoor advertising device that displays advertising content provided by a cloud server (200) on a screen. For example, the advertising device (100) may include digital signage. Digital signage may refer to a device for outdoor advertising using a digital information display (DID). The electronic device (100) may be installed in various outdoor or indoor locations such as department stores, shopping malls, airports, hospitals, restaurants, bars, cafes, subway stations, and streets. The electronic device (100) may be installed in a fixed location, installed in a manner that allows movement over a predetermined distance including a rotating member such as a wheel, installed in a manner that allows movement along with the movement of public transportation such as a bus or taxi, attached to a robot, or attached to various other means of mobility.

[0040]

[0041] FIG. 2 is a block diagram of an electronic device according to one embodiment of the present invention.

[0042] Referring to FIG. 2, an electronic device (100) according to various embodiments of the present invention includes a processor (110), a memory (120), a communication module (130), a camera (140), and a display (150).

[0043] The processor (110) may be composed of one or more cores and may include a processor (110) for data analysis and deep learning, such as a central processing unit (CPU), a general purpose graphics processing unit (GPGPU), or a tensor processing unit (TPU) of the electronic device (100). The processor (110) may read a computer program stored in memory (120) and perform data processing for machine learning according to one embodiment of the present invention. In addition, the processor (110) may control the configuration of the electronic device (100) to operate and implement the operation of the overall system.

[0044] For example, the processor (110) can typically control the overall operation of the electronic device (100). The processor (110) can provide or process appropriate information or functions to the user by processing signals, data, information, etc. that are input or output through the components described above, or by running an application stored in memory (120).

[0045] Additionally, the processor (110) can control at least some of the components of the electronic device (100) to run an application stored in memory (120). Furthermore, the processor (110) can operate at least two or more of the components included in the electronic device (100) in combination with each other to run the application.

[0046] According to one embodiment of the present invention, the processor (110) can perform operations for learning a neural network. The processor (110) can perform operations for learning a neural network, such as processing input data for learning in deep learning (DL), extracting features from input data, calculating errors, and updating the weights of the neural network using backpropagation. At least one of the CPU, GPGPU, and TPU of the processor (110) can process the learning of the network function. For example, the CPU and GPGPU can together process the learning of the network function and data classification using the network function. In addition, the communication module (130) and the camera (140) are of a general configuration, and a detailed description is omitted.

[0047]

[0048] In step 310, the electronic device (100) can identify a user from an image captured using a camera. In step 320, the electronic device (100) can use a processor to verify user characteristic information of the user identified from the captured image, as well as gaze information including the position and direction of the gaze.

[0049] The processor of an electronic device can perform the operations described below using an artificial intelligence model trained by deep learning techniques. The artificial intelligence model can be implemented through various known deep learning modules such as CNN (Convolutional Neural Network), RNN (Recurrent Neural Network), DBN (Deep Belief Network), and GNN (Graph Neural Network). The types of artificial intelligence models are not limited to the examples described above and may include various artificial intelligence models that detect and analyze faces in images in real time.

[0050] An electronic device can use a processor to capture the surroundings through a camera embedded in the device and then use a face recognition algorithm to identify a user included in the image. The processor can detect the user's face region in the captured image and identify the user by comparing it with an existing database or extracting feature vectors. In the aforementioned process, the processor can not only detect a person but also compare it with a previously registered list of users or infer features such as gender, age group, and preferences.

[0051] Referring to FIG. 4, a camera (410) is installed in an electronic device (100) and can capture images of the surroundings of the electronic device (100). The camera (410) may be installed to capture images in the front direction of the display so as to identify the face of a person object looking at an advertisement displayed on the electronic device's display. The electronic device (100) may acquire captured images via a wired or wireless network from a camera module separately provided externally, in which case the camera (410) may be omitted from the electronic device (100).

[0052] The processor of an electronic device can extract user characteristic information and gaze information based on an identified user face image. User characteristic information may include age group, gender, facial expressions (emotional state), past viewing patterns, purchase history, interests, etc. Gaze information refers to data obtained through user eye-tracking technology to determine where, how, and for how long a user gazes at a specific area, object, or GUI element on the screen.

[0053] In step 330, the electronic device (100) can transmit the user characteristic information and the gaze information to the cloud server. The electronic device can transmit the user characteristic information and the gaze information to the cloud server in encrypted form in packets. If necessary, de-identification or additional encryption processes may be performed to protect personal information.

[0054] In step 340, the cloud server (200) can generate a user-customized advertisement video based on user characteristic information. In step 350, the cloud server (200) can transmit the user-customized advertisement video to the electronic device (100).

[0055] Based on user characteristic information received from the electronic device, the cloud server can generate or select an advertising video customized for that user and transmit it to the electronic device. The customized advertising video may be at least one video optimized by considering the user's age group, interests, past viewing history, etc. The electronic device can display the received customized advertising video on the screen instead of existing advertisements or content. Through this, the electronic device can enable the user to watch advertisements selected to match their characteristics in real time.

[0056] In step 360, the cloud server (200) can generate video change information for at least a portion of the customized advertisement video based on gaze information. In step 370, the cloud server (200) can transmit the video change information to the electronic device (100). In step 380, the electronic device can change and display at least a portion of the customized advertisement video according to the video change information.

[0057] During the playback of an advertisement, the electronic device periodically transmits user gaze information to a cloud server, and based on this, the server may provide video change information to the electronic device to modify specific parts of the advertisement. Video change information consists of instruction and command data for changing the personalized advertisement video being played in real time according to user gaze information or reactions, and may include information that enables the electronic device to perform tasks such as scene replacement, adding graphics, or subtitle overlays. In other words, video change information may refer to instructions such as replacing specific scenes within the advertisement video or adding graphic elements.

[0058] At least some of the areas where the display is changed can be determined based on the area the user is looking at. The processor can measure the distance between the user's pupils and the head angle included in the captured image. The processor can determine the user's area of ​​gaze based on the measured distance between the user's pupils, the head angle, and gaze information. The processor can change the display for at least one object present in the user's area of ​​gaze within the customized advertising video. Through the above-described configuration, the advertising effect can be maximized by providing a user-customized interface by changing the display for the object present in the user's area of ​​gaze.

[0059] According to one embodiment, the processor can determine whether at least one object existing in the viewing area is a two-dimensional object. The processor may specify the user's viewing area from a digital image input and then apply image processing techniques and object detection algorithms to analyze shape information existing within the viewing area. The processor may identify objects within the area in the form of bounding boxes or masks through techniques such as a Convolutional Neural Network (CNN) or a Transformer-based vision model. If depth information or 3D structural information of the identified object is insufficient, the processor may determine that the object is a two-dimensional object represented on a two-dimensional plane.

[0060] The processor can convert 2D objects into 3D objects. To convert from 2D images into 3D objects, the processor can apply 3D reconstruction techniques such as NERF (Neural Radiance Fields), SfM (Structure from Motion), and SLAM (Simultaneous Localization and Mapping). In particular, the processor can estimate a depth map from 2D image samples captured from multiple angles or a single image using a deep learning model, and then convert it into a point cloud or a 3D mesh. The processor can generate a 3D object that preserves the visual features of the original 2D image by performing texture mapping on the extracted mesh. These operations can be performed by electronic devices or on a cloud server.

[0061] Referring to FIG. 5, the processor can analyze a two-dimensional image (510) of a product taken at various points in time, separate the two-dimensional image (510) into textureless structural model information (520) of the product and image information (530) containing only texture information, and combine the structural model information and texture information to generate a three-dimensional image of the product viewed at a predetermined point in time. That is, the processor can determine the textureless three-dimensional structure of the product viewed at a predetermined point in time using the structural model information (520), and generate a three-dimensional image of the product viewed at a predetermined point in time by combining the texture information at that point in time with a textureless three-dimensional rendered model.

[0062] The processor can generate a skeleton structure corresponding to the generated 3D object. To automatically generate a skeleton structure (Skeleton or Rig) for a 3D object, the processor can utilize auto-rigging algorithms or ML-based rigging assistance tools. For static objects (e.g., cans, snack bags), the surface shape and geometric features of the mesh can be analyzed to set deformable areas (e.g., parts that can be smoothly bent) as skeleton axes. By automatically distributing and assigning skeleton structures capable of transforming the object's pose (angle or shape), future animation or interaction can be enabled.

[0063] The processor can perform rigging operations to create a skeleton that 3D models a static object (610), and create 3D content by reflecting the movement of parts that can move dynamically according to the type of the static object. For example, referring to FIG. 6, if the static object (610) is assumed to be a snack bag, the processor can create a skeleton of the snack bag through rigging, and then create a dynamic object (620) with the opening (621) of the snack bag open. In this way, the processor can maximize the realism and immersion of 3D content through automatic rigging optimized for the shapes of static and dynamic objects. The processor can display a 3D object combined with a skeleton structure through a display.

[0064] According to one embodiment, a customized advertising video can be generated by the cloud server. The cloud server determines an advertising storyboard based on a pre-set advertising plan based on the identified user characteristic information, and can generate the customized advertising content through a generative AI based on the determined advertising storyboard.

[0065] According to one embodiment, the cloud server calculates the content optimization degree of the customized advertisement video based on the user's response to the customized advertisement video, and if the calculated optimization degree is less than a preset threshold, selects a second customized advertisement video in which parameters related to the user's response can be adjusted, and transmits the second customized advertisement video to the electronic device. Through the above-described configuration, the cloud server can quantitatively evaluate and provide the user-customized advertisement video by providing the user with a video in which the content optimization degree exceeds a preset threshold.

[0066] According to one embodiment, the content optimization level may be determined based on various parameters. The content optimization level may be determined based on a user attention lead time, which indicates the time it takes for the content to attract user attention; a user engagement rate, which indicates the ratio of time spent interacting with the content; a content gaze time, which indicates the average time a user focuses on the content; and the frequency of the user's gaze deviation from the content.

[0067] According to one embodiment, the content optimization degree can be calculated by the following mathematical formula.

[0068]

[0069] O may be content optimization, L may be user focus lead time, T may be user engagement rate, V may be content gaze time, and D may be gaze break frequency.

[0070] In this case, the user attention lead time can be calculated using the user's gaze information as the time from the start of content playback until the user's gaze is fixed on the display. The user engagement rate can be calculated as the ratio of the time from when the content is displayed until a response by the user occurs to the content playback time. The content gaze time can be calculated using the user's gaze information as the ratio of the accumulated time the user fixates on the content in frame units to the total content playback time. The gaze departure frequency can be calculated using the gaze vector generated based on the gaze information and the position of the display as the ratio of the content playback time to the time the gaze vector leaves the display.

[0071] According to one embodiment, the electronic device can generate and transmit to the cloud server first sales data for each item that matches an advertising schedule based on an advertising video provided from the cloud server, and second sales data for a certain period of time including the time when each item that matches the advertising schedule was advertised.

[0072] Accordingly, the cloud server calculates a first score for the sales effect of each item by advertising time according to the advertising schedule based on the first sales data, and based on the first score, can calculate the recommended advertising time for each item matching the advertising schedule, that is, the time when the highest sales volume occurred during advertising because the first score of each item according to the advertising schedule is the highest.

[0073] Subsequently, the cloud server calculates a second score for the sales effect of each of the other items according to each item matched to the advertising schedule based on the second sales data, and based on the second score, can identify at least one first item whose sales increased for a certain period of time according to the advertised time of the advertising item matched to the advertising schedule.

[0074] At this time, if at least one second item is identified among the first items that is included in the advertising item matching the advertising schedule data, the cloud server can generate a recommendation schedule that changes the advertising schedule for the second item according to the advertising schedule of the advertising item.

[0075] According to one embodiment, in generating a recommendation schedule, the cloud server can identify a first time for the time period in which the second item is calculated and a second time after a certain period has elapsed from the first time.

[0076] Specifically, the first time is a time when the sales volume of the second item increases due to the influence of the advertisement item during and after viewing the advertisement item, and the second time may be a time when the influence of the advertisement item ends and the sales volume of the second item drops to the sales volume range prior to the first time.

[0077] Accordingly, when the cloud server changes the advertising schedule for the first time when it is determined that the advertising effect of the second item will increase, it can calculate the first advertising fee by applying a weight proportional to the second score to the advertising fee for the first time of the second item.

[0078] Additionally, when the server (100) changes the advertising schedule for a second time when it is determined that additional advertising of a second item is necessary because the advertising effect of an advertising item has expired, it can calculate the second advertising fee based on the average advertising fee of the second item.

[0079] The cloud server can generate recommended schedules for the first time and the second time.

[0080] Subsequently, when the cloud server receives a schedule update request for a recommendation schedule from an advertiser terminal that matches the second item in the system, it can identify selection information for at least one of the first time and the second time, and determine an advertising fee based on the selected information.

[0081] Furthermore, in calculating the second score, the cloud server can identify a first group of advertising items and sales items for which an increase or decrease in sales volume due to advertising between items has occurred above a certain value, based on the accumulated stored second sales data.

[0082] The cloud server can identify a second group among the first group in which the sales volume of the sales item has increased for the advertising item.

[0083] The cloud server can acquire first sales volume data of the sales items of the second group sold during the advertising time of the matching advertising item, and acquire second sales volume data of the sales items of the second group sold outside the advertising time of the matching advertising item.

[0084] Next, the cloud server calculates a first average sales volume based on the first sales volume data, and based on the first sales volume data, can obtain at least one first sales time period in which a sales volume having a difference of less than or equal to a threshold value from the first average sales volume occurs.

[0085] Accordingly, the cloud server can calculate a second average sales volume for sales volume that occurred during the non-advertising time of the second group of advertising items, which is identical to the first sales time period, based on the second sales volume data.

[0086] At this time, if the second average sales volume differs from the first average sales volume of the same sales time period by more than a threshold, the cloud server can calculate a second score proportional to the difference between the first average sales volume and the second average sales volume.

[0087] Here, the sales time zone may be a unit in which 24 hours are divided into predetermined time intervals.

[0088]

[0089] The above description is merely an illustrative explanation of the technical concept of the present embodiment, and a person skilled in the art to which the present embodiment belongs would be able to make various modifications and variations within the scope of the essential characteristics of the present embodiment. Accordingly, the present embodiments are intended to explain, not limit, the technical concept of the present embodiment, and the scope of the technical concept of the present embodiment is not limited by these embodiments. The scope of protection of the present embodiment shall be interpreted by the claims below, and all technical concepts within an equivalent scope shall be interpreted as being included within the scope of rights of the present embodiment.

Claims

1. In an electronic device that displays user-customized advertising videos, Memory; display; camera; communication module; and Includes a processor, The above memory stores instructions that cause the processor to perform the following operations when the processor is operating, and the operations are: A step of identifying a user from an image captured using a camera; A step of using a processor to identify user characteristic information of a user identified from a captured image, and gaze information including the gaze position and gaze direction; A step of transmitting the above user characteristic information and the above gaze information to the cloud server; A step of receiving a customized advertisement video generated in response to user characteristic information from a cloud server; A step of displaying the customized advertisement video through the display instead of the advertisement video; A step of receiving image change information generated in correspondence with gaze information from the above-mentioned cloud server; A step of changing and displaying at least a portion of the customized advertisement video according to the above video change information; An electronic device including 2. In Paragraph 1, The step of changing and displaying at least a portion of the customized advertisement video according to the above video change information is A step of measuring the distance between the user's pupils and the head angle included in the above-mentioned captured image; A step of determining the user's gaze area based on the distance between the user's pupils, head angle, and gaze information measured above; An electronic device comprising the step of changing and displaying a display for at least one object present in the user's gaze area among the customized advertising videos.

3. In Paragraph 2, The above operations are: A step of checking whether at least one object existing in the above-mentioned observation area is a two-dimensional object; A step of converting the above 2D object into a 3D object; and A step of generating a skeletal structure corresponding to a generated 3D object; An electronic device comprising the step of displaying a three-dimensional object combined with a skeletal structure through a display.

4. In Paragraph 3, The above user characteristic information includes at least one of the user's age and gender, and The above customized advertising video is for the above cloud server: An electronic device that determines an advertising storyboard according to a pre-set advertising plan based on the identified user characteristic information, and generates the customized advertising content through generative AI based on the determined advertising storyboard.

5. In Paragraph 1, The above cloud server is: Based on user response to the above customized ad video, the content optimization degree of the customized ad video is calculated, and If the calculated optimization level is below a preset threshold, select a second customized ad video with adjustable parameters related to user response, and Transmitting the above second customized advertising video to the electronic device, Electronic device.

6. In Paragraph 5, An electronic device in which the above content optimization is determined based on a user attention lead time, which indicates the time it takes for the content to attract user attention; a user engagement rate, which indicates the ratio of time spent interacting with the content; a content gaze time, which indicates the average time the user gazes intently at the content; and the frequency of the user's gaze deviation from the content.

7. In Paragraph 6, The above content optimization degree is calculated by the following mathematical formula, and An electronic device in which L is the user attention lead time, T is the user engagement rate, V is the content gaze time, and D is the gaze break frequency.

8. In Paragraph 7, The above user attention lead time is calculated using user gaze information as the time from the start of content playback until the user's gaze is fixed on the display, and The user engagement rate is calculated as the ratio of the time from when the content is displayed on the screen until a response by the user occurs to the content playback time, and Content gaze time is calculated using user gaze information as the ratio of the accumulated time the user fixated on the content in frame units to the total content playback time, and The gaze departure frequency is calculated as the ratio of the content playback time to the gaze vector departure time from the display, using the gaze vector generated based on gaze information and the display position, Electronic device.

9. A method for controlling an electronic device that displays a user-customized advertising video, A step of identifying a user from an image captured using a camera; A step of using a processor to identify user characteristic information of a user identified from a captured image, and gaze information including the gaze position and gaze direction; A step of transmitting the above user characteristic information and the above gaze information to the cloud server; A step of receiving a customized advertisement video generated in response to user characteristic information from a cloud server; A step of displaying the customized advertisement video through the display instead of the advertisement video; A step of receiving image change information generated in correspondence with gaze information from the above-mentioned cloud server; A method for controlling an electronic device, comprising the step of changing and displaying at least a portion of the customized advertisement video according to the above video change information.

10. A method for controlling a system that provides user-customized advertising videos, wherein the system comprises an electronic device that displays advertising videos and a cloud server that provides advertising videos to the electronic device, and A step of identifying a user from an image captured using a camera by an electronic device; A step of identifying user characteristic information of a user identified from a captured image and gaze information including the gaze position and gaze direction of the gaze, using a processor by means of an electronic device; A step of transmitting the user characteristic information and the gaze information to the cloud server by means of an electronic device; A step of transmitting a customized advertising video generated in response to user characteristic information to an electronic device by a cloud server; A step of displaying the customized advertisement video on the display instead of the advertisement video by means of a cloud server; A step of transmitting image change information generated in response to gaze information by a cloud server; A method for controlling a system that provides a user-customized advertising video, comprising the step of changing and displaying at least a portion of the customized advertising video according to the image change information by means of an electronic device.

Images