Advertising effectiveness analysis apparatus and method
The apparatus uses a vision sensor and deep learning to analyze outdoor ad effectiveness by measuring viewer engagement, providing precise data for strategic advertising adjustments.
Patent Information
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- ADDD INC
- Filing Date
- 2026-02-24
- Publication Date
- 2026-07-07
Smart Images

Figure 2026113459000001_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to an apparatus and method for analyzing the effect of an advertisement presented on an outdoor advertising medium, which analyzes information such as people, vehicles, weather, etc. around the advertising medium acquired through sensors and measures the advertising effect based on this information.
Background Art
[0002] Outdoor advertisements targeted at a plurality of people (pedestrians, people in vehicles, etc.) indoors or outdoors are recognized as an effective advertising method in that they can force the exposure of the advertisement. However, it is difficult to objectively analyze the advertising effect of outdoor advertisements. Conventionally, the effect has been analyzed through conventional estimation based on the floating population around the advertising medium.
[0003] Online advertisements can target viewers (for example, computer users) according to the advertisement content, and are relatively easier to measure the advertising effect compared to outdoor advertisements. However, in the case of outdoor advertising methods, it is not easy to analyze how much people around the advertising medium (for example, pedestrians, people in vehicles, etc.) noticed the advertisement with interest and measure the advertising effect. Therefore, a method for accurately measuring the advertising effect through an outdoor advertising medium based on objective data is required.
Summary of the Invention
Problems to be Solved by the Invention
[0004] In order to solve the above problems, the present invention provides an advertising effect measurement apparatus and method capable of analyzing and measuring the advertising effect using information such as the number of times a person views an advertisement, the viewing time of the advertisement, and the degree of attention.
[0005] An apparatus and method are provided that acquire information on pedestrians and the floating population through a vision sensor such as a camera, analyze and digitize whether a person located within the visual field of the advertising medium recognized and paid attention to the advertisement, and provide the user with meaningful analysis data on the advertising effect.
[0006] Furthermore, by applying deep learning technology to analyze the effectiveness of outdoor advertising media, this method provides a way to improve the efficiency and accuracy of advertising effectiveness measurement.
[0007] The problems that this invention aims to solve are not limited to those described above, and any problems not mentioned will be clearly understood by a person with ordinary skill in the art of this invention from this specification and the accompanying drawings. [Means for solving the problem]
[0008] The present invention provides an advertising effectiveness measuring device for measuring the advertising effectiveness of an advertising medium, comprising: a vision sensor that photographs people and vehicles located within the field of view of the advertising medium; a computing device that receives and analyzes video / images captured by the vision sensor; and a communication device that transmits the analyzed data or advertising effectiveness measurement results to a terminal or server, wherein the computing device includes one or more programs that analyze people and vehicle-related data based on the received video / images.
[0009] The device for measuring advertising effectiveness includes at least one of the following programs: a human status analysis program that analyzes people's exposure, viewing, and attention to advertisements; a human counting program that aggregates the number of people exposed to, viewing, and paying attention to advertisements; a human distribution analysis program that analyzes people's gender and age distribution; a human behavior analysis program that analyzes people's movement paths, dwell time, and inflow / outflow populations; a spatial analysis program that divides the shooting space into areas and analyzes population density, dwelling populations, etc.; a vehicle analysis program that analyzes the number, type, and speed of vehicles; and a potential audience aggregation program that aggregates people inside vehicles within the field of view of the advertising medium as potential audiences.
[0010] The present invention provides a method for measuring the advertising effectiveness of an advertising medium, comprising the steps of: transmitting video / images captured by a vision sensor to a computing device; analyzing the video / images captured by the computing device; and transmitting the analyzed data or advertising effectiveness measurement results to a terminal or server. The vision sensor captures people and vehicles located within the field of view of the advertising medium, and the analysis step uses a program to analyze people and vehicle-related data from the captured video / images. In the analysis step, the computing device or server can analyze the captured video / images, and the present invention provides an advertising effectiveness measurement method that transmits the analyzed data or advertising effectiveness measurement results from the server to a terminal or another server.
[0011] This method provides an advertising effectiveness measurement method that involves analyzing various collected information through a program, combining it with generated data to create metadata, and transmitting it to a terminal or server. The metadata includes exposure population, viewing population, attention population, gender distribution, age distribution, viewership rate, attention rate, day of the week with the highest exposure, maximum exposure time, average number of appearances, average viewing time, average dwell time, etc.
[0012] Furthermore, the system provides an advertising effectiveness measurement method that involves sharing data collected by multiple advertising media sensors among the advertising media to complement and / or improve the accuracy of the data, predicting the flow of people based on this data, preparing necessary advertisements in advance, and displaying the advertisements at the scheduled time.
[0013] The means of solving the problems of the present invention are not limited to the means described above, and means of solving that are not mentioned will be clearly understood by a person skilled in the art to which the present invention pertains from this specification and the accompanying drawings. [Effects of the Invention]
[0014] Through this invention, the effectiveness of advertisements displayed on advertising media can be objectively analyzed, and advertisers, advertising agencies, media companies, media agencies, and other advertising industry professionals, as well as other users (hereinafter referred to as "users"), can check advertising analysis data and advertising effectiveness measurement results in real time. Users can develop various advertising strategies based on the advertising analysis data.
[0015] This invention can also be used for visitor analysis in offline spaces such as exhibitions, event venues, and pop-up stores. By analyzing the gender, age, dwell time, movement patterns, and gaze of visitors staying in pop-up stores, exhibition halls, large shopping centers, and various other stores through cameras and edge computing, the effectiveness of offline marketing can be measured. Furthermore, by utilizing artificial intelligence (AI) technology, the accuracy and efficiency of advertising effectiveness measurement can be improved, providing a basis for advertising solutions and service platforms that enable mutual coexistence among advertisers, media companies, and other users.
[0016] The effects of the invention are not limited to those described above, and any effects not mentioned herein will be clearly understood by a person with ordinary skill in the art to which the invention pertains from this specification and the accompanying drawings. [Brief explanation of the drawing]
[0017] [Figure 1] This diagram illustrates the configuration of a system for analyzing and measuring advertising effectiveness according to an embodiment of the present invention. [Figure 2] This diagram illustrates a method for analyzing and measuring advertising effectiveness according to an embodiment of the present invention. [Figure 3] This flowchart illustrates a method for analyzing and measuring advertising effectiveness using a computing device according to an embodiment of the present invention. [Figure 4] This flowchart illustrates a method for analyzing and measuring advertising effectiveness on a server according to an embodiment of the present invention. [Figure 5]The drawings illustrate the analysis data of the advertising effect and the measurement results of the advertising effect by the embodiments of the present invention in a terminal device. [Figure 6] It is a flowchart for explaining another embodiment of the advertising effect analysis method of the present invention. [Figure 7] The drawing illustrates an embodiment of outputting the outline box of the object of the present invention and the coordinates of the main points of the human body. [Figure 8] It is a flowchart for explaining another embodiment of the advertising effect analysis method of the present invention. [Figure 9] The drawing illustrates the configuration of the advertising effect analysis device by the embodiments of the present invention.
Modes for Carrying Out the Invention
[0018] The object, features, and embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, in describing the embodiments, descriptions of technical content well known in the technical field to which the present invention belongs will be omitted in order to clearly convey without impairing the gist of the present invention.
[0019] The embodiments described in this specification are for explaining the idea of the present invention to those having ordinary knowledge in the technical field to which the present invention belongs. The present invention is not limited to the embodiments described in this specification, and the scope of the present invention should be interpreted to include modifications and changes that do not depart from the idea of the present invention.
[0020] The terms used in this specification are selected as general terms that are currently widely used as much as possible in consideration of the functions in the present invention. However, this may change depending on the intention, precedent, or the emergence of new technologies of those having ordinary knowledge in the technical field to which the present invention belongs. However, in contrast, when specific terms are defined and used in an arbitrary meaning, the meaning of those terms will be separately described. Therefore, the terms used in this specification should not be regarded as mere term names, but should be interpreted based on the substantial meaning of those terms and the overall content of this specification.
[0021] If it is determined that a detailed description of known configurations or functions associated with the present invention in this specification would obscure the gist of the present invention, a detailed description thereof will be omitted.
[0022] Also, the numbers and terms (e.g., first, second, etc.) used in the process of describing the present invention throughout this specification are merely identification symbols for distinguishing one component from another, and do not mean the order of operations, priority, etc. unless otherwise defined in the specification.
[0023] The suffixes “module” or “section” for the components used in the following embodiments are given and / or used for the convenience of preparing the specification, and do not have meanings or roles that distinguish themselves from each other.
[0024] In the following embodiments, the singular form includes the plural form unless the context clearly dictates otherwise.
[0025] The symbol “ / ” used in this specification should be construed to include one or more possible combinations of the related items. For example, it includes one or more possible combinations of the items associated with “and / or”. That is, “transmitting A / B” should be construed to transmit only A, and / or only B, and / or both A and B.
[0026] In the following embodiments, terms such as “include” or “have” mean that the features or components described in the specification exist, and do not exclude the possibility of adding one or more other features, components, and steps.
[0027] The drawings are for facilitating the explanation of the present invention, and the configurations and shapes shown in the drawings can be exaggerated as necessary to assist in understanding the present invention. Therefore, the present invention is not limited to what is shown in the drawings.
[0028] In cases where a particular embodiment can be manifested differently, the order of certain processes may differ from the order in which they are described. For example, two processes described consecutively may be performed substantially simultaneously, or in the reverse order of the description.
[0029] When it is stated that one component is “connected” and / or “linked” to another component, it should be understood that this may include direct connection and / or linkage to the other component, but also the presence of other components in between. On the other hand, when it is stated that one component is “directly connected” and / or “directly linked” to another component, it should be understood that there are no other components in between. Other expressions describing the relationship between components, namely “between” and “immediately between,” or “adjacent to” and “directly adjacent to,” should be interpreted in the same way. For example, when it is stated in this specification that components are electrically connected, it includes not only cases where the components are directly electrically connected, but also cases where other components exist in between and are indirectly or electrically connected.
[0030] The methods described in the embodiments can be embodied in the form of program instructions that can be executed through various computer means and can be recorded on a computer-readable medium. The computer-readable medium may include program instructions, software, algorithms, data files, data structures, etc., individually or in combination. The program instructions recorded on the medium are specifically designed and configured for the embodiments and may be publicly known and usable by those skilled in the computer software art. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROMs, RAMs, and flash memory. Examples of program instructions include not only machine code generated by compilers but also high-level language code that can be executed by a computer using an interpreter or the like.
[0031] These computer program instructions can be implemented in a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing equipment, so that the instructions generate means to perform the functions described in the flowchart blocks. These instructions can also be stored in computer-available or computer-readable memory that can be directed to a computer or other programmable data processing equipment to embody the functions in a particular manner, and the stored instructions can produce a manufactured item containing instruction means to perform the functions described in the flowchart blocks. Computer program instructions can also be implemented on a computer or other programmable data processing equipment, where a series of operational steps are performed to generate a processor that runs on the computer, providing steps to perform the functions described in the flowchart blocks. Hardware devices can be configured to operate with one or more software modules to perform the operations of the embodiments, and vice versa.
[0032] Furthermore, each block may represent a module, segment, or portion of code containing one or more executable instructions to perform a specific logical function. In some alternative execution examples, the functions mentioned in a block may occur out of order. For example, two consecutively shown blocks may occur substantially simultaneously, or the blocks may occur in reverse order according to their respective functions.
[0033] The term "~unit" as used in this invention refers to software or hardware components such as FPGAs (Field Programmable Gate Arrays) or ASICs (Application Specific Integrated Circuits). While a "~unit" performs a specific role, it is not limited to software or hardware. A "~unit" may be configured to reside in an accessible storage medium, or to regenerate one or more processors. Therefore, according to some embodiments, a "~unit" includes components such as software components, object-oriented software components, class components, and task components, as well as processors, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data databases, data structures, tables, arrays, and variables. Components and the functions provided within a "~unit" can be further combined with a smaller number of components and "~units," or further separated into additional components and "~units." Furthermore, components and "~units" can also be embodied to regenerate one or more CPUs within a device or security multimedia card. According to various embodiments of this disclosure, a "~unit" may include one or more processors. The present invention will be described in detail below with reference to the attached drawings.
[0034] Figure 1 is a diagram illustrating the configuration of a system for analyzing and measuring advertising effectiveness according to an embodiment of the present invention.
[0035] As shown in Figure 1, the advertising effectiveness analysis system includes an advertising medium 19 that displays advertisements, a sensor 13 that collects situational information around the advertising medium, a computing device 50 that analyzes the information (video / images, weather, etc.) collected from the sensor 13, a server 30, or a terminal 15 that displays the data analyzed by the computing device 50 and the advertising effectiveness analysis results.
[0036] The advertising medium 19 is a device that displays advertisements in the form of printed materials or videos / images to people (e.g., pedestrians, people in vehicles, etc., a variety of audiences). The advertising medium 19 has the function of continuously showing a single printed material to a person and / or showing various videos / images, information, news, etc. to a person in real time, randomly, or according to a pre-set order / time, or based on the results of advertising effectiveness analysis. The advertising medium 19 includes various types of electronic billboards such as liquid crystal display devices (LCDs) and organic light-emitting display devices (OLEDs), digital signage, and other display-type advertising media. The advertising medium 19 may be equipped with other electronic devices or equipment.
[0037] The advertising medium 19 can be installed inside or outside a building, on a mobile device 20, on the roof or exterior wall of a building, etc., and can also be installed independently on the side of a road (for example, as a roadside billboard). Furthermore, the advertising medium can be installed on or inside a mobile device that travels on a road. In this case, the mobile device 20 includes various means of transportation such as mobile advertising vehicles, passenger cars, buses, trucks, motorcycles, and bicycles. The size of the advertising medium can be changed depending on the location and purpose of use.
[0038] The advertising medium 19 can display advertisements transmitted from the server 30 using the driving pattern of the vehicle 20 and a pedestrian pattern associated with the characteristics of pedestrians around the vehicle 20. In other words, the advertising medium 19 is located outside the vehicle 20 and can display advertisements provided by advertisers based on the driving pattern and pedestrian pattern, and can be applied to adaptive advertising.
[0039] Sensor 13 is a device that collects information about the surrounding environment of the advertising medium 19. Sensor 13 includes a vision sensor that takes pictures of the area around the advertising medium 19, such as a camera, CCTV, or LiDAR (Light Detection and Ranging), as well as various sensors that collect information about the area around the advertising medium, such as a thermometer, a hygrometer, and the weather at the location where the advertising medium 19 is located. The vision sensor can be a commercially available product such as a webcam, camera, or CCTV, and if it is embedded in the advertising medium, an on-device type miniature camera can be used. Weather information includes temperature, humidity, PM2.5, precipitation, snow depth, sunrise information, sunset information, wind direction, wind speed, etc. The sensor may also further include a GPS, a microphone, and a speaker.
[0040] The computing device 50 is a device that analyzes information collected from the sensor 13, combines the collected information with other data, converts it into data in a format required by the user, and transmits advertising effectiveness analysis data and advertising effectiveness measurement results to the terminal 15 or server 30. For example, it analyzes people and vehicle-related data from videos / images captured by a camera. The computing device is installed inside or outside the advertising medium, and / or elsewhere separate from the advertising medium, and can communicate via wired / wireless.
[0041] Server 30 analyzes the information collected by sensor 13 and / or combines it with other data, converts it into data in a format required by the user, and transmits it to terminal 15 or another server. Only one server is used, but multiple servers can be operated as needed, and the server can transmit advertising effectiveness analysis data and advertising effectiveness measurement results to other servers. The server can be a commonly used network server, and various types of servers such as general PCs, high-performance PCs, and cloud servers can be used.
[0042] The computing device 50 and server 30 described above contain various data and one or more programs necessary for advertising effectiveness analysis. The computing device and server analyze people's poses and the direction of pedestrians' movement (whether they are heading towards or away from the advertising medium), etc. They also analyze the number of people heading towards the advertising medium 19, the number of people looking at the advertising medium (people who recognize / pay attention to the advertisements displayed on the advertising medium), and the amount of time people are paying attention to the advertising medium.
[0043] Terminal 15 is a device that displays data analyzed by computing device 50 or server 30 and advertising effectiveness measurement results to the user. Here, the user includes advertisers, advertising agencies, media companies, media agencies, and other advertising industry professionals, as well as other users. The terminal includes various devices such as PCs, tablets, laptops, and smartphones, and provides a web-based and / or app-based program that allows users to easily view advertising analysis data and advertising effectiveness measurement results.
[0044] Information and data collected by sensors on multiple advertising media located in close proximity can be shared amongst the advertising media. This allows for data complementation and / or improved analytical accuracy. For example, by sharing at least one piece of actual audience data, potential audience data, vehicle data, pedestrian movement direction, gender and age data, advertising display information, advertising media environmental information, etc., among multiple advertising media, it is possible to understand information, distance, and environmental information about people near the advertising media and predict the flow of people. For example, advertisements tailored to a person's gender, age, and situation can be prepared in advance and displayed at the appropriate time. This can further enhance advertising effectiveness.
[0045] Figure 2 is a diagram illustrating a method for analyzing and measuring advertising effectiveness according to an embodiment of the present invention.
[0046] As explained in Figure 1, the advertising medium 19 can be installed indoors or outdoors, on a mobile device (e.g., a mobile advertising vehicle), on the rooftop or exterior wall of a building, and can be installed independently on the side of a road. Figure 2 illustrates an example focusing on the advertising medium and related equipment, regardless of the installation location of the advertising medium.
[0047] Sensor 13 includes a vision sensor that takes pictures of the area around the advertising medium, such as a camera, and a sensor that collects information about the area around the advertising medium, such as the weather at the location where the advertising medium is installed. It may also further include GPS, a microphone, a speaker, etc.
[0048] The vision sensor (e.g., a camera) captures people 17 and vehicles 18 located around and within the field of view of the advertising medium. The vision sensor's function can be adjusted based on the size of the space being captured and the measurement range indicating the number of people captured on the screen. The performance of the vision sensor and computing device can be optimized and used according to the size of the space within the field of view of the advertising medium. By determining the resolution, field of view, focal length, IR (Infrared) conversion function, etc., based on the measurement range, it is possible to detect people at a distance and capture images even at night.
[0049] The present invention can be applied by dividing the measurement range into two or more groups. For example, the performance of the vision sensor and computing device can be differentiated / optimized and applied according to the size of the space of the advertising medium's field of view, as follows: a. Small spaces (elevators, menu ordering systems, etc.): For media where the distance to the advertising medium is between 0.5m and 10m, and the number of simultaneous exposures is around 15 people, analysis can be performed using a small PC or a computing board consisting only of a CPU without a GPU, and can be captured using a webcam or small camera with a narrow field of view. b. Medium-sized spaces (digital signage installed in offices, cafeterias, shops, etc., and indoor / outdoor media): These media are suitable for media where the distance to the advertising medium is between 1m and 40m, and the number of people exposed simultaneously is around 50. Analysis can be performed using a small PC, GPU, or NPU computing board, and images can be captured using webcams, miniature cameras, and CCTV cameras with over 1 million pixels. c. Large spaces (e.g., electronic billboards on building exteriors, roadside advertising media / freestanding billboards): These are media where the distance to the advertising medium is between 5m and 130m, and the number of people simultaneously exposed is around 150. Analysis can be performed using high-performance PCs or computing boards such as GPUs and NPUs, and the images can be captured by high-resolution cameras or CCTV cameras with over 2 million pixels.
[0050] The sensor 13 can be installed on the exterior, top, side, or bottom of the advertising medium, and / or inside (built-in) the advertising medium, and can be installed separately from the advertising medium in another location.
[0051] Advertising medium 19 refers to a device that displays advertisements to people in the form of printed materials or video / images. The advertising medium continuously displays a single printed material to people, or displays video / image advertisements and information to people in real time according to a pre-set order / time, in a random manner, or based on the results of advertising effectiveness analysis. If the advertising medium displays not just one advertisement but other advertisements or information in a switching manner, it is an advertising medium in the form of a display, such as an LCD / LED electronic billboard or digital signage.
[0052] The computing device 50 is a device that periodically or in real time analyzes information collected from the sensor 13, and includes one or more programs that combine / transform the collected information with other data or analyze data necessary for measuring advertising effectiveness. For example, it includes a program that analyzes people and vehicle-related data from videos / images captured by a camera. Such programs can be embedded in a computing device, server, or cloud.
[0053] More specifically, the program includes, but is not limited to, at least one of the following programs: a human status analysis program that analyzes people's exposure, viewing, and / or attention status; a human counting program that aggregates the number of people exposed to advertisements, people viewing advertisements, and / or people paying attention to advertisements; a human distribution analysis program that analyzes the distribution of people's gender and / or age; a human behavior analysis program that analyzes people's movement paths, dwell time, and / or inflow and outflow populations; a spatial analysis program that analyzes the density of people or the number of people dwelling in a shooting space; a vehicle analysis program that analyzes the number, type, and / or speed of vehicles; and a potential audience aggregation program that aggregates people inside vehicles within the field of view of the advertising medium as potential audiences.
[0054] The computing device analyzes the exposure status of people around the advertising medium, whether they are in a field of view where they can see the advertisement, whether they are actually viewing the advertisement, and whether they are paying attention to the advertisement for a set period of time or longer (e.g., more than 1 second). Based on this, it is possible to aggregate the number of people exposed to the advertisement, the number of people viewing the advertisement, and the number of people paying attention to the advertisement.
[0055] The system analyzes information such as gender and / or age for the number of people exposed to advertisements, those who view advertisements, and those who pay attention to advertisements, and generates gender and / or age distribution data for each population group. It also analyzes the time people spend in the field of view of advertisements and their movement paths, generating data on dwell time, inflow and outflow population, and movement paths. Furthermore, it divides the shooting space into areas and generates population density and dwell time data for each area.
[0056] Vehicles moving on roads within the advertising medium's field of view are classified by type (passenger cars, buses, trucks, motorcycles, etc.), their numbers are tallied, and vehicle speeds are analyzed to generate data. People within the advertising medium's field of view who recognize and pay attention to the advertisement are classified as the actual audience, while people inside vehicles within the advertising medium's field of view are classified as the potential audience. Weights (e.g., 1.5 people for passenger cars, 10 for buses, 1 for trucks, 1 for motorcycles) are assigned to the aggregated figures by vehicle type to generate advertising exposure data for the potential audience. The aforementioned weights are not always fixed values; they can be changed based on various factors such as the location of the advertising medium, weather, and the user's advertising strategy.
[0057] By applying deep learning technology trained on millions of images, it is possible to detect and track pedestrians and analyze their gender, age, and whether or not they paid attention to advertisements. The deep learning technology and other artificial intelligence (AI) models improve accuracy and efficiency in measuring the flow of people captured by cameras, determining the gender and age of pedestrians, and analyzing whether they recognized / paid attention to advertisements. By quantifying this, the effectiveness of the advertisements can be measured.
[0058] Data analyzed by the computing device or advertising effectiveness measurement results are transmitted via the network to the terminal 15 or server 30. At this time, the data is converted into a format required by the user and transmitted to the terminal or server 30 periodically or in real time.
[0059] The computing device may further include a data merging program that collects advertising information displayed on advertising media, collects advertising environment information such as weather at the location of the advertising media via sensors, analyzes the collected information through the program, and combines it with data to generate metadata.
[0060] The aforementioned metadata includes exposure population, viewing population, attention population, gender distribution, age distribution, viewership rating, attention rating, day of the week with the highest exposure, maximum exposure time, average number of appearances, average viewing time, average dwell time, etc.
[0061] The computing device collects information on advertisements displayed on each advertising medium by date and time (advertising display information), generates metadata by combining it with data or by combining it in a time series, and provides advertisement analysis data and advertising effectiveness measurement results for each advertisement.
[0062] Time series joining refers to the process of aligning and combining two or more time series data sets into a single dataset, using a method that ensures consistent time scales between them. This type of time series joining offers advantages such as improved integrated analysis, data consistency, and enhanced performance of predictive models.
[0063] Metadata is data that describes other data. In other words, it is additional information that helps to better classify and organize raw data, makes it easier to search for specific data, and helps to better understand, analyze, and manage the data. For example, in the case of a book, the data is the content of the book, and the metadata would be the book's title, author, publication year, ISBN number, subject keywords, etc. In the case of a digital photograph, the data is the photographic image, and the metadata would be the date and time of shooting, camera settings (e.g., shutter speed, aperture value (F-number)), and location (GPS coordinates), etc.
[0064] The system collects weather information (e.g., temperature, humidity, PM2.5, precipitation, snow depth, sunrise / sunset information, wind direction, wind speed, etc.) for the location of the advertising medium, broken down by date and time, and generates metadata by combining this information with advertising analysis data. In the case of mobile advertising mediums such as mobile devices, the system updates the current location via GPS or other location sensors, collects speed and surrounding environmental information, and combines this with advertising analysis data.
[0065] Furthermore, the computing device 50 quantifies the performance of the advertising media 19. The computing device 50 combines the advertising display time of the advertising media 19 with a time series and transmits the quantified values of advertising performance by advertisement to the terminal 15 or server 30 periodically or in real time. Advertising performance includes quantified values that combine actual audience-based advertising media performance (e.g., performance by time / date / period / weather, exposure / viewing / attention population, gender / age distribution, etc.) and potential audience-based advertising media performance (e.g., by time / date / period / weather, exposure population, gender / age distribution, etc.).
[0066] The computing device 50 can be installed inside or outside the advertising medium 19, or it can be installed separately from the advertising medium 19 in a different location. The computing device 50 includes edge devices such as PCs, tablets, smartphones, and laptops, and can also perform video analysis deep learning models. The operating system (OS) of the computing device can be Windows, Linux®, Android, or iOS. (Registered trademark) Various software, such as base software, can also be embedded. Deep learning models embedded in edge devices and used for advertising effectiveness analysis are lightweight and optimized for on-device characteristics, so they can run on a CPU (Central Processing Unit) alone without computational accelerator chips such as GPUs (Graphics Processing Units) or NPUs (Neural Processing Units).
[0067] Terminal 15 is a device that displays data analyzed by server 30 or computing device 50 and advertising effectiveness measurement results to the user. Users (e.g., advertisers, advertising agencies, media companies, media agencies, and other advertising industry professionals) can view advertising analysis data and advertising effectiveness measurement results provided by computing device 50 in real time through terminal 15. Based on this, users can develop various advertising strategies. Terminals include various devices such as PCs, tablets, laptops, and smartphones.
[0068] Server 30 collects various data gathered by sensors on advertising media in one place, analyzes the relevant data, combines and transforms the data in a format required by the user, and transmits it to terminal 15 or another server. A computing device 50 connected to the advertising media 19 can analyze the information collected by sensors 13. When there is a large amount of data from advertising media 19, or when providing an advertising effectiveness measurement platform service to users, it can be efficient to use a server for analysis. The server can transmit the advertisements desired by the advertiser to the advertising media 19.
[0069] The computing device 50 can perform calculations with low power consumption. Wired or wireless communication is possible when transmitting video / images from the sensor 13 to the computing device 50, and when using wired communication, it can be connected via a LAN cable, USB cable, etc. The sensor 13 can receive power independently of the advertising medium 19 or the computing device 50, and can also receive power from the computing device 50 via PoE, USB, etc., along with video / image transmission.
[0070] Figure 3 is a flowchart illustrating a method for analyzing and measuring advertising effectiveness using a computing device, based on an example.
[0071] As shown in Figure 3, the vision sensor captures images of people and vehicles located within the field of view from which the advertisement displayed on the advertising medium 19 can be seen (S301), and transmits the captured video / images to the computing device 50 periodically or in real time (S303). Various methods can be used to transmit the video / images from the vision sensor to the computing device, such as wired communication or wireless communication.
[0072] The computing device analyzes the received captured video / image (S305). During this process, the computing device uses one or more programs to analyze the received video / image for the purpose of measuring the effectiveness of the advertisement, including data related to people and vehicles. In the analysis process described above, detection of pedestrians and vehicles within the video / image, object tracking, pose analysis, gender analysis, age analysis, etc., are performed periodically or in real time. The programs can be embedded in the computing device, server, or cloud.
[0073] The program performs at least one of the following: analyzes people's exposure, viewing, and / or attention status; compiles the number of people exposed to advertisements, people viewing advertisements, and / or people paying attention to advertisements; analyzes people's gender and / or age distribution; analyzes people's movement paths, dwell time, and / or inflow and outflow populations; analyzes population density and / or dwell time within the filming space; analyzes the number, type, and / or speed of vehicles; or compiles people inside vehicles within the field of view of the advertising medium as potential audiences.
[0074] The computing device can convert the analyzed data into anonymized data and store it on a storage device, server, or in the cloud (S306). Since the original video / images of people and vehicles are not stored, and only the anonymized data is stored, personal information can be protected. The anonymized data is anonymized data in which people and vehicles are assigned arbitrary IDs during the video analysis process, and gender, age, dwell time, movement patterns, etc. are recorded, making it impossible to identify people and vehicles in the video / images.
[0075] The analyzed data or advertising effectiveness measurement results are transmitted to a terminal or server. The data transmitted at this time includes the state of people, the number of people, the distribution of people, people's behavior, spatial analysis, vehicle analysis, actual / potential audience data, etc.
[0076] The collected data, analyzed data, or advertising effectiveness measurement results are transmitted to the terminal or server periodically (e.g., in units of seconds / minutes / hours, daily, etc.) or in real time (S307). The data transmitted at this time includes actual audience data (e.g., person / vehicle ID, vehicle type, exposure / viewing / attention time, gender, age, dwell time, direction of movement, speed of movement, current time, etc.), potential audience data, advertising display information (advertising ID, advertising type, transmission start time, transmission end time, video length, current time, etc.), advertising media environment information (media ID, location, latitude, longitude, speed of movement, weather, current time, etc.), etc. Various methods are possible for transmitting data from the computing device to the terminal or server, including wired communication and wireless communication.
[0077] The computing device periodically / in real time transmits and updates metadata, advertising media performance, advertising results, relevant statistics, program-analyzed data, and advertising effectiveness measurement results to the terminal or server. Users can check the transmitted data and information in real time through the terminal.
[0078] Figure 4 is a flowchart illustrating a method for analyzing and measuring advertising effectiveness on a server according to an embodiment of the present invention.
[0079] As illustrated in Figure 3, while information collected by computing devices can be analyzed to measure advertising effectiveness, servers can also analyze collected information and measure advertising effectiveness. In this case, the server analyzes the collected information in parallel with or independently of the computing devices.
[0080] As shown in Figure 4, the vision sensor captures people and vehicles located within the field of view of the advertising medium (S401), and transmits the captured video / images to the server periodically or in real time (S403). The server analyzes the received captured video / images (S405). The server uses one or more programs to analyze the received video / images and the people and vehicle-related data necessary for measuring the advertising effectiveness, and the programs can be embedded on the server or in the cloud. The analyzed data can be converted into anonymized data and stored in a storage device, server, or cloud (S406). The analyzed data or advertising effectiveness measurement results can also be transmitted to a terminal, computing device, or other server (S407).
[0081] At this time, the server analyzes the various data collected by the sensors, combines / transforms the data into the format required by the user, and transmits it to a terminal, computing device, or other server. While data can be analyzed on a computing device, if there are many advertising media and a large amount of data, or if you are providing a platform for measuring advertising effectiveness to users, it may be more efficient to use a server for analysis.
[0082] Figure 5 is a diagram illustrating advertising effectiveness analysis data and advertising effectiveness measurement results on a terminal device according to an embodiment of the present invention.
[0083] Data analyzed by the computing device 50 or server 30, as well as advertising effectiveness measurement results, are transmitted to and updated on the terminal 15. Users can check the transmitted data and measurement results in real time through the terminal 15. The data and information that can be checked through the terminal 15 include the location of the advertising medium 19, advertising effectiveness analysis equipment, and location, exposed population, viewing population, attention population, mobile population, gender distribution, age distribution, viewing rate, attention rate, day of maximum exposure, maximum exposure time, average number of impressions, average viewing time, average dwell time, metadata, advertising medium performance, advertising results, other related statistics, and graphs. Alternatively, users can select only the data they want to know and set it to be displayed on the terminal.
[0084] At this time, the terminal visualizes the analyzed data and information to enhance the user's readability and ease of use. The user can specify the date or period they wish to review, and can download advertising analysis data and advertising effectiveness measurement results for the selected period in various report formats (e.g., Word, PDF, Excel, etc.). Through this, the present invention can achieve both data analysis and data visualization in measuring the advertising effectiveness of advertising media.
[0085] Figure 6 is a flowchart illustrating other examples of the advertising analysis method.
[0086] The advertising effectiveness measurement program analyzes people and vehicles in videos / images captured by vision sensors in real time to generate advertising effectiveness measurement data. This program can be installed on computing devices, servers, or in the cloud.
[0087] As shown in Figure 6, the analysis process can be broadly divided into three parts: the data collection part, the object detection and analysis part, and the process optimization part. Within each part, the specific processes can be carried out sequentially or in parallel and simultaneously. The specific processes within each part are as follows:
[0088] First, in the data collection part, vision sensors collect video / image information about people and vehicles in real time.
[0089] Secondly, in the object detection and analysis part, technologies such as object detection, body pose estimation, head pose estimation, person feature extraction and attribute recognition, and vehicle attribute recognition are used to analyze audience characteristics (e.g., gender, age) and behavior (e.g., viewing, attention), as well as vehicle aggregation analysis.
[0090] Object detection technology detects objects such as people and vehicles, outputting the outline of each object in Box coordinates as shown in Figure 7(a). Body pose estimation technology simultaneously outputs the coordinates of major points on a person's body (e.g., 17 points such as eyes, nose, ears, shoulders, elbows, pelvis, knees, and feet) and the outline Box coordinates of the person, as shown in Figure 7(b). At this time, detection ensemble technology is applied to combine the results of object detection and body pose estimation (for example, by finding the intersection of the outline Box coordinates of the person), allowing for more accurate detection of the person's outline. If only object detection or body pose estimation technology is used, the accuracy of the person's outline Box will be low, and there will be problems with intermittent failure to detect people. Therefore, detection ensemble technology can be applied to improve detection accuracy. Furthermore, Detection Ensemble can combine object detection and head pose estimation results, as well as body pose estimation results, to more accurately detect the outline of a person.
[0091] The video / images of people or vehicles detected by Object Detection, along with the corrected video / images of people or vehicles using Detection Ensemble technology, undergo gender and age analysis through Person Feature Extraction + Attribute Recognition technology, and gaze direction is extracted through Head Pose Estimation technology. The vehicle type (passenger car, bus, truck, motorcycle, etc.) is classified through Vehicle Attribute Recognition, and movement speed, direction, etc., are analyzed.
[0092] Thirdly, in the process optimization part, we utilize detection ensemble, object tracking, and reID matching technologies to improve analysis speed and accuracy.
[0093] The captured video is a collection of consecutive images, and when people or vehicles move within the video, their positions change slightly across the consecutive images. Up to the object detection and analysis part described above, object detection and analysis are performed on a single image. If multiple consecutive images that are transmitted are only processed up to the second part, a problem arises where the same person in each image is recognized as a different person. Object tracking technology is used to solve this problem.
[0094] Object tracking technology is a technique that uses a series of images to identify an object detected in a previous image as the same object in subsequent images. In other words, it tracks the same person or vehicle in a series of images and assigns the same ID to the same object. By assigning the same ID to the same object based on the movement of people and vehicles, it is possible to confirm data on the characteristics (e.g., gender and age) and behavior (e.g., viewing, attention, speed, direction of movement, etc.) of the same object.
[0095] If the accuracy of object tracking is not high during this process, it is possible to track objects continuously without interruption, and to improve accuracy, re-identification (ReID Matching) technology is used. Re-identification (ReID Matching) technology is an algorithm that determines whether objects are the same based on the overlapping areas between the outlines (Boxes) of objects detected in a series of images.
[0096] Additionally, prediction accumulation technology can be further utilized. Prediction accumulation technology is a data transformation technology that converts measurement data accumulated by object ID through object tracking into data in a desired format through arithmetic logic, conditional logic, majority rule logic, etc. For example, if, in the data accumulated for ID 176 (e.g., a person) over a period of 0.3 seconds, the gaze direction value was within the viewing angle 5 times and not within the viewing angle 1 time, then ID 176 would be recorded as being in a viewing state for 0.3 seconds. Such majority rule logic is also used for gender and age data analyzed in real time, classifying it in the direction of the majority vote.
[0097] Through this process, it is possible to analyze the characteristics and behavior of people and vehicles in real time and measure the effectiveness of advertising.
[0098] Figure 7 is a diagram illustrating one embodiment that outputs the outline box of an object and the coordinates of major points on a person's body.
[0099] The object detection technology uses a deep learning model to detect people, vehicles, and other objects, outputting the outline of each object in Box coordinates as shown in Figure 7(a). The pose estimation technology simultaneously outputs the coordinates of major points on a person's body (e.g., 17 points such as eyes, nose, ears, shoulders, elbows, pelvis, knees, and feet) and the outline Box coordinates of the person, as shown in Figure 7(b). In this case, the number of points is not limited to 17; it is possible to output coordinates for fewer or more than 17 points depending on the settings.
[0100] Figure 8 is a flowchart illustrating another example of the advertising effectiveness analysis method.
[0101] According to the embodiment shown in Figure 8(a), the vision sensor detects pedestrians and vehicles located around the advertising medium (S801). Figure 8(b) is a diagram showing a screen of one embodiment in which the vision sensor detects pedestrians and vehicles.
[0102] For this type of object detection, it is crucial to detect all individual objects, including people and vehicles, in all videos / images without missing any. Furthermore, accuracy decreases if the detected object boxes point to unexpected coordinates or mistakenly detect other types of objects. This invention provides two technologies (Object Detection and Body Pose Estimation) in parallel. Additionally, the accuracy of the object box coordinates detected by each technology is improved through a Detection Ensemble technique. The Detection Ensemble uses the intersection and coset of all the outline boxes of people output by each model to detect the outline of a person with even greater precision. Furthermore, the input image resolution is upscaled, and the detection critical values for each technology are optimized to ensure that even distant objects are detected. Of course, accuracy will be higher if high-resolution cameras with high-performance GPUs are used for real-time shooting (e.g., 30 FPS or higher).
[0103] The system predicts pedestrian attributes based on the detected information (S803) and analyzes the level of awareness / attention to advertisements (S805). By utilizing an image analysis artificial intelligence model (e.g., Vision Transformer (ViT)), various pedestrian attributes (e.g., exposure time, gender, age, direction of movement, etc.) can be predicted based on images. Exposure time refers to the time the pedestrian was exposed to the advertisement. Gender and age are determined by capturing and analyzing features that can distinguish gender / age from the pedestrian's authenticated clothing through an AI-based deep learning model.
[0104] When analyzing pedestrians' awareness / attention to advertisements (S805), whether or not they are aware of / paying attention to the advertisement can be determined using head pose estimation technology. A three-dimensional vector indicating the direction the head is facing can be predicted from a two-dimensional upper body image. The pedestrian's head pose is estimated and reconstructed in a 3D coordinate system, and through this, the probability of viewing the advertisement is predicted. The duration of advertisement awareness / attention is determined through a pose algorithm to determine whether it is simple recognition or actual attention to the advertisement. Additionally, the size, position of the outdoor advertising medium, and the relative position of the camera are calibrated to define a world coordinate system. By determining whether or not the head direction matches the world coordinate system, it is possible to accurately analyze whether or not the pedestrian is gazing at the advertisement.
[0105] In detecting and tracking pedestrians (S807), if tracking is interrupted, the analysis performance can be compensated for through pedestrian re-identification (S809). Although pedestrian detection and tracking may intermittently fail, a pedestrian re-identification algorithm can be used to compensate for this. By applying the re-identification algorithm and tracking pedestrians over a long period, the information obtained from the analysis is accumulated to improve the final prediction performance.
[0106] Object tracking is crucial in videos and multiple images. Without object tracking in multiple images transmitted sequentially, the same person may be mistakenly identified as different people. Object tracking technology is an algorithm that identifies an object detected in a previous image as the same object in the next image within a sequence of images. Through this, the same object (e.g., person, vehicle, etc.) can be assigned the same ID, and data on the characteristics (e.g., gender, age) and behavior (e.g., viewing, attention, speed, direction of movement, etc.) of each identical object can be secured. In this process, re-identification (ReID Matching) technology can be applied to improve the accuracy of object tracking.
[0107] The system analyzes the effectiveness of advertising based on pedestrian attributes and operates a dashboard that displays advertising analysis data and advertising effectiveness measurement results on user terminals (S811). It can also aggregate vehicle traffic around advertising media based on vehicle information collected through sensors (S813).
[0108] This invention provides machine learning, deep learning, and other technologies that enable the analysis and measurement of advertising effectiveness. Information collected by sensors such as cameras can be transformed by a Machine-Learnable Feature through an AI image recognition algorithm, and a server or computing device can analyze and measure advertising effectiveness based on the AI model.
[0109] Generally known face image-based gender and age classification models lose accuracy at distances of 5 meters or more because the image information (pixels) represented by faces becomes too small. Since people often view outdoor advertising at distances of 5 meters or more, it is important to improve the accuracy of analysis for objects at distances of 5 meters or more.
[0110] The model applied to this invention learns and identifies features of the entire human body, and learns more than 10 times more information than a face image-based model. As a result, it has high analytical accuracy for people at a distance of 5m or more, and can classify gender and age even from just a person's back. To achieve this, the human body can be divided into three parts (for example, upper body, lower body, and legs), and an image analysis artificial intelligence model (for example, Transformer(ViT)) that learns the features of each part to improve inference performance can be applied.
[0111] ViT is a technique for performing image recognition tasks in the field of computer vision. It works well with very large datasets, allows for larger model sizes, and can be applied to a variety of image recognition tasks.
[0112] Furthermore, conventional full-body base models were trained on people worldwide, resulting in relatively low training weighting for East Asians and Koreans, and thus lower accuracy for these groups. To address this issue, the model was trained on the physical characteristics of people by region or country, and in particular, approximately 1 million images of East Asians and Koreans were labeled and added to the training, increasing accuracy from 85% to 95%. This accuracy level is significantly higher than that of other models.
[0113] To improve the efficiency of advertising effectiveness analysis, the processing speed of computing devices and servers is crucial. In particular, model optimization is necessary for computing devices to analyze information in an on-device format on advertising media. Among video analysis deep learning models, models that analyze gender, age, etc., based on the characteristics of the entire human body require a large amount of computation, and therefore require a lot of time for analysis. In order to reduce this analysis time, it is necessary to increase processing speed, and for this purpose, quantization calculations can be applied to weighted calculations in deep learning models, which can reduce the amount of computation by more than double.
[0114] Quantization is a technique that reduces computational complexity. The primary goals of quantization are to lower computational complexity, save storage space, and reduce power consumption. Quantization can significantly improve the efficiency of machine learning models without degrading their performance, helping artificial intelligence models work faster with fewer resources.
[0115] In particular, smartphones and small devices have limited resources, making it necessary to reduce the size and computational complexity of artificial intelligence models through quantization technology.
[0116] Furthermore, even with this weight reduction, the accuracy of the analysis must be maintained compared to before the weight reduction. Through the parallel computing and process flow improvements between the model weight reduction and the analysis model of the present invention, processing time can be reduced by more than 50%. When analyzing a single image for a large space (e.g., at least 50 people), it takes 78.9 ms for the Processor to perform all steps of detection, gender and age classification, attention level, tracking, and re-identification. This represents a processing speed more than three times faster than using a typical method that analyzes 12 images per second.
[0117] JPEG2026113459000002.jpg19170
[0118] Figure 9 is a diagram illustrating the configuration of an advertising effectiveness analysis device according to an embodiment.
[0119] The advertising effectiveness analysis device includes an output unit 901, a sensor unit 903, a control unit 905, an analysis unit 907, a storage unit 909, and a communication unit 911. The output unit outputs advertisements, including advertising media 19 such as a display device. The sensor unit includes various sensors 13, such as a vision sensor and an environmental information collection sensor, to collect information about the surroundings of the advertising media. The control unit controls the output unit, sensor unit, analysis unit, storage unit, communication unit, etc. The analysis unit includes a computing device 50 and performs the function of analyzing the collected data and measuring the advertising effectiveness. The storage unit can store the collected information, analyzed data, advertising effectiveness measurement results, and information necessary for displaying advertisements. The communication unit performs communication functions with various devices such as the sensors 13, terminals 15, advertising media 19, a mobile device 20, and a server 30. However, the various units mentioned above (901 to 909) may be excluded and / or installed and operated elsewhere depending on the advertising media operation method and circumstances.
[0120] While the present invention has been described through limited embodiments and drawings, a person skilled in the art with ordinary skill in the field to which the invention pertains will be able to make various modifications, alterations, applications, and combinations from the above description without departing from the essential features of the technical idea and form of the invention. For example, the described technique may be performed in a different order than described, and / or the described system, structure, apparatus, circuit, and other components may be combined or combined in a different manner than described, and / or substituted with other components or equivalents, and / or replaced, while still achieving the appropriate results. Therefore, any content associated with such modifications, alterations, applications, and combinations, as well as equivalents to other embodiments and claims, should be interpreted as falling within the scope of the claims described below.
Claims
1. An advertising medium installed on a moving device that displays advertisements transmitted from a server, The aforementioned advertising medium is at least, (a) A running pattern acquisition unit that acquires the running pattern of the running device, (b) A pedestrian pattern acquisition unit that acquires a pedestrian pattern associated with the characteristics of pedestrians around the traveling device, (c) An advertisement display control unit that, based on the driving pattern and the pedestrian pattern, selects or switches an advertisement to be displayed from among a plurality of advertisements transmitted from the server, and displays the selected or switched advertisement. Equipped with, The advertising medium is characterized in that the advertising display control unit displays advertisements according to the driving pattern and the pedestrian pattern.
2. In the advertising medium described in claim 1, The advertising medium is an advertising medium in the form of a display including a liquid crystal display device (LCD) or an organic light-emitting display device (OLED).
3. In the advertising medium described in claim 1, The aforementioned mobile device includes advertising media such as mobile advertising vehicles, passenger cars, buses, trucks, motorcycles, or bicycles.
4. In the advertising medium described in claim 1, The aforementioned driving pattern is an advertising medium that includes at least one of the following: driving location, driving route, driving speed, or driving time.
5. In the advertising medium described in claim 1, The pedestrian pattern acquisition unit generates the pedestrian pattern based on information obtained from sensors, including a vision sensor that photographs the area around the traveling device, and is an advertising medium.
6. In the advertising medium described in claim 5, The characteristics of the pedestrian include at least one of the following: gender or age. The pedestrian pattern acquisition unit is an advertising medium that generates a pedestrian pattern based on the characteristics of the pedestrian.
7. In the advertising medium described in claim 5, The pedestrian pattern acquisition unit determines whether a pedestrian is recognizing / paying attention to an advertisement by estimating head pose, and reflects the determination result in the pedestrian pattern, thereby providing an advertising medium.
8. In the advertising medium described in claim 7, The pedestrian pattern acquisition unit calibrates the size, position, and relative position of the outdoor advertising medium, defines a world coordinate system, and analyzes whether a pedestrian is staring at the advertisement based on whether the head direction is accurate to the world coordinate system.
9. A method for displaying advertisements transmitted from a server to an advertising medium installed on a moving device, (a) A step of acquiring the travel pattern of the travel device, (b) A step of acquiring a pedestrian pattern associated with the characteristics of pedestrians around the traveling device, (c) A step of selecting or switching an advertisement to be displayed from among a plurality of advertisements transmitted from the server based on the driving pattern and the pedestrian pattern, (d) The process of displaying the selected or switched advertisement through the advertising medium. A method characterized by including the following.
10. In the method according to claim 9, The method wherein the advertising medium is an advertising medium in the form of a display including a liquid crystal display device (LCD) or an organic light-emitting display device (OLED).
11. In the method according to claim 9, The method includes a mobile advertising vehicle, a passenger car, a bus, a truck, a motorcycle, or a bicycle as the running device.
12. In the method according to claim 9, The aforementioned driving pattern is a method that includes at least one of the following: driving position, driving route, driving speed, or driving time period.
13. In the method according to claim 9, The pedestrian pattern is generated based on information obtained from sensors, including a vision sensor, in a method.
14. In the method according to claim 13, The characteristics of the pedestrian include at least one of the following: gender or age. A method for generating the pedestrian pattern based on the characteristics of the pedestrian.
15. In the method according to claim 13, The method for generating the pedestrian pattern includes determining whether a pedestrian is recognizing / paying attention to an advertisement by head pose estimation, and reflecting the determination result in the pedestrian pattern.