Smart system for displaying targeted advertisement based on collective data using the internet of things and artificial intelligence

Abstract

This invention aims to reduce the error rate of real-time image processing via Internet of Things technologies, artificial intelligence, and also display content that is appropriate for the users by accurately monitoring the surroundings and collecting data using camera, microphone, temperature, humidity, and light sensors and processing the data. First, the business owner aka client will receive the data through the system panel, then, according to the received information and its display content, will define conditions for smart display. Finally, the client can use two options for broadcasting content on the displays: direct display and smart display. The submitted content is displayed based on the specified time, or the analysis of environmental data and the conditions defined by the client. Viewers in the area will be surveyed about the content, and record their opinions with predefined gestures. Image processing and integrated sensors gather data for more targeted advertisement making.

Description

Smart System for Displaying Targeted Advertisement Based on Collective Data Using the Internet of Things and Artificial Intelligence

[0001] A new advertising system utilizing AI and IoT technologies, combined with face recognition and crowd monitoring capabilities is introduced to revolutionize the way advertisements reach consumers. By analyzing demographic data and real-time crowd behavior, businesses can deliver highly personalized ads that resonate with viewers’ interests. This targeted approach not only enhances customer experience but also boosts sales for business owners, making advertising far more effective and meaningful.

[0002] With advanced sensors and face recognition, businesses can identify who is in the vicinity and what products they might like based on previous data. This means fewer irrelevant ads and more engaging content for potential customers. As a result, viewers feel valued and understood, which fosters brand loyalty and encourages repeat visits.

[0003] Moreover, the implementation of this technology for targeted advertising through AI and IoT, helps businesses invest in strategies that directly correspond to their buyer demographics. This increases business profits, and provides advertisements that cater to customer needs and preferences, creating a smoother marketplace for all.

[0004] G06Q 30 / 02

[0005] KR1020180078475

[0006] INTERNET-OF-THINGS SMART ADVERTISEMENT DISPLAY APPARATUS CAPABLE OF REMOTE CONTROL AND METHOD THEREOF

[0007] Disclosed are an Internet-of-things (IoT) smart advertisement display apparatus capable of remote control and a method thereof. The IoT smart advertisement display apparatus comprises: an advertisement content storage module for storing advertisement content; an infrared sensor module for recognizing access of a user; a beacon communication module for transmitting a beacon signal to a smart terminal of the user; and a control module for controlling the beacon communication module to transmit a beacon signal when the access of the user is detected in the infrared sensor module, and controlling the smart terminal to receive an analysis result of a server for an interaction signal transmitted to the server in response to the beacon signal, and to select and output advertisement content stored in the advertisement content storage module in accordance with the received analysis result.

[0008] The mentioned invention also aims to provide customized advertising based on collecting data and user recognition using IoT. However, our design relies on real-time image, sound, temperature, etc. processing and is able to perform face recognition. Therefore, the comprising parts, modules and overall function of the two patents are different.

[0009] IN201711039911

[0010] CONTEXT SENSITIVE IOT PLATFORM FOR ADVERTISEMENT AND USER ENGAGEMENT THROUGH SMART MIRROR

[0011] The various embodiments of the present invention disclose a system for determining an environmental state and a behavioral state of a user by analysis of input signals. The present invention provides a portable device named as Smart mirror to act as the environment and user behavior recorder and analyzer. The smart mirror then functions as an advanced artificial intelligence based compatriot which provides a plurality of information, environmental effects, solutions to problem, mood enhancer effects, memory hub, remote device controller etc. Being switchable between an inert mirror and an interactive display, the smart mirror provides a comfort of a mirror and a data log unit. The smart mirror solves a problem of accessing a data in various distributed devices and provides a single point access.

[0012] The above-mentioned invention relates to a product which is similar to our claimed on in terms of overall function in smart advertising and data collection. However, the design, process and primary focus are different in the two patents. For instance, ours relies on an IoT-based technology and machine learning and does not include a mirror.

[0013] CN117993975

[0014] Internet of Things e-commerce advertisement putting method and system

[0015] The invention discloses an internet-of-things e-commerce advertisement putting method and an internet-of-things e-commerce advertisement putting system. The method comprises the following steps: collecting user interaction behaviors from an e-commerce platform and Internet of Things equipment; preprocessing the user interaction behavior, wherein the preprocessing comprises data cleaning and user interaction behavior analysis to form a user data portrait; processing the obtained user data portrait by using a machine learning model, and identifying a potential demand signal of the user from the user data portrait; according to the identified potential demand signal of the user, adjusting an advertisement target and keyword setting, and pushing the advertisement content to the Internet of Things device to be displayed to the user; adjusting the real-time delivery of the advertisement by using a programmed purchase technology, and optimizing the advertisement bid by using a multi-arm bandit algorithm; real-time advertisement performance monitoring is implemented, and then the spatial precision of advertisement orientation is further analyzed and adjusted in combination with a network topological graph. According to the invention, a more optimized advertisement putting strategy is realized.

[0016] This mentioned invention shares many aspects with our claimed one including overall function, purpose and the use of IoT-based technology to optimize advertisement. Although, the two patents share certain techniques and processing steps, they also have their differences, for instance, this invention focuses primarily on e-commerce while ours has a wider scope of applicability. Moreover, our claimed design utilizes artificial intelligence and monitors the surroundings of the user via processing image, sound, temperature, humidity, etc.

[0017] KR1020210132915

[0018] ADVERTISEMENT-CURATION SIGNAGE SYSTEM BASED ON FACE RECOGNITION AND INTERNET OF THINGS TECHNOLOGY

[0019] The present invention relates to an advertisement-curation signage system using face recognition and Internet of things (IoT) technology to provide suitable and effective information to customers. According to the present invention, an advertisement-curation signage system based on face recognition and IoT technology comprises: a hardware component unit including a camera for facial recognition, a display unit showing information, an interface board for interaction with a customer, a payment system, and a printer for outputting receipts.

[0020] The above-mentioned patent shares common ground with our claimed design in usig face ecognition and IoT technology to optimize advertisement. However, our proposed design also utilizes AI machine learning and uses sound, humidity, temperature as well as image processing for more comprehensive data collection.

[0021] CN115100567

[0022] Media advertisement system based on Internet of Things and use method thereof

[0023] The invention discloses a media advertisement system based on the Internet of Things and a use method of the media advertisement system. The advertisement structure comprises a face recognition module, a database module, a sight tracker, a video storage module, an information processing module, a video playing module, a counting module, a data storage module, a cloud data module, a data analysis module, a merchant App module, an information pushing module, a terminal positioning module, a position recognition module, an advertising board positioning module and a crowd analysis module. The system comprises a video selection module, a cloud downloading module, an advertisement replacement module and a replacement playing module. The system has the beneficial effects that accurate advertisement putting can be performed on bus passengers through the arranged advertisement module, so that the impressions of the passengers on vehicle-mounted advertisements and bus station advertisements are deepened, corresponding advertisement putting can be generated according to the requirements of different passengers, and effective playing of the advertisements is improved.

[0024] The patent discussed above bears a strong resemblance to our claimed design in overall purpose, structure and processing methods. Although, the two patents share many comprising elements, ours also utilizes artificial intelligence machine learning and provides a more detailed data collection method.

[0025] This invention aims to reduce the error rate of real-time image processing using CRM, Internet of Things technologies, artificial intelligence, and also display content that is appropriate for the style, tastes, and interests of the users by accurately monitoring the surroundings and collecting data using camera, microphone, temperature, humidity, and light sensors and processing the received data. First, the business owner, from now on refered to as the client will receive the data through the system panel, then, according to the received information and its display content, will define conditions for smart display.

[0026] Then the client can use two options for broadcasting content on the displays: direct display and smart display. The submitted content is displayed after approval by the expert according to the type of display selected by the client, either directly based on the specified time and the conditions defined by the client, or on smart displays, based on the analysis of environmental data. After displaying the content, overall view numbers are calculated and viewers in the area will be surveyed about the content, and record their opinions with predefined gestures. Finally, using image processing technology, the angle of the head and eyes of the viewers relative to the screen is measured and the relevant feedback data is displayed in the client panel for more targeted and efficient advertisement in the future.

[0027] The proposed system involves accurate recognition of users' faces and identification of information attributed to each user. While prior art in this field, due to the high error of real-time image processing and the distance between the camera and the user, there is a possibility of error in user identification and the information received from the environment may not be efficient enough to display advertising or educational content. Also, with real-time image processing, it is not possible to process information such as occupation, education level, purchase history, annual income, etc. It should be noted that in previous methods for intelligent display of content, the focus has been on image processing, but in this design, in addition to image processing, processing of sound, light, temperature and humidity of the environment is used to display information.

[0028] The advertising systems that implemented in most countries today is often expressed without consulting the client or as an estimated number, and the client, due to lack of accurate data, chooses an environment based on the stated statistics that may not be productive for his business. In addition, the client needs to negotiate with different companies and individuals for each advertising location, and this will increase the cost and time.

[0029] In existing systems, the advertiser has no access to crucial data such as how many users have viewed the ad or whether they liked it. Moreover, the high cost, limited display locations, and the inability to constantly change the images of outdoor advertising cause clients to endorse new emerging advertising methods.Solution of Problem

[0030] After each display is turned on, content is initially played based on the content stored in the external memory. Meanwhile, the central controller monitors the data from the surrounding area using camera, microphone, temperature and humidity sensors and sends the information to the cloud server after processing this data. It should be noted that this information will be sent and stored continuously and online to the cloud server.

[0031] After registering in the system and uploading the specifications related to their video or image content, the client (aka the business owner) can receive suggestions based on the system's smart suggestions based on the collected data and by defining the viewing conditions for their content. They can share their content on the displays according to two methods: displaying advertisements at a specific time or displaying advertisements based on real-time processing of data gathered from the surrounding area. In the smart display method, after expert approval, based on real-time environmental data sent to the cloud server and the conditions defined by the client, the uploaded content is sent online from the cloud server to the displays.

[0032] In the first method, the employer determines the time and place of displaying the content and the content will be broadcast automatically at a specific time. Whereas in the second method, the content will be broadcast based on the processing of the received data regarding the surrounding environment. In both cases, the number of people who have viewed the content will be counted from the content display and they will be surveyed about the broadcast content, and finally this information will be stored and displayed in the content owner's panel. The number of views of the displayed advertisement is calculated via image processing technology based on the angle of the head and eyes of the audience present in the area.

[0033] This invention will make the displayed content fit the style, tastes and interests of the majority of present users, and will ultimately increase the efficiency of this type of advertising. Also, due to the limited time for broadcasting each advertising or educational content in this invention, more opportunities will be created for various businesses. For example, the broadcasting time of each advertising content will be 30 seconds, which will allow 120 advertisements to be displayed per hour, which will create more opportunities for various businesses and reduce advertising costs.

[0034] Clients can also pay for more targeted and frequent advertising. Using this feature, they first define the content specifications and display conditions, and after charging their panel, they can choose their data collection preferences for displaying the advertisement. This feature uses image processing technology to measure the angle of the head and eye of the audience present in the area relative to the display, and the charge will be deducted from the client’s account accordingly.

[0035] Similar advertising systems implemented in most countries today often function without consulting the advertiser. Moreover, the client must talk to an advertising company to register his advertisements in each area and location, whereas this invention, using an artificial intelligence system, automatically provides suggestions to the advertiser based on the collected environmental data of each location, and the advertiser can use these statistics to compile his advertisement and display it in the form of an integrated system on different displays.

[0036] In order to solve the problem of accuracy in collecting data from the surrounding area, in this system, after processing the images of the camera connected to the board, the system first checks the faces of people in the customer database (CRM) and sends all the user information that is already available in the CRM to the server along with effective real-time parameters to display efficient content. If the user information is not available in the databases, the user's demographic data is gathered through online crowd recognition. This features gathers data regarding the user’s appearance such as height, age, gender, weight, etc. and clothing, number of people, duration of presence, facial expressions of people (sad, happy, energetic, etc.), color of skin, hair and eyes, time and location of the person present in the area, movements of people and objects, duration of presence, level of ambient light, number of objects, recognition of license plates, dimensions of objects, coverage of objects, location of objects, sound intensity, etc. This data is processed in real time and content is displayed on the screens in accordance with this received information.

[0037] The information available in CRM can be recorded [Pic. 2] in the following ways:

[0038] 1- Statement by the user himself in various ways, including registration on the site

[0039] 2- Receiving information from other information systems such as stores, medical centers, government databases, and the like through processing the user's face

[0040] 3- Information available on the system from previous user traffic, which can be analyzed and recorded. (For example, a user who passes through different areas in front of the screens, the system will automatically create a user account for the user and save his information. After that, the information attributed to the user will be saved in his user account wherever the user is seen again. In addition, the user can view and complete his information by registering in the system in the future.)

[0041] The information that will be stored in the CRM includes information such as education, age, marital status, demographic information, annual income, religion, occupation, race, height, weight, skin and hair color, eye color, social relationships, nationality, place of residence, educational background, type of clothing, banking information, social network addresses, purchase history, number of visits and purchases from a store, etc. In addition, the CRM system has the ability to connect to all systems that have the ability to connect via API, which allows the information entered by users to be continuously updated and validated.

[0042] Face processing is based on the people present in the environment and on a person-by-person basis, and according to the items mentioned in, the information of the people is processed, and finally the system displays content based on the gathered information and the defined conditions.

[0043] By processing the sounds from the target area, the reoccurring words are processed and this information is sent to the cloud server. This information includes dialect and language. Using this feature, it is possible to display content based on a combination of audio and image of the location, so that advertisements will be displayed based on the words spoken by the users present in the area. It is also possible to adjust the volume of the displays intelligently based on this feature, so that it can be heard in crowded spaces but not be bothersome in quiet areas.

[0044] Temperature and humidity sensors process environmental data and send it to the cloud server through the central board.

[0045] Using the light sensor, the ambient brightness is transmitted to the server, and the client can display its content based on the information and defined conditions. Also, the display brightness is adjusted intelligently based on this, which will reduce energy consumption.

[0046] The number of each sensor varies based on the location. For example, on a highway, there is no need to use a sound sensor, but in a public place such as a train station, all of these sensors are used so that the system can process the surrounding area more accurately.

[0047] In order to increase the accuracy of image processing, the content owner or client can adjust the settings based on the distance between the camera and the people and the quality of the camera by specifying the image processing areas when defining the conditions for displaying content.

[0048] Moreover, when defining the broadcast conditions, the client can specify that if the users are moving or standing, they will be processed and analyzed, and then the content will be displayed accordingly.

[0049] Clients can define information about connecting to IoT-based objects in their panel and enable the use of smart content display by defining the conditions.

[0050] In the direct display method, after registering the content, the client must specify conditions for displaying the content at different times and places. Whereas in the smart display method, the client, using the capabilities of environmental sensors, defines more conditions for more targeted display of its content on the displays, which can be defined based on the input data. These conditions can also be defined based on a Phase Algorithm.

[0051] Given that the connection of the cameras to the controller board is wireless, the image processing cameras can be placedone or more kilometers away from the display to increase accuracy and decision-making in displaying advertisements on highways, and after processing this information, the content is displayed on the display connected to the central board.

[0052] In order to receive audience feedback on the displayed content, the method of processing the audience's facial and hand movements is used. In this method, reactions such as frowning, laughing, surprise, indifference, and the like are identified and transmitted to the system. The user can also send feedback on the broadcast content with hand gestures such as liking (thumbs up), disliking (thumbs down), or indifference.

[0053] By using the ability to connect this invention to online television networks, it is possible to display relevant content accordingly, for example, displaying an educational article "To prevent coronavirus, wash your hands and face every 1 hour" or displaying important results of a sports game.

[0054] This invention includes software and hardware modules. The software part of this invention includes the panel of clients, presenters, relevant experts, ordinary users in the system, as well as image, sound, temperature and humidity, light processing. The hardware part of this invention includes the central board, temperature and humidity sensor, relay (for turning on and off the power supply), cloud server, displays, camera and microphone, lux meter. All these components are interconnected and are connected to the cloud server through the central controller board, and after processing the environmental information, all commands are sent from the cloud server to the display based on the defined conditions.

[0055] Using this invention, we can connect to various displays such as televisions, projectors, large screens, mobile phones, virtual reality glasses, tablets, etc. and display content sent from the cloud server. It should be noted that mobile phones, tablets, and smart TVs that are designed for direct display mode do not require a board hardware component and will be able to connect to the system only by installing the application.Advantage Effects of the Invention

[0056] • Interactive display content

[0057] • Counting the number of views and broadcasts

[0058] • Audio processing, Image processing

[0059] • Displaying ads based on ambient temperature

[0060] • Displaying ads based on light levels

[0061] • Smart consultation

[0062] • Centralization and integration of the smart advertising system and reducing the costs of outdoor advertising

[0063] • Displaying ads on multiple screens

[0064] • Ability to connect to IoT sensors

[0065] • Expandability and ability to connect to other information systems via API

[0066] • Definition of terms and conditions based on business owner preferences

[0067] • End to end encryption

[0068] • Utilizing CRM customer database system

[0069] • Survey after displaying each ad

[0070] • Ability to intelligently display announcements and news and local networks

[0071] • Online streaming

[0072] • Highly customizable and targeted advertisement experience

[0073] Shows a general flowchart of the designed product.

[0074] Depicts a flowchart of the data gathering process.

[0075] Shows a general flowchart of the designed smart display wherein the components and the connection between its parts can be viewed.

[0076] Shows a flowchart of the data gathering process via sensors and the predefined conditions.

[0077] Examples

[0078] The invention can be used to display advertisements in high-traffic areas at very low costs. Animated and attractive videos as well as displaying content based on gathered data regarding the area increase the efficiency of this type of advertising.

[0079] Clients can define the connection information to IoT-based objects in their panel and provide the possibility of using it in displaying smart content by defining conditions.

[0080] For example, a store manager can use an RFID tag for each of his products and record all the product-related specifications in the chip memory. If a product is about to expire (for example, 3 days until expiration), the RFID chip defined on the IoT platform will automatically connect to nearby displays based on the defined conditions and announce to the viewers near the store that this product is being sold at a certain discount percentage. This feature will greatly reduce the additional costs of a store and lead to an increase in the income of the business owner.

[0081] In the direct display method, after registering the content, the client must specify conditions for displaying the content at different times and places, but in the smart display method, the client, using the capabilities of data collection sensors, defines more conditions for a more targeted display of its content on the displays. These conditions can be defined based on input data or a phase algorithm.

[0082] Example 1 - A car manufacturing company can use the brand, model, and license plate of the cars as the basis for displaying its content in the content display conditions definition section and direct users to the nearest sales agency. For example, Saipa Company can define the following conditions to display advertisements for its new product:

[0083] 1. The cars must be unmoving

[0084] 2. 30-second stop time

[0085] 3. Dominant brand of the target product in sight

[0086] 4. Certain city license plates

[0087] 5. Specified time such as 2-8 pm

[0088] Example 2 - Smart screens display content appropriate to that location to users based on information processing. For example, if the weather is sunny, they display an advertisement for sunglasses, and if the weather is rainy, they display an advertisement for an umbrella or a cafe-restaurant near the display. All content is displayed based on the conditions defined by the client and environmental information processing.

[0089] Example 3 - The employer of a women's clothing manufacturer and store wants to display content based on gender, age, and color of the majority among the present crowd at specific times and in locations close to its branches. For example, if the population in front of the screens is mostly women between 6:00 PM and 8:00 PM, if their age is between 20 and 35 years old and the most observed color is blue, a blue dress will be displayed on the screens.

[0090] [Pic. 1] Shows a general schematic of a smart display with verification of the accuracy of the collected information

[0091] 1. Advertiser or Client

[0092] 2. Normal user or viewer

[0093] 3. Connection to other systems via API

[0094] 4. Connection to IoT-based sensors

[0095] 5. Cloud server

[0096] 6. IoT communication network

[0097] 7. Central board

[0098] 8. Display

[0099] [Pic. 1]

[0100]

[0101] [Pic. 2] Illustrates the methods of collecting user information via CRM

[0102] 5. Cloud server

[0103] 9. The first method of collecting information or user input during registration

[0104] 10. The second method of collecting information via face recognition and accessing data banks

[0105] 11. The third method of collecting information using previous data collected from user traffic

[0106] 12. Information systems such as the case registration system

[0107] [Pic. 2]

[0108]

Claims

A smart display system is introduced for displaying customized and targeted advertising content based on collective data, which consists of the following components:1)Connection to CRM information to reduce image processing errors in identifying users in the area.2)Data processing and analysis system using camera, microphone, temperature, humidity and light sensors3)Using inventory information of advertiser's IoT-based products to display content4)Smart recommendation to advertisers based on information collected from the surrounding areaAccording to claim 1, in order to intelligently display content, the system processes and analyzes demographic information such as such as job, nationality, education, annual income, purchase history of individuals, type of clothing, duration of presence in the environment, user distance from the display and camera, etc. to display smart content based on conditions defined by the business owner.According to claim 1, the image processing system displays the most appropriate content by receiving information through sensors and identifying variables such as brands, duration of presence, movement, license plate number, color, brightness level, distance of objects from the camera, etc.According to claim 1, the sound processing and analysis system identifies recuring words spoken in the target area and displays the most appropriate content.According to claim 1, the sound level and ambient light intensity are monitored by the system and the volume of the content and the light intensity of the display are adjusted accordingly.According to claim 1, the image processing and analysis system examines the angle of the audience's head relative to the display and is used to prepare statistics on the visibility of the advertisement.According to claim 1, the image processing and analysis system examines the hand and facial movements of users to express their opinions about the broadcast content, and this information is used to determine the audience's satisfaction with the displayed content.According to claim 1, the data processing and analysis system has the ability to specify the image processing areas in the cameras in order to increase the accuracy of the received information.According to claim 1, the information processing and analysis system has the ability to define viewing conditions by the advertising client or business owner based on collective data from the sensors.

Images