Ai-based active food and beverage curation system utilizing sns activity data and ott content consumption context

Abstract

An AI-based active curation provision system for food and beverages that reflects SNS activity data and OTT content consumption context is provided, and includes a curation service provision server that grants access rights to SNS activity data, OTT (Over the Top) viewing data and order data of delivery applications, and collects multimodal data including user terminals receiving food and beverage recommendations, SNS activity data, OTT viewing data, and order data of user terminals; a Contents-Food Mapping unit that maps the atmosphere of OTT content in OTT viewing data to be converted into gustatory elements; a Personalization unit that calculates recommendation scores reflecting preferences and trends among food and beverages with gustatory elements using order data and SNS activity data; and a Recommendation unit that recommends food and beverages to user terminals in order of highest recommendation scores.

Description

Technology Field

[0001] The present invention relates to an AI-based active curation system for food and beverages that reflects social media activity data and OTT content consumption contexts. It provides a system that determines the optimal timing for food and beverage and marketing execution by integrally analyzing the situation based on social media activity data and OTT content consumption contexts when recommending food and beverages. Background Technology

[0002] While modern food and beverage recommendation services utilize various data to enhance user convenience, most remain limited to static recommendation methods that rely on past order history or simple demographic information, failing to adequately reflect users' current situations and emotional states. In particular, despite food desires shifting significantly depending on what content users are watching, their real-time interests, and environmental factors such as the time of day or weather, conventional systems fail to comprehensively consider these dynamic contexts, resulting in low recommendation accuracy and purchase conversion rates. Furthermore, there is a lack of technology capable of integratively reflecting not only subtle preference elements—such as individual eating habits or brand preferences—but also temporal factors like the timing of hunger onset and repurchase cycles.

[0003] At this time, methods for recommending context-based restaurants or food content through curation have been researched and developed. In this regard, prior art Korean Published Patent No. 2018-0128147 (published December 3, 2018) and Korean Published Patent No. 2020-0104592 (published September 4, 2020) respectively disclose a configuration for collecting environmental information including location, weather, and time, and user behavior data from a user terminal, generating a user behavior pattern, and recommending restaurants suitable for that behavior pattern, and a configuration for classifying elements of food ingredients, weather, emotions, and environment based on recipe information and user profiles, and analyzing query content to recommend recipe content that matches the user's taste.

[0004] However, in the former case, configurations reflecting content-based emotions or consumption contexts are not disclosed, and in the latter case, only configurations recommending recipes tailored to preferences based on user queries and recipe metadata are disclosed, failing to reflect real-time user status or external content data. Thus, general recommendation models perform recommendations based on static user information or limited contexts, which fails to adequately reflect the user's current situation and emotional state, resulting in limitations such as low recommendation accuracy and purchase conversion rates. Therefore, research and development are required for a system capable of actively performing recommendations and marketing by reflecting real-time unstructured data, such as social media activity and OTT content consumption contexts, and by comprehensively considering personal preferences and behavioral patterns. The problem to be solved

[0005] One embodiment of the present invention provides an AI-based active curation system for food and beverages that reflects the context of SNS activity data and OTT content consumption. This system integrates SNS activity data, OTT viewing data, and order data to generate multimodal data, embeds the multimodal data to generate feature vectors, maps the atmosphere of OTT content in the OTT viewing data to gustatory elements, and recommends food and beverages by combining gustatory elements with feature vectors, thereby recommending food and beverages while considering various contexts. Furthermore, by predicting repurchase probability and hunger timing to optimize the timing of food and beverage recommendations, the system goes beyond mere recommendations to dynamically control the timing of marketing, thereby maximizing marketing efficiency. However, the technical problem that this embodiment aims to solve is not limited to the technical problem described above, and other technical problems may exist. means of solving the problem

[0006] As a technical means for achieving the technical problem described above, one embodiment of the present invention includes a curation service providing server comprising: a collection unit that grants access rights to SNS activity data, OTT (Over the Top) viewing data, and order data of a delivery application, and collects multimodal data including SNS activity data, OTT viewing data, and order data of a user terminal receiving recommendations for food and beverages; a Contents-Food Mapping unit that maps the atmosphere of the OTT content of the OTT viewing data to be converted into gustatory elements; a personalization unit that calculates a recommendation score reflecting preferences and trends among food and beverages having gustatory elements using order data and SNS activity data; and a recommendation unit that recommends food and beverages to the user terminal in order of highest recommendation score. Effects of the invention

[0007] According to any one of the means for solving the problem of the present invention described above, food recommendations that precisely reflect the user's current emotions and situation based on SNS activity data and the context of OTT content consumption can be provided, thereby improving recommendation accuracy and purchase conversion rates. Furthermore, marketing efficiency can be maximized by providing customized push notifications and coupons at the optimal time by comprehensively considering personal eating habits, brand preferences, repurchase cycles, and hunger timing, and an active curation service that connects user experience and consumption behavior can be provided beyond simple recommendations. Brief explanation of the drawing

[0008] FIG. 1 is a diagram illustrating an AI-based active food and beverage curation provision system that reflects SNS activity data and OTT content consumption context according to an embodiment of the present invention. Figure 2 is a block diagram illustrating a curation service providing server included in the system of Figure 1. FIGS. 3 and 4 are drawings for illustrating an embodiment in which an AI-based active food and beverage curation service according to an embodiment of the present invention is implemented. FIG. 5 is a flowchart illustrating an operation method for providing an AI-based active curation service for food and beverages according to an embodiment of the present invention. Specific details for implementing the invention

[0009] Embodiments of the present invention are described below with reference to the attached drawings so that those skilled in the art can easily implement the invention. However, the present invention may be embodied in various different forms and is not limited to the embodiments described herein. Furthermore, in order to clearly explain the present invention in the drawings, parts unrelated to the explanation have been omitted, and similar parts throughout the specification are denoted by similar reference numerals.

[0010] Throughout the specification, when a part is described as being "connected" to another part, this includes not only cases where they are "directly connected" but also cases where they are "electrically connected" with other elements interposed between them. Furthermore, when a part is described as "including" a component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components, and it should be understood that this does not preclude the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0011] Terms such as “about,” “substantially,” etc., used throughout the specification, are used to mean at or near the stated value when inherent manufacturing and material tolerances are presented in the stated meaning, and are used to prevent unscrupulous infringers from unfairly exploiting the disclosure in which precise or absolute values ​​are mentioned to aid in understanding the invention. Terms such as “step” or “step of” used throughout the specification of the invention do not mean “step for”.

[0012] In this specification, the term "part" includes a unit realized by hardware, a unit realized by software, and a unit realized using both. Additionally, one unit may be realized using two or more pieces of hardware, and two or more units may be realized by one piece of hardware. Meanwhile, "part" is not limited to software or hardware, and "part" may be configured to reside in an addressable storage medium or configured to run on one or more processors. Accordingly, as an example, "part" includes components such as software components, object-oriented software components, class components, and task components, as well as processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functions provided within the components and "parts" may be combined into a smaller number of components and "parts" or further separated into additional components and "parts." In addition, the components and '~parts' may be implemented to play one or more CPUs within the device or secure multimedia card.

[0013] Some of the operations or functions described herein as being performed by a terminal, device, or device may instead be performed by a server connected to said terminal, device, or device. Likewise, some of the operations or functions described as being performed by a server may also be performed by a terminal, device, or device connected to said server.

[0014] In this specification, some of the operations or functions described as mapping or matching with a terminal may be interpreted as meaning mapping or matching the terminal's unique number or personal identification information, which is the terminal's identifying data.

[0015] The present invention will be described in detail below with reference to the attached drawings.

[0016] FIG. 1 is a diagram illustrating an AI-based active food and beverage curation provision system that reflects SNS activity data and OTT content consumption context according to an embodiment of the present invention. Referring to FIG. 1, the AI-based active food and beverage curation provision system (1) that reflects SNS activity data and OTT content consumption context may include at least one user terminal (100), a curation service provision server (300), and at least one information provision server (400). However, since the AI-based active food and beverage curation provision system (1) that reflects SNS activity data and OTT content consumption context of FIG. 1 is merely an embodiment of the present invention, the present invention is not to be interpreted as being limited through FIG. 1.

[0017] At this time, each component of FIG. 1 is generally connected through a network (Network, 200). For example, as shown in FIG. 1, at least one user terminal (100) can be connected to a curation service provider server (300) through the network (200). Also, the curation service provider server (300) can be connected to at least one user terminal (100) through the network (200). Additionally, at least one information provider server (400) can be connected to the curation service provider server (300) through the network (200).

[0018] Here, a network refers to a connection structure capable of exchanging information among individual nodes, such as multiple terminals and servers. Examples of such networks include Local Area Networks (LANs), Wide Area Networks (WANs), the World Wide Web (WWW), wired and wireless data networks, telephone networks, and wired and wireless television networks. Examples of wireless data communication networks include, but are not limited to, 3G, 4G, 5G, 3GPP (3rd Generation Partnership Project), 5GPP (5th Generation Partnership Project), 5G NR (New Radio), 6G (6th Generation of Cellular Networks), LTE (Long Term Evolution), WIMAX (World Interoperability for Microwave Access), Wi-Fi, Internet, LAN (Local Area Network), Wireless LAN (Wireless Local Area Network), WAN (Wide Area Network), PAN (Personal Area Network), RF (Radio Frequency), Bluetooth network, NFC (Near-Field Communication) network, satellite broadcasting network, analog broadcasting network, DMB (Digital Multimedia Broadcasting) network, etc.

[0019] In the following, the term "at least one" is defined as a term including both singular and plural forms, and it will be obvious that even if the term "at least one" does not exist, each component may exist in a singular or plural form and may mean singular or plural. Furthermore, whether each component is provided in a singular or plural form may be changed according to the embodiment.

[0020] At least one user terminal (100) may be a terminal of a user that grants access rights to SNS activity data, OTT viewing data and order data to a curation service providing server (300) using a web page, app page, program or application related to an AI-based active curation service for food and beverages, and receives and outputs recommendations for food and beverages corresponding thereto.

[0021] Here, at least one user terminal (100) may be implemented as a computer capable of connecting to a remote server or terminal via a network. Here, the computer may include, for example, a navigation system, a laptop equipped with a web browser, a desktop, a laptop, etc. At this time, at least one user terminal (100) may be implemented as a terminal capable of connecting to a remote server or terminal via a network. At least one user terminal (100) may include all kinds of handheld-based wireless communication devices, such as navigation, PCS (Personal Communication System), GSM (Global System for Mobile communications), PDC (Personal Digital Cellular), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), Wibro (Wireless Broadband Internet) terminal, smartphone, smartpad, tablet PC, etc.

[0022] The curation service providing server (300) may be a server that provides an AI-based active curation service web page, app page, program, or application. Furthermore, the curation service providing server (300) may be a server that collects multimodal data including SNS activity data, OTT viewing data, and order data from a user terminal (100), extracts features therefrom, extracts gustatory elements corresponding to the atmosphere of the OTT content in the OTT viewing data, and then inputs them into a recommendation model along with the features to recommend food and beverages. Additionally, the curation service providing server (300) may be a server that recommends food and beverages by checking the repurchase probability and marketing timing of the recommended food and beverages and responding accordingly. Here, the curation service providing server (300) may be implemented as a computer capable of connecting to a remote server or terminal via a network. Here, the computer may include, for example, a laptop, desktop, or laptop equipped with a navigation system and a web browser.

[0023] At least one information providing server (400) may be a server that provides information such as SNS activity data, OTT viewing data, and order data, whether or not a web page, app page, program, or application related to an AI-based active curation service is used. Here, at least one information providing server (400) may be implemented as a computer capable of connecting to a remote server or terminal via a network. Here, the computer may include, for example, a laptop, desktop, or laptop equipped with a navigation system and a web browser.

[0024] FIG. 2 is a block diagram for explaining a curation service providing server included in the system of FIG. 1, and FIG. 3 and FIG. 4 are drawings for explaining an embodiment in which an AI-based active food and beverage curation service according to an embodiment of the present invention is implemented.

[0025] Referring to FIG. 2, the curation service providing server (300) may include a collection unit (310), a content food mapping unit (320), a personalization unit (330), a recommendation unit (340), an environment reflection unit (350), a marketing timing optimization unit (360), a construction unit (370), a dietary restriction reflection unit (380), and an experience curation unit (390).

[0026] When a curation service providing server (300) or another server (not shown) operating in conjunction with one embodiment of the present invention transmits an AI-based active curation service application, program, app page, web page, etc. to at least one user terminal (100), the at least one user terminal (100) may install or open the AI-based active curation service application, program, app page, web page, etc. Additionally, a service program may be operated on at least one user terminal (100) using a script executed in a web browser. Here, a web browser refers to a program that enables the use of web (WWW: World Wide Web) services and receives and displays hypertext described in HTML (Hyper Text Mark-up Language), and includes, for example, Chrome, Microsoft Edge, Safari, Firefox, Whale, UC Browser, etc. In addition, "application" refers to an application on a terminal, and includes, for example, an app running on a mobile terminal (smartphone).

[0027] Referring to FIG. 2, the collection unit (310) can collect multimodal data including SNS activity data, OTT viewing data, and order data of a user terminal (100). The user terminal (100) can grant access rights to SNS activity data, OTT (Over the Top) viewing data, and order data of a delivery application. At this time, the SNS activity data, OTT viewing data, and order data of a delivery application may be as follows, for example.

[0028] SNS activity data Data types example meaning hashtag #SpicyFood, #LateNightSnack, #Chicken Reflects current food interests great I like the tteokbokki video Used as a signal for specific menu preference Search Keywords "Late-night snack recommendation", "Spicy chicken feet" Indicates the state of immediate need arising Post content I'm stressed today, so I want to eat something spicy. Emotion-based food desire extraction possible Follow Follow food influencers Reflects trend sensitivity and personal taste

[0029] OTT viewing data Data types example meaning Content genres Thriller, Romance, Comedy Used for food sentiment mapping Viewing time 11 PM TV Able to determine the timing for late-night snacks City Hall location Home, on the move It is possible to assess the consumption environment. Viewing behavior Immersive (no pauses), lots of skips Emotional state can be estimated specific scene Eating scenes, tense scenes Used as an immediate trigger event

[0030] OTT viewing data Data types example meaning Order menu Chicken, pizza, tteokbokki Basic food preferences can be identified Order time Usually between 10 PM and 12 AM Individual meal patterns can be extracted Order cycle Order every 3 days Repurchase is predictable Order amount Average 20,000 won Price sensitivity analysis is possible Preferred Brand Repeat orders for a specific chicken brand Brand preference can be reflected

[0031] In this context, multimodal data refers to different forms of data (text, images, videos, behaviors, etc.).

[0032] data Modality (form) explanation SNS activity data SNS activity data Hashtags, posts, likes, etc. OTT viewing data Video + Metadata + Behavior Video content, viewing patterns Order data Structured data Menu, time, price, etc.

[0033] To this end, the collection unit (310) can collect structured data such as user order data, payment information, and preferred menus, as well as multimodal data including SNS hashtags and like information, and OTT content genre, running time, and mood information. Kafka or RabbitMQ can be used for data collection, and an Open API or partnership API can be utilized for SNS and OTT integration. The collected data can be stored in PostgreSQL, MongoDB, or Redis.

[0034] Additionally, the collection unit (310) can normalize the collected heterogeneous data based on the user and integrate it into an analyzable form based on time zone, region, and content units. To this end, it uses a data pipeline based on Airflow or Prefect and performs data cleaning through Python Pandas or Spark. The integrated data can be stored in a Data Lake or Cloud Warehouse (BigQuery, Redshift). Furthermore, the collection unit (310) can extract feature values ​​that reflect the current consumption context based on SNS keywords, OTT genres and moods, weather, time zone and regional trends, etc. It can use KoBERT, Sentence-BERT, or a morphological analyzer for natural language processing and utilize rule-based tagging or LLM-based classification models for content metadata analysis, and can collect weather and regional information through external APIs.

[0035] For example, the state vector at user u and time t can be defined as Equation 1.

[0036]

[0037] Xu,t is the state vector of user u at time t, Su,t is the activity vector of SNS activity data, Ou,t is the viewing vector of OTT viewing data, Hu,t is the order vector of order data, Ct is the environment vector of environment data, and Rt is the trend vector of trend information.

[0038]

[0039] Fu,t is an integrated contextual feature vector, and f(·) is a feature extraction function (including NLP, embedding, and normalization). Accordingly, features can be extracted from the user's state vector, Xu,t. This becomes the input data to be fed into the recommendation model when recommending food and beverages.

[0040] The Contents-Food Mapping section (320) can map the atmosphere of the OTT content of the OTT viewing data into gustatory elements. To this end, a rule table for content-food mapping can be constructed, for example, as shown in Table 5 below.

[0041] Content elements Detailed attributes Emotion / Atmosphere gustatory elements converted value Example food Genre thriller Tension, stimulation Spicy flavor spicy=0.9 Chicken feet, tteokbokki Action Excitement, energy Salty taste, high calorie salty=0.8, fat=0.7 Chicken, hamburger Romance Softness, sweetness sweetness sweet=0.9 Cake, pasta comedy Lightness, joy Snack type light=0.8 popcorn, french fries drama Sensibility, warmth Broth / Warmth warm=0.9 Gukbap, ramyeon atmosphere High tension stress stimulating spicy=0.8 Spicy food calmness sense of stability Plainness mild=0.8 salad melancholy Need comfort Sweetness, high calorie sweet=0.7, fat=0.6 chocolate Viewing time nighttime Increased hunger High calorie calorie=0.9 Chicken, pizza afternoon lightness low calorie light=0.7 sandwich weather rain Sensibility, warmth broth warm=0.9 Pajeon, soup dishes heat refreshing sensation coolness cold=0.9 Cold noodles, ice cream Viewing behavior High immersion Concentration Convenience food easy=0.8 finger food Lots of skips boredom stimulating spicy=0.7 Spicy snacks

[0042] In other words, its characteristic feature is that it converts the atmosphere of OTT content into the gustatory attributes of food; by converting video (audiovisual information) into taste (gustatory information), it possesses the characteristic of a rule table that translates the emotions of the content into taste. Mathematically, this can be expressed as follows.

[0043]

[0044] Here, Mu,t is the food preference latent vector (taste, temperature, stimulus, etc.), and g(·) represents the content-food transformation function (rule + embedding-based).

[0045] The personalization unit (330) can calculate a recommendation score that reflects preferences and trends among food and beverages with taste elements using order data and SNS activity data.

[0046]

[0047] Score(u,i,t) is the recommendation score for food and beverage menu i to user u, w1, w2, w3, w4 are weight coefficients, Sim(·) is the similarity function (Cosine Similarity), Mu,t is the food preference latent vector, Ei is the feature vector for food and beverage menu i, Pref(u,i) is the user preference (based on order data), Trend(i,t) is the trend score of food and beverage menu i at time t (based on SNS activity data), and Context(i,t) represents the goodness of fit of food and beverage menu i at time t (weather and situation).

[0048] The personalization unit (330) is described as calculating a recommendation score that reflects preference and trends among food and beverages with taste elements using order data and SNS activity data, which is an explanation of mathematical formula 4. That is, preference refers to the user preference of Pref(u,i), and trend refers to the trend score of food and beverage menu i at time t of Trend(i,t). Also, food and beverages with taste elements are included in Mu,t, because Mu,t is Fu,t transformed by g(·), and Fu,t is derived as [[Su,t, Ou,t, Hu,t, Ct, Rt]→Xu,t→Fu,t]. Accordingly, the higher the similarity between the food preference latent vector and the feature vector of menu i, the higher the probability of recommendation, and ultimately, a recommendation score is calculated so that food and beverages with taste elements corresponding to the atmosphere of the content are extracted and the preference and trends are reflected.

[0049] parameters meaning Sim(M,E) How well this food and beverage menu matches the taste this person wants right now Pref(u,i) Whether this user is someone who originally eats this food and beverage menu Trend(i,t) I wonder if people are eating a lot of it right now Context(i,t) Is the food and beverage menu suitable for the current situation?

[0050] The recommendation unit (340) can recommend food and beverages to the user terminal (100) in order of highest recommendation score. The user terminal (100) can receive recommendations for food and beverages.

[0051] The environment reflection unit (350) can collect environment data including weather and time zones to identify environment information of the user terminal (100) and reflect the environment data when calculating the recommendation score. Accordingly, environment data is collected and reflected as the environment vector Ci of Equation 1, and is ultimately used when calculating the recommendation score. In addition, the environment data described above can be used when the weather is reflected in Context(i,t) of Equation 4.

[0052] The marketing timing optimization unit (360) can optimize the marketing timing for recommending food and beverages based on the repurchase probability model, after predicting the repurchase probability of the food and beverage for which the recommendation score was calculated in the personalization unit before the recommendation unit (340) recommends the food and beverage.

[0053]

[0054] Prepurchase(u,t) is the repurchase probability, σ(·) is the sigmoid function, △tu is the time elapsed since the last purchase, Behavioru is the user behavior pattern vector, and α, β, and γ are learned parameters. In other words, deriving the repurchase probability is the repurchase probability model, and since Equation 5 is merely a conceptual expression of this, α, β, and γ mean that they are parameters obtained as a result of training the repurchase probability model using training data to construct the repurchase probability model. Also, △tu and Behavioru can be extracted from order data.

[0055] A repurchase probability model can consist of a hybrid model comprising, for example, survival analysis, which predicts when a purchase will be made again, and a classification model, which determines whether to buy or not. Survival analysis can be conceptualized mathematically as Prepurchase(t) = 1 - e^-λ(t), and models such as Cox Proportional Hazards, the Weibull Survival Model, and DeepSurv can be utilized. In other words, the model can be configured so that the probability of purchase increases over time. Additionally, for the classification model, Gradient Boosting can be used; for example, XGBoost, LightGBM, and CatBoost can be utilized, but are not limited to these.

[0056] Accordingly, since it is desirable to construct the repurchase probability model as [time-based model] + [behavior-based model], it can be conceptualized as shown in Equation 6 below.

[0057]

[0058] Ptime is the result of survival analysis, Pbehavior is the result of the classification model (machine learning), and α and β are weights. For example, if a user orders chicken every 3 days and 3 days have passed, and assuming the survival analysis model yielded 0.7 and the classification model (XGBoost) yielded 0.6, then P becomes 0.65.

[0059] In addition, marketing timing needs to be optimized, and mathematical formula 7 can be used for this purpose.

[0060]

[0061] T* is the optimal marketing timing (optimal push sending time), and Engagement(u,t) is the probability of user u's response (click or view, etc.) at time t.

[0062]

[0063] If you need to determine not only the timing but also the type of marketing, you can use a method like Equation 8. In this case, θ1, θ2, and θ3 are threshold values.

[0064] Additionally, the marketing timing optimization unit (360) can optimize marketing timing by estimating the hunger timing of the user of the user terminal (100) as real-time situation data and adding it to the repurchase probability. That is, since the repurchase timing (Purchase Cycle) and the hunger timing (Hunger Timing) may be different from each other, the hunger timing can also be considered.

[0065]

[0066] Time(t) represents the time zone, Contextu,t represents the current situation (weather, location, activity, etc.), and Contentu,t represents the state of OTT content. Accordingly, considering both the probability of repurchase and the probability of hunger, it can be expressed as Equation 10.

[0067]

[0068] In other words, the situation of [repurchase possible + current hunger] becomes the best timing. Accordingly, a situational strategy can be established, for example, as shown in Table 7 below.

[0069] situation meaning Action Repurchase ↑ / Hunger ↓ It's not time to eat yet. Advance coupon Repurchase↓ / Hunger↑ I want to eat it right now Instant Recommendation Repurchase↑ / Hunger↑ Optimal timing Strong push Repurchase↓ / Hunger↓ No interest Unexposed

[0070] Additionally, event-based triggers can be added; marketing is not performed when content begins because the user enters a state of immersion, whereas marketing can be applied when the content reaches the middle or ends, as the desire to consume increases.

[0071] The construction unit (370) constructs a rule table in which the atmosphere and gustatory elements of the OTT content are mapped, and when OTT viewing data is collected from the user terminal (100), it can extract gustatory elements corresponding to the atmosphere of the OTT content of the OTT viewing data. Since this is as described above, a redundant explanation is not included.

[0072] The dietary restriction reflection unit (380) receives a non-preferred menu from the user terminal (100) and can reflect the food and beverage corresponding to the non-preferred menu in the recommendation score so that it is not recommended. For example, let's assume that chicken is a user's preference. In this case, if chicken feet are recommended because the user likes chicken, the user may not like it. This is because there may be parts that are not eaten. Also, if tripe or blood soup is recommended because the user likes pork, the user may not like it. By excluding food and beverages that the user explicitly says they dislike in this way, the accuracy of the recommendation can be increased.

[0073] Experience Curation (390) stores food and beverage packages by mapping them to at least one OTT content, and can provide a coupon for the food and beverage package when the OTT content is selected on the user terminal (100). For example, as shown in FIG. 4 (a), when watching a romantic movie, a set of pasta + wine + emotional UI is provided, or when watching a thriller movie, a set of spicy chicken feet + beer + dark UI is provided. Beyond recommending menus, the experience can be recommended by mapping food and beverages of these packages to a specific genre or content. For example, when watching a soccer match, it is recommended that there be beer and chicken. In other words, Experience Curation is not simply recommending [what to eat], but rather recommending [an overall consumption experience] that fits the user's current situation and emotions.

[0074] division General recommendation Experience curation standard menu Situation + Emotion + Content result Chicken Recommendation Rainy night + Thriller → Spicy chicken + Beer range food Food + Drinks + Atmosphere + Timing target select Experience Design

[0075]

[0076] In other words, ① Food refers to menu recommendations, ② Drink refers to beverage recommendations, ③ Mood refers to atmosphere which can be UI color schemes, music, or mood, ④ Timing refers to the timing of when to display it, and ⑤ Context refers to the situation, such as OTT, weather, and time. For example, assuming it is 11 PM, a user is watching thriller content on an OTT platform, and it is a rainy day, whereas previously chicken feet would have been recommended, through experience curation, spicy chicken feet, beer, a dark UI, discount coupons, and "Now Push" can be bundled into a single experience and provided.

[0077] Basic structure Content → Taste → Food and Beverage Expansion structure Content → Emotion → Taste → Food & Beverage + Atmosphere + Timing

[0078] Hereinafter, the operation process according to the configuration of the curation service providing server of FIG. 2 described above will be explained in detail with reference to FIG. 3 and FIG. 4. However, it is obvious that the embodiment is merely one of the various embodiments of the present invention and is not limited thereto.

[0079] Referring to FIG. 3a, the curation service providing server (300) collects multimodal data including SNS activity data, OTT viewing data, and past order data when a user watches thriller genre content on an OTT platform during nighttime hours and simultaneously searches for hashtags related to spicy food on SNS or likes them.

[0080] Then, the curation service providing server (300) normalizes the collected multimodal data based on the user and integrates it by including time zone, weather, and regional information, and derives a feature value, i.e., a contextual feature vector (feature vector), that reflects the current user's emotional state and consumption context. Next, the curation service providing server (300) converts the atmosphere of the OTT content included in the contextual feature vector, i.e., tension and stimulating atmosphere, into gustatory elements such as spiciness and stimulation. Then, the contextual feature vector and the gustatory elements are input into a recommendation model; the gustatory elements may be input by embedding them into a taste vector, or they may be used after being converted into a taste vector by the recommendation model. The recommendation model calculates a recommendation score for each menu by synthesizing the similarity between the taste vector and the feature vector of each menu, user preference, and trends. Here, preference or trends can be extracted from the contextual feature vector. At this time, if the user has a history of dislike for a specific food or brand, that menu may be filtered out and excluded from the recommendation target.

[0081] Subsequently, the curation service provider server (300) calculates the probability of repurchase and the probability of hunger based on the user's repurchase cycle and current hunger state, and determines the optimal marketing timing by considering events such as the start or end of content viewing. Finally, when the recommendation score and trigger conditions are satisfied, the curation service provider server (300) can induce the user's actual purchasing behavior by providing the user with a discount coupon or a promotional message in the form of a push, along with food and beverages suitable for the content currently being viewed. Afterwards, the curation service provider server (300) collects the user's response and uses it as material to retrain each model based on whether the prediction was correct or incorrect, and what the user's response was like.

[0082] Referring to FIG. 4, one embodiment of the present invention provides experience curation as in (a), so that the OTT content and food and beverage package are provided as a single experience rather than simply ending with a menu recommendation, thereby utilizing association so that the food and beverage package always comes to mind when viewing the OTT content. In addition, as in (b), food and beverage ordered by a large number of current viewers of Movie A can be recommended to the user terminal (100), and as in (c), food and beverage can be recommended by reflecting real-time physiological rhythms. Also, as in (d), food and beverage can be recommended using content scene unit triggering.

[0083] Cluster-based dynamic recommendation is a method that forms user clusters with similar characteristics based on users' current status and behavioral data, and dynamically generates recommendation results by reflecting the real-time consumption patterns of those clusters. Additionally, real-time physiological rhythm reflection is a technology that predicts food desires and consumption tendencies at a specific point in time based on user circadian rhythm information, such as time of day, sleep patterns, and activity levels, and incorporates this into recommendations. Furthermore, content scene-based triggers are a trigger method that detects specific scenes or events in OTT content in real time and immediately performs food recommendations or marketing tailored to the emotions or situations of those scenes.

[0084] As for the details regarding the AI-based active curation service provision method of FIGS. 2 to 4 that are not described, they are identical to or can be easily inferred from the details described above regarding the AI-based active curation service provision method of FIG. 1, so further explanation will be omitted.

[0085] FIG. 5 is a diagram illustrating the process of data transmission and reception between each component included in the AI-based active curation provision system for food and beverages reflecting SNS activity data and OTT content consumption context of FIG. 1 according to an embodiment of the present invention. Hereinafter, an example of the process of data transmission and reception between each component will be described through FIG. 5, but the present invention is not to be interpreted as being limited to such an embodiment, and it is obvious to those skilled in the art that the process of data transmission and reception shown in FIG. 5 may be changed according to various embodiments described above.

[0086] Referring to FIG. 5, the curation service providing server collects multimodal data including SNS activity data, OTT viewing data, and order data of a user terminal (S5100), and maps the atmosphere of the OTT content in the OTT viewing data to be converted into gustatory elements (S5200).

[0087] Then, the curation service provider calculates a recommendation score that reflects preferences and trends among food and beverages with taste elements using order data and SNS activity data (S5300), and recommends food and beverages to the user terminal in order of highest recommendation score (S5400).

[0088] The order of the steps described above (S5100~S5400) is merely an example and is not limited thereto. That is, the order of the steps described above (S5100~S5400) may vary, and some of these steps may be executed simultaneously or deleted.

[0089] As for the details regarding the AI-based active curation service provision method of Fig. 5 that are not described, they are identical to or can be easily inferred from the details described above regarding the AI-based active curation service provision method of Figs. 1 to 4, so further explanation will be omitted.

[0090] The method for providing an AI-based active curation service for food and beverages according to one embodiment described through FIG. 5 may also be implemented in the form of a recording medium containing computer-executable instructions, such as an application or program module executed by a computer. A computer-readable medium may be any available medium accessible by a computer and includes both volatile and non-volatile media, as well as removable and inseparable media. Additionally, a computer-readable medium may include all computer storage media. A computer storage medium includes both volatile and non-volatile, removable and inseparable media implemented by any method or technique for storing information, such as computer-readable instructions, data structures, program modules, or other data.

[0091] The method for providing an AI-based active curation service for food and beverages according to one embodiment of the present invention described above may be executed by an application basically installed on a terminal (which may include a program included in a platform or operating system, etc., basically installed on the terminal), or by an application (i.e., a program) directly installed by a user on a master terminal through an application providing server, such as an application store server, an application, or a web server related to the service. In this sense, the method for providing an AI-based active curation service for food and beverages according to one embodiment of the present invention described above may be implemented as an application (i.e., a program) that is basically installed on a terminal or directly installed by a user, and may be recorded on a computer-readable recording medium such as a terminal.

[0092] The foregoing description of the present invention is for illustrative purposes only, and those skilled in the art will understand that other specific forms can be easily modified without altering the technical spirit or essential features of the present invention. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. For example, each component described as a single unit may be implemented in a distributed manner, and components described as distributed may likewise be implemented in a combined form.

[0093] The scope of the present invention is defined by the claims set forth below rather than by the detailed description above, and all modifications or variations derived from the meaning and scope of the claims and equivalent concepts thereof should be interpreted as being included within the scope of the present invention.

Claims

Claim 1 An AI-based active curation system for food and beverages that reflects the context of SNS activity data and OTT content consumption, comprising: a user terminal that grants access rights to SNS activity data, OTT (Over the Top) viewing data, and order data of a delivery application, and receives recommendations for food and beverages; and a curation service providing server comprising: a collection unit that collects multimodal data including SNS activity data, OTT viewing data, and order data of the user terminal; a Contents-Food Mapping unit that maps the atmosphere of the OTT content of the OTT viewing data to be converted into gustatory elements; a personalization unit that calculates a recommendation score reflecting preferences and trends using the order data and SNS activity data among food and beverages having the gustatory elements; and a recommendation unit that recommends the food and beverages to the user terminal in order of highest recommendation score. Claim 2 The AI-based active food and beverage curation providing system reflecting SNS activity data and OTT content consumption context, wherein, in claim 1, the curation service providing server further comprises an environment reflection unit that collects environment data including weather and time zones to identify environment information of the user terminal and reflects the environment data when calculating the recommendation score. Claim 3 The AI-based active curation provision system for food and beverages reflecting SNS activity data and OTT content consumption context, wherein, in claim 1, the curation service providing server further includes a marketing timing optimization unit that, prior to recommending food and beverages in the recommendation unit, predicts the repurchase probability for the food and beverage for which a recommendation score was calculated in the personalization unit using a pre-established repurchase probability model, and then optimizes the marketing timing for recommending the food and beverage based on the repurchase probability. Claim 4 In claim 1, the marketing timing optimization unit is characterized by estimating the user's hunger timing of the user terminal as real-time situational data and optimizing the marketing timing by adding it to the repurchase probability, thereby providing an AI-based active food and beverage curation system reflecting SNS activity data and OTT content consumption context. Claim 5 The AI-based active food and beverage curation provision system reflecting SNS activity data and OTT content consumption context, wherein the curation service providing server further comprises a construction unit that constructs a rule table mapped to the atmosphere and gustatory elements of OTT content, and extracts gustatory elements corresponding to the atmosphere of the OTT content of the OTT viewing data when OTT viewing data is collected from the user terminal. Claim 6 The AI-based active food and beverage curation provision system reflecting SNS activity data and OTT content consumption context, wherein the curation service providing server further includes a dietary constraint reflection unit that receives a non-preferred menu from the user terminal and reflects it in the recommendation score so that food and beverages corresponding to the non-preferred menu are not recommended. Claim 7 The AI-based active curation provision system for food and beverages reflecting SNS activity data and OTT content consumption context, wherein the curation service providing server further comprises an experience curation unit that maps and stores a food and beverage package to at least one OTT content and provides a coupon for the food and beverage package when the OTT content is selected on the user terminal.

Images