Apparatus and method for providing a photography service including an artificial intelligence-based product advertising function.

Abstract

The present invention relates to a photographic method that includes a product advertising function. The method includes the steps of: receiving at least one image from a photographic device or a user terminal; determining whether an image contains an advertising product using an advertising product recognition model that takes an image as input and outputs whether an image contains an advertising product; and providing a reward to the user if the image contains an advertising product.

Description

Technical Field

[0001] The present invention relates to an apparatus and method for providing a photography service including an artificial intelligence-based product advertisement function, and more specifically, to an apparatus and method capable of providing a service that can provide a reward to a user by recognizing an image including the user's advertised product through an artificial intelligence model.

Background Art

[0002] The experience of unmanned photo studios that provide self-photography services is positioned as a new play culture for the younger generation to express and promote themselves. Despite the evolution of smartphones into high-performance cameras, consumers are enthusiastic about printed photos that allow them to preserve memories in analog form rather than digital photos.

[0003] Self-photo booths have rapidly evolved into various forms, ranging from self-studios where users take photos themselves with a remote control, to systems that print photos immediately after shooting at low prices. For example, Photoism, which provides a self-photography service based on the unmanned photo studio concept, operates approximately more than 350 stores nationwide, and the number of stores tends to increase further rapidly.

[0004] Self-photo booths are operated as an evolved version of the print sticker machines that were popular in the past. The difference between self-photo booths and print sticker photos is that they come in various small studio forms. While past print sticker photos mainly focused on character-centered staging, recent self-photo booths enable diverse staging by utilizing a large space. In addition, unlike the past, not only high-resolution cameras are installed, but also IT technologies such as convenient payment systems and print paper remaining amount notification systems are applied, making them not only easier for customers to use but also easier for operators to manage.

[0005] Traditional photo booths generate additional revenue by utilizing the time it takes for photos taken in the self-service photo booth to print, thereby creating revenue through interim advertisements. Furthermore, they are stimulating additional revenue streams beyond photography itself, such as website advertisements for sites where users can download photos taken in the self-service photo booth.

[0006] However, recently, there has been a growing need among advertisers to directly expose their advertised products in photos taken by customers, thereby increasing the effectiveness of their advertising. [Overview of the Initiative] [Problems that the invention aims to solve]

[0007] Traditional self-photography services primarily generate additional revenue through indirect advertising, such as intermediary ad placements and website advertising, which leads to a problem of reduced advertising effectiveness. Therefore, the problem that the present invention aims to solve is to resolve these conventional problems, and there is a need for a new method that can provide a direct advertising effect when customers use a self-photography service by taking pictures of the advertised product together with the product.

[0008] As a result, the inventors of this invention attempted to develop a method to provide a reward to a user if the image taken with a customer's self-photography service includes an advertised product, by determining through an artificial intelligence model whether the image includes the advertised product.

[0009] The problems addressed by the present invention are not limited to those mentioned above, and other problems not mentioned can be clearly understood by those skilled in the art from the following description. [Means for solving the problem]

[0010] To solve the problems of the present invention, a photographic method including a product advertising function according to one embodiment of the present invention is provided. The method includes the steps of: receiving at least one image from a photographic device or a user terminal; determining whether an image contains an advertising product using an advertising product recognition model that takes an image as input and outputs whether an image contains an advertising product; and providing a reward to the user if the image contains an advertising product.

[0011] According to the features of the present invention, the invention further includes the step of providing the photographic device with a guide for photographing the image, wherein the guide for photographing the image including the advertising product may be a photographic guide for at least one of the following: the product exposure range for recognition of the advertising product, the position or size of the advertising product in the image, or the exposure conditions for the characters or product name included in the advertising product.

[0012] According to the features of the present invention, the advertising product recognition model can include an image object recognition model and a character recognition model. According to the features of the present invention, the present invention may further include the steps of receiving information from the camera for an advertising product selected by the user, and providing the user with a recommended background for the image based on an identifier for the advertising product.

[0013] According to another feature of the present invention, the identifier for the advertising product may include information relating to the characteristics of the advertising product, including the hue, type, and target customer segment of the advertising product. According to another feature of the present invention, the step of providing a reward to the user may provide the reward in a differentiated manner based on at least one of the following: the number of advertising products included in the image, the number of images containing the advertising products, or the area ratio occupied by the advertising products in the image.

[0014] According to another feature of the present invention, the step of providing the user reward may include providing a reward of changing an existing shooting frame to a secret frame and shooting it for free, or providing a reward of taking additional shots with secret frames for a fee.

[0015] To solve the problems described above, a system is provided that provides a photography service including a product advertising function according to another embodiment of the present invention. The system includes a camera configured to capture at least one image, a user terminal, and a server configured to take an image as input, determine whether an image contains an advertising product using an advertising product recognition model that outputs whether the image contains an advertising product, and provide a reward to the user if the image contains an advertising product.

[0016] According to the features of the present invention, the photographic device is further configured to receive guidance for capturing the image, the guidance for capturing the image may be a photographic guide for at least one of the following: a product exposure range for recognition of the advertised product; the position or size of the advertised product in the image; or the exposure conditions for characters or product names contained in the advertised product.

[0017] According to another feature of the present invention, the advertising product recognition model may include an image object recognition model and a character recognition model. According to another feature of the present invention, the imaging device is configured to transmit the advertising product selected by the user to a server, and the server may be further configured to provide the user with recommended backgrounds for the image based on an identifier for the advertising product.

[0018] According to another feature of the present invention, the identifier for the advertising product may include information relating to the characteristics of the advertising product, including the hue, type, and target customer segment of the advertising product. According to another feature of the present invention, the server can provide the rewards in a differentiated manner based on at least one of the following: the number of advertising products included in the image, the number of images containing the advertising products, or the area ratio occupied by the advertising products in the image.

[0019] According to another feature of the present invention, the server may be further configured to offer a reward of changing an existing shooting frame to a secret frame and shooting it free of charge, or to offer a reward of taking additional shots with secret frames for a fee.

[0020] To solve the problems described above, a photographic apparatus including a product advertising function according to another embodiment of the present invention is provided. The photographic apparatus includes a display unit, an interface unit, a shooting unit, an output unit, and a control unit, the control unit includes a memory and one or more processors that execute a program loaded into the memory, the program includes instructions for receiving at least one image from a shooting device or a user terminal, instructions for determining whether an image contains an advertising product using an advertising product recognition model that takes an image as input and outputs whether the image contains an advertising product, and instructions for providing a reward to the user if the image contains an advertising product.

[0021] Specific details of other embodiments are included in the detailed description and drawings. [Effects of the Invention]

[0022] According to one embodiment of the present invention, by determining whether an image captured through a camera contains an advertising product and providing a reward to the user if the image contains the advertising product, active user participation can be encouraged, and a high advertising effect can be expected by directly promoting the advertising product in a form in which the user participates.

[0023] Further, according to an embodiment of the present invention, by providing a guide for image capturing to an image capturing device, it is possible to reduce malfunctions of an advertisement product recognition model that takes an image as an input and outputs whether the image contains an advertisement product, thereby improving the accuracy of the advertisement product recognition model.

[0024] Further, according to an embodiment of the present invention, when an advertisement product is included in an image, it is possible to expect a higher advertising effect by further providing the user with an opportunity for self-photography using a secret frame.

[0025] Further, according to an embodiment of the present invention, an operator can maximize the exposure of advertisement products by determining that the user is using an actual SNS, chat application, or the like, and providing a reward. The various and advantageous merits and effects of the present invention are not limited to the contents described above, and further various effects are included in the present specification. [Brief Description of the Drawings]

[0026] [Figure 1a] 1 is a conceptual diagram illustrating a system for providing a photography service including a product advertisement function according to an embodiment of the present invention. [Figure 1b] 1 is a block diagram illustrating an exemplary configuration of a system for providing a photography service according to an embodiment of the present invention. [Figure 2] 1 is a block diagram illustrating a configuration of a user terminal according to an embodiment of the present invention. [Figure 3] 1 is a block diagram illustrating an exemplary hardware configuration in which a server system or each server included in the server system according to an embodiment of the present invention may be implemented. [Figure 4a] 1 is a block diagram illustrating an exemplary configuration of an image capturing device that provides a photography service according to an embodiment of the present invention. [Figure 4b] 1 is a block diagram illustrating an exemplary configuration of an image capturing device that provides a photography service according to an embodiment of the present invention. [Figure 5]This is a conceptual diagram illustrating the processor of a conventional system that provides photography services. [Figure 6] This is an illustrative diagram showing the interface screen of a user terminal that provides a photo-taking service including a product advertising function according to one embodiment of the present invention. [Figure 7] This flowchart schematically illustrates the process for the photographic equipment of a system that provides a photographic service including a product advertising function, according to one embodiment of the present invention. [Figure 8] This flowchart schematically illustrates the mobile process for a system that provides a photo-taking service including a product advertising function, according to another embodiment of the present invention. [Figure 9] This flowchart shows the operation method of a system that provides a photo-taking service including a product advertising function according to one embodiment of the present invention. [Figure 10] This is an illustrative diagram showing the interface screen of a user terminal or kiosk that provides a photo-taking service including a product advertising function according to one embodiment of the present invention. [Figure 11a] This is an illustrative diagram showing the interface screen of a user terminal or kiosk that provides a photo-taking service including a product advertising function according to one embodiment of the present invention. [Figure 11b] This is an illustrative diagram showing the interface screen of a user terminal or kiosk that provides a photo-taking service including a product advertising function according to one embodiment of the present invention. [Figure 12] This is an illustrative diagram showing the interface screen of a user terminal or kiosk that provides a photo-taking service including a product advertising function according to one embodiment of the present invention. [Figure 13] This is an illustrative diagram showing the interface screen of a user terminal or kiosk that provides a photo-taking service including a product advertising function according to one embodiment of the present invention. [Figure 14a] This is an illustrative diagram showing the interface screen of a user terminal or kiosk that provides a photo-taking service including a product advertising function according to one embodiment of the present invention. [Figure 14b]This is an illustrative diagram showing the interface screen of a user terminal or kiosk that provides a photo-taking service including a product advertising function according to one embodiment of the present invention. [Figure 14c] This is an illustrative diagram showing the interface screen of a user terminal or kiosk that provides a photo-taking service including a product advertising function according to one embodiment of the present invention. [Figure 14d] This is an illustrative diagram showing the interface screen of a user terminal or kiosk that provides a photo-taking service including a product advertising function according to one embodiment of the present invention. [Modes for carrying out the invention]

[0027] The advantages and features of the present invention, and the methods for achieving them, will become clearer with reference to the embodiments described below in detail with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but can be embodied in a variety of different forms, and these embodiments are provided merely to complete the disclosure of the present invention and to fully inform a person ordinary in the art to which the present invention belongs of the scope of the invention, and the present invention is defined only by the scope of the claims. In connection with the description of the drawings, similar reference numerals may be used for similar components.

[0028] In this document, expressions such as "have," "may have," "include," or "may include" refer to the existence of the relevant feature (e.g., numerical values, functions, operations, or components such as parts), and do not exclude the existence of further features.

[0029] In this document, expressions such as "A or B," "A or / and B, or one or more of A or / and B" can include all possible combinations of the items listed together. For example, "A or B," "A and B, or A or B, or at least one of A or B" can refer to all cases where (1) at least one A is included, (2) at least one B is included, or (3) at least one A and at least one B are included.

[0030] The terms "first," "second," "first," or "second" used in this document can modify various components regardless of their order and / or importance, and are used only to distinguish one component from another, without limiting it to that component. For example, the first user terminal and the second user terminal may refer to different user terminals, regardless of their order or importance. For example, the first component may be named the second component without exceeding the scope of rights described in this document, and similarly, the second component may be renamed in place of the first component.

[0031] When it is mentioned that a component (e.g., component 1) is "operally or communicatively coupled with / to" or "connected to" another component (e.g., component 2), it should be understood that the component is directly coupled to the other component or can be coupled through another component (e.g., component 3). Conversely, when it is mentioned that a component (e.g., component 1) is "directly coupled" or "directly connected" to another component (e.g., component 2), it should be understood that there is no other component (e.g., component 3) between the component and the other component.

[0032] The expression "configured to" as used in this document may be replaced with other expressions depending on the context, such as "suitable for," "having the capacity to," "designed to," "adapted to," "made to," or "capable of." The term "configured to" does not necessarily mean only something that is "specifically designed to" in terms of hardware. Instead, in some situations, the expression "a device configured to" may mean that the device is "capable" of doing something together with other devices or components. For example, the phrase "a processor configured to perform A, B, and C" may mean a dedicated processor for performing those operations (e.g., an embedded processor), or a generic-purpose processor (e.g., a CPU or application processor) that can perform those operations by running one or more software programs stored in a memory device.

[0033] The terms used herein are used solely to describe specific embodiments and are not intended to limit the scope of other embodiments. Singular expressions may include plural expressions unless the context clearly indicates otherwise. Terms used herein, including technical or scientific terms, may have the same meaning as those generally understood by someone with ordinary skill in the art described herein. Terms used herein that are defined in general dictionaries may be interpreted as having the same or similar meaning as they do in the context of the relevant art, and not as ideal or overly formal unless explicitly defined herein. In some cases, terms defined herein may not be interpreted as excluding the embodiments described herein.

[0034] The features of each of the various embodiments of the present invention are partially or entirely combinable with one another, and are technically capable of a wide range of interdependence and operation, as can be fully understood by those skilled in the art. Each embodiment may be implemented independently of the others or together in relation to one another.

[0035] For clarity of interpretation of this specification, the following definitions of terms used herein are provided. "Photography equipment" refers to an unmanned information terminal for taking photographs. Using photography equipment, captured images can be viewed via a touchscreen, and after taking a photograph, they can be printed immediately using a remote control or timer. Photography equipment may also be referred to as "photography kiosk," "photo kiosk," or "picture kiosk." In this specification, the embodiments of this disclosure are described as being performed by a photography server, but they may also be performed by photography equipment depending on the circumstances.

[0036] "Image" refers to a photograph taken by the camera unit of a photographic device. An image may be taken by one or more users, and a photographic output may contain one or more images. Depending on the example, "Image" may refer to an image output on a photographic output, or it may refer to a shot (cut) of a video.

[0037] "Advertised product" refers to a product for which an advertiser has requested advertising. In this case, advertised products may include, but are not limited to, products such as pharmaceuticals, educational materials, confectionery, cosmetics, beverages, idol merchandise (e.g., photo cards, light sticks, keyrings), corporate logos, and clothing. Depending on the example, advertised products may also include movies, alcoholic beverages, insurance, etc.

[0038] "Photo output" refers to a printed material produced from a photographic device. Photo output may also be called a photo seal and may be in sticker form. The term "application" refers to a photo application that works in conjunction with the server system 200 and / or the camera equipment 300 to provide various functions to users of the self-photography service. For example, users can use the "application" to search for stores throughout South Korea, participate in events promoting advertised products, and receive rewards for doing so.

[0039] Hereinafter, an apparatus and method for providing a photography service including an artificial intelligence-based product advertising function according to the present invention will be described in detail with reference to the drawings. Figure 1a shows a system providing a photo-taking service including a product advertising function according to one embodiment of the present invention, and a screen displayed on a user terminal.

[0040] Referring to Figure 1a, a system 1000 providing a photo-taking service including a product advertising function according to one embodiment of the present invention (hereinafter referred to as the self-photography system) may include a user terminal 100, a user interface providing server 200 (hereinafter referred to as the photo-taking server 200 including the product advertising function), and a photography device 300.

[0041] The user terminal 100 may be a device owned by a user who intends to take a selfie using the camera 300. The user terminal 100 may be a portable digital device equipped with memory and a microprocessor for processing power, such as a smartphone, tablet PC, PC, or laptop computer, and is not limited to any particular form. In various embodiments, the user terminal 100 can download video content, such as captured images, and transmit it to the photo shooting server 200, which includes product advertising functions. In other embodiments, the user terminal 100 can capture images and transmit them to the photo shooting server 200, which includes product advertising functions.

[0042] The photo-taking server 200, which includes a product advertising function, can take an image taken by a user from the camera 300 as input and determine whether the image contains an advertising product through an advertising product recognition model. In this case, if the image contains an advertising product, the photo-taking server 200, which includes a product advertising function, can provide a reward to the user terminal 100. In the embodiment, the photo-taking server 200 can provide a reward to the user terminal 100 when the user uploads the image they have taken to a social networking service (SNS) or sets it as a profile in a chat application, etc., automatically, semi-automatically, or manually. For example, a user can upload the image they have taken to an SNS, capture the post uploaded to the SNS, and register it in the application of the camera equipment and / or operator providing the photo-taking service according to the embodiment of this disclosure. In this case, the photo-taking server 200 or the application can determine whether it is a post by recognizing the characteristics of the post uploaded to the SNS, the user's ID, tags, etc., using OCR or image recognition. If the recognized user's uploaded post or profile contains an advertising product, the photo-taking server 200 can provide a reward to the user terminal 100. By rewarding users based on their actual use of social media, chat apps, etc., operators can maximize the exposure of their advertised products.

[0043] Here, the rewards provided to the user terminal 100 may include, but are not limited to, points or secret frames. For example, they may include lottery tickets that allow the operator to enter a drawing for an event being run, or lottery tickets that increase the probability of winning an ongoing event provided by an advertiser. On the other hand, the photo shooting server 200, which includes product advertising functionality, can receive and store images, camera equipment information, user data, matching data, etc. The photo shooting server 200, which includes product advertising functionality, can provide the user with an identification code that allows access to the location where the photos taken through the camera equipment 300 are stored. The identification code that allows access to the location where the taken photos are stored may include barcodes and / or two-dimensional codes. The identification code may be printed on the photo output together with the photos. The photo shooting server 200, which includes product advertising functionality, can also provide images to the user terminal 100 in response to the user terminal 100 requesting the captured images by identifying the identification code. The photo shooting server 200, which includes product advertising functionality, may include a device DB server, a CMS (content management system) DB server, a storage server, and a mobile DB server. A more detailed configuration of the photo-taking server 200, including the product advertising function, is disclosed in detail through Figure 2.

[0044] The camera 300 is configured to provide a self-service photography service that allows users to take photos themselves and print the photos they have taken. After taking a photo, the camera 300 can generate a resource (e.g., the captured image) and upload it to the touchscreen. The camera 300 can print a photographic output that includes the image and an identification code. In one embodiment, the camera 300 can provide the image in digital file form to the photo-taking server 200 and simultaneously use AI to determine whether the image contains an advertising product and provide it to the user terminal 100. The camera 300 can also provide the captured image to the user in digital file form while simultaneously providing the printed photographic output. A more detailed configuration of the camera 300 is disclosed in detail through Figures 3a and 3b.

[0045] Figure 1b is a block diagram showing an exemplary configuration of a system providing a photography service according to one embodiment of the present invention. Referring to Figure 1b, the photo-taking server 200, which includes product advertising functionality, may include a device DB server 210, a CMS DB server 220, a storage server 230, and a mobile DB server 240 for managing and storing their respective data. In Figure 2, each server is shown as a separate configuration, but the present invention is not limited thereto, and each server shown does not necessarily have to be embodied as a separate configuration. Two or more servers may be embodied in a single server or configured as modules that perform the relevant functions within a single server.

[0046] The device DB server 210 may be a server for communicating with and managing the camera 300. In this case, the device DB server 210 can receive and store device information of the camera 300 from the camera 300 and provide it to the camera administrator. The device information of the camera 300 may include basic information of the camera 300 (e.g., ID, hardware information), information necessary for communication with the camera 300, information on the printer status (e.g., remaining paper, printer error), software program operating status information, software program update information, etc. Furthermore, the device DB server 210 can manage the operating status of the camera 300 in real time (e.g., normal operation, idle state, error occurrence, etc.). In this embodiment, the device DB server 210 can use AI to determine whether an image is included in an advertising product.

[0047] The CMS DB server 220 may be a server for storing user data and / or storage information (e.g., URLs). CMS (content management system) refers to a system that allows the camera equipment administrator to intuitively manage content. User data may include captured image data, data on whether the image contains advertising products determined through an advertising product recognition model, data for frames selected by the user, payment information data, matching data matching resources and member information, etc. The CMS DB server 220 can receive and store images captured from the camera equipment 300 and provide an API for uploading the images to the storage server 230.

[0048] The storage server 230 may be a server that communicates with the CMS DB server 220 and / or the mobile DB server 240, etc., to store images captured by the imaging device 300. Based on matching data stored in the CMS DB server 220, the storage server 230 can provide the user terminal 100 with images captured by the relevant user from among the stored resources (e.g., captured images).

[0049] The mobile DB server 240 may be a server for storing member information and / or file address information. For example, the mobile DB server 240 can receive an image video request from the user terminal 100 and transmit the member information stored in the CMS DB server 220. The mobile DB server 240 may not store the captured image files, but may store file address information that allows the user to download them directly from an application or storage server 230.

[0050] The photographic equipment 300, the photographic server 200 including product advertising functions, and the individual servers 210, 220, 230, 240 and user terminal 100 shown in Figure 1b can operate to exchange data with each other through a communication network. The communication network according to one embodiment of this disclosure can be configured regardless of the mode of communication, such as wired communication or wireless communication, and can consist of various types of communication networks, such as local area networks (LANs), metropolitan area networks (MANs), and wide area networks (WANs). Preferably, the communication network referred to herein may be the well-known Internet or World Wide Web (WWW). However, the communication network is not necessarily limited thereto and may include, in at least part, well-known wired and wireless data communication networks, well-known telephone networks, or well-known wired and wireless television communication networks. For example, the communication network is a wireless data communication network that may, at least in part, embody conventional communication methods such as Wi-Fi communication, Wi-Fi-Direct® communication, Long Term Evolution (LTE) communication, 5G communication, Bluetooth communication (including Low Energy Bluetooth® (BLE) communication), infrared communication, and ultrasonic communication. To give another example, the communication network is an optical communication network that may, at least in part, embody conventional communication methods such as Light Fidelity (LiFi).

[0051] Figure 2 is a block diagram showing the configuration of a user terminal according to one embodiment of the present invention. Referring to Figure 2, the user terminal 100 may include a memory interface 110, one or more processors 120, and peripheral interfaces 130. The various components within the user terminal 100 may be connected by one or more communication buses or signal lines.

[0052] The memory interface 110 is connected to the memory 150 and can transmit various types of data to the processor 120. Here, the memory 150 can include at least one type of storage medium from among flash memory type, hard disk type, multimedia card micro type, card type memory (e.g., SD or XD memory), RAM, SRAM, ROM, EEPROM, PROM, network storage, cloud, and blockchain database.

[0053] In various embodiments, the memory 150 can store at least one of the following: an operating system 151, a communication module 152, a graphic user interface module (GUI) 153, a sensor processing module 154, a telephone module 155, and an application module 156. Specifically, the operating system 151 can include commands for processing basic system services and commands for performing hardware tasks. The communication module 152 can communicate with at least one other device, computer, and server. The graphic user interface module (GUI) 153 can process a graphic user interface. The sensor processing module 154 can process sensor-related functions (e.g., processing audio input received using one or more microphones 192). The telephone module 155 can process telephone-related functions. The application module 156 can perform a variety of user application functions, such as electronic messaging, web browsing, media processing, exploration, imaging, and other process functions. In addition, the user terminal 100 can store one or more software applications 156-1, 156-2 associated with a single type of service in the memory 150. In this case, application 156-1 refers to a photo application that provides various functions to users using the self-photography service on the user terminal 100. Here, the user can use application 156-1 to search for stores that offer self-photography services, check events promoting advertised products, upload images containing advertised products that the user has photographed, and receive rewards for doing so.

[0054] In various embodiments, the memory 150 can store a digital assistant client module 157 (hereinafter referred to as the DA client module), thereby storing command words for performing the client-side functions of the digital assistant and various user data 158 (e.g., user data, information on events the user has participated in, event winning information, and other data).

[0055] On the other hand, the DA client module 157 can acquire user voice input, text input, touch input, and / or gesture input through various user interfaces (e.g., I / O subsystem 140) provided on the user terminal 100.

[0056] Furthermore, the DA client module 157 can output data in audiovisual and tactile forms. For example, the DA client module 157 can output data consisting of at least two, or one or more combinations, of voice, sound, notifications, text messages, menus, graphics, video, animation, and vibration. In addition, the DA client module 157 can communicate with a digital assistant server (not shown) using the communication subsystem 180.

[0057] In various embodiments, the DA client module 157 can collect additional information about the environment surrounding the user terminal 100 from various sensors, subsystems, and peripheral devices to constitute a context related to user input. For example, the DA client module 157 can provide context information along with user input to the digital assistant server to infer the user's intent. Here, the context information obtained along with user input can include sensor information, such as light, ambient noise, ambient temperature, images of the surrounding environment, video, etc. Another example is that the context information can include the physical state of the user terminal 100 (e.g., device orientation, device position, device temperature, power level, speed, acceleration, motion pattern, cellular signal strength, etc.). Yet another example is that the context information can include information related to the software state of the user terminal 100 (e.g., processes running on the user terminal 100, installed programs, past and present network activity, background services, error logs, resource usage, etc.).

[0058] In various embodiments, the memory 150 may include added or deleted instruction words, and the user terminal 100 may also include additional configurations or omit some configurations in addition to the configuration shown in Figure 2.

[0059] The processor 120 can control the overall operation of the user terminal 100 and execute a variety of instructions to implement the photo-taking service interface, including the artificial intelligence-based product advertising function, for applications or programs stored in the memory 150.

[0060] The processor 120 could be a computing device such as a CPU (Central Processing Unit) or an AP (Application Processor). Furthermore, the processor 120 could be embodied in the form of an integrated chip (IC), such as a System on Chip (SoC), which integrates various computing devices, including an NPU (Neural Processing Unit).

[0061] The peripheral interface 130 is connected to various sensors, subsystems, and peripheral devices, and can provide data to enable the user terminal 100 to perform various functions. Here, the performance of a function by the user terminal 100 can be understood as being performed by the processor 120.

[0062] The peripheral interface 130 can receive data from the motion sensor 160, the lighting sensor (light sensor) 161, and the proximity sensor 162, through which the user terminal 100 can perform orientation, light, and proximity sensing functions, etc. Another example is that the peripheral interface 130 can receive data from other sensors 163 (positioning system - GPS receiver, temperature sensor, biometric recognition sensor), through which the user terminal 100 can perform functions related to the other sensors 163.

[0063] In various embodiments, the user terminal 100 may include a camera subsystem 170 connected to a peripheral interface 130 and an optical sensor 171 connected thereto, through which the user terminal 100 can perform various shooting functions such as taking photographs and recording video clips.

[0064] In various embodiments, the user terminal 100 may include a communication subsystem 180 connected to a peripheral interface 130. The communication subsystem 180 may consist of one or more wired / wireless networks and may include various communication ports, radio frequency transceivers, and optical transceivers.

[0065] In various embodiments, the user terminal 100 includes an audio subsystem 190 connected to a peripheral interface 130, and such an audio subsystem 190 includes one or more speakers 191 and one or more microphones 192, enabling the user terminal 100 to perform voice-activated functions, such as speech recognition, voice duplication, digital recording, and telephone functions.

[0066] In various embodiments, the user terminal 100 may include an I / O subsystem 140 connected to a peripheral interface 130. For example, the I / O subsystem 140 can control a touchscreen 143 included in the user terminal 100 through a touchscreen controller 141. As an example, the touchscreen controller 141 can detect user contact and movement or interruption of contact and movement using one of several touch sensing technologies, such as capacitive, resistive, infrared, surface acoustic wave technology, proximity sensor arrays, etc. In another example, the I / O subsystem 140 can control other input / control devices 144 included in the user terminal 100 through other input controllers 142. As an example, the other input controllers 142 can control one or more buttons, rocker switches, thumbwheels, infrared ports, USB ports, and pointer devices such as styluses.

[0067] Figure 3 is a block diagram showing an exemplary hardware configuration that can be embodied in a server system or each server included in a server system according to one embodiment of the present invention. Referring to Figure 3, the photo-taking server 200, which includes product advertising functionality, may include a communication interface 210, memory 220, I / O interface 230, and processor 240, and each component may communicate with one or more communication buses or signal lines. Referring to Figures 1a to 1b, the photo-taking service provider server 200 may be the photo-taking server 200 which includes product advertising functionality.

[0068] The communication interface 210 can connect to imaging equipment and / or user terminals via a wired / wireless network to exchange data. For example, the communication interface 210 can receive user information, payment information, images, and camera status information from the camera. The communication interface 210 can also receive image requests from the user terminal. In other examples, the communication interface 210 can transmit control information to the camera. The communication interface 210 can transmit images to the user terminal. Furthermore, the communication interface 210 can transmit to the user terminal whether the image contains an advertising product, as determined through the advertising product recognition model. Additionally, if the image contains an advertising product, the communication interface 210 can offer a reward to the user. In this case, the reward may include points or a secret frame.

[0069] On the other hand, the communication interface 210 that enables the transmission and reception of such data includes a wired communication port 211 and a wireless circuit 212, where the wired communication port 211 may include one or more wired interfaces, such as Ethernet®, USB, FireWire, etc. The wireless circuit 212 can transmit and receive data with an external device via RF signals or optical signals. In addition, the wireless communication may use at least one of several communication standards, protocols and technologies, such as GSM, EDGE, CDMA, TDMA, Bluetooth®, Wi-Fi, VoIP, Wi-MAX®, or any other suitable communication protocol.

[0070] Memory 220 can store a variety of data used in the self-service photography service. For example, memory 220 can store equipment information of the photography device, user data, storage information, captured images, member information, result values ​​output from the advertising product recognition model, and / or file address information.

[0071] In various embodiments, the memory 220 may include volatile or non-volatile recording media capable of storing various types of data, instructions, and information. For example, the memory 220 may include at least one type of storage medium from among flash memory type, hard disk type, multimedia card micro type, card type memory (e.g., SD or XD memory), RAM, SRAM, ROM, EEPROM, PROM, network storage, cloud, and blockchain database.

[0072] In various embodiments, the memory 220 can store at least one of the following configurations: the operational system 221, the communication module 222, the user interface module 223, and one or more applications 224.

[0073] The operational structure 221 (e.g., built-in operational structures such as LINUX®, UNIX®, MAC OS, WINDOWS®, VxWorks®, etc.) can include a variety of software components and drivers for controlling and managing general system operations (e.g., memory management, storage device control, power management, etc.) and can support communication between various hardware, firmware, and software components.

[0074] The communication module 222 can support communication with other devices through the communication interface 210. The communication module 222 can include a variety of software components for processing data received by the wired communication port 211 or the wireless circuit 212 of the communication interface 210.

[0075] The user interface module 223 can receive user requests or input from a keyboard, touchscreen, mouse, microphone, etc., via the I / O interface 230, and provide a user interface on the display.

[0076] Application 224 may include a program or module configured to run on one or more processors 240. Here, the application for the photo-taking service may be implemented on a server farm.

[0077] The I / O interface 230 can connect at least one of the input / output devices (not shown) of the photo-taking service server 200, such as a display, keyboard, touchscreen, and microphone, to the user interface module 223. Together with the user interface module 223, the I / O interface 230 can receive user input (e.g., voice input, keyboard input, touch input, etc.) and process commands based on the received input.

[0078] The processor 240 is connected to the communication interface 210, memory 220, and I / O interface 230 to control the overall operation of the photo shooting service server 200, and can execute a variety of commands for downloading captured images or their corresponding time-lapse videos to a user terminal through applications or programs stored in memory 220.

[0079] Processor 240 could be a computing device such as a CPU (Central Processing Unit) or AP (Application Processor). Alternatively, processor 240 could be embodied in the form of an integrated chip (IC), such as a System on Chip (SoC), which integrates various computing devices. Or, processor 240 could include a module for computing artificial neural network models, such as a Neural Processing Unit (NPU).

[0080] Figure 3 is merely one embodiment of the configuration of server 200. Server 200 may have some components omitted from Figure 3, additional components not shown in Figure 3, or configurations or arrangements that combine two or more components. Components that can be included in server 200 may be embodied in hardware, software, or a combination of both hardware and software, including integrated circuits specialized for one or more signal processing or applications.

[0081] Figures 4a and 4b are block diagrams showing an exemplary configuration of a photographic device that provides a photographic service according to one embodiment of the present invention. Referring to Figure 4a, the imaging device 300 according to one embodiment of the present invention may include a display unit 301, an interface unit 302, a power supply unit 303, a communication unit 304, a settlement unit 305, a control unit 310, an imaging unit 321, an output unit 322, a sensor unit 323, an illumination unit 324, and the like.

[0082] The display unit 301 can display a variety of information related to the operation of the shooting device 300. For example, the display unit 301 can display a menu related to the shooting service, the status of the subject being photographed, etc. In addition, the display unit 301 can display both UI (User Interface) and GUI (Graphical User Interface) information using the displayed image. The specific information displayed by the display unit 301 described above is merely an example, and the present invention is not limited thereto.

[0083] The display unit 301 may include at least one of the following: liquid crystal display (LCD), thin film transistor-liquid crystal display (TFT LCD), organic light-emitting diode (OLED), flexible display, 3D display, e-ink display, or light-emitting diode (LED).

[0084] The interface unit 302 is configured to receive user input necessary for shooting services such as taking pictures and printing, and can be, but is not limited to, a physical keypad, dome switch, touchpad, stick, lever, etc. In one embodiment of the present invention, the display unit 301 can be realized as a touchscreen by forming a layered structure with or integrating it with a touch sensor. Such a touchscreen can function as the display unit 301 of the shooting device 300 and at the same time as the interface unit 302.

[0085] The power supply unit 303 can supply power to all or some of the components of the imaging equipment 300. For example, the power supply unit 303 may include a power management system, one or more power sources such as a battery or alternating current (AC), a charging system, a power failure detection circuit, a power converter or inverter, a power status indicator, or any other components for power generation, management, and distribution.

[0086] The communication unit 304 allows the imaging device 300 to send and receive data to and from a server system or external device via a communication network. The imaging device 300 can send and receive data to and from the server system in both directions via the communication unit 304, including user information of captured images and links to storage where image data is stored.

[0087] The payment unit 305 allows users to pay for services provided through the camera device 300 using cards, cash, mobile payments, cryptocurrencies, coupons, barcodes, 2D barcodes, etc.

[0088] The control unit 310 can perform overall control of the imaging device 300. In one embodiment of the present invention, when the user selects imaging conditions through the interface unit 302, the settlement unit 305 processes the settlement, and an imaging command is input through the interface unit 302, the control unit 310 can control the imaging unit 321 to image the subject. The control unit 310 can receive the input of the image and transmit it to the server system through the communication unit 304. The control unit 310 can receive a download web address link for accessing the image from the server system and control the output unit 322 to issue a two-dimensional code based on it.

[0089] In one embodiment of the present invention, the control unit 310 may include a processor 311 and a memory 312. The control unit 310 may include one or more processors 311. The one or more processors 311 included in the control unit 310 may be integrated on a single chip or they may be physically separated. Alternatively, the processor 311 and memory 312 may be implemented on a single chip. The memory 312 may include not only volatile memory such as S-RAM and D-RAM, but also non-volatile memory such as flash memory, ROM (Read Only Memory), and Erasable Programmable Read Only Memory (EPROM).

[0090] The imaging unit 321 can capture a subject located in the imaging space of the imaging device 300 by means of a control signal from the control unit 310. The output unit 322 can print a photographic output containing the captured photographic image and a two-dimensional code that allows access to the image stored in the server system. In one embodiment of the present invention, the shooting unit 321 and the output unit 322 may each include a camera and a printer.

[0091] The sensor unit 323 can sense whether or not a user is positioned in front of the imaging device 300, or the movements of a user positioned in front of the imaging device 300. The lighting unit 324 may include the lighting necessary for the shooting unit 321 to perform shooting operations.

[0092] Referring to Figure 4b, the photographic device 300 according to one embodiment of the present invention may include a PC 330, a camera 331, a printer 332, a user control unit 341, an I / O board 342, a card payment device 343, a banknote recognition device 344, etc., and the components included in the photographic device 300 can exchange various kinds of information with each other.

[0093] Camera 331 can provide captured images to PC 330. The camera can also receive camera on / off control signals from I / O board 342. Printer 332 can receive output control information from PC 330 and send printer status information to PC 330.

[0094] The I / O board 342 can receive control information transmitted from PC330 to each component or from each component to PC330. The user control unit 341 may include a touch monitor 30 and / or a camera remote control 31. The touch monitor 30 can transmit touch coordinate information entered by the user to the PC 330. The camera remote control 31 can transmit button press information to the PC 330 via the I / O board 342.

[0095] The card payment terminal 343 can send the card information (e.g., card number) entered by the user to the PC 330 if the user selects card payment. The banknote recognition machine 344 can send banknote insertion information of the banknotes inserted by the user to the PC 330 via the I / O board 342 if the user selects cash payment.

[0096] PC330 can perform overall control of the configuration included in the imaging device 300. PC330 can transmit user information, payment information, images, imaging device status information, etc., collected from various configurations to the external server 400. Here, PC330 can receive the results of the advertising product recognition model of the image from the external server 400. On the other hand, referring to Figures 1 and 2, the external server 400 may be a photography server 200 that includes product advertising functions.

[0097] Figure 5 is a conceptual diagram illustrating the processor of a conventional system providing a photo-taking service. In a conventional photo-taking service, the advertiser 510 requests the self-photography equipment operator 520 to advertise a product (e.g., product A). In this case, the self-photography equipment operator 520 can generate a photo frame suitable for the product. As another example, the self-photography equipment operator 520 can generate an advertising video to be provided after the advertiser 510 has taken a self-photo and while the photo is being output. The self-photography equipment operator 520 can provide the self-photography frame or advertising video suitable for the product to the company's stores or franchise stores 530.

[0098] Users can take self-portraits through a camera 550 installed in a directly managed store or affiliated store 530. The camera 550 can provide the user with a photo frame containing advertising products and print the photo output 540 that the user has taken and requested. As another example, the camera 550 can provide advertising videos received from a conventional photo service while the taken photo 540 is being printed.

[0099] Therefore, traditional photography services can provide indirect advertising effects by offering users photographic frames related to the advertised product. Furthermore, traditional photography services can provide advertising effects for the advertised product by providing images related to the product while the customer is waiting for their photos to be printed.

[0100] Figure 6 is an illustrative diagram showing the interface screen of a user terminal that provides a photo-taking service including a product advertising function according to one embodiment of the present invention. An application providing a photo-taking service that includes product advertising functionality can provide users with user interfaces 610, 620, and 630 for participating in events promoting advertised products. As an example, a user can upload event photos by clicking "Event Photos" on the first user interface screen 610. These event photos can be received from a camera. Alternatively, the event photos may be photographs taken from the user's terminal.

[0101] When a user uploads an event photo, a user interface 620 may be provided that displays the event photo, including a product advertisement function. The event photo may include an image 621 containing an advertised product. In this case, the photo-taking server including the product advertisement function according to one embodiment of the present disclosure can input the image 621 containing the advertised product as an input value to an advertised product recognition model. The photo-taking server including the product advertisement function can use the advertised product recognition model to determine whether the event photo includes an image 621 containing an advertised product. In this case, if the event photo includes an image 621 containing an advertised product, the photo-taking server including the product advertisement function can provide a reward to the user terminal. If an image 621 containing an advertised product is included in the event photo, a user interface 630 may be provided that displays 1000 points provided as a reward.

[0102] A more detailed explanation of the photography service, including product advertising functionality, will be provided in the following examples. Figure 7 is a flowchart illustrating the process for the photographic equipment of a system that provides a photographic service including a product advertising function according to one embodiment of the present invention.

[0103] In S701, when the user completes taking a photograph through the camera 710, the camera 710 can upload the user IDUID and the captured image and / or video to the CMS. At this time, the camera 710 can transmit the user IDUID and the captured image and / or video to the CMS DB server 722. The CMS DB server 722 can store the user IDUID and the image and / or video transmitted from the camera 710.

[0104] In S702, the CMS DB server 722 can send the user ID, image, and / or video to the storage server 723. The API (or API server) provided by the CMS DB server 722 can upload the user ID, image, and / or video files to the storage server 723 upon receiving them.

[0105] In S703, the storage server 723 can, in response to receiving the user ID and image and / or video, send the URL (or file address) to which the image and / or video file will be uploaded to the CMS DB server 722.

[0106] In S704, the CMS DB server 722 can match the user IDUID with the URL (or file address) to which the image and / or video file is uploaded.

[0107] In S705, the CMS DB server 722 can send the URL matched with the UID to the camera device 710. With S706, the imaging device 710 can generate a QR code (registered trademark) containing UID information.

[0108] In S707, the user terminal 730 can use AI to determine whether or not an advertising product is included in the received image and / or video file. This determination can be made using AI comprised of an advertising product recognition model.

[0109] In S708, the user terminal 730 can upload the parsed UID, member information, and AI-sensed content to the CMS DB server 722. In S709, the CMS DB server 722 can upload the parsed UID, member information, and AI-sensed content to the storage server 723.

[0110] With the S710, the camera 710 can output a QR code (registered trademark) and / or a photograph to a download page that includes the UID. In S711, the user terminal 730 can parse the UID through the QR code (registered trademark) provided by the camera 710.

[0111] In S712, the user terminal 730 can transmit the parsed UID and the member information logged in by the user terminal 730 to the CMS DB server 722. In S713, the CMS DB server 722 can match the received UID with the member information.

[0112] In S714, the CMS DB server 722 can send the updated UID, member information, URL, and AI detection content to the mobile DB server 724. With S715, the camera 710 uses AI to determine whether or not the advertising product is included in the image. If the image does not contain the advertising product, it can offer the user the opportunity to retake the photo.

[0113] In S716, the user terminal 730 can send requests to the mobile DB server 724 for resource viewing and image download. In S717, the user terminal 730 can send the file address to the mobile DB server 724.

[0114] In S718, the storage server 723 can provide the user terminal 730 with the download of the image file. Figure 8 is a flowchart illustrating a mobile process for a system that provides a photo-taking service including a product advertising function, according to another embodiment of the present invention.

[0115] In S801, when the user completes taking a photograph through the camera 810, the camera 810 can upload the user IDUID and the captured image and / or video to the CMS. At this time, the camera 810 can transmit the user IDUID and the captured image and / or video to the CMS DB server 822. The CMS DB server 822 can store the user IDUID and the image and / or video transmitted from the camera 810.

[0116] In S802, the CMS DB server 822 can send the user ID, image, and / or video to the storage server 823. The API (or API server) provided by the CMS DB server 822 can upload the user ID, image, and / or video files to the storage server 823 upon receiving them.

[0117] In S803, storage server 823 can, in response to receiving the user ID and image and / or video, send a URL (or file address) to which the image and / or video file will be uploaded to CMS DB server 822.

[0118] In S804, the CMS DB server 822 can match the user IDUID with the URL (or file address) to which the image and / or video file is uploaded.

[0119] In S805, the CMS DB server 822 can send the URL matched with the UID to the camera device 810. With S806, the imaging device 810 can generate a QR code (registered trademark) containing UID information.

[0120] In S807, the user terminal 830 can upload the parsed UID and AI type, etc., to the CMS DB server 822. In S808, the CMS DB server 822 can send the parsed UID and AI detection details to the storage server 823.

[0121] With S809, the camera 810 can output a QR code (registered trademark) and / or a photograph to a download page that includes the UID. With S810, the user terminal 830 can parse the UID through the QR code (registered trademark) provided by the camera 810.

[0122] In S811, user terminal 830 can send the parsed UID and member information logged in by user terminal 730 to the CMS DB server 822. In S812, the CMS DB server 822 can match the received UID with the member information.

[0123] In S813, the CMS DB server 822 can send the updated UID, member information, and URL to the mobile DB server 824. In S814, the user terminal 830 can send requests to the mobile DB server 824 for viewing resources and downloading images.

[0124] In S815, the user terminal 830 can send the file address to the mobile DB server 824. In S816, the user terminal 830 can use AI to determine whether or not an advertising product is included in the received image and / or video file. In this case, the determination of whether or not an advertising product is included can be made using AI composed of an advertising product recognition model.

[0125] In S817, the storage server 823 can provide the user terminal 830 with the download of the image file. In S818, the user terminal 830 can request the CMS DB server 822 to issue a secret frame coupon if an advertising product is included in the image. For example, if the user terminal 830 uses AI to determine whether an advertising product is included in the image and it is found to be included, it can request the CMS DB server 822 to issue a secret frame coupon. On the other hand, if the user terminal 830 determines that the AI ​​detection failed to detect an advertising product, it can terminate the process.

[0126] In S819, when the user terminal 830 (or user account) is provided with a secret frame code and / or random number, the user terminal 830 can receive the secret frame coupon provided by the CMS DB server 822.

[0127] When a user is given a secret frame coupon, or when a user uses a secret frame coupon, the kiosk is populated with a secret frame code and / or a random number, and the user can proceed with taking photos using the corresponding secret frame within a predetermined time.

[0128] Figure 9 is a flowchart showing the operation method of a system that provides a photo-taking service including a product advertising function according to one embodiment of the present invention, and Figures 10 to 14 are illustrative diagrams showing the interface screen of a user terminal that provides a photo-taking service including a product advertising function according to one embodiment of the present invention.

[0129] First, referring to Figure 9, the photo-taking server 200, which includes product advertising functionality, receives an image (S910). At this time, the image may be a four-shot photograph including a shot of the advertised product. Here, the photo-taking server 200, which includes product advertising functionality, can receive an image from a camera or a user terminal. Here, the image may be a four-shot photograph taken from a camera. As another example, the image may be a photograph taken from a user terminal. At this time, the photo-taking server 200, which includes product advertising functionality, can provide the user terminal with an image shooting frame.

[0130] On the other hand, the photo-taking server 200, which includes product advertising functionality, can receive data for matched users from the camera. For example, when the photo-taking server 200, which includes product advertising functionality, receives an image from the camera, it can also receive data indicating that the user has received the image.

[0131] Referring to Figure 10 for more details, the photo-taking server 200, which includes product advertising functionality, can receive event photo upload requests through interaction with the event photo area 1010. At this time, the photo-taking server 200, which includes product advertising functionality, can provide a user interface for uploading event photos to the user terminal. Here, it can be confirmed that the mileage 1020 of the user currently logged into the user terminal is 0 points.

[0132] Referring to Figure 11a, the user interface of the user terminal can display an image 1110 that has been captured by the user terminal and uploaded to the server by the user. As another example, the image 1110 displayed on the user terminal may be an image captured by a camera and received from the camera. Here, the user terminal can receive the captured image through a QR code (registered trademark) provided by the camera. Alternatively, the user terminal can log in to the camera with the same user and receive the captured image through the server. In this case, the method by which the user terminal receives the image from the camera is not limited to the above case and can be configured in various ways.

[0133] In another embodiment, referring to Figure 11b, the user interface of the camera can display a screen asking the user whether or not to upload the image 1110 captured by the camera to the photo shooting server 200 for event participation. If the user enters "Yes" 1120, the photo shooting server 200, which includes product advertising functionality, can receive the event photo upload request. The user can also log in to the camera with the account to which they wish to receive rewards.

[0134] In another embodiment, the photo-taking server 200, which includes a product advertising function, can provide the camera with guidance for capturing images that include the advertised product. In this case, the guidance for capturing images that include the advertised product may include guidance for the product exposure range, the position of the advertised product in the image, and the exposure range of the text or product name included in the advertised product. Here, the product exposure range may be the ratio of the area exposed in the image to the total area of ​​the image corresponding to the advertised product. The position or size of the advertised product in the image may be the position or size of the advertised product in the image. In this case, the exposure conditions for the text or product name included in the advertised product may be conditions for the text or product name that must be included in the advertised product. On the other hand, the guidance for capturing images that include the advertised product may be provided to improve the accuracy of the advertised product recognition model of the photo-taking server 200, which includes a product advertising function, or to improve the advertising effect of the advertised product.

[0135] For example, the photo-taking server 200, which includes product advertising functionality, can provide guidance that 70% of the area of ​​the advertised product image must be exposed within the image. It can also provide guidance that the product must not be on the outermost edge but must be located in the center. Another example is the provision of shooting guidance that the closer the advertised product is to the center of the image and the larger its size, the greater the reward offered. Additionally, if the product name is "Hokkesu," the photo-taking server 200 can provide guidance that "Hokke" must be exposed. However, the guidance provided by the photo-taking server 200 for image capture including advertised products is not limited to the examples given above.

[0136] Returning to Figure 9, the photo-taking server 200, which includes a product advertising function, uses an advertising product recognition model to determine whether an image contains an advertising product, taking the image as input (S920). In this case, the advertising product recognition model may be an AI model configured to determine whether an image contains an advertising product, taking the image as input. Here, the advertising product recognition model may consist of an object recognition model and a character recognition model.

[0137] More specifically, referring to Figure 12, the photo-taking server 200, which includes a product advertising function, can determine whether an image contains an advertising product by taking an image as input through an advertising product recognition model. At this time, the photo-taking server 200, which includes a product advertising function, can recognize the advertising product 1210 through the object recognition model of the advertising product recognition model. Here, the photo-taking server 200, which includes a product advertising function, can recognize the characters or product name 1220 contained in the advertising product through the character recognition model of the advertising product recognition model.

[0138] In this case, the object recognition model can be composed of one of the object detection models. For example, the object recognition model can be composed of a YOLO model. In this case, the object recognition model can include a loss function. Here, the object recognition model is an object detection model without a mainstream anchor, and the loss function can include classification loss, box regression loss, object loss, etc. On the other hand, the object recognition model can be composed of an artificial neural network model, a convolutional neural network (CNN), etc. As another example, the object recognition model can be machine-learned using methods such as supervised learning, unsupervised learning, and reinforcement learning. In this context, machine learning algorithms used to perform machine learning may include decision trees, Bayesian networks, support vector machines, artificial neural networks, Ada-boost, perceptrons, genetic programming, and clustering. On the other hand, character recognition models may be constructed using methods such as OCR (Optical Character Recognition).

[0139] Returning to Figure 9, the photo-taking server 200, which includes a product advertising function, provides a reward to the user terminal based on whether the image contains an advertising product (S930). At this time, if the photo-taking server 200, which includes a product advertising function, determines that the image contains an advertising product, it can provide a reward to the user terminal. Here, the reward may be points provided to the user logged into the user terminal. On the other hand, if the photo-taking server 200, which includes a product advertising function, determines that the image does not contain an advertising product, it can provide a request to the user terminal and / or the camera to retake the photo.

[0140] Another example is a photo-taking server 200, which includes a product advertising function, that can provide rewards to users in a secret frame. In this case, the secret frame may be a self-portrait photo frame created by a self-portrait equipment operator at the request of an advertiser. For example, the secret frame may include frames with altered hues, frames with altered poses of people in the frame, frames with altered people in the frame, and so on.

[0141] Here, the photo-taking server 200, which includes product advertising functionality, can provide a secret frame code and / or random number if it determines that the image contains an advertised product. At this time, the user may be offered a secret frame coupon or a photo-taking coupon that allows for immediate shooting. On the other hand, the photo-taking server 200, which includes product advertising functionality, can terminate the photo-taking process if it determines that the image does not contain an advertised product.

[0142] When a user is given a secret frame coupon, or when a user uses a secret frame coupon, the kiosk is populated with a secret frame code and / or a random number, and the user can proceed with taking a photo using the corresponding secret frame.

[0143] For example, if the photo-taking server 200, which includes a product advertising function, recognizes an advertising product through an advertising product recognition model as a cheering stick for idol a, it can offer the user a secret frame coupon for idol a as a reward.

[0144] On the other hand, the photo-taking server 200, which includes a product advertising function, can provide the user with a change coupon, such as a free coupon, which allows the user to change the frame of the self-portrait taken to one of the existing frames and / or a secret frame, if the advertised product is recognized through the advertised product recognition model.

[0145] Furthermore, the photo-taking server 200, which includes product advertising functionality, can provide users with additional photo-taking coupons in the form of paid coupons or the like, which allow them to take additional self-portraits with a secret frame for a fee, if an advertised product is recognized through the advertised product recognition model. Another example is that the photo-taking server 200, which includes product advertising functionality, can provide rewards to user terminals with differences. Depending on the embodiment, the photography device 300 can provide rewards with differences. In this case, the photo-taking server 200, which includes product advertising functionality, can provide rewards with differences based on the number of advertised products, the number of images containing advertised products, the area ratio occupied by advertised products in an image, etc. In this case, the number of advertised products may be the number of advertised products included in a single image. For example, if there are 3 advertised products in one shot of an image, the number of advertised products may be 3. Here, the number of images containing advertised products may be the number of images containing advertised products out of 4 shots of an image. For example, if 3 out of 4 shots of an image contain advertised products, the number of images containing advertised products may be 3. Furthermore, the area ratio occupied by the advertising product in an image may be the ratio of the advertising product's area to the total area of ​​the image. For example, if the advertising product occupies only 30% of the area in an image, the area ratio occupied by the advertising product in the image may be 30%.

[0146] Referring to Figure 13 for more details, the photo-taking server 200, which includes a product advertising function, can provide a reward to the user terminal if the image contains an advertised product. At this time, the user terminal can confirm that the user's mileage 1310 has increased to 500 points due to the provided reward.

[0147] In another embodiment, the photo-taking server 200, which includes a product advertising function, can provide the user with an image background suitable for the advertised product. In other words, the photo-taking server 200, which includes a product advertising function, can provide the user with a recommended image background based on an identifier for the advertised product. In this case, the identifier for the advertised product may include information about the characteristics of the advertised product, including the type of product, hue, and target customer segment.

[0148] Referring to Figure 14a in more detail, the photo-taking server 200, which includes product advertising functionality, can provide the camera with a user interface 1410 for the user to select from multiple advertising products. The photo-taking server 200, which includes product advertising functionality, can receive the advertising products selected by the user from the camera.

[0149] Referring to Figure 14b, the photo-taking server 200, which includes product advertising functionality, can provide a basic background determined through an identifier for the advertised product. In this case, the photo-taking server 200, which includes product advertising functionality, can provide a basic background 1420 determined based on the type, hue, and target customer segment of the advertised product. Here, the photo-taking server 200, which includes product advertising functionality, can receive the basic background from the advertiser. On the other hand, the photo-taking server 200, which includes product advertising functionality, can provide an image in which the background of a photographed image has been changed to the basic background.

[0150] Referring to Figure 14c, the photo-taking server 200, which includes a product advertising function, can provide a user interface that allows the user to select from a plurality of backgrounds determined based on an identifier for the advertised product. In this case, the photo-taking server 200, which includes a product advertising function, can include backgrounds 1430 determined based on the hue of the advertised product, backgrounds 1440 determined based on the atmosphere of the advertised product, and backgrounds 1450 determined based on the type of advertised product. For example, if the advertised product is a naturalistic toner, the photo-taking server 200, which includes a product advertising function, can provide options such as a white background determined to harmonize with the toner's hue, a background containing leaves that harmonize with naturalism, and a background containing water droplets depending on the type of toner. On the other hand, the photo-taking server 200, which includes a product advertising function, can provide images in which the background of the captured photograph has been changed to each of the plurality of backgrounds.

[0151] Depending on the embodiment, the secret frame may be provided as a random frame. The random subject may be a member of an idol group, an artist's costume, etc., and the random frame can be set in a variety of ways. Referring to Figure 14d, a member of idol a may be randomly selected and provided as a frame. In this case, the secret frame may be provided in the form of a hidden frame 1460 in which only the silhouette of the randomly selected member is visible, and it is not possible to confirm which member it is. If the user succeeds in the mission (e.g., by taking a photo using idol a's cheering stick), the user can select a member and print a photo output with the selected member. Conversely, if the user fails the mission, the user can print a photo output with a randomly selected member of idol a. The programs executed by the terminals and servers described in this disclosure may be embodied in hardware components, software components, and / or combinations of hardware components and software components. The programs may be executed by any system capable of executing computer-readable instructions.

[0152] Software can include computer programs, code, instructions, or a combination of one or more of these, and can configure a processing unit to operate as desired, or instruct the processing unit independently or collectively. Software can be embodied in a computer program containing instructions stored on computer-readable storage media. Examples of computer-readable storage media include magnetic storage media (e.g., ROM (Read-Only Memory), RAM (Random-Access Memory), floppy disks, hard disks, etc.) and optically readable media (e.g., CD-ROMs, DVDs (Digital Versatile Discs)). Computer-readable storage media can be distributed across a network of computer systems, and computer-readable code can be stored and executed in a distributed manner. The media can be read by a computer, stored in memory, and executed by a processor.

[0153] Computer-readable storage media may be provided in the form of non-transitory storage media. Here, "non-transitory" simply means that the storage medium is tangible and does not contain signals; it does not distinguish whether the data is stored on the storage medium semi-permanently or temporarily.

[0154] Furthermore, the programs relating to the embodiments of this disclosure may be provided as part of a computer program product. Computer programs may be traded as goods between sellers and buyers. A computer program may include a software program and a storage medium readable by a computer on which the software program is stored. For example, a computer program may include a product in the form of a software program (e.g., a downloadable application) that is distributed electronically through a device manufacturer or an electronic marketplace (e.g., Google Play Store, application store). For electronic distribution, at least a portion of the software program may be stored in or temporarily generated on a storage medium. In this case, the storage medium may be a server of a manufacturer, a server of an electronic marketplace, or a storage medium of an intermediary server that temporarily stores the software program.

[0155] A computer program may be part of a system comprising a server and a device, and may include a storage medium for the server or a device. Alternatively, if a third device (e.g., a smartphone) is connected to the server or device, the computer program may include a storage medium for the third device. Or, the computer program may include the software program itself, which is transmitted from the server to the device or the third device, or from the third device to the device. In this case, one of the server, device, and third device can execute the computer program and perform the method according to the disclosed embodiment. Alternatively, two or more of the server, device, and third device can execute the computer program and perform the method according to the disclosed embodiment in a distributed manner. For example, a server can execute a computer program stored on the server and control a device connected to the server to perform the method according to the disclosed embodiment. Another example is a third device that can execute a computer program and control a device connected to the third device to perform the method according to the disclosed embodiment. When the third device executes a computer program, the third device can download the computer program from the server and execute the downloaded computer program. Alternatively, the third device may perform the method according to the disclosed embodiment by running a computer program provided in a pre-loaded state.

[0156] As described above, the embodiments have been explained with limited embodiments and drawings, but a person with ordinary skill in the art can make various modifications and variations from the above description. For example, the described technology may be carried out in a different order than described, and / or the components of the described computer system or module may be combined or assembled in a different form than described, or substituted or replaced by other components or equivalents, and the appropriate results may be achieved. [Explanation of Symbols]

[0157] 100: A system that provides photography services including product advertising functionality. 110: Imaging equipment 120: Server system 121: Device DB Server 122: CMS DB Server 123: Storage Server 124: Mobile DB Server 200: Photo shooting server with product advertising functionality 300: Filming equipment

Claims

1. A photographic method including a product advertising function embodied by a processor, A step of receiving at least one image from a camera or user terminal; A step of determining whether an image contains an advertising product using an advertising product recognition model that takes an image as input and outputs whether the image contains an advertising product; and A photography method including a product advertising function, comprising the step of providing a reward to the user if the image includes an advertised product.

2. The steps further include providing the imaging device with guidance for capturing the image; The guidelines for photographing images including the aforementioned advertising products are as follows: A photographic method including a product advertising function according to claim 1, comprising a photographic guide for at least one of the following: a product exposure range for recognizing the advertised product; the position or size of the advertised product within the image; and exposure conditions for characters or product names included in the advertised product.

3. The aforementioned advertising product recognition model, A photographic method including a product advertising function according to claim 1, comprising an object recognition model and a character recognition model for the aforementioned image.

4. A step of receiving information about the advertising product selected by the user from the aforementioned camera; A photographic method including a product advertising function according to claim 1, further comprising the step of providing the user with a recommended background for the image based on an identifier for the advertised product.

5. The identifier for the advertising product is A photographic method that includes a product advertising function according to claim 4, comprising information on the characteristics of the advertised product, including the hue, type, and target customer segment of the advertised product.

6. The step of providing the aforementioned user with a reward is: A photographic method including a product advertising function according to claim 5, wherein the reward is provided based on at least one of the number of the advertising products included in the image, the number of the images containing the advertising products, and the area ratio occupied by the advertising products in the images.

7. The step of providing the aforementioned user reward is: Offer a reward of changing an existing shooting frame to a secret frame for free, or A photographic method including a product advertising function according to claim 6, comprising the step of offering a reward of additional photography with a secret frame for a fee.

8. A system that provides a photography service including a product advertising function, A photographic device configured to capture at least one image; User terminal; and Using an advertising product recognition model that takes an image as input and outputs whether the image contains an advertising product, the system determines whether the image contains an advertising product based on the input image. A photography system including product advertising functionality, including a server configured to provide a reward to the user if the aforementioned image contains an advertising product;

9. The aforementioned photographic equipment is Further configured to receive guidance for capturing the aforementioned image, The guide for capturing the aforementioned image is: A photographic system comprising a product advertising function according to claim 8, the product advertising function comprising a photographic guide for at least one of the following: a product exposure range for recognition of the advertised product, the position or size of the advertised product in the image, and exposure conditions for characters or product names included in the advertised product.

10. The aforementioned advertising product recognition model, A photographic system including a product advertising function according to claim 8, comprising an object recognition model and a character recognition model for the aforementioned image.

11. The aforementioned photographic equipment is The system is configured to transmit information about the advertising product selected by the user to the server, The aforementioned server, A photography system including a product advertising function according to claim 8, further configured to provide the user with a recommended background for the image based on an identifier for the advertising product.

12. The identifier for the advertising product is A photographic system that includes a product advertising function according to claim 11, comprising information on the characteristics of the advertised product, including the hue, type and target customer segment of the advertised product.

13. The aforementioned server, A photographic system including a product advertising function according to claim 8, which provides the rewards based on at least one of the number of the advertised products included in the image, the number of the image containing the advertised products, and the area ratio occupied by the advertised products in the image.

14. The aforementioned server, Offer a reward of changing an existing shooting frame to a secret frame for free, or A photography system including a product advertising function according to claim 13, further configured to offer a reward of additional photography with a secret frame for a fee.

15. A photographic device that includes a product advertising function, Display unit; Interface section; Photography Department; Output section; and Includes a control unit, The control unit includes a memory and one or more processors that execute the program loaded into the memory. The aforementioned program, Instructions for receiving at least one image from a camera or user terminal; Instructions for determining whether an image contains an advertising product, using an advertising product recognition model that takes an image as input and outputs whether the image contains an advertising product; A photographic apparatus including a product advertising function, which includes instructions for providing a reward to the user if the image contains an advertised product.

Images