Apparatus and method for providing photographing service, including artificial intelligence-based product advertisement function

Abstract

The present invention relates to a photographing method including a product advertisement function. The method comprises the steps of: receiving at least one image from a photographing device or a user terminal; determining whether the image includes an advertisement product by using an advertisement product recognition model that takes an image as an input and determines whether the image includes an advertisement product as an output; and providing a reward to the user when the image includes an advertisement product.

Description

Device and method for providing a photo shooting service including an artificial intelligence-based product advertising function

[0001] The present invention relates to a device and method for providing a photo shooting service including an artificial intelligence-based product advertising function, and more particularly, to a device and method capable of providing a service that provides a reward to a user by recognizing an image containing a user's advertised product through an artificial intelligence model.

[0002] The experience of unmanned photo studios offering self-portraits is establishing itself as a new form of entertainment for young people, offering a way to express themselves and appeal to them. Despite the advancement of smartphones into powerful cameras, consumers are still enthusiastic about printed photos, allowing them to preserve their memories in analog form rather than digital.

[0003] Self-photo booths have rapidly evolved into various forms, from self-studios where you can take pictures yourself with a remote control to inexpensive methods where you can print the pictures right after taking them.

[0004] For example, Photoism, which provides a self-photography service with an unmanned photo studio concept, operates around 350 stores nationwide, and the number of stores is increasing rapidly.

[0005] Self-photo booths are an evolved version of the sticker cameras that were popular in the past. What sets them apart from sticker photography is their compact, studio-style design. While sticker photography of the past focused on people, modern self-photo booths utilize a spacious space to accommodate a variety of settings. Furthermore, unlike past models, they incorporate high-resolution cameras, convenient payment systems, and even paper-remaining notifications, making them more convenient for customers and easier for operators to manage.

[0006] Additional revenue models for existing photo booths include using the printing time of photos taken in self-photo booths to run mid-roll advertisements, generating revenue from these. Furthermore, additional revenue streams outside of photo booths, such as website advertising for downloading photos taken in self-photo booths, are being promoted.

[0007] However, there is a growing need among advertisers to increase advertising effectiveness by directly displaying the advertised product in photos taken by customers.

[0008] The existing methods of generating additional revenue in self-photography services mainly involve indirect advertising, such as mid-roll advertisements and website advertisements, which has the problem of low advertising effectiveness.

[0009] Accordingly, the problem that the present invention seeks to solve is to solve such existing problems, and a new method is required that can provide a direct advertising effect by having a customer take a picture of the advertised product together when using a self-photography service.

[0010] As a result, the inventors of the present invention sought to develop a method for providing a reward to a user when an advertised product is included in a photographed image by determining whether the photographed image contains an advertised product through a customer's self-photography service using an artificial intelligence model.

[0011] The tasks of the present invention are not limited to the tasks mentioned above, and other tasks not mentioned will be clearly understood by those skilled in the art from the description below.

[0012] To solve the problem of the present invention, a photography method including a product advertising function according to one embodiment of the present invention is provided. The method comprises the steps of: receiving at least one image from a photographing device or a user terminal; using an advertising product recognition model that inputs the image and outputs whether the image contains an advertising product; determining whether the image contains an advertising product using the image as input; and providing a reward to the user if the image contains an advertising product.

[0013] According to a feature of the present invention, the method further includes a step of providing assistance for taking pictures of the image to the photographing device, wherein the assistance for taking pictures of the image including the advertising product may be assistance for taking pictures of at least one of a product exposure range for recognizing the advertising product, a position or size of the advertising product within the image, and an exposure condition of characters or product names included in the advertising product.

[0014] According to a feature of the present invention, the advertising product recognition model may include an object recognition model and a character recognition model of the image.

[0015] According to a feature of the present invention, the method may further include a step of receiving information about an advertising product selected by the user from the photographing device, and a step of providing a recommended background of the image to the user based on an identifier for the advertising product.

[0016] According to another feature of the present invention, the identifier for the advertising product may include information about the characteristics of the advertising product, including the color, type, and demand customer base of the advertising product.

[0017] According to another feature of the present invention, the step of providing a reward to the user may provide the reward differentially based on at least one of the number of the advertising products included in the image, the number of the images including the advertising products, and the ratio of the area occupied by the advertising products in the image.

[0018] According to another feature of the present invention, the step of providing the user reward may include a step of providing a reward for shooting by changing an existing shooting frame to a secret frame for free, or a step of providing a reward for shooting additionally with a secret frame for a fee.

[0019] In order to solve the aforementioned problem, a system for providing a photo shooting service including a product advertising function according to another embodiment of the present invention is provided. The system includes a photographing device configured to capture at least one image, a user terminal, and a server configured to determine whether the image contains an advertising product using an advertising product recognition model that inputs the image and outputs whether the image contains an advertising product, and to provide a reward to the user if the image contains an advertising product.

[0020] According to a feature of the present invention, the photographing device is further configured to receive help for photographing the image, and the help for photographing the image may be a photographing help for at least one of a product exposure range for recognizing the advertising product, a position or size of the advertising product within the image, and an exposure condition of characters or product names included in the advertising product.

[0021] According to another feature of the present invention, the advertising product recognition model may include an object recognition model and a character recognition model of the image.

[0022] According to another feature of the present invention, the photographing device is configured to transmit an advertising product selected by the user to a server, and the server may be further configured to provide a recommended background of the image to the user based on an identifier for the advertising product.

[0023] According to another feature of the present invention, the identifier for the advertising product may include information about the characteristics of the advertising product, including the color, type, and demand customer base of the advertising product.

[0024] According to another feature of the present invention, the server can provide the reward differentially based on at least one of the number of the advertising products included in the image, the number of the images including the advertising products, and the ratio of the area occupied by the advertising products in the image.

[0025] According to another feature of the present invention, the server may be further configured to provide a reward for shooting by changing an existing shooting frame into a secret frame for free, or to provide a reward for shooting additionally with a secret frame for a fee.

[0026] In order to solve the problem described above, a photography device including a product advertising function according to another embodiment of the present invention is provided. The photography device includes a display unit, an interface unit, a photography unit, an output unit, and a control unit, and the control unit includes a memory and one or more processors that execute a program loaded in the memory, wherein the program includes an instruction for receiving at least one image from a photography device or a user terminal, an instruction for determining whether the image includes an advertising product by using an advertising product recognition model that inputs an image and outputs whether the image includes an advertising product, and an instruction for providing a reward to the user when the image includes an advertising product.

[0027] Specific details of other embodiments are included in the detailed description and drawings.

[0028] According to one embodiment of the present invention, by determining whether an image captured by a camera device includes an advertising product and providing a reward to the user if the image includes an advertising product, active participation of the user can be induced, and a high promotional effect can be expected by directly promoting the advertising product in a form in which the user participates.

[0029] In addition, according to one embodiment of the present invention, by providing assistance in capturing an image to a photographing device, the malfunction of an advertising product recognition model that inputs an image and outputs whether the image contains an advertising product can be reduced, thereby increasing the accuracy of the advertising product recognition model.

[0030] In addition, according to one embodiment of the present invention, when an advertisement product is included in an image, a higher promotional effect can be expected by providing users with more opportunities to take self-photos with a secret frame.

[0031] Additionally, according to one embodiment of the present invention, the operator can maximize the exposure of the advertising product by providing rewards based on the user's judgment of actual SNS, chat app, etc.

[0032] The various advantageous advantages and effects of the present invention are not limited to those described above, and further diverse effects are included in the present specification.

[0033] FIG. 1A is a conceptual diagram illustrating a system that provides a photo shooting service including a product advertising function according to one embodiment of the present invention.

[0034] FIG. 1b is a block diagram illustrating an exemplary configuration of a system providing a photo shooting service according to one embodiment of the present invention.

[0035] Figure 2 is a block diagram showing the configuration of a user terminal according to one embodiment of the present invention.

[0036] FIG. 3 is a block diagram illustrating an exemplary hardware configuration in which a server system or each server included in the server system according to one embodiment of the present invention can be implemented.

[0037] FIGS. 4A and 4B are block diagrams illustrating an exemplary configuration of a photographing device that provides a photographing service according to one embodiment of the present invention.

[0038] Figure 5 is a conceptual diagram schematically illustrating a processor of a system providing a conventional photo shooting service.

[0039] FIG. 6 is an exemplary diagram showing an interface screen of a user terminal that provides a photo shooting service including a product advertising function according to one embodiment of the present invention.

[0040] FIG. 7 is a flowchart schematically illustrating a process for a photographing device of a system that provides a photo shooting service including a product advertising function according to one embodiment of the present invention.

[0041] FIG. 8 is a flowchart schematically illustrating a mobile process of a system that provides a photo shooting service including a product advertising function according to another embodiment of the present invention.

[0042] FIG. 9 is a flowchart illustrating an operation method of a system providing a photo shooting service including a product advertising function according to one embodiment of the present invention.

[0043] FIGS. 10 to 14 are exemplary diagrams showing interface screens of a user terminal or kiosk that provides a photo shooting service including a product advertising function according to one embodiment of the present invention.

[0044] The advantages and features of the present invention, and the methods for achieving them, will become clearer with reference to the embodiments described in detail below together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms. These embodiments are provided solely to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the invention, and the present invention is defined only by the scope of the claims. In connection with the description of the drawings, similar reference numerals may be used for similar components.

[0045] In this document, the expressions "has," "may have," "includes," or "may include" indicate the presence of a feature (e.g., a number, function, operation, or component such as a part), but do not exclude the presence of additional features.

[0046] In this document, the expressions "A or B," "at least one of A and / or B," or "one or more of A or / and B" can include all possible combinations of the listed items. For example, "A or B," "at least one of A and B," or "at least one of A or B" can all refer to cases where (1) at least one A is included, (2) at least one B is included, or (3) at least one A and at least one B are included.

[0047] The expressions "first," "second," "first," or "second," used in this document, may describe various components, regardless of order and / or importance, and are only used to distinguish one component from another, and do not limit the components. For example, a first user terminal and a second user terminal may represent different user terminals, regardless of order or importance. For example, without departing from the scope of the rights set forth in this document, a first component may be referred to as a second component, and similarly, a second component may also be referred to as a first component.

[0048] When it is said that a component (e.g., a first component) is "(operatively or communicatively) coupled with / to" or "connected to" another component (e.g., a second component), it should be understood that the component is directly coupled to the other component, or can be connected via another component (e.g., a third component). Conversely, when it is said that a component (e.g., a first component) is "directly coupled to" or "directly connected to" another component (e.g., a second component), it should be understood that no other component (e.g., a third component) exists between the first component and the other component.

[0049] The expression "configured to" as used herein can be used interchangeably with, for example, "suitable for," "having the capacity to," "designed to," "adapted to," "made to," or "capable of." The term "configured to" does not necessarily mean something is "specifically designed to" in hardware. Instead, in some contexts, the expression "a device configured to" can mean that the device, together with other devices or components, is "capable of." For example, the phrase "a processor configured (or set) to perform A, B, and C" may mean a dedicated processor (e.g., an embedded processor) for performing those operations, or a general-purpose processor (e.g., a CPU or application processor) that can perform those operations by executing one or more software programs stored in a memory device.

[0050] The terms used in this document are used only to describe specific embodiments and may not be intended to limit the scope of other embodiments. The singular expression may include the plural expression unless the context clearly indicates otherwise. Terms used herein, including technical or scientific terms, may have the same meaning as commonly understood by those of ordinary skill in the art described in this document. Terms defined in general dictionaries among the terms used in this document may be interpreted as having the same or similar meaning in the context of the relevant technology, and shall not be interpreted in an idealized or overly formal sense unless explicitly defined in this document. In some cases, even if a term is defined in this document, it cannot be interpreted to exclude the embodiments of this document.

[0051] The individual features of the various embodiments of the present invention can be partially or wholly combined or combined with each other, and as can be fully understood by those skilled in the art, various technical connections and operations are possible, and each embodiment can be implemented independently of each other or can be implemented together in a related relationship.

[0052] For clarity in the interpretation of this specification, the terms used in this specification are defined below.

[0053] "Photography device" refers to an unmanned information terminal for taking pictures. Using the photography device, users can review captured images via a touchscreen and instantly print them after taking them using a remote control or timer. The photography device may also be referred to as a "photography kiosk," a "photo kiosk," or a "photography kiosk." While embodiments of the present disclosure are described herein as being performed by a photo taking server, they may also be performed by the photography device in some cases.

[0054] "Image" refers to a photograph(s) captured by the camera unit of a camera device. An image may capture one or more users, and a photographic output may include one or more images. Depending on the embodiment, "image" may refer to images captured on a photographic output, or may refer to a portion of a video cut.

[0055] "Advertised product" refers to a product for which an advertiser has requested promotion. Advertised products may include pharmaceuticals, workbooks, snacks, cosmetics, beverages, idol merchandise (e.g., photocards, cheering sticks, keychains), corporate logos, clothing, and other products. However, these are merely examples and are not intended to be limiting. Depending on the specific example, advertised products may include movies, alcohol, insurance, and other products.

[0056] "Photographic print" refers to a print produced from a photographic device. Photographic prints may also be referred to as photo seals or may take the form of stickers.

[0057] "Application" refers to a photography application that provides various functions to users using the self-photography service by linking with a server system (200) and / or a photographic device (300). For example, users can use the "Application" to search for stores nationwide, participate in events promoting advertised products, and receive rewards for doing so.

[0058] Hereinafter, a device and method for providing a photo shooting service including an artificial intelligence-based product advertising function according to the present invention will be described in detail with reference to the drawings.

[0059] FIG. 1a is a diagram illustrating a system that provides a photo shooting service including a product advertising function according to one embodiment of the present invention and a screen displayed on a user terminal.

[0060] Referring to FIG. 1A, a system (hereinafter, a self-photography system) (1000) providing a photo shooting service including a product advertisement function according to one embodiment of the present invention may include a user terminal (100), a user interface providing server (200) (hereinafter, a photo shooting server (200) including a product advertisement function), and a photo shooting device (300).

[0061] The user terminal (100) may be a device possessed by a user who wishes to take a self-photograph using the photographing device (300). The user terminal (100) may be a portable digital device equipped with a memory means and a microprocessor and having computing power, such as a smartphone, tablet PC, PC, or laptop, and is not limited to a specific form. In various embodiments, the user terminal (100) may download video material, such as the photographed image(s), and transmit it to a photographing server (200) that includes a product advertising function. In another embodiment, the user terminal (100) may capture image(s) and transmit it to a photographing server (200) that includes a product advertising function.

[0062] A photo shooting server (200) including a product advertising function can input image(s) taken by a user from a photographing device (300) and determine whether the image contains an advertising product through an advertising product recognition model. At this time, if the image contains an advertising product, the photo shooting server (200) including a product advertising function can provide a reward to the user terminal (100). According to an embodiment, the photo shooting server (200) can provide a reward to the user terminal (100) if the user automatically, semi-automatically, or manually uploads the image(s) taken by the user to a SNS or sets them as a profile of a chat app. For example, a user can upload the image(s) taken by the user to a SNS, capture the post uploaded to the SNS, and register it in an application of an operator who provides a photographing device and / or a photo shooting service according to an embodiment of the present disclosure. At this time, the photo shooting server (200) or the application can determine whether to post by recognizing the characteristics of the post uploaded to the SNS, the user's ID, tags, etc., using OCR or image recognition, etc. The photo capture server (200) can provide a reward to the user terminal (100) if the recognized user's uploaded post or profile contains an advertisement product. As the user determines the actual SNS, chat app, etc., and provides the reward, the operator can maximize the exposure of the advertisement product.

[0063] Here, the reward provided to the user terminal (100) may include points or secret frames, and the present disclosure is not limited thereto. For example, it may include a raffle ticket that can be used to draw in an event run by an operator, a raffle ticket that increases the probability of drawing in an event being held by an advertiser, etc. Meanwhile, the photo shooting server (200) including a product advertising function may receive and store images, information about the photographing device, user data, matching data, etc. The photo shooting server (200) including a product advertising function may provide the user with an identification code that allows access to the location where the photos taken by the photographing device (300) are stored. The identification code that allows access to the location where the photos are stored may include a barcode and / or a two-dimensional code. The identification code may be printed together with the photos on a photo printout. In addition, the photo shooting server (200) including a product advertising function may provide the image(s) to the user terminal (100) in response to the user terminal (100) requesting the captured image(s) upon identifying the identification code. A photo shooting server (200) including a product advertising function may include a device DB server, a CMS (content management system) DB server, a storage server, and a mobile DB server. A more detailed configuration of the photo shooting server (200) including a product advertising function is disclosed in detail through FIG. 2.

[0064] The photographing device (300) is configured to provide a self-photography service in which a user can take and print photos by himself / herself. The photographing device (300) can create resources (e.g., the photographed image(s)) after taking photos and upload them to a touch screen. The photographing device (300) can print photo outputs including the image(s) and an identification code. According to an embodiment, the photographing device (300) can provide the image(s) in digital file form to the photo taking server (200) and at the same time, use AI to determine whether the image contains an advertising product and provide the result to the user terminal (100). In addition, the photographing device (300) can provide the printed photo outputs and at the same time provide the photographed image(s) to the user in digital file form. A more detailed configuration of the photographing device (300) is disclosed in detail through FIGS. 3A and 3B .

[0065] FIG. 1b is a block diagram illustrating an exemplary configuration of a system providing a photo shooting service according to one embodiment of the present invention.

[0066] Referring to FIG. 1b, a photo shooting server (200) including a product advertisement function may include a device DB server (210), a CMS DB server (220), a storage server (230), and a mobile DB server (240) to manage and store each data. In FIG. 2, each server is illustrated as a separate configuration, but the present invention is not limited thereto, and each illustrated server does not necessarily need to be implemented as a separate configuration, and two or more servers may be implemented as one server or may be configured as modules that perform the corresponding functions within one server.

[0067] The device DB server (210) may be a server for communicating with the photographing device (300) and managing the photographing device (300). At this time, the device DB server (210) may receive and store device information of the photographing device (300) from the photographing device (300) and provide it to the photographing device manager. The device information of the photographing device (300) may include basic information of the photographing device (300) (e.g., ID, hardware information), information required for communication with the photographing device (300), information about the printer status (e.g., remaining paper, printer error), information about the software program operation status, and update information about the software program. In addition, the device DB server (210) may manage the operation status of the photographing device (300) in real time (e.g., normal operation, idle state, error occurrence, etc.). According to an embodiment, the device DB server (210) may use AI to determine whether an image includes an advertising product.

[0068] The CMS DB server (220) may be a server for storing user data and / or storage information (e.g., URL). A CMS (content management system) refers to a system that allows a camera device manager to intuitively manage content. User data may include data on captured image(s), data on whether an image contains an advertisement product determined through an advertisement product recognition model, data on a frame selected by the user, payment information data, matching data that matches resources and member information, etc. The CMS DB server (220) may receive and store image(s) captured by the camera device (300) and provide an API for uploading the image(s) to the storage server (230).

[0069] The storage server (230) may be a server for storing image(s) captured by a photographing device (300) by communicating with a CMS DB server (220) and / or a mobile DB server (240). The storage server (230) may provide the captured image(s) of the corresponding user to the user terminal (100) among the stored resources (e.g., captured image(s)) based on matching data stored in the CMS DB server (220).

[0070] The mobile DB server (240) may be a server for storing member information and / or file address information. For example, the mobile DB server (240) may receive an image video request from a user terminal (100) and transmit member information stored in the CMS DB server (220). The mobile DB server (240) may store file address information that allows a user to directly download the captured image(s) files from an application or storage server (230) without storing the captured image(s) files.

[0071] The photographing device (300) illustrated in FIG. 1B, the photo shooting server (200) including a product advertising function, the individual servers (210, 220, 230, 240) constituting the same, and the user terminal (100) may operate to exchange data with each other via a communication network. The communication network according to one embodiment of the present disclosure may be configured regardless of the communication mode such as wired communication or wireless communication, and may be configured with various communication networks such as a local area network (LAN), a metropolitan area network (MAN), and a wide area network (WAN). Preferably, the communication network referred to herein may be the well-known Internet or the World Wide Web (WWW). However, the communication network is not necessarily limited thereto, and may include at least a portion of a well-known wired and wireless data communication network, a well-known telephone network, or a well-known wired and wireless television communication network. For example, the communication network may be a wireless data communication network that implements, at least in part, a conventional communication method such as WiFi communication, WiFi-Direct communication, Long Term Evolution (LTE) communication, 5G communication, Bluetooth communication (including Bluetooth Low Energy (BLE) communication), infrared communication, or ultrasonic communication. As another example, the communication network may be an optical communication network that implements, at least in part, a conventional communication method such as LiFi (Light Fidelity).

[0072] Figure 2 is a block diagram showing the configuration of a user terminal according to one embodiment of the present invention.

[0073] Referring to FIG. 2, the user terminal (100) may include a memory interface (110), one or more processors (120), and a peripheral interface (130). Various components within the user terminal (100) may be connected by one or more communication buses or signal lines.

[0074] The memory interface (110) is connected to the memory (150) and can transmit various data to the processor (120). Here, the memory (150) can include at least one type of storage medium among flash memory type, hard disk type, multimedia card micro type, card type memory (e.g., SD or XD memory, etc.), RAM, SRAM, ROM, EEPROM, PROM, network storage, cloud, and blockchain database.

[0075] In various embodiments, the memory (150) may store at least one of an operating system (151), a communication module (152), a graphical user interface (GUI) module (153), a sensor processing module (154), a telephone module (155), and an application module (156). Specifically, the operating system (151) may include instructions for processing basic system services and instructions for performing hardware operations. The communication module (152) may communicate with at least one of one or more other devices, computers, and servers. The graphical user interface (GUI) module (153) may process a graphical user interface. The sensor processing module (154) may process sensor-related functions (e.g., processing voice input received using one or more microphones (192). The telephone module (155) may process telephone-related functions. The application module (156) may perform various functions of a user application, such as electronic messaging, web browsing, media processing, navigation, imaging, and other processing functions. In addition, the user terminal (100) can store one or more software applications (156-1, 156-2) associated with a certain type of service in the memory (150). At this time, the application (156-1) refers to a photo application that provides various functions to a user using a self-photography service on the user terminal (100). Here, the user can use the application (156-1) to search for a store that provides a self-photography service, check an event promoting an advertised product, upload an image containing an advertised product taken by the user, and receive a reward for the same.

[0076] In various embodiments, the memory (150) may store a digital assistant client module (157) (hereinafter, DA client module), and accordingly, may store commands for performing client-side functions of the digital assistant and various user data (158) (e.g., user data, information on events in which the user participated, information on winning events, etc.).

[0077] Meanwhile, the DA client module (157) can obtain the user's voice input, text input, touch input, and / or gesture input through various user interfaces (e.g., I / O subsystem (140)) provided in the user terminal (100).

[0078] Additionally, the DA client module (157) can output data in audiovisual and tactile forms. For example, the DA client module (157) can output data consisting of a combination of at least two or more of voice, sound, notification, text message, menu, graphic, video, animation, and vibration. In addition, the DA client module (157) can communicate with a digital assistant server (not shown) using a communication subsystem (180).

[0079] In various embodiments, the DA client module (157) may collect additional information about the surroundings of the user terminal (100) from various sensors, subsystems, and peripheral devices to construct a context associated with the user input. For example, the DA client module (157) may provide context information along with the user input to a digital assistant server to infer the user's intent. Here, context information that may accompany the user input may include sensor information, such as lighting, ambient noise, ambient temperature, images of the surrounding environment, videos, etc. As another example, the context information may include the physical state of the user terminal (100) (e.g., device orientation, device position, device temperature, power level, speed, acceleration, motion patterns, cellular signal strength, etc.). As yet another example, the context information may include information related to the software state of the user terminal (100) (e.g., processes running on the user terminal (100), installed programs, past and present network activity, background services, error logs, resource usage, etc.).

[0080] In various embodiments, the memory (150) may include added or deleted instructions, and further, the user terminal (100) may include additional configurations other than the configuration illustrated in FIG. 2, or may exclude some configurations.

[0081] The processor (120) can control the overall operation of the user terminal (100) and can execute various commands to implement a photo shooting service interface including an artificial intelligence-based product advertising function using an application or program stored in the memory (150).

[0082] The processor (120) may correspond to a computing device such as a CPU (Central Processing Unit) or an AP (Application Processor). In addition, the processor (120) may be implemented in the form of an integrated chip (IC), such as a SoC (System on Chip) that integrates various computing devices such as an NPU (Neural Processing Unit).

[0083] The peripheral interface (130) can be connected to various sensors, subsystems, and peripheral devices to provide data so that the user terminal (100) can perform various functions. Here, the function performed by the user terminal (100) can be understood as being performed by the processor (120).

[0084] The peripheral interface (130) can receive data from a motion sensor (160), a light sensor (light sensor) (161), and a proximity sensor (162), through which the user terminal (100) can perform orientation, light, and proximity detection functions, etc. For another example, the peripheral interface (130) can receive data from other sensors (163) (positioning system - GPS receiver, temperature sensor, biometric sensor), through which the user terminal (100) can perform functions related to the other sensors (163).

[0085] In various embodiments, the user terminal (100) may include a camera subsystem (170) connected to a peripheral interface (130) and an optical sensor (171) connected thereto, through which the user terminal (100) may perform various photographing functions such as taking pictures and recording video clips.

[0086] In various embodiments, the user terminal (100) may include a communication subsystem (180) connected to a peripheral interface (130). The communication subsystem (180) may be comprised of one or more wired / wireless networks and may include various communication ports, radio frequency transceivers, and optical transceivers.

[0087] In various embodiments, the user terminal (100) includes an audio subsystem (190) connected to a peripheral interface (130), and the audio subsystem (190) includes one or more speakers (191) and one or more microphones (192), thereby enabling the user terminal (100) to perform voice-activated functions, such as voice recognition, voice replication, digital recording, and telephone functions.

[0088] In various embodiments, the user terminal (100) may include an I / O subsystem (140) connected to a peripheral interface (130). For example, the I / O subsystem (140) may control a touch screen (143) included in the user terminal (100) via a touch screen controller (141). As an example, the touch screen controller (141) may detect a user's contact and movement or cessation of contact and movement using any one of a plurality of touch sensing technologies, such as capacitive, resistive, infrared, surface acoustic wave, or proximity sensor array. As another example, the I / O subsystem (140) may control other input / control devices (144) included in the user terminal (100) via other input controller(s) (142). As an example, the other input controller(s) (142) may control one or more buttons, rocker switches, thumb-wheels, infrared ports, USB ports, and pointer devices such as a stylus.

[0089] FIG. 3 is a block diagram illustrating an exemplary hardware configuration in which a server system or each server included in the server system according to one embodiment of the present invention can be implemented.

[0090] Referring to FIG. 3, a photo shooting server (200) including a product advertising function may include a communication interface (210), a memory (220), an I / O interface (230), and a processor (240), and each component may communicate with each other through one or more communication buses or signal lines. Referring to FIGS. 1A and 1B, a photo shooting service providing server (200) may be a photo shooting server (200) including a product advertising function.

[0091] The communication interface (210) can be connected to a photographing device and / or a user terminal via a wired / wireless communication network to exchange data.

[0092] For example, the communication interface (210) can receive user information, payment information, image(s), camera device status information, etc. from the camera device. In addition, the communication interface (210) can receive an image request from the user terminal. In another example, the communication interface (210) can transmit control information for controlling the camera device to the camera device. The communication interface (210) can transmit an image to the user terminal. In addition, the communication interface (210) can transmit to the user terminal whether the image contains an advertisement product, determined through an advertisement product recognition model. In addition, the communication interface (210) can provide a reward to the user if the image contains an advertisement product. In this case, the reward can include points or a secret frame.

[0093] Meanwhile, the communication interface (210) that enables transmission and reception of such data includes a wired communication port (211) and a wireless circuit (212), wherein the wired communication port (211) may include one or more wired interfaces, for example, Ethernet, Universal Serial Bus (USB), FireWire, etc. In addition, the wireless circuit (212) may transmit and receive data with an external device via an RF signal or an optical signal. In addition, the wireless communication may use at least one of a plurality of communication standards, protocols, and technologies, for example, GSM, EDGE, CDMA, TDMA, Bluetooth, Wi-Fi, VoIP, Wi-MAX, or any other suitable communication protocol.

[0094] The memory (220) can store various data used in the self-photography service. For example, the memory (220) can store device information of the photographing device, user data, storage information, captured image(s), member information, output values ​​from an advertising product recognition model, and / or file address information.

[0095] In various embodiments, the memory (220) may include a volatile or non-volatile storage medium capable of storing various data, commands, and information. For example, the memory (220) may include at least one type of storage medium among a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., an SD or XD memory, etc.), RAM, SRAM, ROM, EEPROM, PROM, network storage, cloud, and a blockchain database.

[0096] In various embodiments, the memory (220) may store configurations of at least one of an operating system (221), a communication module (222), a user interface module (223), and one or more applications (224).

[0097] An operating system (221) (e.g., embedded operating systems such as LINUX, UNIX, MAC OS, WINDOWS, VxWorks, etc.) may include various software components and drivers to control and manage general system operations (e.g., memory management, storage device control, power management, etc.) and may support communication between various hardware, firmware, and software components.

[0098] The communication module (222) can support communication with other devices through the communication interface (210). The communication module (222) can include various software components for processing data received by the wired communication port (211) or wireless circuit (212) of the communication interface (210).

[0099] The user interface module (223) can receive a user's request or input from a keyboard, touch screen, keyboard, mouse, microphone, etc. through an I / O interface (230) and provide a user interface on the display.

[0100] The application (224) may include a program or module configured to be executed by one or more processors (240). Here, the application for the photo shooting service may be implemented on a server farm.

[0101] The I / O interface (230) can connect at least one of an input / output device (not shown) of the photo shooting service providing server (200), such as a display, a keyboard, a touch screen, and a microphone, to the user interface module (223). The I / O interface (230) can receive user input (e.g., voice input, keyboard input, touch input, etc.) together with the user interface module (223) and process a command according to the received input.

[0102] The processor (240) is connected to a communication interface (210), a memory (220), and an I / O interface (230) to control the overall operation of the photo shooting service providing server (200), and can perform various commands to download images captured through an application or program stored in the memory (220) or time-lapse videos corresponding thereto from a user terminal.

[0103] The processor (240) may correspond to a computing device such as a Central Processing Unit (CPU) or an Application Processor (AP). Furthermore, the processor (240) may be implemented in the form of an integrated chip (IC), such as a System on Chip (SoC) in which various computing devices are integrated. Alternatively, the processor (240) may include a module for calculating an artificial neural network model, such as a Neural Processing Unit (NPU).

[0104] FIG. 3 is only one embodiment regarding the configuration of the server (200), and the server (200) may have a configuration or arrangement in which some of the components illustrated in FIG. 3 are omitted, additional components not illustrated in FIG. 3 are further provided, or two or more components are combined. Components that may be included in the server (200) may be implemented as hardware, software, or a combination of both hardware and software, including one or more signal processing or application-specific integrated circuits.

[0105] FIGS. 4A and 4B are block diagrams illustrating an exemplary configuration of a photographing device that provides a photographing service according to one embodiment of the present invention.

[0106] Referring to FIG. 4A, a photographing device (300) according to one embodiment of the present invention may include a display unit (301), an interface unit (302), a power supply unit (303), a communication unit (304), a payment unit (305), a control unit (310), a photographing unit (321), an output unit (322), a sensor unit (323), a lighting unit (324), etc.

[0107] The display unit (301) can display various information related to the operation of the photographing device (300). For example, the display unit (301) can display menus related to photographing services, the appearance of the subject being photographed, etc. In addition, the display unit (301) can display UI (User Interface) and GUI (Graphical User Interface) information according to the displayed image(s). The specific description of the information displayed by the display unit (301) described above is only an example, and the present invention is not limited thereto.

[0108] The display unit (301) may include at least one of a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode (OLED), a flexible display, a 3D display, an e-ink display, and a light-emitting diode (LED).

[0109] The interface unit (302) is a component for receiving user input required for photographing services such as shooting and printing from the user, and may be implemented as a physical key pad, dome switch, touch pad, stick, lever, etc., but is not limited thereto. In one embodiment of the present invention, the display unit (301) may be implemented as a touch screen by forming a mutual layer structure with a touch sensor or forming it as an integral part. Such a touch screen may function as the display unit (301) of the photographing device (300) and also as the interface unit (302).

[0110] The power supply (303) may supply power to all or part of the components of the photographing device (300). For example, the power supply (303) may include a power management system, one or more power sources such as a battery or alternating current (AC), a charging system, a power failure detection circuit, a power converter or inverter, a power status indicator, or any other components for power generation, management, and distribution.

[0111] The communication unit (305) enables the photographing device (300) to transmit and receive data with a server system or external devices via a communication network. The photographing device (300) can bidirectionally transmit and receive data regarding the captured image(s), user information, links to storage locations where the image(s) data is stored, etc., with the server system via the communication unit (305).

[0112] The payment unit (305) allows users to pay for services provided through a photographing device (300) using a card, cash, mobile pay, virtual currency, coupon, barcode, 2D barcode, etc.

[0113] The control unit (310) can perform overall control of the photographing device (300). In one embodiment of the present invention, when a photographing condition is selected by a user through the interface unit (302), payment is processed by the payment unit (305), and then a photographing command is input again through the interface unit (302), the control unit (310) can control the photographing unit (321) to photograph a subject. The control unit (310) can receive the photographed image(s) and transmit them to a server system through the communication unit (305). The control unit (310) can receive a download web address link for accessing the corresponding image(s) from the server system and control the output unit (322) to issue a two-dimensional code based on the download web address link.

[0114] In one embodiment of the present invention, the control unit (310) may include a processor (311) and a memory (312).

[0115] The control unit (310) may include one or more processors (311). The one or more processors (311) included in the control unit (310) may be integrated into a single chip or may be physically separated. In addition, the processor (311) and the memory (312) may be implemented as a single chip. The memory (312) may include not only volatile memory such as S-RAM and D-RAM, but also non-volatile memory such as flash memory, ROM (Read Only Memory), and EPROM (Erasable Programmable Read Only Memory).

[0116] The shooting unit (321) can shoot a subject located in the shooting space of the shooting device (300) according to the control signal of the control unit (310).

[0117] The output unit (322) can print photographic prints containing a two-dimensional code that can access the photographed image(s) and the image(s) stored in the server system. In one embodiment of the present invention, the photographing unit (321) and the output unit (322) may each include a camera and a printer.

[0118] The sensor unit (323) can detect whether a user is positioned in front of the photographing device (300) or the movement of a user positioned in front of the photographing device (300).

[0119] The lighting unit (324) may include lighting(s) required for the shooting unit (321) to perform the shooting operation.

[0120] Referring to FIG. 4b, a photographing device (300) according to one embodiment of the present invention may include a PC (330), a camera (331), a printer (332), a user control unit (341), an I / O board (342), a card payment machine (343), a banknote recognition machine (344), etc., and the components included in the photographing device (300) may exchange various information with each other.

[0121] The camera (331) can provide the captured image(s) to the PC (330). In addition, the camera can receive a camera on / off control signal from the I / O board (342).

[0122] The printer (332) can receive output control information from the PC (330) and transmit printer status information to the PC (330).

[0123] The I / O board (342) can receive control information(s) transmitted from the PC (330) to each component or transmitted from each component to the PC (330). I

[0124] The user control unit (341) may include a touch monitor (30) and / or a shooting remote control (31). The touch monitor (30) may transmit touch coordinate information input by the user to the PC (330). The shooting remote control (31) may transmit button press information to the PC (330) via the I / O board (342).

[0125] If the user selects card payment, the card payment machine (342) can transmit card information (e.g., card number) of the card entered by the user to the PC (330). If the user selects cash payment, the banknote recognition machine (344) can transmit banknote insertion information of the banknote inserted by the user to the PC (330) via the I / O board (342).

[0126] The PC (330) can perform overall control of the components included in the photographing device (300). The PC (330) can transmit user information, payment information, images, photographing device status information, etc. collected from various components to an external server (400). Here, the PC (330) can receive the results of an advertisement product recognition model of the image from the external server (400). Meanwhile, referring to FIGS. 1 and 2, the external server (400) may be a photo shooting server (200) including a product advertisement function.

[0127] Figure 5 is a conceptual diagram schematically illustrating a processor of a system providing a conventional photo shooting service.

[0128] According to a conventional photo shooting service, an advertiser (510) requests an advertisement for an advertised product (e.g., Product A) from a self-photography device operator (520). At this time, the self-photography device operator (520) can create a shooting frame suitable for the advertised product. As another example, the self-photography device operator (520) can create an advertising video to be provided while the self-photography is being printed after the advertiser (510) takes a self-photo. The self-photography device operator (520) can provide a self-photography frame or advertising video suitable for the advertised product to a direct store or franchise store (530).

[0129] Users can take self-photos using a photography device (550) installed in a direct store or franchise store (530). At this time, the photography device (550) can provide the user with a photo frame containing an advertising product and print a photo printout (540) that the user has taken and requested to be printed. As another example, the photography device (530) can provide an advertising video received from a conventional photo shooting service during the printing time of the taken photo (540).

[0130] Therefore, conventional photography services can provide indirect advertising by providing users with a photo frame related to the advertised product. Furthermore, conventional photography services can provide advertising effects for the advertised product by providing video of the advertised product while the customer's photo is being developed.

[0131] FIG. 6 is an exemplary diagram showing an interface screen of a user terminal that provides a photo shooting service including a product advertising function according to one embodiment of the present invention.

[0132] An application that provides a photo-taking service that includes a product advertising function may provide user interfaces (610, 620, 630) for users to participate in an event promoting the advertised product. For example, a user may upload an event photo by clicking "Event Photo" on the first user interface screen (610). The event photo may be received from a photographing device. As another example, the event photo (620) may be a photo taken from a user terminal.

[0133] When a user uploads an event photo, a user interface (620) displaying the event photo including a product advertisement function may be provided. The event photo may include an image (621) including an advertisement product. At this time, a photo shooting server including a product advertisement function according to an embodiment of the present disclosure may input the image (621) including the advertisement product as an input value of an advertisement product recognition model. Here, the photo shooting server including a product advertisement function may determine whether the event photo includes an image (621) including an advertisement product using the advertisement product recognition model. At this time, the photo shooting server including a product advertisement function may provide a reward to the user terminal if the event photo includes an image (621) including an advertisement product. If the event photo includes an image (621) including an advertisement product, a user interface (630) displaying 1,000 points provided as a reward may be provided.

[0134] A more detailed description of the photo shooting service including the product advertising function will be described in the following embodiments.

[0135]

[0136] FIG. 7 is a flowchart schematically illustrating a process for a photographing device of a system that provides a photo shooting service including a product advertising function according to one embodiment of the present invention.

[0137] In S701, when a user completes taking a picture using the photographing device (710), the photographing device (710) can upload the user ID (UID) and the captured image and / or video to the CMS. At this time, the photographing device (710) can transmit the user ID (UID) and the captured image and / or video to the CMS DB server (722). The CMS DB server (722) can store the user ID (UID) and the image and / or video received from the photographing device (710).

[0138] In S702, the CMS DB server (722) can transmit a user ID (UID) and an image and / or video file to the storage server (723). When the API (or API server) provided by the CMS DB server (722) receives the user ID (UID) and an image and / or video file, it can upload the file to the storage server (723).

[0139] In S703, the storage server (723) may, in response to receiving a user ID (UID) and an image and / or video, transmit a URL (or file address) to which an image and / or video file is uploaded to the CMS DB server (722).

[0140] In S704, the CMS DB server (722) can match the user ID (UID) with the URL (or file address) where the image and / or video file is uploaded.

[0141] In S705, the CMS DB server (722) can transmit a URL matching the UID to the shooting device (710).

[0142] In S706, the photographing device (710) can generate a QR code containing UID information.

[0143] In S707, the user terminal (730) can use AI to determine whether the received image and / or video file contains an advertising product. In this case, whether an advertising product is included can be determined using AI configured with an advertising product recognition model.

[0144] In S708, the user terminal (730) can upload the parsed UID, member information, and AI detection content to the CMS DB server (722).

[0145] In S709, the CMS DB server (722) can upload the parsed UID, member information, and AI detection content to the storage server (723).

[0146] In S710, the photographing device (710) can output a QR code and / or a photo to a download page including a UID.

[0147] In S711, the user terminal (730) can parse the UID through the QR code provided by the photographing device (710).

[0148] In S712, the user terminal (730) can transmit the parsed UID and member information logged in from the user terminal (730) to the CMS DB server (722).

[0149] In S713, the CMS DB server (722) can match the received UID with member information.

[0150] In S714, the CMS DB server (722) can transmit the updated UID, member information, URL, and AI detection content to the mobile DB server (724).

[0151] In S715, the shooting device (710) can use AI to determine whether an advertisement product is included and, when the advertisement product is not included in the image, provide the user with an opportunity to reshoot.

[0152] In S716, the user terminal (730) can transmit a resource view and image download request to the mobile DB server (724).

[0153] In S717, the user terminal (730) can transmit a file address to the mobile DB server (724).

[0154] In S718, the storage server (723) can provide a download for an image file to the user terminal (730).

[0155] FIG. 8 is a flowchart schematically illustrating a mobile process of a system that provides a photo shooting service including a product advertising function according to another embodiment of the present invention.

[0156] In S801, when a user completes taking a picture using the photographing device (810), the photographing device (810) can upload the user ID (UID) and the captured image and / or video to the CMS. At this time, the photographing device (810) can transmit the user ID (UID) and the captured image and / or video to the CMS DB server (822). The CMS DB server (822) can store the user ID (UID) and the image and / or video received from the photographing device (810).

[0157] In S802, the CMS DB server (822) can transmit a user ID (UID) and an image and / or video file to the storage server (823). When the API (or API server) provided by the CMS DB server (822) receives the user ID (UID) and an image and / or video file, it can upload the file to the storage server (823).

[0158] In S803, the storage server (823) may, in response to receiving a user ID (UID) and an image and / or video, transmit a URL (or file address) to which an image and / or video file is uploaded to the CMS DB server (822).

[0159] In S804, the CMS DB server (822) can match the user ID (UID) with the URL (or file address) where the image and / or video file is uploaded.

[0160] In S805, the CMS DB server (822) can transmit a URL matching the UID to the shooting device (810).

[0161] In S806, the photographing device (810) can generate a QR code containing UID information.

[0162] In S807, the user terminal (830) can upload the parsed UID and AI type, etc. to the CMS DB server (822).

[0163] In S808, the CMS DB server (822) can transmit the parsed UID and AI detection contents, etc. to the storage server (823).

[0164] In S809, the photographing device (810) can output a QR code and / or a photo to a download page including a UID.

[0165] In S810, the user terminal (830) can parse the UID through the QR code provided by the photographing device (810).

[0166] In S811, the user terminal (830) can transmit the parsed UID and member information logged in from the user terminal (730) to the CMS DB server (822).

[0167] In S812, the CMS DB server (822) can match the received UID with member information.

[0168] In S813, the CMS DB server (822) can transmit the updated UID, member information, and URL to the mobile DB server (824).

[0169] In S814, the user terminal (830) can transmit a resource view and image download request to the mobile DB server (824).

[0170] In S815, the user terminal (830) can transmit a file address to the mobile DB server (824).

[0171] In S816, the user terminal (830) can use AI to determine whether the received image and / or video file contains an advertising product. At this time, whether an advertising product is included can be determined using AI configured with an advertising product recognition model.

[0172] In S817, the storage server (823) can provide a download for an image file to the user terminal (830).

[0173] In S818, the user terminal (830) may request the CMS DB server (822) to issue a secret frame coupon when an advertisement product is included in the image. For example, if the user terminal (830) determines whether an advertisement product is included in the image using AI and determines that an advertisement product is included in the image, the user terminal (830) may request the CMS DB server (822) to issue a secret frame coupon. Meanwhile, if the user terminal (830) determines that the AI ​​detection did not detect an advertisement product, the process may be terminated.

[0174] In S819, when a secret frame code and / or a random number is provided to a user terminal (830) (or user account), the user terminal (830) can receive a secret frame coupon provided by the CMS DB server (822).

[0175] When a secret frame coupon is given to a user, or when a user uses a secret frame coupon, a secret frame code and / or a random number is entered into the kiosk, and the user can take a photo using the secret frame within a preset time.

[0176] FIG. 9 is a flowchart illustrating an operation method of a system providing a photo shooting service including a product advertising function according to one embodiment of the present invention, and FIGS. 10 to 14 are exemplary diagrams showing an interface screen of a user terminal providing a photo shooting service including a product advertising function according to one embodiment of the present invention.

[0177] First, referring to FIG. 9, a photo server (200) including a product advertising function receives an image (S910). At this time, the image may be a four-cut photo including a cut that includes an advertised product. Here, the photo server (200) including a product advertising function may receive the image from a photographing device or a user terminal. Here, the image may be a four-cut photo taken by a photographing device. As another example, the image may be a photo taken by a user terminal. At this time, the photo server (200) including a product advertising function may provide an image shooting frame to the user terminal.

[0178] Meanwhile, a photo-taking server (200) including a product advertising function may receive data about a matched user from a photographing device. For example, when the photo-taking server (200) including a product advertising function receives an image from a photographing device, it may also receive data indicating that the image has been received from the user.

[0179] More specifically, referring to FIG. 10, a photo capture server (200) including a product advertisement function may receive an event photo upload request due to interaction with an event photo area (1010). At this time, the photo capture server (200) including a product advertisement function may provide a user interface for uploading event photos to a user terminal. Here, it can be confirmed that the mileage (1020) of the user currently logged into the user terminal is 0 points.

[0180] Referring to FIG. 11A, the user interface of the user terminal may display an image (1110) captured by the user terminal and uploaded to a server by the user. As another example, the image (1110) displayed on the user terminal may be an image captured by a photographing device and received from the photographing device. Here, the user terminal may receive the captured image through a QR code provided by the photographing device. Additionally, the user terminal may log in to the photographing device as the same user and receive the captured image through the server. In this case, the method by which the user terminal receives the image from the photographing device is not limited to the above case and may be configured in various ways.

[0181] In another embodiment, referring to FIG. 11b, the user interface of the photographing device may display a screen asking the user whether to upload an image (1110) captured by the photographing device to the photographing server (200) for event participation. If the user selects "Yes" (1120), the photographing server (200), which includes a product advertising function, may receive a request to upload an event photo. Additionally, the user may log into the photographing device with an account for which they wish to receive a reward.

[0182] In another embodiment, a photo-taking server (200) including a product advertisement function may provide help for taking images including an advertisement product to a photographing device. At this time, the help for taking images including an advertisement product may include shooting help for the product exposure range, the location of the advertisement product within the image, and the exposure range of characters or product names included in the advertisement product. Here, the product exposure range may be the ratio of the area exposed in the image to the total area of ​​the image corresponding to the advertisement product. In addition, the location or size of the advertisement product within the image may be the location or size of the advertisement product within the image. At this time, the exposure condition for characters or product names included in the advertisement product may be a condition for characters or product names that must be included among characters or product names included in the advertisement product. Meanwhile, help for taking images including an advertisement product may be provided to improve the accuracy of the advertisement product recognition model of the photo-taking server (200) including a product advertisement function or to enhance the promotional effect of the advertisement product.

[0183] For example, a photo shooting server (200) including a product advertisement function may provide help indicating that 70% of the area of ​​an image of an advertised product must be exposed within the image. In addition, a photo shooting server (200) including a product advertisement function may provide help indicating that a product must not be located at the outermost part but must be located in the center. As another example, a photo shooting help indicating that the closer the advertised product is to the center of the image and the larger the product is in the image, the more rewards are provided may be provided. In addition, a photo shooting server (200) including a product advertisement function may provide help indicating that if the product name is '허개수', up to '허개' must be exposed. Meanwhile, help regarding taking images including an advertised product of a photo shooting server (200) including a product advertisement function may be configured without being limited to the examples above.

[0184] Returning to Figure 9, the photo capture server (200) including the product advertising function uses an advertising product recognition model to determine whether an image contains an advertising product (S920). The advertising product recognition model may be an AI model configured to determine whether an image contains an advertising product. The advertising product recognition model may be comprised of an object recognition model and a character recognition model.

[0185] More specifically, referring to FIG. 12, a photo-taking server (200) including a product advertisement function can determine whether an image contains an advertisement product by inputting an image through an advertisement product recognition model. At this time, the photo-taking server (200) including a product advertisement function can recognize an advertisement product (1210) through an object recognition model of the advertisement product recognition model. Here, the photo-taking server (200) including a product advertisement function can recognize characters or product names (1220) included in an advertisement product through a character recognition model of the advertisement product recognition model.

[0186] At this time, the object recognition model can be composed of one of the object detection models. For example, the object recognition model can be composed of a YOLO model. At this time, the object recognition model can include a loss function. Here, the object recognition model is an object detection model without a mainstream anchor, and the loss function can include classification loss, box regression loss, object loss, etc. Meanwhile, the object recognition model can be composed of an artificial neural network model, a convolutional neural network (CNN), etc. As another example, the object recognition model can be machine learned according to a method such as supervised learning, unsupervised learning, or reinforcement learning. At this time, machine learning algorithms for performing machine learning may include decision trees, Bayesian networks, support vector machines, artificial neural networks, Ada-boost, perceptrons, genetic programming, and clustering. Meanwhile, character recognition models may be composed of optical character recognition (OCR), etc.

[0187] Returning to Figure 9, the photo capture server (200) including a product advertising function provides a reward to the user terminal based on whether the image includes an advertised product (S930). At this time, the photo capture server (200) including a product advertising function may provide a reward to the user terminal if it determines that the image includes an advertised product. Here, the reward may be points provided to a user logged into the user terminal. Meanwhile, the photo capture server (200) including a product advertising function may request a re-capture to the user terminal and / or the photographing device if it determines that the image does not include an advertised product.

[0188] As another example, a photo-taking server (200) with a product advertising function may provide a reward to a user as a secret frame. The secret frame may be a self-photographed photo frame created by a self-photographing device operator upon request from an advertiser. For example, the secret frame may include a frame with a different color, a frame with a different pose, a frame with a different person, etc.

[0189] Here, the photo-taking server (200) including a product advertising function may provide a secret frame code and / or a random number if it determines that the image contains an advertised product. At this time, the user may be provided with a secret frame coupon or a photo-taking ticket that allows immediate photography. Meanwhile, the photo-taking server (200) including a product advertising function may terminate the photo-taking process if it determines that the image does not contain an advertised product.

[0190] At this time, if a secret frame coupon is given to the user, or if the user uses a secret frame coupon, a secret frame code and / or a random number is entered into the kiosk, and the user can take a photo using the secret frame.

[0191] For example, a photo shooting server (200) including a product advertisement function can provide a secret frame coupon for idol A as a reward to a user when the advertisement product recognized through the advertisement product recognition model is a cheering stick of idol A.

[0192] Meanwhile, the photo shooting server (200) including the product advertising function can provide the user with a change right in the form of a free coupon or the like to change the frame of the taken self-photo to one of the existing frames and / or a secret frame when the advertisement product is recognized through the advertisement product recognition model.

[0193] In addition, the photo-taking server (200) including a product advertisement function may provide the user with an additional photo-taking right to take additional self-photos with a secret frame for a fee in the form of a paid coupon, etc., when the advertisement product is recognized through the advertisement product recognition model. As another example, the photo-taking server (200) including a product advertisement function may provide rewards to the user terminal in a differentiated manner. According to an embodiment, the photographing device (300) may provide rewards in a differentiated manner. At this time, the photo-taking server (200) including a product advertisement function may provide rewards in a differentiated manner based on the number of advertisement products, the number of images including the advertisement products, the area ratio occupied by the advertisement products in the image, etc. At this time, the number of advertisement products may be the number of advertisement products included in one image. For example, if there are three advertisement products in one image, the number of advertisement products may be three. Here, the number of images including advertisement products may be the number of images including advertisement products among four images. For example, if three of the four images contain an advertisement product, the number of images containing the advertisement product can be three. Furthermore, the area ratio of the advertisement product in the image can be the percentage of the total area of ​​the image that the advertisement product occupies. For example, if the advertisement product occupies 30% of the image, the area ratio of the advertisement product in the image can be 30%.

[0194] More specifically, referring to FIG. 13, a photo capture server (200) including a product advertising function can provide a reward to a user terminal if an advertised product is included in the image. At this time, the user terminal can confirm that the user's mileage (1310) has increased to 500 points due to the provided reward.

[0195] In another embodiment, a photo capture server (200) including a product advertising function may provide a user with a background image suitable for the advertised product. In other words, the photo capture server (200) including a product advertising function may provide a recommended background image to the user based on an identifier for the advertised product. The identifier for the advertised product may include information about the characteristics of the advertised product, including its type, color, and target customer base.

[0196] More specifically, referring to FIG. 14A, a photo-taking server (200) including a product advertisement function may provide a user interface (1410) to a photographing device for a user to select from among multiple advertisement products. The photo-taking server (200) including a product advertisement function may receive an advertisement product selected by the user from the photographing device. At this time,

[0197] Referring to FIG. 14b, a photo capture server (200) including a product advertising function can provide a basic background determined through an identifier for an advertised product. At this time, the photo capture server (200) including a product advertising function can provide a basic background (1420) determined based on the type, color, and customer base of the advertised product. Here, the photo capture server (200) including a product advertising function can receive the basic background from the advertiser. Meanwhile, the photo capture server (200) including a product advertising function can provide an image in which the background of a photograph taken is changed to a basic background.

[0198] Referring to FIG. 14c, the photo-taking server (200) including a product advertising function may provide a user interface for selecting from among a plurality of backgrounds determined based on an identifier for the advertised product. At this time, the photo-taking server (200) including a product advertising function may include a background (1430) determined based on the color of the advertised product, a background (1440) determined based on the atmosphere of the advertised product, and a background (1450) determined based on the type of the advertised product. For example, if the advertised product is a naturalistic toner, the photo-taking server (200) including a product advertising function may provide as options a white background determined to be a color that matches the toner, a background including leaves that match the naturalistic style, a background including water droplets according to the type of toner, etc. Meanwhile, the photo-taking server (200) including a product advertising function may provide an image in which the background of a taken photo is changed to each of a plurality of backgrounds.

[0199] According to an embodiment, the secret frame may be provided as a random frame. The random target may be a member of an idol group, an artist's costume, etc., and the random frame may be set in various ways. Referring to FIG. 14D, a member of an idol group may be randomly selected and provided as a frame. In this case, the secret frame may be provided in the form of a hidden frame (1460) that only shows the silhouette of the randomly selected member and cannot identify the member. If the user succeeds in the mission (e.g., if the user takes a picture using an idol group's cheering stick), the user may select a member and print a photo print using a photo taken with the selected member. On the other hand, if the user fails the mission, the user may print a photo print using a photo taken with a randomly selected member of an idol group. The program executed by the terminal and server described in the present disclosure may be implemented as hardware components, software components, and / or a combination of hardware components and software components. The program may be executed by any system capable of executing computer-readable instructions.

[0200] Software may include a computer program, code, instructions, or a combination of one or more of these, and may configure a processing device to perform a desired operation or command the processing device, either independently or collectively. Software may be implemented as a computer program including instructions stored on a computer-readable storage medium. Examples of the computer-readable storage medium include magnetic storage media (e.g., read-only memory (ROM), random-access memory (RAM), floppy disks, hard disks, etc.) and optical reading media (e.g., CD-ROMs, digital versatile discs (DVDs)). The computer-readable storage medium may be distributed across network-connected computer systems so that the computer-readable code is stored and executed in a distributed manner. The medium may be readable by a computer, stored in a memory, and executed by a processor.

[0201] Computer-readable storage media may be provided in the form of non-transitory storage media. Here, "non-transitory" simply means that the storage medium does not contain signals and is tangible, but does not distinguish between whether data is stored semi-permanently or temporarily on the storage medium.

[0202] Additionally, a program according to embodiments of the present disclosure may be provided as a computer program product. The computer program may be traded as a product between a seller and a buyer. The computer program may include a software program and a computer-readable storage medium storing the software program. For example, the computer program may include a product in the form of a software program (e.g., a downloadable application) distributed electronically by a device manufacturer or through an electronic marketplace (e.g., Google Play Store, App Store). For electronic distribution, at least a portion of the software program may be stored on a storage medium or temporarily generated. In this case, the storage medium may be a storage medium of a manufacturer's server, an electronic marketplace server, or a relay server that temporarily stores the software program.

[0203] In a system comprising a server and a device, the computer program may include the storage medium of the server or the storage medium of the device. Alternatively, if a third device (e.g., a smartphone) is present and is communicatively connected to the server or the device, the computer program may include the storage medium of the third device. Alternatively, the computer program may include a software program itself that is transmitted from the server to the device or the third device, or from the third device to the device. In this case, one of the server, the device, and the third device may execute the computer program to perform the method according to the disclosed embodiments. Alternatively, two or more of the server, the device, and the third device may execute the computer program to perform the method according to the disclosed embodiments in a distributed manner. For example, a server may execute a computer program stored on the server and control a device communicatively connected to the server to perform the method according to the disclosed embodiments. As another example, a third device may execute a computer program and control a device communicatively connected to the third device to perform the method according to the disclosed embodiments. When the third device executes the computer program, the third device may download the computer program from the server and execute the downloaded computer program. Alternatively, the third device may execute a computer program provided in a pre-loaded state to perform the method according to the disclosed embodiments.

[0204] Although the embodiments described above have been described with limited examples and drawings, those skilled in the art will appreciate that various modifications and variations can be made based on the above description. For example, appropriate results can still be achieved even if the described techniques are performed in a different order than described, and / or components such as the described computer system or modules are combined or combined in a different manner than described, or are replaced or substituted with other components or equivalents.

[0205]

[0206] [Explanation of symbols]

[0207] 100: A system that provides a photo shooting service that includes a product advertising function.

[0208] 110: Camera 120: Server system

[0209] 121: Device DB Server 122: CMS DB Server

[0210] 123: Storage Server 124: Mobile DB Server

[0211] 200: Photo server with product advertising function

[0212] 300: Filming device

Claims

1. A photo taking method including a product advertising function implemented by a processor, A step of receiving at least one image from a photographing device or a user terminal; A step of determining whether an image contains an advertisement product by using an advertisement product recognition model that inputs an image and outputs whether the image contains an advertisement product; and A method for taking pictures including a product advertising function, comprising: providing a reward to the user if the image includes an advertising product.

2. In paragraph 1, further comprising a step of providing assistance for capturing the image to the above-mentioned photographing device; For help with taking images of the above-mentioned products, A photography method including a product advertising function, which is a photography help for at least one of product exposure range for recognizing the advertised product, position or size of the advertised product within the image, and exposure conditions of characters or product names included in the advertised product.

3. In paragraph 1, The above advertising product recognition model is, A method for taking a photo including a product advertising function, the method including an object recognition model and a character recognition model of the above image.

4. In paragraph 1, A step of receiving information about an advertising product selected by the user from the above photographing device; A photo taking method including a product advertising function, further comprising the step of providing a recommended background of the image to the user based on an identifier for the advertised product.

5. In paragraph 4, The above identifier for the above advertising product is, A photographing method including a product advertising function, wherein the photographing method includes information on the characteristics of the advertised product, including the color, type, and customer base of the advertised product.

6. In paragraph 5, The step of providing a reward to the above user is: A photo shooting method including a product advertising function, wherein the reward is provided differentially based on at least one of the number of the advertising products included in the image, the number of the images including the advertising products, and the ratio of the area occupied by the advertising products in the image.

7. In paragraph 6, The steps for providing the above user rewards are: We offer rewards for shooting by changing existing shooting frames to secret frames for free, or A method of taking pictures including a product advertising function, comprising the step of providing a reward for taking additional pictures with a secret frame for a fee.

8. A system that provides a photo shooting service that includes a product advertising function. A photographing device configured to photograph at least one image; User terminal; and Using an advertising product recognition model that takes an image as input and determines whether the image contains an advertising product as output, the image is input and whether the image contains an advertising product is determined. A photo-taking system including a product advertising function, comprising a server configured to provide a reward to a user if the image includes an advertising product.

9. In paragraph 8, The above photographing device, It is further configured to receive help on shooting the above image, For help with taking the above image, A photography system including a product advertising function, which is a photographing help for at least one of a product exposure range for recognizing the advertised product, a position or size of the advertised product within the image, and an exposure condition of characters or product names included in the advertised product.

10. In paragraph 8, The above advertising product recognition model is, A photography system including a product advertising function, the photography system including an object recognition model and a character recognition model of the above image.

11. In paragraph 8, The above photographing device, It is configured to transmit information about the advertising product selected by the user to the server, The above server, A photo shooting system including a product advertising function, further configured to provide a recommended background of the image to the user based on an identifier for the advertised product.

12. In paragraph 11, The above identifier for the above advertising product is, A photo shooting system including a product advertising function, which includes information on the characteristics of the advertised product, including the color, type, and customer base of the advertised product.

13. In paragraph 8, The above server, A photo shooting system including a product advertising function that provides the reward differentially based on at least one of the number of the advertising products included in the image, the number of the images including the advertising products, and the ratio of the area occupied by the advertising products in the image.

14. In paragraph 13, The above server, We offer rewards for shooting by changing existing shooting frames to secret frames for free, or A photo shooting system including a product advertising function, further configured to provide a reward for taking additional photos with a secret frame for a fee.

15. A photo-taking device including a product advertising function, Display section; Interface section; filming department; Output section; and Including a control unit, The control unit includes a memory and one or more processors for executing a program loaded into the memory, The above program is, An instruction for receiving at least one image from a photographing device or a user terminal; Instructions for determining whether an image contains an advertisement product using an advertisement product recognition model that takes an image as input and determines whether the image contains an advertisement product as output; A photo-taking device including a product advertising function, comprising instructions for providing a reward to the user if the image includes an advertising product.

Images