System and method for automatically generating tag of content on basis of image analysis

Abstract

A server for providing a service on the basis of a fashion image for which a tag has been generated, according to an aspect, comprises: a communication module that performs communication with a user terminal; a memory storing at least one program; and a processor that performs a calculation by executing the at least one program, wherein the processor: acquires a first fashion image collected from a social network service server corresponding to the user terminal and a user account; selects, from the first fashion image, a second fashion image that is a target of tag generation; tags the second fashion image with at least one tag from among tags pre-stored in a tag database, by using an image evaluation engine; and provides a result of matching a user and a fashion brand, on the basis of the tagged second fashion image.

Description

System and method for automatically generating tags for content based on image analysis

[0001] The following embodiments relate to a system and method for automatically generating tags for content based on image analysis.

[0002] With the proliferation of the Internet and social media, the number of users who directly produce and create their own content and provide various image and video information through social media platforms such as Facebook, Twitter, Instagram, and YouTube, as well as portal site platforms such as Naver, Google, and Daum is increasing explosively.

[0003] Accordingly, new types of advertising, such as through influencers, have emerged. Influencers are users who already have a solid following and are easily able to attract other consumers. They cultivate a fandom through various social media platforms, allowing companies to secure potential customers and conduct marketing campaigns.

[0004] Accordingly, a service is being provided that analyzes the tendencies of fashion brands and influencers and recommends influencers for effective advertising of fashion brands.

[0005] However, because words used to describe fashion images are often subjective and ambiguous, it's extremely difficult to provide optimal matching results between fashion brands and influencers based on text. This means recommending fashion influencers who meet the needs of fashion brands expressed in text is challenging.

[0006] Accordingly, there is a need for a method to recommend influencers that meet the needs of fashion brands by analyzing the characteristics of the image itself.

[0007] The background technology described above is technical information that the inventor possessed for the purpose of deriving the present invention or acquired in the process of deriving the present invention, and cannot necessarily be considered as publicly known technology disclosed to the general public prior to the application for the present invention.

[0008] The present invention provides a system and method for automatically generating tags for content based on image analysis.

[0009] The problems addressed by the present invention are not limited to those mentioned above. Other problems and advantages of the present invention not mentioned above can be understood through the following description and will be more clearly understood through the embodiments of the present invention. Furthermore, it will be appreciated that the problems and advantages addressed by the present invention can be realized by the means and combinations thereof set forth in the claims.

[0010] As a technical means for achieving the above-described technical task, a first aspect of the present disclosure may provide a method, including: a step of obtaining a first fashion image collected from a social network service server corresponding to a user terminal and a user account; a step of selecting a second fashion image to be a target of tag generation from the first fashion image; a step of tagging the second fashion image data with at least one tag among tags previously stored in a tag database using an image evaluation engine; and a step of providing a matching result of the user and a fashion brand based on the tagged second fashion image.

[0011] A second aspect of the present disclosure provides a server that provides a service based on a fashion image in which a tag is generated, the server comprising: a communication module for performing communication with a user terminal; a memory storing at least one program; and a processor for performing an operation by executing the at least one program, wherein the processor obtains a first fashion image collected from a user terminal and a social network service server of the user, selects a second fashion image to be a target of tag generation from the first fashion image, tags the second fashion image data with at least one tag among tags previously stored in a tag database using an image evaluation engine, and provides a matching result of the user and a fashion brand based on the tagged second fashion image.

[0012] A third aspect of the present disclosure can provide a computer-readable recording medium having recorded thereon a program for executing the method according to the first aspect on a computer.

[0013] According to the problem solving means of the present disclosure described above, by automatically generating tags for image content related to fashion, there is an effect of being able to classify and analyze images based on tags.

[0014] Additionally, by classifying and analyzing images based on tags, it is possible to match fashion brands with creators optimized for them, which can help fashion brands produce efficient advertisements.

[0015] Additionally, by analyzing and tagging the image itself, we can address potential distortions that may arise when the meaning of adjectives describing a particular image is unclear and subjective.

[0016] FIG. 1 is a diagram illustrating an example of a system that provides a service based on a fashion image in which a tag is generated according to one embodiment.

[0017] FIG. 2 is a configuration diagram illustrating an example of a server that provides a service based on a fashion image in which a tag is generated according to one embodiment.

[0018] FIG. 3 is a flowchart illustrating an example of a method for providing a service based on a fashion image in which a tag is generated according to one embodiment.

[0019] FIG. 4 is a diagram for explaining a content database according to one embodiment.

[0020] FIG. 5 is a diagram illustrating a process for performing preprocessing, image evaluation, tag mapping, and tag information input on a fashion image according to one embodiment.

[0021] FIG. 6 is a drawing illustrating an example of a method for tagging a fashion image according to one embodiment.

[0022] FIG. 7 is a diagram showing an example of a product registration screen in which a fashion brand can register a product it wishes to sponsor, according to one embodiment.

[0023] FIG. 8 is a drawing showing an example of a screen for a fashion brand terminal according to one embodiment to input persona information sought by a fashion brand.

[0024] FIG. 9 is a diagram showing an example of a screen in which a matching result between a fashion brand and a user is provided to a fashion brand terminal according to one embodiment.

[0025] Figure 10 is a drawing showing an example of a screen that displays information of a selected user on a fashion brand terminal.

[0026] A first aspect of the present disclosure provides a method, comprising: obtaining a first fashion image collected from a social network service server corresponding to a user terminal and a user account; selecting a second fashion image to be a target of tag generation from the first fashion image; tagging the second fashion image data with at least one tag among tags previously stored in a tag database using an image evaluation engine; and providing a matching result between the user and a fashion brand based on the tagged second fashion image.

[0027] A second aspect of the present disclosure provides a server that provides a service based on a fashion image in which a tag is generated, the server comprising: a communication module for performing communication with a user terminal; a memory storing at least one program; and a processor for performing an operation by executing the at least one program, wherein the processor obtains a first fashion image collected from a user terminal and a social network service server of the user, selects a second fashion image to be a target of tag generation from the first fashion image, tags the second fashion image data with at least one tag among tags previously stored in a tag database using an image evaluation engine, and provides a matching result of the user and a fashion brand based on the tagged second fashion image.

[0028] The advantages and features of the present invention, and the methods for achieving them, will become clearer with reference to the embodiments described in detail with the accompanying drawings. However, the present invention is not limited to the embodiments presented below, but can be implemented in various different forms, and it should be understood that all modifications, equivalents, and alternatives fall within the spirit and technical scope of the present invention.

[0029] When a part of the specification is said to "include" a component, this does not exclude other components, but rather implies the inclusion of other components, unless otherwise specifically stated. Furthermore, terms such as "~ engine" and "~ module" used in the specification mean a unit that processes at least one function or operation, which may be implemented in hardware, software, or a combination of hardware and software.

[0030] Additionally, terms including ordinal numbers, such as "first" or "second," used in the specification may be used to describe various components, but the components should not be limited by the terms. The terms may be used to distinguish one component from another.

[0031] The terminology used in this application is only used to describe specific embodiments and is not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, it should be understood that the terms "comprise" or "have" indicate the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, but do not preclude the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0032] Some embodiments of the present disclosure may be represented by functional block configurations and various processing steps. Some or all of these functional blocks may be implemented by various hardware and / or software configurations that perform specific functions. For example, the functional blocks of the present disclosure may be implemented by one or more microprocessors or by circuit configurations for a given function. Furthermore, for example, the functional blocks of the present disclosure may be implemented using various programming or scripting languages. The functional blocks may be implemented by algorithms that execute on one or more processors. Furthermore, the present disclosure may employ conventional techniques for electronic configuration, signal processing, and / or data processing. Terms such as "mechanism," "element," "means," and "configuration" may be used broadly and are not limited to mechanical and physical configurations.

[0033] Additionally, the connecting lines or connecting members between components depicted in the drawings are merely exemplary representations of functional connections and / or physical or circuit connections. In an actual device, connections between components may be represented by various functional connections, physical connections, or circuit connections that may be replaced or added.

[0034] The present disclosure will be described in detail with reference to the attached drawings below.

[0035] FIG. 1 is a diagram illustrating an example of a system that provides a service based on a fashion image in which a tag is generated according to one embodiment.

[0036] Referring to FIG. 1, a system (10) according to one embodiment may include a user terminal (120), a social network service server (130), a fashion brand terminal (140), and a tag generation server (100).

[0037] The user terminal (120) and the fashion brand terminal (140) may be computing devices equipped with a display device and a device for receiving user input (e.g., a keyboard, a mouse, etc.), and including a memory and a processor. For example, the user terminal (120) may be, but is not limited to, a notebook PC, a desktop PC, a laptop, a tablet computer, a smart phone, etc. The tag generation server (100) may be implemented as a computer device or multiple computer devices that communicate over a network to provide commands, codes, files, content, services, etc.

[0038] The tag generation server (100) may be a device that communicates with a user terminal (120), a social network service server (130), and a fashion brand terminal (140) using a network. Alternatively, the tag generation server (100) may be a computing device that includes a memory and a processor and has its own computing capabilities. If the tag generation server (100) is a computing device, the tag generation server (100) may perform at least some of the operations of a server that provides a service based on a fashion image in which a tag is generated, which will be described later with reference to FIGS. 2 to 6. For example, the tag generation server (100) may be a cloud server, but is not limited thereto.

[0039] The tag generation server (100) can obtain fashion images from a user terminal (120) and a social network service server (130) through a network, tag the obtained fashion images, and provide a matching result of a user and a fashion brand to a fashion brand terminal (140) based on the tagged fashion images.

[0040] The fashion image is obtained from a user terminal (120) and / or a social network service server (130), and may be a single image frame or multiple image frames included in a video. The fashion image may include fashion image content voluntarily uploaded by a user to a tag creation server (100) or a social network service server (130) corresponding to a user account after registering as a member, and fashion image content created and registered by wearing a sponsored item after matching with a fashion brand.

[0041] Users can refer to creators or influencers who directly produce and create fashion image content and provide various image and video information through social media platforms such as Facebook, Twitter, Instagram, and YouTube, as well as portal site platforms such as Naver, Google, and Daum.

[0042] A fashion brand can request sponsorship management for products handled by the fashion brand through a fashion brand terminal (140). For example, the fashion brand terminal (140) can receive information about products that are the subject of sponsorship advertisements and information about the fashion mood pursued by the fashion brand (hereinafter referred to as "persona information") from the fashion brand terminal and transmit the information to the tag generation server (100). The fashion brand can select an image or input a keyword for the persona information through the input / output interface (keyboard, mouse, etc.) of the fashion brand terminal (140).

[0043] Here, if persona information is input in the form of an image selection, the tag generation server (100) can provide matching results with the user based on the tags of the selected images. For example, the tag generation server (100) can calculate tag similarity between the tags of the selected images and the tags of the users' fashion images, and provide recommendation information of creators of fashion images whose tag similarity exceeds a threshold value to the fashion brand terminal (140).

[0044] Sponsorship management is a type of service provided by the tag generation server (100) based on the tagging results, and may include recruitment-type sponsorship management and external sponsorship management. Recruitment-type sponsorship management may be a service that receives applications from users who wish to receive sponsorship for products of a corresponding fashion brand, selects the optimal user among the applicants, and provides the selection result to the fashion brand. According to one embodiment, the tag generation server (100) tags the fashion images of users who have applied for recruitment-type sponsorship, and recommends at least one user to the fashion brand based on the tag similarity between the tagged image and the fashion brand's persona information.

[0045] At this time, the tag generation server (100) may acquire a predetermined number of fashion images from among the fashion image contents uploaded by users who applied for sponsorship to the tag generation server (100) or the social network service server (130) corresponding to the user account, and determine the acquired fashion images as the representative images of the sponsorship applicants. The tag generation server (100) may tag the representative images and analyze the applicants' tendencies based on the tag information of the representative images.

[0046] The external sponsorship management may include a service that outputs fashion images to the display section of a fashion brand terminal, allows the user to select one or more fashion images with a desired mood, and searches for the optimal user based on tag similarity with the selected fashion image.

[0047] At this time, the fashion images output may be images previously stored in the tag generation server (100) and may be images with matching tag information. Here, tag similarity may be determined by a similarity measure such as a Euclidean distance measure or cosine similarity.

[0048] According to one embodiment, the tag generation server (100) may tag a fashion image with at least one tag among tags previously stored in the tag generation server (100) using an image evaluation engine. Here, the image evaluation engine may be a CNN (Convolutional Neural Network)-based model trained to extract a feature vector from a fashion image and classify the feature vector into a previously stored tag by using image data, a feature vector of the image data, and tag data of the image stored in the tag generation server (100) as learning data.

[0049] According to one embodiment, by applying a CNN-based model to a fashion image and assigning tags based on the extracted feature vector, the problem that adjectives describing a specific image are subjective and have unclear meanings, resulting in inaccurate matching results between fashion brands and users can be solved.

[0050] The network includes a local area network (LAN), a wide area network (WAN), a value-added network (VAN), a mobile radio communication network, a satellite communication network, and a combination thereof, and is a comprehensive data communication network that enables each network component illustrated in Fig. 1 to communicate smoothly with each other, and may include wired Internet, wireless Internet, and mobile radio communication networks. In addition, wireless communication may include, but is not limited to, wireless LAN (Wi-Fi), Bluetooth, Bluetooth low energy, Zigbee, WFD (Wi-Fi Direct), UWB (ultra wideband), infrared communication (IrDA, infrared Data Association), NFC (Near Field Communication), etc., for example.

[0051] Referring to FIG. 1, a system (10) according to one embodiment may further include an analysis database (150). The analysis database (150) may be implemented in the memory of the tag generation server (100), or may be implemented as embedded in a separate server.

[0052] The analysis database (150) can store the preferences and sponsorship selection rates of fashion brands for recommended users based on the tag similarity between tagged images and the fashion brand's persona information. Alternatively, it can store the results of analyzing the advertising effectiveness of sponsored advertisements created by selected users. The image evaluation engine of the tag generation server (100) can learn the information stored in the analysis database (150) and adjust the criteria for kernels and weights used to extract feature maps from fashion images.

[0053]

[0054] FIG. 2 is a configuration diagram illustrating an example of a server that provides a service based on a fashion image in which a tag is generated according to one embodiment.

[0055] The server (200) of FIG. 2 may be a tag generation server (100). The server (200) illustrated in FIG. 2 only shows components related to the present embodiments, and it is apparent to those skilled in the art that other general components may be included in addition to the components illustrated in FIG. 2.

[0056] Referring to FIG. 2, the server (200) may include a processor (210), a communication module (220), and a database (230). Only components related to the embodiment are illustrated in the server (200) of FIG. 2. Therefore, those skilled in the art will understand that other general components may be included in addition to the components illustrated in FIG. 2.

[0057] The processor (210) controls the overall operation of the server (200). For example, the processor (210) can control the input / output interface (not shown), the display (not shown), the communication module (220), the DB (230), etc., by executing programs stored in the DB (230). The processor (210) can control the operation of the server (200) by executing programs stored in the DB (230).

[0058] The processor (210) may acquire a first fashion image collected from a social network service server corresponding to a user terminal and a user account, and perform preprocessing to select a second fashion image to be the target of tag generation from the first fashion image. In addition, the processor (210) may tag the second fashion image with at least one tag among the tags previously stored in a database using an image evaluation engine. The processor (210) may provide a matching result between a user and a fashion brand based on the tagged second fashion image.

[0059] For example, the processor (210) may determine a fashion image that matches or is similar to the persona information and tag information pursued by the fashion brand among fashion images acquired from a social network service server corresponding to the user terminal and user account. In this case, the persona information pursued by the fashion brand may be information input from the fashion brand terminal and may be input in the form of text or images.

[0060] The processor (210) may be implemented using at least one of application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, micro-controllers, microprocessors, and other electrical units for performing functions.

[0061] The communication module (220) may include one or more components that enable wired / wireless communication with other nodes. For example, the communication module (220) may include at least one of a short-range communication unit (not shown), a mobile communication unit (not shown), and a broadcast receiving unit (not shown).

[0062] DB (230) is hardware that stores various data processed within the server (200), and can store programs for processing and controlling the processor (210). In addition, DB (230) can store fashion image content, tags, and matching information between fashion images and tags.

[0063] DB (230) may include random access memory (RAM) such as dynamic random access memory (DRAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), CD-ROM, Blu-ray or other optical disk storage, hard disk drive (HDD), solid state drive (SSD), or flash memory.

[0064] Hereinafter, with reference to FIGS. 3 to 6, an example of a method for providing a service based on a fashion image in which a tag is generated by operating a tag generation server (100) of the present invention or a server (200) that provides a service based on a fashion image in which a tag is generated will be described. The method for providing a service based on a fashion image in which a tag is generated of FIGS. 3 to 6 is composed of steps that are processed in time series in the server (100, 200) or the processor (210) illustrated in FIGS. 1 and 2. Therefore, even if the content is omitted below, the content described above with respect to the server (100, 200) or the processor (210) illustrated in FIG. 1 can also be applied to the methods of FIGS. 3 to 6.

[0065]

[0066] FIG. 3 is a flowchart illustrating an example of a method for providing a service based on a fashion image in which a tag is generated according to one embodiment.

[0067] Referring to FIG. 3, in step 310, the processor (210) obtains a first fashion image collected from a social network service server corresponding to a user terminal and a user account. According to one embodiment, the processor (210) may obtain the first fashion image from a database (230) included in a server (200) that provides a service based on a fashion image for which a tag has been generated. The database (230) may include a content database that stores the first fashion image collected from the social network service server and a tag database that stores pre-stored tags, tagged images, etc.

[0068] In another embodiment, the content database may be implemented via an external content database server implemented separately from the server (200) that provides services based on fashion images for which tags are generated. A specific example of a first fashion image stored in the tag database is described below in FIG. 4.

[0069]

[0070] FIG. 4 is a diagram for explaining a content database according to one embodiment.

[0071] Referring to FIG. 4, the content database (400) can store fashion image content voluntarily registered by a user on the server (200), fashion image content created and registered as a sponsored advertisement of a fashion brand, and fashion image content uploaded by a user on an SNS channel. Here, the fashion image content voluntarily registered by a user may be fashion image content stored in a user terminal that the user uploaded to the database (230) of the server (200). In addition, the fashion image content uploaded by a user on an SNS channel refers to image content uploaded to an SNS server corresponding to a user account registered by the user upon signing up for membership.

[0072] Referring again to FIG. 3, in step 320, the processor (210) selects a second fashion image to be the target of tag generation from the first fashion image.

[0073] According to one embodiment, the processor (210) may use an object detection algorithm to determine whether the first fashion image satisfies a preset condition, and in response to satisfying the preset condition, classify the first fashion image as the second fashion image. The preset condition may include at least one of a first condition related to a subject of the first fashion image, a second condition related to a category of a fashion item worn by the subject, and a third condition related to identifying the upper / lower parts of the fashion item.

[0074] The object detection (OD) algorithm can be implemented using known image analysis techniques, such as Faster R-CNN, YOLO (You Only Look Once), or SSD (Single Shot Multibox Detector). The object detection (OD) algorithm can be pre-trained using an image dataset in which the categories and item names of the top and bottom fashion items worn by a person are labeled. The object detection (OD) algorithm can be trained to locate the person in the first fashion image and identify the top and bottom clothing worn by the person.

[0075] For example, a first condition may be that the subject included in the image is a person and that the subject will wear a fashion item. The processor (210) may use an object detection algorithm to detect the person object and the fashion item object overlaid on the person object in the image to determine whether the first condition is satisfied. A second condition may be that the category of the fashion item worn by the person object is identifiable. In addition, a third condition may be that the names of at least the top and bottom items among the fashion items are identifiable.

[0076] The processor (210) may classify the first fashion image as a second fashion image if the first fashion image satisfies at least one of the first to third conditions. Preferably, the processor (210) may classify the first fashion image as a second fashion image if the first fashion image satisfies all of the first to third conditions. The processor (210) may process only the classified second fashion images using the image evaluation engine.

[0077] In step 330, the processor (210) tags the second fashion image with at least one tag among the tags previously stored in the tag database using an image evaluation engine.

[0078] Here, the image evaluation engine may be trained to extract feature vectors from fashion images and classify the feature vectors into pre-stored tags by using image data stored in a tag database, feature vectors of the image data, and tag data of the image as learning data.

[0079] According to one embodiment, the processor (210) may input a second fashion image into a convolutional neural network (CNN), obtain at least one feature map, and generate at least one first feature vector from the at least one feature map. Then, the processor (210) may compare the first feature vector with feature vectors of previously stored tags to determine tags matching the second fashion image.

[0080] That is, the processor (210) inputs the preprocessed image (second fashion image) into an image evaluation engine trained with a CNN algorithm, and extracts and quantifies a feature map and a feature vector. Thereafter, the processor (210) can compare the feature vector (first feature vector) extracted from the second fashion image with a preset numerical range.

[0081] A single tag can correspond to one or more values. In other words, even if different feature vectors are extracted from different fashion images, the same tag can be assigned to each image. The preset numerical ranges are intervals of the feature vector values ​​that enable the image evaluation engine to assign specific tags to images, based on training data containing tagged images.

[0082] According to one embodiment, the processor (210) may determine whether at least one first feature vector belongs to at least one of the numerical intervals set for each of the pre-stored tags, and in response to the at least one first feature vector belonging to at least one of the numerical intervals, map the pre-stored tag corresponding to the numerical interval to the first feature vector. Thereafter, the tagged fashion image may be stored in a tag database.

[0083] At this time, the pre-saved tags may include words expressing a mood related to the fashion image. For example, the pre-saved tags may be adjectives indicating fashion styles such as [Hip], [Easy Casual], [Casual], [Street], [Sporty], [Vintage], [Modern], [Feminine], [Mannish], [Girlish], [Boyish], [Classic], [Formal], [Minimal], [Normcore], and [Natural].

[0084] Additionally, the pre-stored tags may include words indicating the pose of the subject included in the fashion image, the type of fashion item worn by the subject, the tone / color of the fashion item, the seasonality, coordination information according to the combination of one or more fashion items (e.g., a tone-on-tone look), etc. The pre-stored tags are not limited to the examples described and may include all types of keywords that describe the fashion image.

[0085] In another embodiment, the processor (210) may output a manual input notification in response to the first feature vector not falling within any of the preset numerical ranges. If the feature vector extracted from the second fashion image does not fall within the preset numerical ranges and thus image classification is not possible, the processor (210) may control the server to output a notification prompting the administrator to manually input a tag.

[0086] In step 340, the processor (210) provides a matching result of the user and the fashion brand based on the tagged second fashion image.

[0087] According to one embodiment, the server (200) may propose a method for creating image content optimized for products handled by a fashion brand based on a tagged second fashion image. For example, the server (200) may recommend creator candidates optimized for the fashion brand and provide the creator with suitable fashion brand information. The processor (210) may compare tag information of image content registered by the creator or uploaded to social media with tag information selected by the fashion brand, thereby optimizing the matching between the creator and the fashion brand.

[0088] For example, the server (200) may generate a blended image based on at least one persona image selected by a fashion brand and tag the blended image. Thereafter, the server may calculate the similarity between the tags of the blended image and the tags of the second fashion image, and provide a list of users who match the fashion brand based on the calculated similarity. Here, the persona images may be multiple images selected from a content list by the fashion brand that have a similar style or atmosphere to the products it wishes to sponsor.

[0089] A content list is a list of images provided to a fashion brand's terminal when registering a product, in order to request sponsorship management for the brand's products. The content list can be a random image list generated based on basic product information, or it can be an image list determined based on the brand's previous history of selecting sponsored models.

[0090] The server (200) generates a content list and provides it to the fashion brand terminal, through which the fashion brand can select a persona image. The selected persona image is transmitted to the server (200), and the server (200) can create a blended image based on the content list and tag the blended image.

[0091] The user list may refer to a candidate list of sponsorship models containing information about multiple users. The server (200) may display user information about multiple users and generate a user interface that allows users to select a sponsorship model from among the multiple users, and transmit the generated user information to the fashion brand terminal. The user information may include at least one of similarity, the user's content status, content efficiency, and response rate. Furthermore, the server (200) may generate graphics that visually display the fashion brand's direction and the user's direction by contrasting attributes, and transmit the generated graphics to the fashion brand terminal. A specific example of the matching result is described below in FIG. 10.

[0092] FIG. 5 is a diagram illustrating a process for performing preprocessing, image evaluation, tag mapping, and tag information input on a fashion image according to one embodiment.

[0093] Referring to FIG. 5, in the preprocessing step (501), the processor (210) uses an object algorithm to classify fashion images acquired from a user terminal or SNS server corresponding to a user account into non-matching images and matching images. Here, non-matching images may be images that do not contain objects recognized as people or images in which fashion items cannot be identified. The processor (210) may determine only images to which tags can be applied as matching images.

[0094] Thereafter, in the image evaluation (502) step, the processor (210) may input only the matching image into a CNN-based image evaluation engine, extract a feature map, and convert the feature map into a one-dimensional feature vector to be digitized. Then, in the tag mapping step (503), the processor (210) may determine whether the extracted feature vector belongs to at least one of the preset numerical intervals for each pre-stored tag, and in response to the feature vector belonging to at least one of the numerical intervals, may map the pre-stored tag corresponding to the corresponding numerical interval to the feature vector. The image to which the tag is mapped may be stored in a tag database.

[0095] The processor (210) may determine that tag mapping is impossible in response to the extracted feature vector not belonging to any of the preset numerical ranges. If a feature vector for which tag mapping is impossible is extracted, the processor (210) may output a tag manual input notification in the tag information input step (504). The administrator of the server (200) may manually assign tags to images to which tags are not mapped based on the manual input notification. In addition, the administrator may modify tags of images to which tags are mapped.

[0096]

[0097] FIG. 6 is a drawing illustrating an example of a method for tagging a fashion image according to one embodiment.

[0098] Referring to FIG. 6, a single fashion image (600) may be tagged with tags according to a plurality of tag attributes (601, 602, 603, 604, 605, 606, 607). Pre-stored tags may be classified and stored by a plurality of tag attributes. For example, the tag attributes (601, 602, 603, 604, 605, 606, 607) may include seven attributes: demographic information of a subject (610) included in the fashion image, pose, color of a fashion item worn by the subject (610), tone, look keyword of the fashion image, mood, and fashion item name. The classification and number of tag attributes illustrated in FIG. 6 are merely examples, and tags may be classified in various ways.

[0099] According to one embodiment, the image evaluation engine may be a convolutional neural network (CNN)-based model trained to extract feature vectors associated with a plurality of tag attributes from a fashion image and classify each feature vector into a pre-stored tag. In this case, the image evaluation engine may include a number of convolutional layers corresponding to the number of tag attributes, and the processor (210) may extract feature vectors associated with each tag attribute using the image evaluation engine.

[0100] In this way, by categorizing previously stored tags into multiple tag attributes and assigning tags to fashion images based on each tag attribute, tag information for a single fashion image can be diversely secured. Furthermore, tag accuracy can be improved compared to assigning multiple tags to a single fashion image without distinguishing tag attributes.

[0101]

[0102] FIG. 7 is a diagram showing an example of a product registration screen in which a fashion brand can register a product it wishes to sponsor, according to one embodiment.

[0103] Referring to Figure 7, fashion brands can enter basic product information, such as product category, product name, product price, SKU (Stock Keeping Unit) code, product description, and desired product launch date. Additionally, fashion brands can register a main thumbnail image and detailed images for the product, and add product registration tags.

[0104] At this time, product registration tags are keywords arbitrarily entered by the fashion brand to describe the product and can be utilized by influencers when searching for products. Product registration tags are distinct from the tags automatically generated by the image evaluation engine according to an embodiment of the present invention, as they are tags that the brand directly designates product characteristics or related information. Furthermore, the entered product registration tags can be used as hashtags in posts produced by creators who receive sponsorship and promote the product.

[0105] Once basic product information is entered, a preview image (700) can be viewed, allowing a preview of what the product will look like when actually registered. The preview image (700) is generated based on the information and image entered by the fashion brand, and provides a function for pre-reviewing the final registration status of the product.

[0106]

[0107] FIG. 8 is a drawing showing an example of a screen for a fashion brand terminal according to one embodiment to input persona information sought by a fashion brand.

[0108] Referring to FIG. 8, a fashion brand can input persona information by selecting one or more images with a similar style or atmosphere to the product (800) the fashion brand wishes to sponsor from the provided content list (801). The content list (801) may be a random image list determined based on registered product basic information. Alternatively, the image list may be determined by further considering the fashion brand's sponsorship history of selecting sponsored models.

[0109] The selected images are then automatically blended by AI, and the resulting blended image (802) is displayed on the screen in real time. The blended image (802) visually represents the persona pursued by the fashion brand, and the fashion brand can use it to determine whether it matches the image it desires.

[0110] In one embodiment, the blended image (802) may be utilized as a reference image for matching users (influencers). That is, the tag generation server may extract tags from the blended image (802), extract tags from images uploaded by users, and provide matching results between fashion brands and users based on the similarity of the extracted tags.

[0111]

[0112] FIG. 9 is a diagram showing an example of a screen in which a matching result between a fashion brand and a user is provided to a fashion brand terminal according to one embodiment.

[0113] The tag generation server can generate tags for persona images or blended images selected by fashion brands and for images uploaded by influencers. Furthermore, matching results between fashion brands and users (influencers) can be provided based on the similarity of the generated image tags.

[0114] In one embodiment, the tag generation server may recommend users in order of highest product information consistency and highest look information consistency. The product information consistency refers to the degree of agreement between product registration tags registered as basic product information and product information entered by sponsor applicants. Furthermore, the look information consistency refers to the degree of similarity between tags in the persona image and tags in the user image.

[0115] Although the product information and look information consistency rates of the recommended users in FIG. 9 are shown as 30% and 80%, respectively, multiple recommended users may have different consistency rates. For example, even if the product information consistency rate is low, if the look information consistency rate is high and the combined consistency rate satisfies a preset standard, the user may be recommended. Alternatively, all users whose look information exceeds a preset standard may be recommended, and matching results may be provided in order of users with the highest product information consistency rate. The method for providing matching results is not limited to the described examples, and the present disclosure may include various methods for providing recommended information based on tag similarity.

[0116] Fashion brands can select users (901, 902) to be sponsored models from the list of users provided as a result of matching.

[0117]

[0118] Figure 10 is a drawing showing an example of a screen that displays information of a selected user on a fashion brand terminal.

[0119] Referring to FIG. 10, the fashion brand terminal can display user information such as product information consistency and look information consistency, content status, content efficiency, member response rate, and channel tag of the selected user.

[0120] Here, content status can include the number of recently uploaded content, topics, and response rates. Content effectiveness can include views relative to the user's follower count, sponsorship selection rates, and advertising effectiveness. Additionally, channel tags represent topics or styles primarily covered on a user's channel (e.g., social media, portal sites, etc.).

[0121] Additionally, the fashion brand terminal may provide an area (1001) that visually displays an applicant tag, which is a fashion mood of a selected user. According to one example, the applicant tag may be determined based on the mood of an image uploaded by a user selected as a sponsored model. According to another embodiment, the applicant tag may be determined based on the mood of products for which the selected user applied for sponsorship.

[0122] Additionally, the fashion brand terminal can visually display a selection tag determined based on uploaded images of users previously selected as sponsored models by the fashion brand.

[0123] In addition, the fashion brand terminal may provide an area (1002) that indicates which of the contrasting attributes the brand is closer to (the area indicated by the filled circle in FIG. 9) and indicates which of the contrasting attributes the selected user is more likely to support (the area indicated by the dotted circle in FIG. 9).

[0124] For example, a fashion brand terminal can show whether the fashion brand targets the young or the mature demographic, and indicate which demographic the selected member is more likely to support through sponsorship, thereby clearly identifying the degree to which the fashion brand and user's direction align based on specific attributes.

[0125]

[0126] Embodiments of the present invention may be implemented in the form of a computer program that can be executed through various components on a computer, and such a computer program may be recorded on a computer-readable medium. In this case, the medium may include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specifically configured to store and execute program instructions, such as ROMs, RAMs, and flash memories.

[0127] Meanwhile, the computer program may be specifically designed and constructed for the present invention, or may be one known and available to those skilled in the computer software field. Examples of computer programs may include not only machine language code, such as that generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like.

[0128] According to one embodiment, the method according to various embodiments of the present disclosure may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) through an application store (e.g., Play Store™) or directly between two user devices. In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

Claims

1. A method for providing a service based on a fashion image in which a tag has been created. A step of obtaining a first fashion image collected from a social network service server corresponding to a user terminal and a user account; A step of selecting a second fashion image to be the target of tag creation from the first fashion image; A step of tagging the second fashion image with at least one tag among tags previously stored in a tag database using an image evaluation engine; A method comprising: providing a matching result between a user and a fashion brand based on the tagged second fashion image.

2. In paragraph 1, The above selection step is, A step of determining whether the first fashion image satisfies a preset condition using an object detection algorithm; and In response to satisfying the above preset condition, a step of classifying the first fashion image into the second fashion image is included; The above conditions are set, A method comprising at least one of a first condition related to a subject of the first fashion image, a second condition related to a category of a fashion item worn by the subject, and a third condition related to identification of the upper / lower parts of the fashion item.

3. In paragraph 1, The above tagging step is: A step of inputting the second fashion image into a CNN (Convolutional Neural Network) to obtain at least one feature map; generating at least one first feature vector from at least one feature map; and A method comprising: a step of comparing the first feature vector with the feature vector of the previously stored tag to determine a tag matching the second fashion image.

4. In paragraph 3, The steps for determining the above tags are: A step of determining whether at least one of the first feature vectors belongs to at least one of the numerical ranges set for each of the previously stored tags; and A method comprising: a step of mapping the pre-stored tag corresponding to the numerical interval to the first feature vector in response to the at least one first feature vector belonging to at least one of the numerical intervals; 5. In paragraph 4, The steps for determining the above tags are: A method comprising: a step of outputting a manual input notification in response to the first feature vector not belonging to any of the preset numerical ranges; 6. In paragraph 1, The above tagged fashion images are, stored in the above tag database, The above image evaluation engine, A method for extracting feature vectors from fashion images and learning to classify feature vectors into the previously stored tags by using image data stored in the tag database, feature vectors of the image data, and tag data of the image as learning data.

7. In paragraph 1, The above previously saved tags are: A method comprising words expressing a mood associated with a fashion image.

8. In paragraph 1, The step of providing the above matching result is: A step of tagging a blended image generated based on the persona image of the above fashion brand; A step of calculating the similarity between the tag of the blended image and the tag of the second fashion image; and A method comprising: providing a list of users who match the fashion brand based on the similarity.

9. In paragraph 8, The steps for tagging the above blended image are: A step of generating a content list based on at least one of product information of the fashion brand and sponsorship history of the fashion brand; A step of providing the above content list to a fashion brand terminal so that the fashion brand selects the above persona image; A step of obtaining the selected persona image and generating the blended image; and A method comprising: a step of tagging at least one of the previously stored tags to the generated blended image.

10. In paragraph 8, The step of providing the above user list is: A method comprising the steps of: generating a user interface that displays user information of a plurality of users and allows a user to select a sponsorship model from among the plurality of users.

11. In Article 10, The above user information is: A method comprising at least one of the above similarity, the user's content status, content efficiency and response rate.

12. In paragraph 1, The step of providing the above matching result is: A method comprising the step of generating graphics that visually display the direction of the fashion brand and the direction of the user, respectively, by contrasting attributes.

13. A computer-readable recording medium having recorded thereon a program for executing the method of Article 1 on a computer.

14. In a server that provides services based on fashion images for which tags have been created, A communication module that performs communication with a user terminal; memory in which at least one program is stored; and A processor comprising: a processor that performs a calculation by executing at least one program; The above processor, Obtaining a first fashion image collected from a social network service server corresponding to a user terminal and a user account, From the above first fashion image, a second fashion image that is to be the target of tag creation is selected, Using an image evaluation engine, tagging the second fashion image with at least one tag among the tags previously stored in the tag database, A server that provides matching results between users and fashion brands based on the above-mentioned tagged second fashion images.

15. In paragraph 14, The above processor, Using an object detection algorithm, it is determined whether the first fashion image satisfies a preset condition, In response to satisfying the above preset condition, classifying the first fashion image as the second fashion image, The above conditions are set, A server comprising at least one of a first condition related to a subject of the first fashion image, a second condition related to a category of a fashion item worn by the subject, and a third condition related to identification of the upper / lower parts of the fashion item.

16. In paragraph 14, The above processor, By inputting the above second fashion image into a CNN (Convolutional Neural Network), at least one feature map is obtained, Generating at least one first feature vector from at least one feature map, A server that compares the first feature vector with the feature vectors of the previously stored tags to determine a tag matching the second fashion image.

17. In paragraph 16, The above processor, Determine whether at least one of the first feature vectors above belongs to at least one of the numerical ranges preset for each of the previously stored tags, A server that maps the stored tag corresponding to the numerical interval to the first feature vector in response to the at least one first feature vector belonging to at least one of the numerical intervals.

18. In paragraph 17, The above processor, A server configured to output a manual input notification in response to the first feature vector not belonging to any of the preset numerical ranges.

19. In paragraph 14, The above tagged fashion images are, stored in the above tag database, The above image evaluation engine, A server that learns to extract feature vectors from fashion images and classify the feature vectors into the previously stored tags by using image data stored in the tag database, feature vectors of the image data, and tag data of the image as learning data.

20. In paragraph 14, The above previously saved tags are: A server that contains words expressing the mood associated with a fashion image.

Images