Information processing device, information processing method, and recording medium

Abstract

An information processing device according to one aspect of the present disclosure is provided with: an image acquisition means for acquiring an image included in a post on a social networking site (SNS); an extraction means for extracting, from the image, a portion including an article and a portion including the background; a search means for searching for similar images that include portions similar to each of the extracted portions and are images posted on the SNS; an attribute information acquisition means for acquiring attribute information assigned to posts including similar images; and an assignment means for assigning attribute information to posts on the SNS on the basis of the attribute information.

Description

Information Processing Apparatus, Information Processing Method, and Recording Medium

[0001] The present disclosure relates to an information processing apparatus, an information processing method, and a recording medium.

[0002] There is a technique of attaching information indicating the content of an image to the image using the result of image search.

[0003] For example, Patent Document 1 discloses a technique of performing image recognition processing on an image to be searched and obtaining tags associated with similar images.

[0004] Japanese Unexamined Patent Application Publication No. 2010-218248

[0005] By the way, in order to analyze the preferences of users of SNS (Social Networking Service), it is desired to attach attribute information to posts. For example, when attaching attribute information to a post based on an image included in the post, the method disclosed in Patent Document 1 can be used. However, when using the result of a similar image search targeting the entire image as in the method disclosed in Patent Document 1, there is a possibility that attribute information related to a specific element in the image cannot be attached to the post. That is, there is a possibility that attribute information deviating from the purpose of analysis is attached to the post and appropriate attribute information for analysis is not attached.

[0006] One of the objects of the present disclosure is to attach appropriate attribute information to posts on SNS.

[0007] An information processing apparatus according to an aspect of the present disclosure includes an image acquisition unit that acquires an image included in a post on SNS, an extraction unit that extracts a part including an article and a part including a background from the image, a search unit that searches for a similar image that is an image posted on SNS and includes a part similar to each of the extracted parts, an attribute information acquisition unit that acquires attribute information attached to a post including the similar image, and an attachment unit that attaches attribute information to a post on SNS based on the attribute information.

[0008] An information processing method in one aspect of this disclosure involves acquiring an image contained in a post on a social networking service (SNS), extracting portions containing an item and a background from the image, searching for similar images that include portions similar to the extracted portions and that have been posted on the SNS, acquiring attribute information attached to posts containing similar images, and attaching attribute information to posts on the SNS based on the attribute information.

[0009] A recording medium in one aspect of this disclosure records a program that causes a computer to perform the following processes: acquire images contained in posts on social networking services (SNS), extract portions containing items and backgrounds from the images, search for similar images that include portions similar to each of the extracted portions and that are posted on SNS, acquire attribute information attached to posts containing similar images, and attach attribute information to posts on SNS based on the attribute information.

[0010] According to this disclosure, appropriate attribute information can be added to posts on social networking services (SNS).

[0011] This is a block diagram showing an example of the configuration of the information processing system related to this disclosure. This is a first block diagram showing an example of the configuration of the information processing device related to this disclosure. This is a first flowchart showing an example of the operation of the information processing device related to this disclosure. This is a second block diagram showing an example of the configuration of the information processing device related to this disclosure. This is a first example of a screen displayed by the terminal device related to this disclosure. This is a second flowchart showing an example of the operation of the information processing device related to this disclosure. This is a block diagram showing an example of the hardware configuration of the information processing device related to this disclosure.

[0012] The embodiments of this disclosure will be described in detail below with reference to the drawings. In each of the embodiments described in the drawings and specification, the same reference numerals are used for similar components, and detailed descriptions will not be repeated.

[0013] (First embodiment)

[0014] (Configuration Example) The configuration example of the information processing system 10 will be described below with reference to Figure 1. Figure 1 is a block diagram showing an example of the configuration of the information processing system 10 in this disclosure. The information processing system 10 comprises at least a terminal device 20, an SNS service provision system 30, a management device 40, an image database 50, an attribute information database 60, and an information processing device 100.

[0015] Information processing system 10 is a system that assigns attribute information to posts containing images on social networking services (SNS) based on those images. By assigning attribute information to posts, information processing system 10 makes it possible to analyze the preferences of users who have shown positive reactions to the posts, such as liking or commenting on them, based on that attribute information.

[0016] Here, attribute information refers to information that indicates a person's preferences regarding an item. For example, attribute information is information that indicates the attributes of an item. In one example, attribute information may include information indicating at least one of the following: the type of item, the color of the item, the brand of the item, the impression the item gives, and the circumstances in which the item is used.

[0017] Furthermore, "articles" refer to items worn by a person. Examples of articles include clothing, hats, shoes, jewelry, bags, wallets, and watches.

[0018] For example, if a social networking service (SNS) user frequently reacts to posts that have "T-shirt" attached as attribute information, it can be inferred that the type of item the user is interested in is "T-shirt." The information processing system 10 can be said to provide information that supports the analysis of SNS users' fashion preferences by attaching attribute information to posts on SNS.

[0019] For example, an image is at least an image of an object. Alternatively, the image may be, for example, an image of a person and the object the person is wearing. In one example, attribute information regarding the appearance of an object can be identified from an image of an object. Attribute information regarding the appearance of an object may include, in one example, the type of object, the color of the object, and the brand of the object. The information processing system 10 analyzes the image to identify the attribute information of the object worn by the person in the image and assigns this attribute information to the post containing the image. Note that the examples of objects are not limited to those exemplified here.

[0020] Furthermore, the area of ​​an image excluding the objects is the background. For example, attribute information regarding the context in which the objects in the image are used can be identified from the background. One example of attribute information regarding the context in which the objects are used is the event, season, and time of day. For example, if a party venue is visible in the background, it can be seen that one of the events suitable for the use of the items worn by the person in the image is a "party." Note that the examples of backgrounds are not limited to those exemplified here.

[0021] The information processing system 10 analyzes the image to identify the attribute information of the items worn by the person in the image, and then adds the attribute information regarding the items worn by the person in the image to the post containing the image.

[0022] Terminal device 20 is a terminal device operated by a user of the social networking service (SNS). Terminal device 20 connects to the internet and accesses the SNS, for example. Terminal device 20 also posts to the SNS according to the operator's instructions. Terminal device 20 also receives the operator's reactions to posts made by other SNS users, according to the operator's instructions. In one example, terminal device 20 implements these functions using a web browser or an installed application.

[0023] Here, SNS refers to a type of SNS where users can view posts from an unspecified number of other users.

[0024] For example, the terminal device 20 includes an interface for receiving operations from the operator. In one example, the terminal device 20 includes at least one of a touch panel and a physical button. However, the examples of interfaces provided by the terminal device 20 are not limited to those illustrated here.

[0025] Furthermore, the terminal device 20 includes, for example, means for providing visual information to the operator. In one example, the terminal device 20 includes a display.

[0026] In one example, the terminal device 20 is an electronic device such as a smartphone, tablet, or PC (Personal Computer). However, the examples of the terminal device 20 are not limited to those exemplified here.

[0027] The SNS service provision system 30 is a system that provides SNS services.

[0028] For example, when the SNS service provision system 30 receives a reaction from the operator of the terminal device 20 to a post containing an image, it notifies the management device 40 that it has received a reaction to the post.

[0029] The management device 40 is a management device that has the function of managing attribute information assigned to users and posts on the SNS. In one example, the management device 40 is a device managed by an entity that manages attribute information assigned to posts on the SNS, which is different from the entity that provides the SNS service. However, if the entity that provides the SNS service and the entity that manages attribute information assigned to posts on the SNS are the same, the management device 40 may be part of the SNS service provision system 30.

[0030] For example, when the management device 40 receives notification from the SNS service provision system 30 that it has received a reaction to a post containing an image, it sends an instruction to the information processing device 100 to assign attribute information to the post and the identification information of the post.

[0031] In one example, when the management device 40 receives notification from the SNS service provision system 30 that it has received a reaction to a post containing an image, it sends an instruction to the information processing device 100 to assign attribute information to the post and information indicating the ID (Identifier) ​​of the post.

[0032] Furthermore, for example, the management device 40 may send an instruction to the information processing device 100 to add attribute information if attribute information has not been added to a post that has received a reaction from a user. In one example, the management device 40 sends an instruction to the information processing device 100 to add attribute information if the attribute information added to a post that has received a reaction from a user is not recorded in the attribute information database 60.

[0033] The image database 50 is a database that records images included in posts on social networking services (SNS). For example, the SNS service provision system 30 records data of posts on SNS in the image database 50. In one example, the image database 50 is a database provided by the SNS service provision system 30 that records data of posts.

[0034] For example, the image database 50 records images included in posts on social networking services (SNS) in association with the identification information of those posts. In one example, the image database 50 records images included in posts on SNS in association with an ID used to identify those posts in the SNS system.

[0035] Furthermore, the image database 50 may record images included in posts on multiple social networking services (SNS). That is, the image database 50 records images posted on SNS other than the SNS on which the post to which attribute information is to be assigned was posted. This increases the amount of data that the information processing device 100 can refer to when assigning attribute information to a post, compared to the case where the image database 50 records images included in posts on a single SNS. Therefore, the likelihood of assigning more appropriate attribute information to a post increases.

[0036] The information processing system 10 may include multiple image databases 50. Each image database 50 records images included in posts on different social networking services (SNS).

[0037] The attribute information database 60 is a database that records attribute information assigned to users and posts on the SNS. In one example, the attribute information database 60 is managed by an entity that manages attribute information assigned to posts on the SNS, which is different from the entity that provides the SNS service. However, if the entity that provides the SNS service and the entity that manages attribute information assigned to posts on the SNS are the same, the attribute information database 60 may be part of the SNS service provision system 30.

[0038] For example, the attribute information database 60 records attribute information assigned to users and posts on the SNS, associating it with the identification information of the user and the post. In one example, the attribute information database 60 records attribute information assigned to users and posts on the SNS, associating it with the ID used to identify the user and the post on the SNS.

[0039] The image database 50 and the attribute information database 60 may be the same database. In this case, the database will record at least the images included in a post and the attribute information assigned to that post in association with each other.

[0040] The information processing device 100 is an information processing device that analyzes the content of images included in a post and assigns attribute information to the post based on the analysis results. In one example, the information processing device 100 is a device managed by an entity that manages attribute information assigned to posts on the SNS, which is different from the entity that provides the SNS service. However, if the entity that provides the SNS service and the entity that manages attribute information assigned to posts on the SNS are the same, the information processing device 100 may be part of the SNS service provision system 30.

[0041] An example of each component of the information processing apparatus 100 will be described with reference to FIG. 2. FIG. 2 is a block diagram showing an example of the configuration of the information processing apparatus 100 in the present disclosure. The information processing apparatus 100 includes an image acquisition unit 110, an extraction unit 120, a search unit 130, an attribute information acquisition unit 140, and an assignment unit 150. The image acquisition unit 110 acquires an image included in a post to a social networking service (SNS). The extraction unit 120 extracts a portion in which an article is depicted and a portion in which a background is depicted from the image. The search unit 130 searches for similar images that are images posted to the SNS and include portions similar to each of the extracted portions. The attribute information acquisition unit 140 acquires attribute information assigned to a post including the similar image. The assignment unit 150 assigns attribute information to a post including an image based on the attribute information assigned to a post including the similar image.

[0042] The image acquisition unit 110 acquires an image included in a post on the SNS, which is an object to which attribute information is to be assigned. For example, when the image acquisition unit 110 receives an instruction to assign attribute information to a post from the management device 40, the image acquisition unit 110 acquires the data of the image included in the post.

[0043] In one example, when the image acquisition unit 110 receives an instruction to assign attribute information to a post from the management device 40, the image acquisition unit 110 acquires the data of the image included in the post from the image database 50 based on the identification information of the post transmitted from the management device 40. Note that the image acquisition unit 110 may acquire an image by receiving the data of the image included in the post that is an object to which attribute information is to be assigned and is transmitted from the management device 40.

[0044] The extraction unit 120 extracts a portion in which an article is depicted and a portion in which a background is depicted from the image. For example, the extraction unit 120 extracts a portion in which an article is depicted and a portion in which a background is depicted from the image acquired by the image acquisition unit 110.

[0045] In one example, the extraction unit 120 detects a region in the image in which an article is depicted and extracts the region as the portion in which the article is depicted. Further, the extraction unit 120 extracts a portion other than the region in the image in which the article is depicted as the portion in which the background is depicted. Note that, as a specific means for detecting a region in the image in which an article is depicted, known image recognition techniques such as image recognition using a learned model may be used.

[0046] Note that the extraction unit 120 may extract a plurality of portions in the image where the article is depicted.

[0047] The search unit 130 searches for similar images that are images posted on the SNS and include portions similar to each of the extracted portions. For example, the search unit 130 searches for similar images that are images posted on the SNS and include portions similar to each of the portion where the article is depicted and the portion where the background is depicted, which are extracted by the extraction unit 120.

[0048] In one example, the search unit 130 calculates the similarity of feature amounts between the images recorded in the image database 50 for each of the portion where the article is depicted and the portion where the background is depicted, and sets an image with a similarity of feature amounts equal to or greater than the threshold as a similar image. Note that the threshold is appropriately set based on the type of the article in the image or the like. Note that, as a means for searching for similar images, known image recognition technologies such as image recognition using a learned model may be used.

[0049] Also, in one example, the search unit 130 sets an image as a similar image if the image recorded in the image database 50 includes a portion where the similarity of feature amounts with the portion where the article is depicted and the portion where the background is depicted is equal to or greater than the threshold. Note that the threshold is appropriately set based on the type of the article in the image or the like. Note that, as a means for searching for similar images, known image recognition technologies such as image recognition using a learned model may be used.

[0050] The attribute information acquisition unit 140 acquires the attribute information attached to the post including the similar image. For example, the attribute information acquisition unit 140 acquires the attribute information attached to the post including the similar image searched by the search unit 130 from the attribute information database 60.

[0051] In one example, the attribute information acquisition unit 140 uses the identification information of the post including the similar image searched by the search unit 130 to acquire the attribute information attached to the post from the attribute information database 60.

[0052] The attribute information acquisition unit 140 may also acquire hashtags associated with posts containing similar images as attribute information. For example, the attribute information acquisition unit 140 may acquire strings contained in hashtags in the text of posts containing similar images from a database (not shown) as attribute information.

[0053] Here, a hashtag is a function on social media used to organize posts related to a specific theme. For example, hashtags act as links between keywords included in a post and keywords related to that post, assisting in filtering and searching for posts. Therefore, the keywords included in a hashtag may indicate the theme intended by the person who made the post.

[0054] The attribute assignment unit 150 assigns attribute information to posts that are the target of attribute information assignment, based on the attribute information assigned to posts containing similar images. For example, the attribute assignment unit 150 associates the attribute information assigned to posts containing similar images, which was acquired by the attribute information acquisition unit 140, with the posts that are the target of attribute information assignment, and records this in the attribute information database 60.

[0055] In one example, the attribute assignment unit 150 associates attribute information relating to the appearance of an item among the attribute information assigned to a post containing similar images of the part in which the item is pictured with the post to which the attribute information is assigned, and records it in the attribute information database 60. In addition, the attribute assignment unit 150 associates attribute information relating to the circumstances under which the item is used among the attribute information assigned to a post containing similar images of the part in which the background is pictured with the post to which the attribute information is assigned, and records it in the attribute information database 60.

[0056] Furthermore, if there are multiple similar images, the attribute assignment unit 150 may assign some of the attribute information assigned to multiple posts containing similar images to the post to which the attribute information is to be assigned.

[0057] For example, the attribute assignment unit 150 assigns common attribute information, which has been assigned to multiple posts containing similar images, to the post to which the attribute information is to be assigned.

[0058] In one example, the assignment unit 150 calculates the number of posts containing similar images to which each attribute piece of attribute information has been assigned, and assigns the attribute information to posts that are equal to or greater than a predetermined number to the post to which the attribute information is to be assigned. The predetermined number is set appropriately based on the number of posts containing similar images, etc. Furthermore, the assignment unit 150 may also choose not to assign attribute information that has low commonality among multiple posts containing similar images to the post to which the attribute information is to be assigned.

[0059] For example, the attribute assignment unit 150 assigns attribute information that has been assigned to multiple posts containing similar images to the post to which attribute information is to be assigned, according to the degree of similarity between the image in the post to which attribute information is to be assigned and similar images.

[0060] In one example, the attribute assignment unit 150 assigns attribute information to a post that contains similar images whose similarity to the images in that post is to be assigned attribute information is above a threshold. Furthermore, if attribute information assigned to a post containing similar images whose similarity is below a threshold has been assigned to a post containing a predetermined number or more of similar images, the attribute assignment unit 150 may also assign this attribute information to the post to be assigned attribute information.

[0061] (Example of Operation) An example of the operation of the information processing device 100 will be explained below with reference to Figure 3. Figure 3 is a flowchart showing an example of the operation of the information processing device 100 in this disclosure.

[0062] First, the image acquisition unit 110 acquires images included in posts on social networking services (SNS) that are to which attribute information will be assigned (S110). In one example, when the image acquisition unit 110 receives an instruction from the management device 40 to assign attribute information to a post, it acquires image data included in the post from the image database 50.

[0063] Next, the extraction unit 120 extracts the portion of the image in which the object is visible and the portion of the image in which the background is visible (S120). In one example, the extraction unit 120 detects the region in the image in which the object is visible and extracts that region as the portion in which the object is visible. The extraction unit 120 also extracts the portion of the image other than the region in which the object is visible as the portion of the background.

[0064] Next, the search unit 130 searches for similar images that include parts similar to each of the extracted parts and that have been posted on social media (S130). In one example, the search unit 130 searches the image database 50 for similar images that contain parts with a high degree of feature matching for both the part showing the object and the part showing the background.

[0065] Next, the attribute information acquisition unit 140 acquires attribute information assigned to posts containing similar images (S140). In one example, the attribute information acquisition unit 140 uses the identification information of posts containing similar images that the search unit 130 has searched for to acquire attribute information assigned to those posts from the attribute information database 60.

[0066] Next, the attribute assignment unit 150 assigns attribute information to the target post based on the attribute information assigned to the post containing similar images (S150). In one example, the attribute assignment unit 150 assigns attribute information relating to the appearance of the item from the attribute information assigned to the post containing similar images of the part in which the item is pictured to the target post. The attribute assignment unit 150 also assigns attribute information relating to the circumstances in which the item is used from the attribute information assigned to the post containing similar images of the part in which the background is pictured to the target post.

[0067] The information processing device 100 can assign appropriate attribute information to posts on social networking services (SNS). This is because the information processing device 100 acquires images included in posts to which attribute information is to be assigned, extracts parts containing items and parts containing backgrounds from the images, searches for similar images posted on SNS that include parts similar to the extracted parts, acquires attribute information assigned to posts containing similar images, and assigns attribute information to posts to which attribute information is to be assigned based on this attribute information.

[0068] (Modification 1) A modification of the information processing device 100 will be described. This modification is an example in which the information processing device 100 assigns attribute information to a post based on the portion of the image in which a person is depicted. This modification may be applied to other embodiments and modifications to the extent that it does not create inconsistencies.

[0069] For example, by examining similar images of the portion of an image that shows a person, attribute information about a person who might use the item the person in the image is wearing can be identified. In one example, attribute information about a person who might use the item includes the ratio of the size of each part of the body to the rest of the body and the hairstyle. The ratio of the size of each part of the body to the rest of the body and the hairstyle of the person in the image can be inferred to be characteristics of a person who would suit the item the person is wearing.

[0070] The image acquisition unit 110 acquires images included in posts on social networking services (SNS), which are the targets to which attribute information is to be assigned. The operation of the image acquisition unit 110 is the same as in the first embodiment, so a detailed explanation is omitted.

[0071] The extraction unit 120 extracts from the image the parts containing objects, the parts containing people, and the parts containing the background. For example, the extraction unit 120 extracts from the image acquired by the image acquisition unit 110 the parts containing objects, the parts containing people, and the parts containing the background.

[0072] In one example, the extraction unit 120 detects areas in the image where objects are depicted and extracts these areas as parts containing objects. The extraction unit 120 also detects areas in the image where people are depicted and extracts these areas as parts containing people. Furthermore, the extraction unit 120 extracts areas in the image other than those containing objects and people as parts containing the background. As a specific means of detecting areas containing objects and people in an image, known image recognition techniques such as image recognition using a trained model may be used.

[0073] Furthermore, when the extraction unit 120 extracts a portion of an image in which a person is visible, it may also extract a portion in which a part of the person's body is visible. For example, the extraction unit 120 extracts a portion in which the person's head is visible. As a specific means of detecting a region in an image in which a part of a person's body is visible, known image recognition techniques such as image recognition using a trained model may be used.

[0074] The search unit 130 searches for similar images that include parts similar to each of the extracted parts and that have been posted on social media. For example, the search unit 130 searches for similar images that include parts similar to each of the parts extracted by the extraction unit 120—parts showing items, parts showing people, and parts showing the background—and that have been posted on social media.

[0075] In one example, the search unit 130 calculates the similarity of features between the image data recorded in the image database 50 and the parts containing objects, people, and backgrounds, and identifies images that contain parts with a feature similarity above a threshold as similar images. The threshold is set appropriately based on the type of object in the image, etc. Known image recognition techniques, such as image recognition using a trained model, may be used as means for searching for similar images.

[0076] The attribute information acquisition unit 140 acquires attribute information attached to posts containing similar images. The operation of the attribute information acquisition unit 140 is the same as in the first embodiment, so a detailed explanation is omitted.

[0077] The attribute assignment unit 150 assigns attribute information to posts that are the target of attribute assignment, based on the attribute information assigned to posts containing similar images. The operation of the attribute assignment unit 150 is the same as in the first embodiment, so a detailed explanation is omitted.

[0078] The information processing device 100 can assign appropriate attribute information to posts on social networking services (SNS). This is because the information processing device 100 acquires images included in posts to which attribute information is to be assigned, extracts parts containing objects, parts containing people, and parts containing the background from the images, searches for similar images posted on SNS that include parts similar to the extracted parts, acquires attribute information assigned to posts containing similar images, and assigns attribute information to posts to which attribute information is to be assigned based on this attribute information.

[0079] (Modification 2) A modification of the information processing device 100 will be described. This modification is an example in which the information processing device 100 includes images used on a website in the scope of searching for similar images. This modification may be applied to other embodiments and modifications to the extent that it does not create a contradiction.

[0080] For example, a website is a website used for selling or advertising goods. One example is an online shopping site that sells goods. Another example is a company homepage that sells goods.

[0081] For example, images used on a website are product images used on an e-commerce site. In one example, images used on a website are thumbnail images of products used on an e-commerce site. In another example, images used on a website are images showing how to use a product used on an e-commerce site.

[0082] Furthermore, for example, the images used on a website are product images used on the website of the company that sells the goods. In one example, the images used on a website are thumbnail images of products used on the company's website. In another example, the images used on a website are images showing how to use the product, used on the company's website.

[0083] The image database 50 is further a database that records images used on a website. For example, the image database 50 records data on images used on a website by a system (not shown) that manages the website.

[0084] For example, the image database 50 records images used on a website, associating them with identification information for those images. In one example, the image database 50 records images used on a website, associating them with an ID used to identify those images within the website's system.

[0085] The image database 50 may also record images used on multiple websites.

[0086] The attribute information database 60 is a database that further records attribute information assigned to images used on a website.

[0087] For example, the attribute information database 60 records attribute information assigned to images used on a website, associating it with the identification information of those images. In one example, the attribute information database 60 records attribute information assigned to images used on a website, associating it with an ID used to identify those images in the website's system.

[0088] The image acquisition unit 110 acquires images included in posts on social networking services (SNS), which are the targets to which attribute information is to be assigned. The operation of the image acquisition unit 110 is the same as in the first embodiment, so a detailed explanation is omitted.

[0089] The extraction unit 120 extracts the portion of the image that contains the object and the portion that contains the background. The operation of the extraction unit 120 is the same as in the first embodiment, so a detailed explanation is omitted.

[0090] The search unit 130 searches for similar images that contain parts similar to each of the extracted parts. For example, the search unit 130 searches for similar images that contain parts similar to each of the parts showing the item and the background extracted by the extraction unit 120.

[0091] In one example, the search unit 130 calculates the similarity of feature quantities between the portion containing the object and the portion containing the background, respectively, and the image data recorded in the image database 50. Images containing portions with a feature quantity similarity above a threshold are considered similar images. The threshold is set appropriately based on the type of object in the image, etc. Known image recognition techniques, such as image recognition using a trained model, may be used as means for searching for similar images.

[0092] The attribute information acquisition unit 140 acquires attribute information attached to posts containing similar images. For example, the attribute information acquisition unit 140 acquires attribute information attached to at least one of the posts containing similar images found by the search unit 130 and one of the images used on the website from the attribute information database 60.

[0093] In one example, the attribute information acquisition unit 140 uses the identification information of images used in posts and websites that contain similar images found by the search unit 130 to acquire attribute information attached to those images used in the posts and websites from the attribute information database 60.

[0094] The attribute assignment unit 150 assigns attribute information to posts that are the target of attribute assignment, based on the attribute information assigned to posts containing similar images. The operation of the attribute assignment unit 150 is the same as in the first embodiment, so a detailed explanation is omitted.

[0095] The information processing device 100 can assign appropriate attribute information to posts on social networking services (SNS). This is because the information processing device 100 acquires images included in posts to which attribute information is to be assigned, extracts parts containing items and parts containing backgrounds from the images, searches for similar images posted on SNS that include parts similar to the extracted parts, acquires attribute information assigned to posts containing similar images, and assigns attribute information to posts to which attribute information is to be assigned based on this attribute information.

[0096] (Second Embodiment) The second embodiment will now be described. This embodiment is an example of a case in which an information processing device outputs attribute information attached to a post. In the second embodiment, terms used in the first embodiment will have the same meaning as terms used in the first embodiment unless otherwise specified. This embodiment may be applied to other embodiments and modifications to the extent that it does not cause inconsistencies.

[0097] (Configuration Example) In the configuration shown in Figure 1, the information processing system of this embodiment includes an information processing device 200 instead of the information processing device 100.

[0098] An example of the configuration of the information processing device 200 will be explained using Figure 4. Figure 4 is a block diagram showing an example of the configuration of the information processing device 200 in this disclosure. The information processing device 200 comprises an image acquisition unit 210, an extraction unit 220, a search unit 230, an attribute information acquisition unit 240, an assignment unit 250, and an output unit 260. The image acquisition unit 210 acquires images included in posts to social networking services (SNS). The extraction unit 220 extracts parts of the image that show the object and parts that show the background. The search unit 230 searches for similar images that include parts similar to each of the extracted parts and are images posted to SNS. The attribute information acquisition unit 240 acquires attribute information assigned to posts that include similar images. The assignment unit 250 assigns attribute information to posts that include images based on the attribute information assigned to posts that include similar images. The output unit 260 outputs the attribute information assigned to the posts.

[0099] The operation of the image acquisition unit 210 is the same as that of the image acquisition unit 110 in the first embodiment, so a detailed explanation is omitted. The operation of the extraction unit 220 is the same as that of the extraction unit 120 in the first embodiment, so a detailed explanation is omitted. The operation of the search unit 230 is the same as that of the search unit 130 in the first embodiment, so a detailed explanation is omitted. The operation of the attribute information acquisition unit 240 is the same as that of the attribute information acquisition unit 140 in the first embodiment, so a detailed explanation is omitted. The operation of the assignment unit 250 is the same as that of the assignment unit 150 in the first embodiment, so a detailed explanation is omitted.

[0100] The output unit 260 outputs the attribute information that the assignment unit 250 has assigned to the post to which attribute information is to be assigned.

[0101] For example, the output unit 260 outputs to a terminal device (not shown) a display showing the attribute information assigned to the post to which attribute information is to be assigned, and a display that accepts input of information indicating whether or not it is appropriate for the attribute information to be assigned to the post. The output unit 260 may also output to the SNS service provision system 30 an instruction to the terminal device to display the attribute information assigned to the post to which attribute information is to be assigned, and to display the display that accepts input of information indicating whether or not it is appropriate for the attribute information to be assigned to the post. The output unit 260 may also output an instruction to the management device 40 to perform the above instruction.

[0102] In one example, the output unit 260 outputs an instruction to the terminal device 20 to display the attribute information attached to the post to which attribute information is to be attached, and to display an instruction to accept input of information indicating whether or not it is appropriate for the attribute information to be attached to the post. That is, the output unit 260 outputs an instruction to the terminal device of the SNS user who reacted to the post to display the attribute information attached to the post to which attribute information is to be attached, and to display an instruction to accept input of information indicating whether or not it is appropriate for the attribute information to be attached to the post.

[0103] In one example, the output unit 260 outputs instructions to the terminal device of the SNS user who made the post, indicating a display showing the attribute information assigned to the post to which attribute information is to be assigned, and a display that accepts input of information indicating whether or not it is appropriate for the attribute information to be assigned to the post.

[0104] In one example, the output unit 260 outputs instructions to the terminal device of the person managing the SNS, indicating a display showing the attribute information assigned to the post to which attribute information is to be assigned, and a display that accepts input of information indicating whether or not it is appropriate for the attribute information to be assigned to the post.

[0105] In one example, the output unit 260 outputs instructions to the terminal device of the person managing the attribute information attached to posts on the SNS, indicating a display showing the attribute information attached to the post to which the attribute information is to be attached, and a display that accepts input of information indicating whether or not it is appropriate for the attribute information to be attached to the post.

[0106] Figure 5 shows an example of a screen displayed by the terminal device based on the instructions of the output unit 260 in this disclosure. For example, the terminal device displays a string indicating attribute information attached to a post and a string prompting the user to input information indicating whether or not it is appropriate that such attribute information was attached to the post.

[0107] In one example, Figure 5 displays the attribute information "One Piece," "Pink," "Night," and "Spring" as attributes attached to the post. Additionally, the text prompting the user to input information indicating whether the attribute information was appropriate for the post is displayed as, "Please select all keywords that you think are appropriate for the content of this post." In this example, the terminal device accepts input indicating that the attribute information was appropriate for the post when the area corresponding to the attribute information displayed on the terminal device's screen is touched by the terminal device operator.

[0108] Furthermore, if the output unit 260 receives input from a terminal device indicating whether or not it is appropriate for attribute information to be attached to a post, it updates the attribute information of the post based on the input information. For example, the output unit 260 removes attribute information that the terminal device has determined to be inappropriate for being attached to the post from the attribute information of the post.

[0109] In one example, the output unit 260 deletes attribute information from the attribute information database 60 that has been assigned to a post, for which information indicating that it was inappropriate to assign the attribute information to the post has been input from the terminal device.

[0110] Furthermore, if the output unit 260 outputs to multiple terminal devices a display showing the attribute information assigned to a post to which attribute information is to be assigned, and a display accepting input of information indicating whether or not it is appropriate for the attribute information to be assigned to the post, the output unit 260 may delete attribute information that is deemed inappropriate for the post in a number of terminal devices exceeding a threshold. In one example, the output unit 260 deletes attribute information that is deemed inappropriate for the post in a number of terminal devices exceeding a predetermined proportion of the multiple terminal devices.

[0111] Furthermore, the output unit 260 may output an instruction to the terminal device to display the attribute information assigned to the post to which the attribute information is to be assigned. Alternatively, the output unit 260 may output an instruction to the SNS service provision system 30 to instruct the terminal device to display the attribute information assigned to the post to which the attribute information is to be assigned.

[0112] Figure 6 shows an example of a screen displayed by the terminal device based on the instructions of the output unit 260 in this disclosure. For example, the terminal device displays a string of characters indicating attribute information attached to the post.

[0113] In one example, Figure 6 shows the attribute information attached to the post as "One Piece," "Pink," "Night," and "Spring."

[0114] (Example of Operation) An example of the operation of the information processing device 200 will be explained below with reference to Figure 7. Figure 7 is a flowchart showing an example of the operation of the information processing device 200 in this disclosure.

[0115] First, the image acquisition unit 210 acquires images included in posts on social media, which are the targets to which attribute information will be assigned (S210).

[0116] Next, the extraction unit 220 extracts the portion of the image that contains the object and the portion that contains the background (S220).

[0117] Next, the search unit 230 searches for similar images that include parts similar to each of the extracted parts and that have been posted on social media (S230).

[0118] Next, the attribute information acquisition unit 240 acquires attribute information attached to posts containing similar images (S240).

[0119] Next, the attribute assignment unit 250 assigns attribute information to the posts to which attribute information should be assigned, based on the attribute information assigned to posts containing similar images (S250).

[0120] Next, the output unit 260 outputs the attribute information attached to the post to the terminal device (S260). Specifically, it outputs instructions to the terminal device to display the attribute information attached to the post to which the attribute information is to be attached, and to display an input indicating whether or not it is appropriate that the attribute information was attached to the post.

[0121] The information processing device 200 can assign appropriate attribute information to posts on social networking services (SNS). Furthermore, the information processing device 200 can improve the appropriateness of the attribute information assigned to the posts. This is because the information processing device 200 can assign to the posts attribute information that the person who made the post and the people who reacted to the post consider to be appropriate to be assigned to the post.

[0122] (Hardware Configuration) The hardware constituting each of the above-described information processing devices will now be explained. Figure 8 is a block diagram showing an example of the hardware configuration of a computer 300 that implements the information processing devices 100 and 200 according to this disclosure. The computer 300 includes a processor 310, a memory 320, an auxiliary storage device 330, a communication interface 340, and an input / output interface 350. The memory 320 includes a RAM (Random Access Memory) 321 and a ROM (Read Only Memory) 322. The computer 300 implements the procedures shown in each of the above embodiments by executing a program 400 that realizes the functions of the information processing devices 100 and 200.

[0123] In one example, the processor 310 executes the program 400 using the RAM 321.

[0124] In one example, the processor 310 is at least one of a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit). The processor 310 may be any other type of processor, and is not limited to those exemplified here.

[0125] Here, program 400 is a program that causes the computer to function as a part of the apparatus shown in each of the embodiments described above.

[0126] The program 400 may be recorded in the ROM 322 or stored in the auxiliary storage device 330.

[0127] The communication interface 340 communicates with external devices.

[0128] The input / output interface 350 is connected to a device that receives information input to the information processing device and a device that outputs information from the information processing device.

[0129] The functions of each part of the device shown in each of the above embodiments are realized by the processor 310 executing the program 400. Alternatively, the processor 310 may realize the functions of each part of the device shown in each of the above embodiments by reading programs and data stored on a recording medium mounted on a drive device or the like into RAM.

[0130] The processing method for recording a program to realize each configuration in the function of each embodiment on a recording medium, reading the program recorded on the recording medium, and executing it on a computer is also included in the scope of each embodiment. Furthermore, the recording medium on which the above program is recorded, which is readable by a computer, and the program itself are also included in the scope of each embodiment.

[0131] The configuration described as a single device in the embodiments described above may be distributed and provided across multiple devices. For example, the information processing devices 100 and 200 may be implemented by multiple computer devices using cloud computing technology or the like.

[0132] The embodiments and modifications described above have been presented as exemplary examples. However, the disclosure is not limited to these embodiments and modifications. Within the scope of the disclosure, the disclosure may include embodiments that apply various modifications or applications that are so-called to those skilled in the art. The disclosure may also include embodiments that combine or substitute the matters described herein as appropriate. For example, matters described using a particular embodiment may be applied to other embodiments to the extent that they do not cause a contradiction.

[0133] (Note) All or part of this disclosure may also be described as follows. However, this disclosure is not necessarily limited to the form described in this note. (Note 1) An information processing device comprising: an image acquisition means for acquiring an image included in a post on an SNS; an extraction means for extracting a portion containing an article and a portion containing a background from the image; a search means for searching for similar images that include portions similar to each of the extracted portions and are images posted on an SNS; an attribute information acquisition means for acquiring attribute information attached to a post containing the similar image; and an attachment means for attaching attribute information to a post on an SNS based on the attribute information. (Note 2) The information processing device according to Note 1, further comprising an output means for outputting attribute information attached to a post on an SNS to a terminal device. (Note 3) The information processing device according to Note 2, wherein the output means further outputs a display to the terminal device indicating whether or not it is appropriate for the attribute information attached to the post on the SNS to be attached to the post on the SNS, and deletes the attribute information attached to the post on the SNS from the terminal device for which it has received input indicating that it is not appropriate for the attribute information attached to the post on the SNS. (Note 4) The information processing device according to Note 3, wherein when the output means outputs a display to multiple terminal devices indicating whether or not it is appropriate for the attribute information attached to the post on the SNS to be attached to the post on the SNS to be attached to the post on the SNS, it deletes the attribute information for which it has received input indicating that it is not appropriate for the attribute information attached to the post on the SNS from a number of terminal devices equal to or greater than a threshold. (Note 5) The information processing device according to any one of Notes 1 to 4, wherein the extraction means further extracts the portion in which a person is pictured from the image. (Note 6) The extraction means is an information processing device according to Note 5, further extracting a portion of a person's body. (Note 7) The attribute information acquisition means is an information processing device according to any one of Notes 1 to 6, acquiring a hashtag associated with a post containing the similar image as attribute information.(Note 8) The information processing device according to Notes 1 to 7, wherein the assignment means, when there are multiple similar images, assigns some of the attribute information assigned to multiple posts containing the similar images to the posts on the SNS. (Note 9) The information processing device according to Note 8, wherein the assignment means assigns common attribute information assigned to multiple posts containing the similar images to the posts on the SNS. (Note 10) The information processing device according to Note 9, wherein the assignment means assigns attribute information assigned to a number of posts exceeding a threshold among multiple posts containing the similar images to the posts on the SNS. (Note 11) The information processing device according to Note 8, wherein the assignment means assigns attribute information assigned to multiple posts containing the similar images to the posts on the SNS according to the degree of similarity between the images included in the posts on the SNS and the similar images. (Note 12) The information processing device according to Note 11, wherein the assignment means assigns attribute information to posts containing similar images whose similarity to an image included in a post on the SNS is greater than or equal to a threshold. (Note 13) The information processing device according to Note 12, wherein the assignment means assigns common attribute information, which has been assigned to multiple posts containing similar images, from among the attribute information assigned to posts containing similar images whose similarity to an image included in a post on the SNS is less than a threshold. (Note 14) The information processing device according to Note 13, wherein the assignment means assigns attribute information, which has been assigned to posts containing similar images whose similarity to an image included in a post on the SNS is less than a threshold, to posts whose attribute information has been assigned to a predetermined number or more. (Note 15) The information processing device according to any one of Notes 2 to 4, wherein the terminal device is the terminal device of an SNS user who has reacted to a post on the SNS. (Note 16) The terminal device is the information processing device described in any of Notes 2 to 4, which is the terminal device of the SNS user who made the post on the SNS. (Note 17) The terminal device is the information processing device described in any of Notes 2 to 4, which is the terminal device of the person who manages the SNS.(Note 18) The terminal device is an information processing device as described in any of Notes 2 to 4, which is a terminal device of a person who manages attribute information attached to posts on the SNS. (Note 19) An information processing method for acquiring an image included in a post on an SNS, extracting parts containing an item and parts containing a background from the image, searching for similar images that include parts similar to each of the extracted parts and are posted on the SNS, acquiring attribute information attached to posts containing the similar images, and attaching attribute information to posts on the SNS based on the attribute information. (Note 20) A recording medium that records a program that causes a computer to execute the process of acquiring an image included in a post on an SNS, extracting parts containing an item and parts containing a background from the image, searching for similar images that include parts similar to each of the extracted parts and are posted on the SNS, acquiring attribute information attached to posts containing the similar images, and attaching attribute information to posts on the SNS based on the attribute information.

[0134] Furthermore, some or all of the configurations described in Appendices 2 to 18, which are dependent on Appendice 1 above, may also be dependent on Appendices 19 and 20 in the same way as those described in Appendices 2 to 18. Moreover, not limited to Appendices 1, 19, and 20, some or all of the configurations described as appendices may also be dependent on various hardware, software, various recording means for recording software, or systems, without departing from the embodiments described above.

[0135] 10 Information processing system 20 Terminal device 30 SNS service provision system 40 Management device 50 Image database 60 Attribute information database 100 Information processing device 110 Image acquisition unit 120 Extraction unit 130 Search unit 140 Attribute information acquisition unit 150 Assignment unit 200 Information processing device 210 Image acquisition unit 220 Extraction unit 230 Search unit 240 Attribute information acquisition unit 250 Assignment unit 260 Output unit 300 Computer 310 Processor 320 Memory 321 RAM 322 ROM 330 Auxiliary storage device 340 Communication interface 350 Input / output interface 400 Program

Claims

1. An information processing device comprising: an image acquisition means for acquiring images included in posts on social networking services (SNS); an extraction means for extracting portions containing items and portions containing backgrounds from the images; a search means for searching for similar images that include portions similar to each of the extracted portions and that are posted on SNS; an attribute information acquisition means for acquiring attribute information attached to posts containing the similar images; and an attachment means for attaching attribute information to posts on SNS based on the attribute information.

2. The information processing apparatus according to claim 1, further comprising output means for outputting attribute information attached to posts on the SNS to a terminal device.

3. The information processing apparatus according to claim 2, wherein the output means further outputs a display to the terminal device indicating whether or not it is appropriate for the attribute information attached to the post on the SNS to be attached to the post on the SNS to be attached to the terminal device, and deletes the attribute information attached to the post on the SNS for which the terminal device has received input indicating that it is not appropriate for the attribute information attached to the post on the SNS to be attached to the post on the SNS.

4. The information processing apparatus according to claim 3, wherein the output means outputs a display to multiple terminal devices indicating whether or not it is appropriate for attribute information attached to a post on the SNS to be attached to the SNS, and in a number of terminal devices exceeding a threshold, the attribute information for which input indicating that it is inappropriate for the attribute information attached to the post on the SNS has been received is deleted.

5. The information processing apparatus according to claim 1 or 2, wherein the extraction means further extracts a portion containing a person from the image.

6. The information processing apparatus according to claim 5, wherein the extraction means further extracts a portion of the image showing a part of a person's body.

7. The information processing apparatus according to claim 1 or 2, wherein the attribute information acquisition means acquires hashtags associated with posts containing similar images as attribute information.

8. The information processing apparatus according to claim 1 or 2, wherein, if there are multiple similar images, the assigning means assigns some of the attribute information assigned to multiple posts containing the similar images to the posts on the SNS.

9. The information processing apparatus according to claim 8, wherein the assignment means assigns common attribute information assigned to a plurality of posts including the similar images to a post on the SNS.

10. The information processing apparatus according to claim 9, wherein the assignment means assigns attribute information assigned to a number of posts, including the similar image, that is greater than or equal to a threshold, to the posts on the SNS.

11. The information processing apparatus according to claim 8, wherein the assigning means assigns attribute information assigned to a plurality of posts containing the similar image to the posts on the SNS, according to the degree of similarity between the image contained in the posts on the SNS and the similar image.

12. The information processing apparatus according to claim 11, wherein the assigning means assigns attribute information assigned to posts containing similar images whose similarity to an image included in a post on the SNS is equal to or greater than a threshold, to the post on the SNS.

13. The information processing apparatus according to claim 12, wherein the assignment means assigns to a post on the SNS common attribute information that has been assigned to a plurality of posts containing the similar image, from among the attribute information assigned to posts containing similar images whose similarity to the image included in the post on the SNS is less than a threshold.

14. The information processing apparatus according to claim 13, wherein the assignment means assigns attribute information to posts that include similar images whose similarity to an image included in a post on the SNS is less than a threshold, and which has been assigned to a predetermined number or more of such posts.

15. The information processing device according to claim 2 or 3, wherein the terminal device is the terminal device of an SNS user who reacted to a post on the SNS.

16. The information processing apparatus according to claim 2 or 3, wherein the terminal device is the terminal device of the SNS user who made the post on the SNS.

17. The information processing apparatus according to claim 2 or 3, wherein the terminal device is the terminal device of the person managing the SNS.

18. The information processing apparatus according to claim 2 or 3, wherein the terminal device is a terminal device of a person who manages attribute information attached to posts on the SNS.

19. An information processing method comprising: obtaining an image contained in a post on social media; extracting parts containing an item and parts containing a background from the image; searching for similar images posted on social media that include parts similar to each of the extracted parts; obtaining attribute information attached to the post containing the similar image; and attaching attribute information to the post on social media based on the attribute information.

20. A recording medium that records a program that causes a computer to perform the following processes: acquire images contained in posts on social networking services (SNS); extract portions containing items and portions containing backgrounds from the images; search for similar images that include portions similar to the extracted portions and that are posted on SNS; acquire attribute information attached to posts containing the similar images; and attach attribute information to the posts on SNS based on the attribute information.

Images