Information processing apparatus, information processing method, and information processing program

Abstract

To provide thumbnail images reflecting intent of a search by a user.SOLUTION: An information processing device of the present invention comprises a search unit, a selection unit, and a provision unit. The search unit searches for a product based on a search query specified by a user. The selection unit selects a candidate image most relevant to the search query among multiple candidate images associated with search targets included in a result of the search by the search unit from each search target. The provision unit provides the candidate images selected by the selection unit in the form of thumbnail images of the search targets.SELECTED DRAWING: Figure 5

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing method, and an information processing program.

Background Art

[0002] Conventionally, techniques related to image search using a VSE (Visual-Semantic Embedding) model have been disclosed. For example, in image search using a VSE model, a specified search keyword and an image to be searched are each converted into a feature vector, and an image corresponding to the similarity between the two feature vectors is output as a search result (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the prior art, only images whose search keywords and feature vectors are similar are searched, and consideration has not been given to providing thumbnail images that reflect the search intention of the user.

[0005] The present invention has been made in view of the above, and an object thereof is to provide an information processing apparatus, an information processing method, and an information processing program capable of providing a thumbnail image that reflects the search intention of the user.

Means for Solving the Problems

[0006] In order to solve the above-described problems and achieve the object, an information processing apparatus according to the present invention includes a search unit that searches for a search target based on a search query specified by a user, a selection unit that selects, from among a plurality of candidate images associated with the search target included in the search result by the search unit, the candidate image having the highest relevance to the search query for each of the search targets, and a provision unit that provides the candidate image selected by the selection unit as a thumbnail image of the search target.

Effects of the Invention

[0007] According to the present invention, it is possible to provide a thumbnail image that reflects the search intention of the user.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

[0009] Hereinafter, a mode (hereinafter referred to as "embodiment") for implementing the information processing apparatus, information processing method, and information processing program according to the present application will be described in detail with reference to the drawings. Note that the information processing apparatus, information processing method, and information processing program according to the present application are not limited by this embodiment. Further, hereinafter, the case where the search target is a product handled in an electronic shopping street will be described.

[0010] [Embodiment] [1.1 System] First, the provision system shown in FIG. 1 will be described. FIG. 1 is a diagram showing a configuration example of the provision system according to the embodiment. As shown in FIG. 1, the provision system includes an information processing apparatus 1 and a user terminal 10. The information processing apparatus 1 and the user terminal 10 are communicably connected by wire or wirelessly via a predetermined communication network (network N).

[0011] The information processing apparatus 1 provides an e-commerce service that provides (searches, sells, etc.) products including clothing (also referred to as fashion items, including clothes, underwear, footwear (also referred to as shoes), hats (such as caps and hats), bags, ornaments (also referred to as accessories), etc.), cosmetics (also referred to as fashion items), etc. Further, the information processing apparatus 1 provides a coordination service that accepts submissions from users of contents (images, videos, articles, etc.) showing coordination using a plurality of pieces of clothing, contents showing makeup using one or more cosmetics, etc., and provides (searches, distributes, etc.) them to other users. For example, the information processing apparatus 1 can be realized by a server apparatus, a cloud system, or the like.

[0012] In addition, as will be described later, the information processing apparatus 1 provides various services to the user by using the pre-learned distributed representation space. In the distributed representation space according to the embodiment, an image and a keyword (including tags and the like) are projected, and the proximity between an image and an image, an image and a keyword, and a keyword and a keyword is projected. Also, the closer the two are semantically, the closer they are learned to be on the distributed representation space (the reverse also holds). That is, the similarity between an image and an image, an image and a keyword, and a keyword and a keyword can be measured as the distance on the distributed representation space. Note that the proximity on the distributed representation space according to the embodiment includes, for example, proximity based on Euclidean distance and proximity based on cosine distance.

[0013] In addition, the information processing apparatus 1 may have a function as a web server that provides a website related to the service. Also, the information processing apparatus 1 may be a device that distributes information to be displayed on applications related to various services installed on the user terminal 10 to the user terminal 10. Also, the information processing apparatus 1 may be a device that distributes the application data itself.

[0014] In addition, the information processing apparatus 1 may function as a distribution device that distributes control information to the user terminal 10. Here, the control information is described, for example, by a script language such as JavaScript (registered trademark) or a style sheet language such as CSS (Cascading Style Sheets). Note that the application itself distributed from the information processing apparatus 1 may be regarded as control information.

[0015] The user terminal 10 is an information processing apparatus used by the user. The user terminal 10 is realized, for example, by a smartphone, a tablet-type terminal, a notebook PC (Personal Computer), a desktop PC, a mobile phone, a PDA (Personal Digital Assistant), or the like. Also, the user terminal 10 displays information distributed by the information processing apparatus 1, a server apparatus that provides a predetermined service, or the like on a web browser or an application.

[0016] [[1.2 Information Processing]] Next, with reference to FIG. 2, the information processing according to the embodiment will be described. FIG. 2 is a diagram showing an example of the information processing according to the embodiment. As shown in FIG. 2, the information processing apparatus 1 functions as a web server that provides information regarding fashion items, and first, receives a specification of a search query designated by a user from the user terminal 10 (step S1).

[0017] The information processing apparatus 1 searches for products based on such a search query (step S2). For example, products tagged with keywords or the like corresponding to the search query are output as search results.

[0018] Subsequently, the information processing apparatus 1 selects a thumbnail image for each product included in the search results (step S3). More specifically, the information processing apparatus 1 selects, from among a plurality of candidate images associated with each product, the candidate image having the highest relevance to the search query for each product included in the search results.

[0019] For example, the information processing apparatus 1 selects a thumbnail image of each product based on the similarity in the distributed representation space between the search query (keyword) and each candidate image. More specifically, the search query indicates the search intention of the user, and a thumbnail image clicked (viewed) by the user who specified the search query (it may also be a thumbnail image that led to a purchase) can be regarded as an image that more reflects the search intention of the user. For example, the information processing apparatus 1 can project each search query, candidate image, etc. into the distributed representation space by using techniques related to the distributed representation space, such as VSE (Visual-Semantic Embedding) and CLIP (Contrastive Language-Image Pre-training).

[0020] Therefore, for example, the information processing apparatus 1 can use the search query specified by the user and the thumbnail image clicked by the user as learning data, and thus select a thumbnail image that has a high degree of relevance to the search query, that is, reflects the user's search intention. Further, for example, the information processing apparatus 1 may use the search query specified by the user and the thumbnail image not clicked by the user as negative example learning data. In the present embodiment, in addition to the search query and the candidate images, a thumbnail image is selected from the candidate images based on the attributes of the user who specified the search query.

[0021] Here, the distributed representation space according to the embodiment will be described with reference to FIG. 3. FIG. 3 is an explanatory diagram of the distributed representation space according to the embodiment. Here, the process of selecting a thumbnail image from a plurality of candidate images associated with one product will be described.

[0022] As shown in FIG. 3, first, the information processing apparatus 1 embeds a plurality of candidate images associated with a product into the distributed representation space (step S11), and embeds the search query and the user attributes into the distributed representation space, respectively (steps S12 and S13). Note that the user attributes include information regarding, for example, the user's age, gender, body type, foot type, skin color, residence, click logs, search logs (search histories), purchase logs (purchase histories), and browsing logs (browsing histories) by the user. Further, the user attributes may include information regarding, for example, the age, gender, body type, foot type, skin color, place of residence, click logs, search logs, purchase logs, and browsing logs of a user who has a predetermined relationship (including a parent-child relationship, etc.) with the user.

[0023] In the example shown in FIG. 3, the feature vectors of the respective candidate images are indicated as "image vectors Vi1 to Vi5", the feature vector of the search query is indicated as "query vector Vq", and the feature vector of the user attributes is indicated as "user vector Vu". Hereinafter, when the image vectors Vi1 to Vi5 are not distinguished, they will be denoted as "image vector Vi".

[0024] For example, the information processing apparatus 1 creates a ranking according to the positional relationship between the query vector Vq and the user vector Vu among the image vectors Vi1 to Vi5. For example, the information processing apparatus 1 creates a ranking in order from the image vector Vi with the smallest area of the space (triangle) formed by connecting each of the image vectors Vi1 to Vi5 and the query vector Vq and the user vector Vu. Note that the information processing apparatus 1 may generate a ranking in order of similarity to either the query vector Vq or the user vector Vu among the image vectors Vi1 to Vi5.

[0025] Then, the information processing apparatus 1 selects the candidate image ranked first in the ranking as the thumbnail image. That is, the information processing apparatus 1 selects, as the thumbnail image, the candidate image corresponding to the image vector Vi that is similar to the query vector Vq and the user vector Vu among each of the image vectors Vi.

[0026] That is, the information processing apparatus 1 selects, as the thumbnail image, a candidate image corresponding to each candidate image associated with a product according to the search query and user attributes. More specifically, among the image vectors Vi corresponding to each candidate image, the image vector Vi that is similar to the query vector Vq corresponding to the search query and the user vector Vu corresponding to the user attributes is selected and selected as the thumbnail image.

[0027] Thereby, the information processing apparatus 1 can select a thumbnail image that reflects the search intention of the user from a plurality of candidate images associated with the product. In the example shown in FIG. 3, it shows that the area of the space connecting the image vector Vi4 and the query vector Vq and the user vector Vu is the smallest, and it shows that the candidate image corresponding to the image vector Vi4 is selected as the thumbnail image.

[0028] When the information processing apparatus 1 selects a thumbnail image from each candidate image for each product included in the search result in step S2, as shown in FIG. 2, it provides the selected thumbnail image to the user terminal 10 (step S4).

[0029] In addition, for example, when there is no image vector Vi that satisfies the selection condition of the thumbnail image among the respective image vectors Vi in the distributed representation space, the information processing apparatus 1 can also generate a thumbnail image for the product. That is, for a product for which there is no appropriate thumbnail image, the information processing apparatus 1 newly generates a thumbnail image based on the search query. Note that the information processing apparatus 1 may register a plurality of candidate images generated in advance, and when there is no thumbnail image that satisfies the selection condition, select and provide a thumbnail image from the candidate images registered in advance.

[0030] FIG. 4 is a schematic diagram of the thumbnail image generation process according to the embodiment. For example, the information processing apparatus 1 generates a composite image Ic, which is a thumbnail image obtained by combining and synthesizing a product image It of a product for which there is no appropriate thumbnail image and a synthesis image Is corresponding to the search query.

[0031] For example, the synthesis image Is includes candidate images associated with other products and posted images (for example, coordination images) posted by each user to a coordination service or the like. The posted images include photos in which fashion items being worn by each user are tagged.

[0032] For example, the information processing apparatus 1 extracts each synthesis image Is from candidate images associated with other products and posted images posted by each user based on the search query. For example, the information processing apparatus 1 extracts the synthesis image Is from each image according to the similarity between the image vector Vi of each image in the distributed representation space and the query vector Vq of the search query.

[0033] Subsequently, the information processing apparatus 1 synthesizes a composite image Ic obtained by combining the product image It and the synthesis image Is using various image generation AIs.

[0034] At this time, for example, the information processing apparatus 1 repeatedly executes generation of the composite image Ic until the area of the space connecting the image vector Vi of the composite image Ic, the query vector Vq of the search query, and the user vector Vu of the user attributes becomes equal to or less than a threshold value, and generates a composite image Ic that conforms to the search query specified by the user.

[0035] In this way, for products without a thumbnail image corresponding to the search query, the information processing apparatus 1 automatically generates a thumbnail image. Thereby, the information processing apparatus 1 can provide an appropriate thumbnail image corresponding to the search query input by the user.

[0036] 〔2. Information Processing Apparatus〕 Next, with reference to FIG. 5, a configuration example of the information processing apparatus 1 according to the embodiment will be described. FIG. 5 is a block diagram showing a configuration example of the information processing apparatus 1 according to the embodiment. As shown in FIG. 5, the information processing apparatus 1 includes a communication unit 2, a storage unit 3, and a control unit 4. Note that the information processing apparatus 1 may have an input unit (for example, a keyboard or a mouse) that receives various operations from an administrator or the like who uses the information processing apparatus 1, and a display unit (for example, a liquid crystal display) that displays various information.

[0037] The communication unit 2 is realized by, for example, a NIC (Network Interface Card). The communication unit 2 is connected to a communication network such as 4G (4th Generation) or 5G (5th Generation) by wire or wirelessly, and transmits and receives information to and from each of the user terminals 10 and the like via the communication network.

[0038] The storage unit 3 is realized by, for example, a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, or a storage device such as a hard disk or an optical disk. The storage unit 3 includes a user information storage unit 31, a product information storage unit 32, and a model storage unit 33.

[0039] The user information storage unit 31 stores information about each user. FIG. 6 is a diagram showing an example of the information stored in the user information storage unit 31 according to the embodiment. As shown in FIG. 6, the user information storage unit 31 stores information of items such as "user ID" and "user information" in association with each other.

[0040] "User ID" indicates identification information for identifying a user. "User information" indicates user information. In the example shown in FIG. 6, an example in which conceptual information such as "user information #1" and "user information #2" is stored in "user information" is shown. Actually, information on user attributes such as age and gender, identification information of products (items) owned by the user, etc. are stored.

[0041] The product information storage unit 32 stores product information. The product information storage unit 32 is information about products handled in the e-commerce street provided by the information processing apparatus 1. FIG. 7 is a diagram showing an example of the information stored in the product information storage unit 32 according to the embodiment.

[0042] As shown in FIG. 7, the product information storage unit 32 stores information of items such as "product ID" and "image information" in association with each other. "Product ID" indicates identification information for identifying a product. "Image information" is image information of the product identified by the corresponding product ID. In the present embodiment, a plurality of candidate images are stored as image information for each product.

[0043] The model storage unit 33 stores models. For example, the model is a learning model that projects each image, search query (keyword), and user information into the same distributed representation space. For example, the learning model is a model learned by a learning unit 42 described later, and for example, a VSE model can be adopted as the learning model.

[0044] Next, the control unit 4 will be described. The control unit 4 is a controller, and for example, it is realized by various programs (corresponding to an example of an information processing program) stored in a storage device inside the information processing apparatus 1 being executed with the RAM as a work area by a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or the like. Further, the control unit 4 is, for example, a controller and is realized by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0045] As shown in FIG. 5, the control unit 4 includes an acquisition unit 41, a learning unit 42, a search unit 43, a selection unit 44, a generation unit 45, and a provision unit 46, and realizes or executes the functions and operations of information processing described below. Note that the internal configuration of the control unit 4 is not limited to the configuration shown in FIG. 5, and any other configuration may be used as long as it can perform the information processing described later. Also, the connection relationship between the respective processing units included in the control unit 4 is not limited to the connection relationship shown in FIG. 5, and other connection relationships may be used.

[0046] The acquisition unit 41 acquires various types of information. For example, the acquisition unit 41 acquires user information from the user terminal 10 and registers it in the user information storage unit 31. Further, the acquisition unit 41 acquires product information from the terminal of a store owner (not shown) in an electronic shopping street and registers it in the product information storage unit 32.

[0047] Also, the acquisition unit 41 accepts submissions from users such as content (images, videos, articles, etc.) showing coordinates using a plurality of pieces of clothing and content showing makeup using one or more cosmetics. For example, such content is provided (searched for, distributed, etc.) to other users through a platform such as an SNS. Note that such a platform may be a platform provided by the information processing apparatus 1.

[0048] The learning unit 42 performs learning of the model (learning model) stored in the model storage unit 33. First, the method of collecting learning data will be described. FIG. 8 is a diagram showing an example of the method of collecting learning data according to the embodiment.

[0049] As shown in FIG. 8, for example, the information processing apparatus 1 provides an annotation application to the user terminal 10 and acquires learning data of the learning model through the annotation application. For example, as shown in FIG. 8, in the annotation application, a search query to be learned and a plurality of images are displayed.

[0050] Then, the user selects, through the annotation application, an image that matches the search query, and the information processing apparatus 1 acquires learning data in which the search query is associated with the image selected by the user. In the example shown in FIG. 8, the search query is "golf", and the user will select an image that associates with "golf". Note that the search query may be "golf knit sweater", and the user may select an image that associates with "golf knit sweater".

[0051] The information processing apparatus 1 can collect learning data of the image corresponding to each search query by having the user select, through the annotation application, the image corresponding to each search query.

[0052] Note that the annotation application is not limited to the example shown in FIG. 8. The number of images to be displayed simultaneously may be two, the arrangement of a plurality of images may be changed according to the user, the arrangement of a plurality of images may be displayed in a circular shape, or the arrangement of a plurality of images may be displayed while constantly moving. Thereby, the bias that makes it easier to be selected depending on the arrangement of the images can be removed.

[0053] In addition to the above example, the information processing apparatus 1 may, for example, collect learning data from the log information in the e-commerce street of each user. FIG. 9 is a diagram showing an example of learning data according to the embodiment.

[0054] As shown in FIG. 9, the training data has items such as "user", "search query", "clicked image", and "unclicked image". "User" indicates the attributes of the user. "Search query" indicates the search query specified by the corresponding user. Note that the training data may also include information such as the time taken for the user to select an image, the images displayed simultaneously, and the position (display position) of the image selected by the user.

[0055] The "clicked image" is an image of a product included in the search results for the corresponding search query by the corresponding user, and indicates the clicked image (for example, a thumbnail image), and the "unclicked image" is an image of a product included in the search results for the corresponding search query by the corresponding user, and indicates the unclicked image.

[0056] That is, in this case, it becomes possible to collect training data based on the logs of each user in the e-commerce street. After collecting these training data, the learning unit 42 learns the learning model based on the collected training data. Specifically, the learning unit 42 performs learning of the learning model so that the feature vector of the search query and the feature vector of the image associated with the search query are similar feature vectors in the distributed representation space.

[0057] More specifically, the learning unit 42 performs learning of the learning model so that similar images have similar feature vectors, similar search queries (keywords) have similar feature vectors, and the search query serving as the training data and the image associated with the search query have similar feature vectors.

[0058] In addition, the learning unit 42 performs learning of a learning model that can project the user attributes of each user into the distributed representation space. For example, the user attributes include so-called demographic attributes such as the user's age, gender, and residence, and the user logs in the e-commerce street, and the user logs include search history, browsing history, purchase history, and the like.

[0059] In addition, the learning unit 42 generates a pseudo-ranking of thumbnail images representing search intentions by a method such as Serial Rank, and evaluates the learning model. Further, the learning unit 42 learns the learning model so that similar user attributes become similar feature vectors in the distributed representation space. Note that the learning data is not limited to the above example, and may include the logs of each user in the coordination service. The user logs in the coordination service include browsing history, search history, posting history, purchase history of products through the coordination service, and the like.

[0060] The search unit 43 searches for a search target (product) based on a search query specified by the user. Specifically, the search unit 43 acquires the user ID and the search query from the user terminal 10, and searches the product information storage unit 32 for products that match the search query specified by the user. Then, the search unit 43 passes the user ID and the product information included in the search result to the selection unit 44.

[0061] The selection unit 44 selects, from among a plurality of candidate images associated with the products included in the search result by the search unit 43, the candidate image having the highest relevance to the search query for each product. Specifically, the selection unit 44 projects the candidate image, the user attribute, and the search query for each product into the distributed representation space using the learning model, and calculates the area of the space connecting each image vector Vi, user vector Vu, and query vector Vq in the distributed representation space (see Figure 2).

[0062] For example, the selection unit 44 selects, as the thumbnail image, the candidate image corresponding to the image vector Vi having the smallest such area. At this time, if the area of the space connecting the image vector Vi, the user vector Vu, and the query vector Vq is equal to or greater than a threshold value, that is, when there is no thumbnail image corresponding to the search query, the selection unit 44 passes the information about such a product to the generation unit 45 and instructs the generation of the thumbnail image.

[0063] The generation unit 45 generates thumbnail images of products included in the search results by the search unit 43 based on the relevance between the search query and the images. The generation unit 45 generates thumbnail images for products for which the generation of thumbnail images is instructed by the selection unit 44, that is, products for which a plurality of candidate images associated with the products do not satisfy the selection conditions for the thumbnail images.

[0064] For example, the generation unit 45 generates a coordination image including the products included in the search results by the search unit 43 as a thumbnail image. The coordination image is an image using a plurality of clothing items. For example, the generation unit 45 generates a coordination image by combining a plurality of candidate images associated with each product corresponding to each category such as tops, bottoms, and shoes.

[0065] First, the generation unit 45 extracts a composite image based on the search query. For example, the generation unit 45 projects product images of products handled in the e-commerce street and posted images posted by users into a dispersion representation space using a learning model, and extracts an image similar to the query vector Vq of the search query in the dispersion representation space as the composite image.

[0066] The generation unit 45 extracts, for example, an image similar to the search query and suitable for the product in the dispersion representation space based on the product image It that is the target of generating the thumbnail image and the search query as the composite image. For example, the generation unit 45 extracts composite images of a plurality of different patterns according to the skeleton, style, season, etc. of the model shown in the posted image. Note that the generation unit 45 may extract composite images of a plurality of different patterns according to the compatibility between the product (for example, tops, etc.) and the model shown in the posted image, the compatibility with other products (for example, skirts, etc.) worn by the model shown in the posted image, and the compatibility with the background shown in the posted image.

[0067] When the generation unit 45 extracts a composite image from a posted image posted by a user, for example, a posted image including products of brands not handled in the e-commerce street may be excluded from the target of the composite image.

[0068] For example, the posted images are tagged with respect to the products (items) each user is wearing, and the generation unit 45 can determine posted images including products not handled in the e-commerce street based on such tags.

[0069] Note that the generation unit 45 may overwrite products not handled in the e-commerce street to generate a composite image when generating a composite image using, for example, the composite image without excluding all posted images from the target of the composite image. That is, by not posting products of brands not handled in the e-commerce street in the thumbnail image, for example, infringement of copyright can be suppressed.

[0070] Note that the generation unit 45 may prepare images corresponding to each search query in advance and extract (select) a composite image from such images.

[0071] After finishing the extraction of the composite image, the generation unit 45 generates a composite image from the product image It of the target product and the composite image using the image generation AI. For example, the generation unit 45 arranges each product image (each item image) on the composite image to generate a collage image, and then generates a natural thumbnail image from the collage image. At this time, the generation unit 45 may extract cutout images such as used items and backgrounds from each composite image and then generate a thumbnail image.

[0072] In addition, the generation unit 45 may change the instruction to the image generation AI according to the user or the search query. For example, when the user is male, the generation unit 45 instructs the image generation AI to generate an image with a male as a model.

[0073] Also, when the user who is the search source is male and the search query is "female, dress, fashionable", assume that the male is searching for a present for a female. In this case, the generation unit 45 instructs the image generation AI to generate an image of a female as the model based on "female" in the search query. Regarding which to prioritize, the user or the search query, for example, it may be determined in advance based on rules.

[0074] Also, when generating the composite image, the generation unit 45 may, for example, include products purchased by the user in the past in the composite image. The products purchased by the user in the past can be specified, for example, from the user's purchase history in the e-commerce street, but may also be specified from the posted images (coordination images) posted by the user.

[0075] FIG. 10 is a diagram showing an example of a composite image according to the embodiment. For example, as shown in FIG. 10, the generation unit 45 extracts product images of products purchased by the user in the past from the user information storage unit 31, and generates a composite image including the extracted product images. At this time, the generation unit 45 may generate a composite image including the product images of the products selected from the user's logs. In this case, the generation unit 45 may extract product images related to products registered as favorites by the user, products viewed in the past, or products purchased by users similar to the target user. In the example shown in FIG. 10, the products purchased by the user in the past are a bag and a necklace. The generation unit 45 further uses the product image Ib corresponding to the bag and the product image In corresponding to the necklace as composite images, and generates a composite image Ic.

[0076] That is, in this case, the composite image Ic is a composite image Ic coordinated with the bag and necklace purchased by the user in the past for the product corresponding to the search query specified by the user, in other words, the product image (a dress in FIG. 10) that matches the user's search intention.

[0077] That is, in the composite image Ic, it is possible to present a sample to the user of how the dress that is the target of this sale would look when worn together with the bag and necklace that the user currently owns.

[0078] As described above, since the generation unit 45 can generate the composite image Ic that is individually optimized for the user, an improvement in the user's purchasing desire is expected. Note that, for example, the generation unit 45 may generate the composite image Ic by combining the products specified by the user. In this case, the user may select products from the products they have purchased in the past, or for example, may select an image of a product photographed by the user.

[0079] The providing unit 46 provides the thumbnail image selected by the selection unit 44 to the user. At this time, for products for which the thumbnail image has not been selected by the selection unit 44, the providing unit 46 provides the composite image Ic generated by the generation unit 45 as the thumbnail image.

[0080] Thereby, the providing unit 46 can provide a thumbnail image that reflects the search intention of the user.

[0081] 〔3. Processing Flow〕 Next, with reference to FIG. 11, the processing procedure executed by the information processing apparatus 1 according to the embodiment will be described. FIG. 11 is a flowchart showing an example of the processing procedure of the information processing according to the embodiment. The following processing is repeatedly executed by the information processing apparatus 1 at a predetermined cycle every time a search query is acquired.

[0082] As shown in FIG. 11, the information processing apparatus 1 receives a search query from the user (step S101). Subsequently, the information processing apparatus 1 searches for products that match the search query (step S102).

[0083] Subsequently, the information processing apparatus 1 embeds a plurality of candidate images associated with the products included in the search results, the search query, and the user attributes into the distributed representation space (step S103). Subsequently, the information processing apparatus 1 selects a thumbnail image from among the candidate images based on each feature vector in the distributed representation space (step S104).

[0084] Subsequently, the information processing apparatus 1 determines whether there is a product for which the candidate image does not satisfy the selection condition (step S105). If there is a product that does not satisfy the selection condition (step S105: Yes), a thumbnail image of the product is generated (step S106).

[0085] Also, when thumbnail images can be selected from all the products (step S105; No), the information processing apparatus 1 proceeds to the process of step S107. Then, the information processing apparatus 1 provides the thumbnail images to the user (step S107) and ends the process.

[0086] [4. Modification Example] In the above-described embodiment, the case where the information processing apparatus 1 provides a thumbnail image related to a product has been described, but the present invention is not limited thereto. For example, the present invention may be applied when providing thumbnail images of various web services such as SNSs and video sites. Also, when the information processing apparatus 1 provides a thumbnail image related to coordination in a coordination service, the present invention may be provided. For example, the selection unit may select, from among a plurality of candidate images (posted images) associated with the coordination included in the search results by the search unit, the candidate image having the highest degree of relevance to the search query from each coordination and provide it as a thumbnail image.

[0087] In the above-described embodiment, it was explained that the providing unit 46 provides the composite image Ic generated by the generating unit 45 as a thumbnail image for products for which thumbnail images are not selected by the selecting unit 44. However, the present invention is not limited to this. Even when there is no candidate image reflecting the search intention of the user among a plurality of candidate images associated with a product, if there is a posted image showing a coordination in which the product is used posted on a coordination site and there is a posted image reflecting the search intention of the user among them, the posted image may be provided as a thumbnail image. Specifically, the information processing apparatus 1 embeds a plurality of posted images showing coordinations using products included in the search results, a search query, and user attributes in a distributed representation space. Subsequently, the information processing apparatus 1 selects a thumbnail image from among the posted images based on each feature vector in the distributed representation space.

[0088] Note that the information processing apparatus 1 may not embed all the posted images showing coordinations using products included in the search results in the distributed representation space, but may embed in advance the posted images included in the search results obtained by searching on the coordination site with the same search query in the distributed representation space.

[0089] That is, the search target by the search unit 43 may be extended to, for example, coordination images posted on a coordination service in addition to products handled in an e-commerce street. By thus extending the search target, it is possible to provide a search result reflecting the search intention of the user.

[0090] 〔5. Effects〕 The information processing apparatus 1 according to the embodiment includes a search unit 43 that searches for products based on a search query specified by a user, a selection unit 44 that selects, from among a plurality of candidate images associated with the products included in the search results by the search unit 43, the candidate image having the highest relevance to the search query for each product, and a providing unit that provides the candidate image selected by the selection unit 44 as a thumbnail image of the product.

[0091] In addition, the selection unit 44 projects the search query and candidate images into the distributed representation space, and selects candidate images that are similar to the search query in the distributed representation space. Further, the selection unit 44 selects candidate images based on the user information of the user who specified the search query, and in the distributed representation space obtained by projecting the user information, selects the candidate image with the smallest area of the space connecting the user information, the search query, and the candidate images.

[0092] In addition, in the learning stage, the selection unit 44 uses a model learned so that a candidate image pre-annotated as an image corresponding to a search query and the search query become similar vectors in the distributed representation space to select a candidate image.

[0093] In addition, in the learning stage, the selection unit 44 uses a model learned so that a search query specified by a user and an image selected by the user when using the search query become similar vectors in the distributed representation space to select a candidate image.

[0094] Further, the information processing apparatus 1 includes a generation unit 45 that generates a composite image using a composite image that satisfies the search condition for a product that does not satisfy the selection condition of the candidate image by the selection unit 44, and the provision unit 46 provides the composite image generated by the generation unit 45 as a thumbnail image.

[0095] By any one or combination of the above-described processes, the information processing apparatus according to the present application can provide a thumbnail image that reflects the search intention of the user.

[0096] 〔6. Hardware Configuration〕 In addition, the information processing apparatus 1 according to the above-described embodiment is realized by a computer 1000 configured as shown in FIG. 12, for example. FIG. 12 is a hardware configuration diagram showing an example of a computer that realizes the functions of the information processing apparatus according to the embodiment. The computer 1000 includes a CPU 1100, a RAM 1200, a ROM 1300, an HDD 1400, a communication interface (I / F) 1500, an input / output interface (I / F) 1600, and a media interface (I / F) 1700.

[0097] The CPU 1100 operates based on a program stored in the ROM 1300 or the HDD 1400 and controls each part. The ROM 1300 stores a boot program executed by the CPU 1100 when the computer 1000 is started up, a program dependent on the hardware of the computer 1000, and the like.

[0098] The HDD 1400 stores a program executed by the CPU 1100, data used by such a program, and the like. The communication interface 1500 receives data from other devices via a network (communication network) N and sends it to the CPU 1100, and transmits data generated by the CPU 1100 to other devices via the network N.

[0099] The CPU 1100 controls an output device such as a display and a printer, and an input device such as a keyboard and a mouse (in FIG. 12, the output device and the input device are collectively referred to as an "input / output device") via the input / output interface 1600. The CPU 1100 acquires data from the input device via the input / output interface 1600. Further, the CPU 1100 outputs the generated data to the output device via the input / output interface 1600.

[0100] The media interface 1700 reads a program or data stored in the recording medium 1800 and provides it to the CPU 1100 via the RAM 1200. The CPU 1100 loads such a program from the recording medium 1800 onto the RAM 1200 via the media interface 1700 and executes the loaded program. The recording medium 1800 is, for example, an optical recording medium such as a DVD (Digital Versatile Disc), a PD (Phase change rewritable Disk), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory, etc.

[0101] For example, when the computer 1000 functions as the information processing apparatus according to the embodiment, the CPU 1100 of the computer 1000 realizes the functions of the control unit 4 by executing the program loaded on the RAM 1200. The CPU 1100 of the computer 1000 reads and executes these programs from the recording medium 1800. However, as another example, these programs may be acquired from other devices via the network N.

[0102] [7. Others] As described above, the embodiments of the present application have been described, but the present invention is not limited by the contents of these embodiments. Further, the above-described constituent elements include those that can be easily assumed by those skilled in the art, those that are substantially the same, and those within the so-called equivalent range. Furthermore, the above-described constituent elements can be combined as appropriate. Further, various omissions, substitutions, or changes of the constituent elements can be made without departing from the gist of the above-described embodiments.

[0103] In addition, among the processes described in the above embodiments, all or part of the processes described as being automatically performed can also be manually performed, or all or part of the processes described as being manually performed can be automatically performed by a known method. In addition, the processing procedures, specific names, and information including various data and parameters shown in the above documents and drawings can be arbitrarily changed unless otherwise specified. For example, the various information shown in each figure is not limited to the illustrated information.

[0104] In addition, each component of each device shown in the drawings is conceptually functional and does not necessarily have to be physically configured as shown in the drawings. That is, the specific form of the distribution and integration of each device is not limited to that shown in the drawings, and all or part of it can be functionally or physically distributed and integrated in any unit according to various loads and usage situations.

[0105] For example, the information processing apparatus described above may be implemented by a plurality of server computers, and depending on the functions, the configuration can be flexibly changed, such as by calling an external platform or the like through an API (Application Programming Interface) or network computing.

[0106] In addition, the above-described embodiments and modified examples can be appropriately combined as long as the processing contents do not conflict.

[0107] In addition, the "section, module, unit" described above can be read as "means", "circuit", etc. For example, the acquisition unit can be read as an acquisition means or an acquisition circuit.

Description of Reference Numerals

[0108] 1 Information processing apparatus 2 Communication unit 3 Storage unit 4 Control unit 10 User terminal 31 User information storage unit 32 Commodity Information Storage Unit 33 Model Storage Unit 41 Acquisition Unit 42 Learning Unit 43 Search Unit 44 Selection Unit 45 Generation Unit 46 Provision Unit

Claims

1. A search unit that searches for a search target based on a search query specified by a user; A selection unit that selects, from among a plurality of candidate images associated with the search target included in the search result by the search unit, the candidate image having the highest degree of relevance to the search query for each search target; A providing unit that provides the candidate image selected by the selection unit as a thumbnail image of the search target comprising: The selection unit: selects an image including clothing as the candidate image; The providing unit: provides the image including the clothing as the thumbnail image characterized information processing apparatus.

2. The image including the clothing is an image including clothing that can be purchased in an e-commerce service characterized information processing apparatus according to claim 1.

3. An information processing method executed by a computer, comprising: a search step of searching for a search target based on a search query specified by a user; a selection step of selecting, from among a plurality of candidate images associated with the search target included in the search result by the search step, the candidate image having the highest degree of relevance to the search query for each search target; a providing step of providing the candidate image selected by the selection step as a thumbnail image of the search target including: The selection step: selects an image including clothing as the candidate image; The providing step: provides the image including the clothing as the thumbnail image characterized information processing method.

4. A search procedure for searching for a search target based on a search query specified by a user; a selection procedure for selecting, from among a plurality of candidate images associated with the search target included in the search result by the search procedure, the candidate image having the highest degree of relevance to the search query for each search target; a providing procedure for providing the candidate image selected by the selection procedure as a thumbnail image of the search target causing a computer to execute, The selection procedure: selects an image including clothing as the candidate image; The providing procedure: provides the image including the clothing as the thumbnail image characterized information processing program.

Images