Personalized Image Recognition

Abstract

To provide an image recognition technique for supporting a user to describe an item.SOLUTION: A computing device includes at least one processor and at least one memory. The at least one memory stores instructions. The at least one processor executes the instructions to perform operations. The operations comprise: obtaining an input image containing an object and a user identifier; preprocessing the input image to generate image feature data; preprocessing user information corresponding to the user identifier to generate user feature data; determining a user group on the basis of the user feature data to obtain group feature data of the user group; conducting a search of an image database on the basis of the image feature data and the group feature data to search for one or a plurality of images; and generating object information on the basis of image information of the one or the plurality of images.SELECTED DRAWING: Figure 4

Description

[Background technology]

[0001] Unless otherwise stated herein, the approaches described in this section are not prior art to the claims of this application, and are not admitted to be prior art by their inclusion in this section.

[0002] In recent years, commerce through networks, called e-commerce, has been increasing, and the market is expected to expand and gradually replace traditional offline markets in the future. To advertise or sell an item online, typically, one or more images and specifications of the item are posted on the online market. The specifications of the item may include, for example, the category, the brand name, the item name, and other details of the item. Since there are various types of products and the specifications are different for each product, specifying the item to be posted and describing the specifications is time-consuming and labor-intensive for the user, and the user may make errors in the description. On the other hand, when a user tries to search for an item for purchase purposes, he or she needs to create a search query to specify the item, which may cause the same problems as above. Summary of the Invention [Problem to be solved by the invention]

[0003] The use of image recognition techniques has been explored to assist users in describing items. In some instances, users can upload images containing items for recognition, and the image recognition system can generate predictions about the item's details from the uploaded image. However, it has been found that for many of the uploaded images, the system is unable to make predictions with a sufficiently high degree of confidence. For example, for some reasons, the items shown in the uploaded images may be difficult to distinguish (e.g., men's jeans and women's jeans), or such items may belong to multiple categories (e.g., some toys belong to both soft toys and character goods). [Means for solving the problem]

[0004] The techniques generally described herein relate to providing a personalized image recognition scheme. Various exemplary methods using one or more computing devices are described. An exemplary method may include: obtaining, using one or more computing devices, an input image including an object and a user identifier for a user; preprocessing, using the one or more computing devices, the input image to generate image feature data for the input image; preprocessing, using the one or more computing devices, user information corresponding to the user identifier to generate user feature data for the user, the user information and the user feature data being stored in a user database; determining, using the one or more computing devices, a user group based on the user feature data to obtain group feature data for the user group, the user groups and the group feature data being stored in the user database; performing, using the one or more computing devices, a search of an image database based on the image feature data and the group feature data to retrieve a first one or more images; and generating, using the one or more computing devices, first object information for the object based on image information for the first one or more images. In some examples, the method may further include using the one or more computing devices to perform a search of the image database based on the image feature data to retrieve a second one or more images; and using the one or more computing devices to generate second object information for the object based on image information of the second one or more images.In some examples, generating the first object information may include determining a trust value for the first object information, and the method may further include comparing the trust value with a predetermined value; and in response to a determination that the trust value is higher than the predetermined value, selecting the first object information as object information for the object.

[0005] In some examples, a computing device for providing image recognition is described. An exemplary computing device may include at least one processor; and at least one memory. The at least one memory may store instructions, and the at least one processor may execute the instructions to perform operations. The operations may include obtaining an input image including an object and a user identifier for a user; pre-processing the input image to generate image feature data for the input image; pre-processing user information corresponding to the user identifier to generate user feature data for the user, the user information and the user feature data being stored in a user database; determining a user group based on the user feature data to obtain group feature data for the user group, the user group and the group feature data being stored in the user database; performing a search of an image database based on the image feature data and the group feature data to retrieve a first one or more images; and generating first object information for the object based on the image information of the first one or more images.

[0006] In some examples, a non-transitory computer-readable storage medium is described. An exemplary non-transitory computer-readable storage medium may store instructions executable by the computing device to cause the computing device to perform operations. The operations may include obtaining an input image including an object and a user identifier for a user; pre-processing the input image to generate image feature data for the input image; pre-processing user information corresponding to the user identifier to generate user feature data for the user, the user information and the user feature data being stored in a user database; determining a user group based on the user feature data to obtain group feature data for the user group, the user groups and the group feature data being stored in the user database; performing a search of an image database based on the image feature data and the group feature data to retrieve a first one or more images; and generating first object information for the object based on the image information of the first one or more images.

[0007] The above summary is illustrative only and is not intended to be in any way limiting. In addition to the illustrative aspects, embodiments, and features described above, further aspects, embodiments, and features will become apparent by reference to the drawings and the following detailed description. [Brief description of the drawings]

[0008] The above and other features of the present disclosure will become more fully apparent from the following description and appended claims, taken in conjunction with the accompanying drawings, in which: The present disclosure will be described with additional specificity and detail using the accompanying drawings, with the understanding that these drawings only depict certain embodiments in accordance with the present disclosure and are therefore not to be considered limiting of its scope.

[0009] [Figure 1] FIG. 1 illustrates an example image recognition system in which a user device is configured to communicate with a server over a network, as arranged in accordance with at least some embodiments described herein.

[0010] [Figure 2A] 1 illustrates an example in which two user devices each send an input image including an object to a server, configured in accordance with at least some embodiments described herein. [Figure 2B] 1 illustrates an example in which two user devices each receive object information from a server in response to transmitting an input image, the example being configured in accordance with at least some embodiments described herein.

[0011] [Diagram 3] FIG. 1 illustrates a block diagram of an example server, configured in accordance with at least some embodiments described herein.

[0012] [Figure 4] FIG. 1 illustrates an example flow diagram of a process for providing image recognition, arranged in accordance with at least some embodiments described herein.

[0013] [Diagram 5] FIG. 1 illustrates another example flow diagram of a process for providing image recognition, arranged in accordance with at least some embodiments described herein.

[0014] [Figure 6] FIG. 10 shows an example flow diagram of a further process for providing image recognition, arranged in accordance with at least some embodiments described herein.

[0015] [Figure 7A] 1 illustrates a portion of a user database configured in accordance with at least some embodiments described herein. [Figure 7B] 1 illustrates another portion of a user database, configured in accordance with at least some embodiments described herein.

[0016] [Figure 8] 1 illustrates a computer program product that may be utilized to perform image recognition, configured in accordance with at least some embodiments described herein.

[0017] [Figure 9] FIG. 1 is a block diagram illustrating an example computing device that may be configured for image recognition, arranged in accordance with at least some embodiments described herein. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0018] In the following detailed description, reference is made to the accompanying drawings, which form a part of this specification. In the drawings, like symbols typically identify like components unless the context dictates otherwise. The exemplary embodiments described in the detailed description, drawings, and claims are not intended to be limiting. Other embodiments may be utilized, and other changes may be made, without departing from the spirit or scope of the subject matter presented herein. Aspects of the present disclosure, as generally described herein and illustrated in the drawings, can be configured, substituted, combined, separated, and designed in a wide variety of different ways, all of which are expressly contemplated herein.

[0019] The present disclosure is directed generally to methods, systems, apparatus, and computer-readable storage media relating specifically to image recognition.

[0020] Briefly, in general, techniques for providing a personalized image recognition scheme are described. In some examples, a user may wish to post an object, such as one of his / her possessions, for sale on an online marketplace by using a user device. The posting of the object may include object information describing the object. The object information may include, for example, a category, a brand name, and / or some details of the object. The user may use the user device to select an input image including the object. The input image may be captured by a camera associated with the user device, read from a memory of the user device, or obtained over a network such as the Internet. In some embodiments, the user device may transmit the input image to a server along with a user identifier for the user.

[0021] The server may receive the input image together with the user identifier. The server may preprocess the input image and extract image feature data from the preprocessed input image. Further, the server may obtain user information corresponding to the user identifier from a user database. The user database may store user identifiers and user information of a plurality of users and group the plurality of users into a plurality of user groups. The server may preprocess the obtained user information to generate user feature data for the user. The server may determine a user group based on the user feature data and obtain group feature data for the determined user group.

[0022] The server may perform a search of the image database based on the image feature data and the user-related data to retrieve (i.e., obtain) the first one or more images. For example, the user-related data may include, but is not limited to, a user identifier, user information, user feature data, group feature data, and / or any information related to the determined user group. In one example, the server may perform the search based on the image feature data and the group feature data. The server may generate first object information of the object based on the image information of the first one or more images. Furthermore, the server may determine a confidence value of the first object information. Meanwhile, the server may perform a search of the image database based only on the image feature data (i.e., not based on the group feature data) to retrieve the second one or more images. The server may generate second object information of the object based on the image information of the second one or more images. If the determined confidence value is higher than a predetermined value, the server may select the first object information as the object information of the object. Otherwise, the server may select the second object information as the object information. The selected object information may be utilized for prediction of detailed item information. In some examples, the server can provide object information to a user device, which can present the object information on its screen to aid in user input.

[0023] In some other embodiments, the user device may pre-process the input image and extract image feature data from the pre-processed input image instead of sending the input image to the server. In such embodiments, the user device may obtain and / or store user group data corresponding to the user. In some examples, the user device may receive the user group data from the server and update the user group, for example, periodically. The user device may perform a search of the image database based on the image feature data and the user-related data to search for the first one or more images, and generate first object information for the object based on the first one or more images. Further, the user device may determine a confidence value of the first object information. If the determined confidence value is higher than a predetermined value, the user device may present the first object information on its screen to assist the user's input. Otherwise, the user device may transmit the image-related data (e.g., the input image and / or image feature data) and the user-related data (e.g., a user identifier, user information, user feature data, group feature data, and / or any information related to the determined user group) to the server. The server may then perform a search of the image database based solely on the image feature data (i.e., not based on the group feature data) to retrieve the second image or images. The server may generate second object information for the object based on the image information of the second image or images and return the second object information to the user device.

[0024] In other cases where the user desires to purchase an object, the server may formulate a search query to search for one or more objects related to the object included in the input image based on the object information, and provide a search result for the searched object or objects to the user device.

[0025] 1 illustrates a diagram of an example image recognition system in which a user device is configured to communicate with a server over a network, as configured in accordance with at least some embodiments described herein. As shown, image recognition system 100 may include user device 110, network 120, and server 130.

[0026] The user device 110 may be any type of computing device, such as user device 112, 114, ... or 116, configured to communicate with the server 130 via the network 120. The user device 110 may download application programs providing image recognition services according to the present disclosure from the server 130 and / or other external devices (not shown), such as application providing servers such as Google Play Store, App Store, etc., via the network 120. Various types of user device 110 include, but are not limited to, desktop computers, mobile computers (e.g., laptops, ultrabooks, netbooks), mobile phones (including smartphones), tablet computers, and / or personal digital assistants (PDAs).

[0027] User device 110 may access server 130 through network 120. Network 120 may be any wired network, wireless network, or combination thereof. In addition, network 120 may include, but is not limited to, a personal area network, a local area network, a wide area network, a cable network, a satellite network, a cellular telephone network, or combinations thereof. For example, network 120 may be a publicly accessible network of linked networks, possibly operated by various separate parties, such as the Internet. In some embodiments, network 120 may be a private or semi-private network, such as a corporate or university intranet. Network 120 may include one or more wireless networks, such as a Code Division Multiple Access (CDMA) network, a Long Term Evolution (LTE) network, a Long Term Evolution Advanced (LTE-A) network, a Wireless Fidelity for Microwave Access (WiMax) network, a Wireless Fidelity (Wi-Fi) network, and any other type of wireless network. For example, protocols used by network 120 may include HyperText Transfer Protocol (HTTP), HTTP Secure (HTTPS), Message Queuing Telemetry Transport (MQTT), Constrained Application Protocol (CoAP), etc. Protocols and components for communicating over the Internet or other types of communication networks mentioned above are well known to those skilled in the art and thus will not be described in further detail herein.

[0028] The server 130 may be implemented as a single computing device, a server farm including multiple servers, a distributed network, a cloud computing configuration, and / or the like. The server 130 may include one or more processors and memory. The server 130 and / or other computing devices communicatively coupled to the server 130 may include an image database and / or a user database. In some examples, the image database may store multiple images, each including an item. Additionally, the image database may store image feature data for each stored image. The image feature data may include, for example, at least one of a feature vector, a graph, and an image. In some examples, the user database may store user identifiers and user information for multiple users. The user information may include at least one of demographic information and behavioral information. The demographic information may include, for example, age, gender, location, address, occupation, and the like. The behavioral information may include, for example, categories visited, preferred brand names, browsing history, application usage frequency, purchase history, and the like.

[0029] In some examples, the server 130 may be configured to generate user feature data for each user of the plurality of users based on the user information of that user, as described in more detail below. Additionally, the server 130 may be configured to group the plurality of users stored in the user database into a plurality of user groups based on the user information of each user of the plurality of users, and extract group feature data for each user group based on the user information of one or more users in the same group. Thus, in addition to the user identifier and the user information, the user database may also store user feature data for each user, group information for each user group of the plurality of user groups, and a plurality of group feature data. The user feature data and the group feature data may include, for example, one or more feature vectors. The server 130 may be configured to update the user database at a predetermined frequency. The server 130 may update at least one of the user information, the user feature data, the user group, and the group feature data.

[0030] The user device 110 may obtain an input image. The input image may include, for example, an object to be displayed, advertised, or purchased. The user device 110 may display the input image on a display screen associated with the user device 110 so that a user can review the input image. In some embodiments, the user device 110 may transmit the input image to the server 130 over the network 120. The server 130 may be configured to obtain the input image and a user identifier. The user identifier may be associated with a user of the user device 110 or with the user device 110. The server 130 may be configured to receive the input image along with the user identifier.

[0031] The server 130 may be configured to pre-process the input image to generate image feature data for the input image. Pre-processing of the input image may include, for example, resizing the input image, segmenting the input image, cropping the input image, transforming the input image, modifying the color of the input image, the brightness of the input image, the contrast of the input image, etc., to facilitate generation of the image feature data. The server 130 may pre-process the input image and extract image feature data from the input image. The extracted image feature data may relate to objects included in the input image. The image feature data may include, for example, at least one of a vector, a graph, and an image.

[0032] The server 130 may be configured to pre-process the user information corresponding to the user identifier to generate user feature data for the user. In some examples, the server 130 may retrieve the user identifier from a user database and read the user information corresponding to the user identifier. The server 130 may encode the user information, for example, using a multi-hot encoding technique, to generate the user feature data and quantize the user feature data to adjust or reduce a size or length of the user feature data. The server 130 may store the generated user feature data in the user database.

[0033] The server 130 may be configured to determine a user group associated with the user based on the user characteristic data. In some examples, the server 130 may read each group characteristic data of each user group of the multiple user groups stored in the user database and calculate a difference, e.g., a vector distance, between the user characteristic data and the group characteristic data. The server 130 may determine whether the calculated difference is within a predetermined value, and if the calculated difference is within a predetermined value, the server 130 may determine such user group as a user group associated with the user. In some examples, when two or more user groups are determined, the server 130 may select one user group with a smallest difference as the user group associated with the user.

[0034] The server 130 may be configured to perform a search of the image database based on the image feature data and the user-related data to retrieve the first one or more images. For example, the user-related data may include, but is not limited to, a user identifier, user information, user feature data, group feature data, and / or any information related to the determined user group. In one example, the server 130 may perform a search based on the image feature data and the group feature data of the determined user group. In some examples, the user database may store listing image history and / or listing item history for each user group. The server 130 may read the listing image history and / or listing item history based on the group feature data to determine the plurality of images stored in the image database. The server 130 may then compare the image feature data associated with the object in the input image to the feature data of each image of the determined plurality of images. If the difference between the image feature data related to the object and the feature data of the images stored in the image database is within a predetermined value, the server 130 may determine the corresponding image as one of the first one or more images related to the object.

[0035] The server 130 may be configured to generate a first object information of the object based on the image information of the first one or more images. The image database may store image information for each image, and the image information may include, for example, a title of the image, a category of an item in the image, a brand name of the item, a detailed description of the item, and the like. The server 130 may read the image information of the first one or more images from the image database. In some examples, the server 130 may determine one of the first one or more images based on a similarity value with the input image, and obtain the image information of the determined image as the object information. In some other examples, the server 130 may obtain the image information of the first one or more images as the object information. The server 130 may select information about each feature of the object, such as a title, a category, a brand name, a detailed description, and the like, to generate the object information.

[0036] In some further embodiments, in addition to searching the first one or more images, the server 130 may be configured to perform a search of the image database based on the image feature data, but not based on the user feature data or the group feature data. In some examples, the server 130 may compare the image feature data for the object in the input image with the feature data of the images stored in the image database. If the difference between the image feature data for the object and the feature data of the images stored in the image database is within a predetermined value, the server 130 may determine the corresponding image as one of the second one or more images for the object, and generate second object information for the object based on the image information of the second one or more images.

[0037] In some examples, when generating the first object information, the server 130 can determine a confidence value of the first object information. The server 130 can compare the confidence value with a predetermined confidence value. If the confidence value of the first object information is higher than the predetermined confidence value, the server 130 can select the first object information as a result of the image recognition. Otherwise, if the confidence value of the first object information is not higher than the predetermined confidence value, the server 130 can select the second object information as a result of the image recognition.

[0038] In some examples where a user of user device 110 is seeking to post or sell an object, server 130 may transmit at least a portion of the object information to user device 110. In other examples where a user of user device 110 is seeking to purchase an item related to an object included in the input image, server 130 may formulate a search query based on the object information to search for one or more objects related to the object (included in the input image). In such examples, server 130 may further perform a search for the one or more objects based on the search query and return search results to user device 110.

[0039] Although the above describes the server 130 preprocessing the input image, other embodiments are possible in which the user device 110 preprocesses the input image to generate the first object information. In some other embodiments, instead of sending the input image to the server, the user device 110 may preprocess the input image and extract image feature data from the preprocessed input image. The user device 110 may obtain and / or store user group data corresponding to the user. In some examples, the user device 110 may receive user group data from the server 130 and update the user group, for example periodically. The user device 110 may perform a search of the image database based on the image feature data and the user-related data to search for the first one or more images, and generate first object information for the object based on the first one or more images. Furthermore, the user device 110 may determine a confidence value for the first object information. If the determined confidence value does not exceed a predetermined value, user device 110 may send image related data (e.g., the input image and / or image feature data) and user related data (e.g., a user identifier, user information, user feature data, group feature data, and / or any information associated with the determined user group) to server 130. Server 130 may then perform a search of the image database based on the image feature data, but not based on the user feature data or the group feature data, to retrieve a second image or images. Server 130 may generate second object information for the object based on the image information of the second image or images and return the second object information to user device 110.

[0040] 2A illustrates an example in which two user devices each transmit an input image including an object to a server, and FIG. 2B illustrates an example in which two user devices each receive object information from a server in response to transmitting the input image, configured in accordance with at least some embodiments described herein. As shown in FIG. 2A, two different users of user devices 212 and 214 may upload the same input image, e.g., an image of pants, and the users of user devices 212 and 214 may transmit the input image to server 230 via network 220.

[0041] The server 230 can obtain the input image and the user identifier. The user of the user device 212 has different characteristics than the user of the user device 214. That is, the user information of the user of the user device 212 is different from the user information of the user of the user device 214. The server 230 can pre-process the input image and the user identifier of the user of the user device 212 to generate an image feature vector and a first user feature data. Furthermore, the server 230 can pre-process the input image and the user identifier of the user of the user device 214 to generate an image feature vector and a second user feature data. The server 230 can perform a search of the image database to search for the first one or more images and generate the first object information. Furthermore, the server 230 can perform a search of the image database to search for the second one or more images and generate the second object information.

[0042] 2B, in response to transmitting the input image, the user device 212 can receive object information of the pants shown in the input image: "Gender: Female; Category: Jeans; Brand: Alpclo; Size: S; Texture: ...." Additionally, the user device 214 can receive different object information of the pants: "Gender: Male; Category; Casual; Brand: Toms; Size: L; Texture: ...."

[0043] In typical image recognition techniques, information of objects contained in an image can be extracted from the image itself. However, because there are various types of items, such extracted information may have a wide range or may contain inaccurate information. The characteristics of a user may affect trends and / or statistics about items related to the user. Such trends and / or statistics can be more clearly understood by understanding the user groups of multiple users related to the user. In this regard, when performing image recognition, the accuracy of object information can be improved by utilizing not only the image itself but also information about the user and / or user groups.

[0044] 3 illustrates a block diagram of an example server configured in accordance with at least some embodiments described herein. As illustrated, the server 300 may include a network module 310, a pre-processing module 320, a user database 330, an image database 340, and a first image recognition module 350. Additionally or optionally, the server 300 may further include a second image recognition module 360 ​​and / or a result selection module 370. The server 300 may be implemented as a single computing device, a server farm including multiple servers, a distributed network, a cloud computing configuration, or the like.

[0045] In some embodiments, the network module 310 may be configured to exchange data with a user device (such as, for example, user device 110 of FIG. 1) over a network (such as, for example, network 120 of FIG. 1). A user of the user device may select an input image by using the user device, and the user device may send information about the input image to the server 300. The network module 310 may obtain the input image from the user device and a user identifier of the user.

[0046] The pre-processing module 320 may include an image vector extractor 322 and a user vector extractor 324. The image vector extractor 322 may be configured to pre-process the input image to generate image feature data for the input image. The image vector extractor 322 may perform one or more pre-processing operations including, for example, resizing the input image, segmenting the input image, cropping the input image, transforming the input image, modifying the color of the input image, the brightness of the input image, the contrast of the input image, etc., to facilitate the generation of the image feature data. The image vector extractor 322 may extract the image feature data using the pre-processed input image. The extracted image feature data may relate to objects included in the input image. The image feature data may include, for example, at least one of a vector, a graph, and an image. The image vector extractor 322 may store the image feature data in an image database 340.

[0047] The user vector extractor 324 may be configured to pre-process the user information corresponding to the user identifier to generate user feature data for the user. The user database 330 may be configured to store the user identifier and the user information for a plurality of users. The user information may include at least one of demographic information and behavioral information. The demographic information may include, for example, age, gender, location, address, occupation, and the like. The behavioral information may include, for example, categories visited, preferred brand names, browsing history, application usage frequency, purchase history, and the like. Furthermore, the user database 330 may be further configured to store an image listing history and / or an item listing history for each user.

[0048] In some examples, the user vector extractor 324 may retrieve a user identifier from the user database 330 and read user information corresponding to the user identifier. The user vector extractor 324 may encode the user information, for example using a multi-hop encoding technique, to generate user feature data and quantize the user feature data to adjust or reduce a size or length of the user feature data. The user vector extractor 324 may store the generated user feature data in the user database 330.

[0049] In some examples, the pre-processing module 320 may process the plurality of users stored in the user database 330 into a plurality of user groups UG based on the user information of each user of the plurality of users. 1 , U.G. 2 , U.G. 3 , … , U.G. n The user database 330 may group the users into groups and extract group characteristic data for each user group based on the user information of one or more users in the same group. In some examples, the group characteristic data for each user group may be obtained by combining the user characteristic data of one or more users in the same group. Thus, in addition to the user identifier and the user information, the user database 330 may store user characteristic data for each user, a user group UG, and a user identifier and user information for each user. 1 , U.G. 2 , U.G. 3 , … , U.G. n Group information for each group, and user group UG 1 , U.G. 2 , U.G. 3 , … , U.G. nThe pre-processing module 320 may store a plurality of group feature data in the user database 330. The user feature data and the group feature data may include, for example, one or more feature vectors. The pre-processing module 320 may be configured to update the user database 330 at a predetermined frequency. The pre-processing module 320 may store the user information, the user feature data, the group information of the user groups, and the group feature data of the user groups. 1 , U.G. 2 , U.G. 3 , … , U.G. n In addition, the user database 330 may be configured to further store listing image history and / or listing item history for each user group.

[0050] The image database 340 may be configured to store a plurality of images, each of which includes an item. Additionally, the image database 340 may store image feature data for each stored image. The image feature data may include, for example, at least one of a feature vector, a graph, and an image.

[0051] The first image recognition module 350 may be configured to determine a user group associated with the user based on the user characteristic data. In some examples, the first image recognition module 350 may be configured to determine a user group associated with the user based on the user characteristic data. 1 , U.G. 2 , U.G. 3 , … , U.G. nThe first image recognition module 350 may read each group feature data of each user group and calculate a difference between the user feature data and the group feature data. The difference may be calculated, for example, by calculating a vector distance. The first image recognition module 350 may determine whether the calculated difference is within a predetermined value, and if the calculated difference is within a predetermined value, the first image recognition module 350 may determine such user group as a user group associated with the user. In some examples, when two or more user groups are determined, the first image recognition module 350 may select one user group having a smallest difference as the user group associated with the user.

[0052] The first image recognition module 350 may be configured to perform a search of the image database 340 based on the image feature data and the user-related data to retrieve the first one or more images. For example, the user-related data may include, but is not limited to, a user identifier, user information, user feature data, group feature data, and / or any information related to the determined user group. In some examples, the first image recognition module 350 may perform a search based on the image feature data and the group feature data of the determined user group. In some examples, the user database 330 may store listing image history and / or listing item history for each user group. The first image recognition module 350 may read the listing image history and / or listing item history based on the group feature data to determine the plurality of images stored in the image database 340. The first image recognition module 350 may then compare the image feature data associated with the object in the input image to the feature data of each image of the determined plurality of images. If the difference between the image feature data associated with the object and the feature data of the images stored in the image database 340 is within a predetermined value, the first image recognition module 350 can determine the corresponding image as one of the first one or more images for the object.

[0053] The first image recognition module 350 may be configured to generate first object information of the object based on image information of the first one or more images. The image database 340 may store image information for each image, and the image information may include, for example, a title of the image, a category of an item in the image, a brand name of the item, a detailed description of the item, and the like. The first image recognition module 350 may read the image information of the first one or more images from the image database 340. Similarly, the object information may include at least one of the title, category, brand name, detailed description, and the like of the object. In some examples, the first image recognition module 350 may determine one of the first one or more images based on a similarity value with the input image, and obtain the image information of the determined image as the object information. In some other examples, the first image recognition module 350 may obtain the image information of the first one or more images. The first image recognition module 350 may select information about each feature of the object, such as a title, a category, a brand name, a detailed description, and the like, to generate the object information.

[0054] Additionally or optionally, the second image recognition module 360 ​​may be configured to perform a search of the image database based on the image feature data but not on the group feature data. In some examples, the second image recognition module 360 ​​may compare the image feature data obtained by the user vector extractor 324 with the feature data of the images stored in the image database 340. If the difference between the image feature data relating to the object and the image feature data of the images stored in the image database 340 is within a predetermined value, the second image recognition module 360 ​​may determine the corresponding image as one of the second one or more images associated with the object and generate second object information for the object based on the image information of the second one or more images.

[0055] In some further examples, the first image recognition module 350 may determine a confidence value of the first object information when generating the first object information. The confidence value may be related to a similarity value between the input image and the first one or more images. The result selection module 370 may compare the confidence value with a predetermined confidence value. If the confidence value of the first object information is higher than the predetermined confidence value, the result selection module 370 may select the first object information as the object information of the object. Otherwise, if the confidence value of the first object information is not higher than the predetermined confidence value, the result selection module 370 may select the second object information as the object information of the object.

[0056] In some examples, the network module 310 may transmit at least a portion of the object information to the user device. In other examples, the server 300 may formulate a search query to search for one or more objects associated with the object based on the object information. In such examples, the server 300 may further perform a search for the one or more objects based on the search query, and the network module 310 may transmit the search results to the user device.

[0057] FIG. 4 illustrates an example flow diagram of a process for providing image recognition, configured in accordance with at least some embodiments described herein. The process 400 of FIG. 4 may be implemented using one or more computing devices, such as the server 130, 230, or 300 described with reference to FIG. 1, 2A, 2B, or 3, which may be in operative communication with a user device, such as the user device 110, 212, or 214 described with reference to FIG. 1, 2A, or 2B. The process 400 may include one or more operations, actions, or functions illustrated by one or more of blocks S410, S420, S430, S440, S450, and / or S460. Although illustrated as separate blocks, the various blocks may be divided into additional blocks, combined into fewer blocks, supplemented with other blocks, or removed, depending on the specific implementation. In some further examples, the various described blocks may be implemented as parallel processes instead of sequential processes, or combinations thereof. For example, blocks S420 and S430 may be performed in parallel.Process 400 may begin with block S410 "Get an input image including an object and a user identifier."

[0058] In block S410, one or more computing devices may obtain an input image and a user identifier. In some examples, the one or more computing devices may obtain the input image through various methods, as described above. The input image may include an object. The user identifier may be associated with a user of the user device. Block S410 may be followed by block S420, “Preprocessing the Input Image to Generate Image Feature Data.”

[0059] In block S420, the one or more computing devices may preprocess the input image to generate image feature data for the input image. In some examples, the one or more computing devices may preprocess the input image to extract image feature data from the input image. The image feature data may include, for example, at least one of a vector, a graph, and an image. Block S420 may be followed by block S430, “Preprocess User Information Corresponding to a User Identifier.”

[0060] In block S430, the one or more computing devices may preprocess the user information corresponding to the user identifier to generate user characteristic data for the user. In some examples, the one or more computing devices may read the user information based on the user identifier and encode the user information to generate the user characteristic data. The one or more computing devices may store the generated user characteristic data in a user database, such as, for example, user database 330 of FIG. 3. Block S430 may be followed by block S440, “Determine User Groups and Obtain Group Characteristic Data for User Groups.”

[0061] In block S440, the one or more computing devices may determine a user group based on the user feature data to obtain group feature data for the user group. In some examples, the one or more computing devices may read each group feature data for each user group of the plurality of user groups stored in the user database and calculate a difference, e.g., a vector distance, between the user feature data and the group feature data. The one or more computing devices may determine the user groups for which the calculated difference is within a predetermined value. Block S440 may be followed by block S450, "Perform a search of an image database to retrieve one or more images."

[0062] In block S450, the one or more computing devices may perform a search of the image database based on the image feature data and the user-related data to retrieve the first one or more images. In some examples, the one or more computing devices may perform the search based on the image feature data and the group feature data of the determined user group. In some examples, the user database may store listing image history and / or listing item history for each user group. The one or more computing devices may read the listing image history and / or listing item history based on the group feature data, and in response to the listing image history and / or listing item history, the one or more computing devices may determine a plurality of images stored in the image database. The one or more computing devices may then compare the image feature data from the input image with the feature data of each image of such determined plurality of images. If a difference between the image feature data from the input image and the feature data of an image from the image database is within a predetermined value, the one or more computing devices may determine the corresponding image as one of the first one or more images associated with the object. Block S450 may be followed by block S460, "Generate Object Information for Object."

[0063] In block S460, the one or more computing devices may generate first object information for the object based on the image information of the first one or more images. In some examples, the one or more computing devices may determine one of the first one or more images based on a similarity value with the input image and obtain the image information of the determined image as the object information. In some other examples, the one or more computing devices may obtain the image information of the first one or more images. The one or more computing devices may select information for each feature of the object to generate the object information.

[0064] Additionally or optionally, in some examples where a user is browsing or seeking to purchase objects, the one or more computing devices may formulate a search query to search for one or more objects associated with the object (included in the input image) based on the object information, and the one or more computing devices or other computing devices may perform an image search using the search query and return search results to the user device.

[0065] FIG. 5 illustrates another example flow diagram of a process for providing image recognition, configured in accordance with at least some embodiments described herein. Process 500 of FIG. 5 may be implemented using one or more computing devices, such as server 130, 230, or 300 described with reference to FIG. 1, 2A, 2B, or 3, that may be in operative communication with a user device, such as user device 110, 212, or 214 described with reference to FIG. 1, 2A, or 2B. Process 500 may include one or more operations, actions, or functions illustrated by one or more of blocks S510, S520, S530, S540, S550, S560, S570, S580, and / or S590. Although illustrated as separate blocks, various blocks may be divided into additional blocks, combined into fewer blocks, supplemented with other blocks, or removed, depending on the specific implementation. In some further examples, the various described blocks may be implemented as parallel processes instead of sequential processes, or as a combination thereof. The detailed description of blocks S410, S420, S430, S440, S450, and S460 of FIG. 4 is applicable to blocks S510, S520, S530, S540, S550, and S560, respectively, and repetitive descriptions are omitted for brevity. Block S520 “Preprocessing the input image to generate image feature data” is followed by block S530, which is also followed by block S570 “Performing a search of an image database to retrieve a second image or images.”

[0066] In block S570, the one or more computing devices may perform a search of an image database based on the image feature data. In some examples, the one or more computing devices may compare the image feature data obtained in block S520 with feature data of images stored in the image database. If a difference between the image feature data associated with the object and the feature data of the images stored in the image database is within a predetermined value, the one or more computing devices may determine the corresponding image as one of a second one or more images associated with the object. Block S570 may be followed by block S580 "Generate Second Object Information."

[0067] In block S580, the one or more computing devices may generate second object information for the object based on the image information of the second one or more images. In some examples, the one or more computing devices may determine one of the second one or more images based on a similarity value with the input image and obtain the image information of the determined image as the second object information. In some other examples, the one or more computing devices may obtain the image information of the second one or more images and select information for each feature of the object to generate the second object information.

[0068] Here, as shown in FIG. 5, the one or more images retrieved by the one or more computing devices in block S550 are first one or more images, and the generated object information generated by the one or more computing devices in block S560 is first object information. In block S560, the one or more computing devices may determine a confidence value of the first object information. The confidence value may be related to a similarity value between the input image and the first one or more images. Blocks S560 and S580 may be followed by block S590 “Compare confidence values ​​and select the first or second one or more images.”

[0069] In block S590, the one or more computing devices may compare the trust value with a predetermined trust value. If the trust value of the first object information is higher than the predetermined trust value, the one or more computing devices may select the first object information as the object information of the object. Otherwise, if the trust value of the first object information is not higher than the predetermined trust value, the one or more computing devices may select the second object information as the object information of the object.

[0070] FIG 6 illustrates an example flow diagram of a further process for providing image recognition, arranged in accordance with at least some embodiments described herein. Process 600 of FIG 6 may be implemented using one or more computing devices, such as servers 130, 230 or 300 described with reference to FIG 1, FIG 2A, FIG 2B or FIG 3. Process 600 may be performed before, after and / or during process 400 illustrated in FIG 4 or process 500 illustrated in FIG 5.

[0071] Process 600 may include one or more operations, actions, or functions as illustrated by one or more of blocks S610, S620, and / or S630. Although illustrated as separate blocks, the various blocks may be divided into additional blocks, combined into fewer blocks, supplemented with other blocks, or eliminated, depending on the specific implementation. In some further examples, the various described blocks may be implemented as parallel processes instead of sequential processes, or combinations thereof. Process 600 may begin with block S610, “Group a plurality of users into a plurality of user groups.”

[0072] In block S610, the one or more computing devices may group a plurality of users into a plurality of user groups. In some examples, the one or more computing devices and / or other computing devices communicatively coupled to such one or more computing devices may include a user database. The user database may store user identifiers and user information for a plurality of users. The user information may include at least one of demographic information and behavioral information. The one or more computing devices may group a plurality of users stored in the user database into a plurality of user groups based on the user information of each of the plurality of users. Block S610 may be followed by block S620 “Extract group characteristic data for each user group based on the user information of one or more users in the same group.”

[0073] In block S620, the one or more computing devices may extract group feature data for each user group based on the user information of one or more users in the same group. The one or more computing devices may generate user feature data for each user based on the user information and extract the group feature data, for example, by combining the user feature data of one or more users in the same group. The user feature data and the group feature data may include, for example, one or more feature vectors. Block S620 may be followed by block S630, “update user database at a predetermined frequency.”

[0074] In block S630, the one or more computing devices may update the user database at a predetermined frequency. The one or more computing devices may update at least one of user information, user characteristic data, user groups, and group characteristic data of the user groups.

[0075] FIG. 7A illustrates a portion of a user database, and FIG. 7B illustrates another portion of the user database, configured in accordance with at least some embodiments described herein. As illustrated in FIG. 7A, the user database may include multiple columns of data for each user. The multiple columns may include, for example, a user identifier and user information regarding age, gender, categories visited, preferred brand names, image / listing item history, and the like. In one example, a computing device according to the present disclosure may extract a user vector using the user information. As illustrated in FIG. 7A, to extract a user vector for user 12353, the computing device may read user information such as age 34, gender M, categories A, B, and C, and brand names a and b from the user database. The computing device may encode the user information to generate a user vector of 34 10 0001000101000000 000000100000001000..., as illustrated at the bottom of FIG. 7A, as user characteristic data according to the present disclosure. In some examples, the computing device may use any suitable encoding technique, such as, for example, a multi-hot encoding technique. Additionally, the computing device may quantize the generated user vector to adjust or shorten the size or length of the user vector. For example, a user vector of 34 10 0001000101000000 000000100000001000 may be quantized to a quantized user vector 2.45.6.23.... In some examples, the computing device may store the generated user vector as user feature data for each user in a user database.

[0076] The computing device may group the plurality of users 12353, 13412, 54212, 43347, etc., stored as shown in FIG. 7A into a plurality of user groups, group 1, group 2, etc., based on the user information, and extract a group vector for each user group, as shown in FIG. 7B, in accordance with the present disclosure. The group vector is an example of group feature data in accordance with the present disclosure. In one example, the computing device may group the plurality of users 12353, 13412, 54212, 43347, etc., into group 1, group 2, etc., by, for example, comparing the user vectors of the users. Further, the computing device may generate a group vector, for example, by combining the user vectors of users in the same group. As shown in FIG. 7B, the user database may include multiple columns of data for each user group. The multiple columns may include, for example, a group name (identifier), group information about the group vector, a user list for the user group, an image vector along with image / listing item history, categories, preferred brand names, etc. In one example, the computing device may compare the user vector with the user group vector for each group, group 1, group 2, etc; determine the user group (group 1); obtain a group vector for 2.45.7.12...; and determine a number of images having image vectors 66.23.45.76.12..., 67.73.54.19.65..., etc. The computing device may then compare the image vector extracted from the input image with each of the image vectors 66.23.45.76.12..., 67.73.54.19.65..., etc. to search for one or more images.

[0077] 8 illustrates a computer program product 800 that may be utilized to provide image recognition according to at least some embodiments described herein. The computer program product 800 may include a signal-bearing medium 802. The signal-bearing medium 802 may include one or more instructions 804 that, for example, in response to execution by one or more processors, may provide the functionality and features described above with respect to FIGS. As an example, the instructions 804 may include at least one of: one or more instructions for obtaining an input image including an object and a user identifier for a user; one or more instructions for preprocessing the input image to generate image feature data for the input image; one or more instructions for preprocessing user information corresponding to the user identifier to generate user feature data for the user, where the user information and the user feature data are stored in a user database; one or more instructions for determining a user group based on the user feature data and obtaining group feature data for the user group, where the user group and the group feature data are stored in a user database; one or more instructions for performing an image database search to retrieve a first one or more images based on the image feature data and the group feature data; or one or more instructions for generating first object information for the object based on image information for the first one or more images. Thus, for example, with reference to Figures 1-3, server 130, 230, or 300 may undertake one or more of the blocks illustrated in Figures 4-5 in response to the instructions 804.

[0078] In some implementations, the signal bearing medium 802 may include a non-transitory computer-readable medium 806, such as a hard disk drive, a compact disc (CD), a digital video disc (DVD), digital tape, a memory, etc. In some implementations, the signal bearing medium 802 may include a recordable medium 808, such as, but not limited to, a memory, a read / write (R / W) CD, a R / W DVD, etc. In some implementations, the signal bearing medium 802 may include a communication medium 810, such as, but not limited to, a digital and / or analog communication medium (e.g., fiber optic cable, a wave guide, a wired communication link, a wireless communication link, etc.). Thus, for example, the program product 800 may be conveyed to one or more modules of a computing device, such as, for example, the server 130, 230, or 300, by an RF signal bearing medium 802, where the signal bearing medium 802 is conveyed by a wireless communication medium 810 (e.g., a wireless communication medium conforming to the IEEE 802.11 standard).

[0079] FIG. 9 is a block diagram illustrating an example computing device that may be utilized for image recognition, configured in accordance with at least some embodiments described herein.

[0080] In a very basic configuration 902, a computing device 900 typically includes one or more processors 904 and a system memory 906. A memory bus 908 may be used for communication between the processors 904 and the system memory 906.

[0081] Depending on the desired configuration, the processor 904 may be of any type, including but not limited to a microprocessor (μP), a microcontroller (μC), a digital signal processor (DSP), or any combination thereof. The processor 904 may include one or more levels of caching, such as a level 1 cache 910 and a level 2 cache 912, a processor core 914, and registers 916. An exemplary processor core 914 may include an arithmetic logic unit (ALU), a floating point unit (FPU), a digital signal processing core (DSP core), or any combination thereof. An example memory controller 918 may be used with the processor 904, or in some implementations, the memory controller 918 may be an integral part of the processor 904.

[0082] Depending on the desired configuration, system memory 906 may be of any type, including but not limited to, volatile memory (e.g., RAM), non-volatile memory (e.g., ROM, flash memory, etc.), or any combination thereof. System memory 906 may include an operating system 920, one or more applications 922, and program data 924.

[0083] The application 922 may include an image recognition module 926 that may be configured to perform the operations and / or functions described herein, including the actions described with respect to the architecture of the server 130, 230, or 300 as shown in FIGS. 1-3, or the actions described with respect to the flowcharts shown in FIGS. 4-6. The program data 924 may include any data that may be useful for providing an image recognition scheme as described herein. In some examples, the application 922 may be configured to operate with the program data 924 on the operating system 920 such that a personalized image recognition scheme as described herein may be provided.

[0084] The computing device 900 may have additional features or functions and additional interfaces to facilitate communication between the basic configuration 902 and any necessary devices and interfaces. For example, a bus / interface controller 930 may be used to facilitate communication between the basic configuration 902 and one or more data storage devices 932 via a storage interface bus 934. The data storage devices 932 may be removable storage devices 936, non-removable storage devices 938, or a combination thereof. Removable and non-removable storage devices include magnetic disk drives such as floppy disk drives and hard disk drives (HDDs), optical disk drives such as compact disk (CD) drives or digital versatile disk (DVD) drives, solid state drives (SSDs), and tape drives, to name a few. Exemplary computer storage media may include volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules, or other data.

[0085] System memory 906, removable storage 936, and non-removable storage 938 are examples of computer storage media including, but not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and that can be accessed by computing device 900. Any such computer storage media may be part of computing device 900.

[0086] The computing device 900 may also include an interface bus 940 to facilitate communication from various interface devices (e.g., output devices 942, peripheral interface 944, and communication devices 946) to the basic configuration 902 via the bus / interface controller 930. Exemplary output devices 942 include a graphics processing unit 948 and an audio processing unit 950, which may be configured to communicate to various external devices, such as a display or a speaker, via one or more A / V ports 952. Exemplary peripheral interface 944 includes a serial interface controller 954 or a parallel interface controller 956, which may be configured to communicate with external devices, such as input devices (e.g., keyboard, mouse, pen, voice input device, touch input device, etc.) or other peripheral devices (e.g., printer, scanner, etc.) via one or more I / O ports 958. Exemplary communication device 946 includes a network controller 960, which may be configured to facilitate communication with one or more other computing devices 962 over a network communication link via one or more communication ports 964.

[0087] A network communication link may be one example of a communication medium. Communication media may typically be embodied by computer-readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave or other transport mechanism, and may include any information delivery media. A "modulated data signal" may be a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media may include wired media, such as a wired network or direct-wired connection, and wireless media, such as acoustic, radio frequency (RF), microwave, infrared (IR) and other wireless media. As used herein, the term "computer-readable media" may include both storage media and communication media.

[0088] The present disclosure should not be limited with respect to the specific embodiments described in this application. The embodiments are intended as illustrations of various aspects. Many modifications and variations can be made without departing from the spirit and scope thereof. From the above description, functionally equivalent methods and apparatuses within the scope of the present disclosure are possible, in addition to those recited herein. Such modifications and variations are intended to fall within the scope of the appended claims. The present disclosure is limited only by the terms of the appended claims and the full scope of equivalents to which such claims are entitled. The present disclosure is not limited to particular methods, reagents, compounds, compositions or biological systems, which can, of course, vary. The terminology used herein is merely for the purpose of describing specific embodiments and is not intended to be limiting.

[0089] The subject matter described herein sometimes shows different components that are included within or connected to different other components. Such illustrated architectures are merely examples, and in fact many other architectures that achieve the same functionality can be implemented. In a conceptual sense, any arrangement of components to achieve the same functionality is substantially "associated" such that the desired functionality is achieved. Thus, any two components that are combined herein to achieve a particular functionality, whether architectural or intermediate components, can be considered to be "associated" with each other such that the desired functionality is achieved. Similarly, two components that are so associated can also be considered to be "operably connected" or "operably coupled" to each other to achieve the desired functionality, and any two components that can be so associated can also be considered to be "operably coupled" to each other to achieve the desired functionality. Specific examples of operably coupleable include, but are not limited to, physically matable and / or physically interacting components, and / or wirelessly interacting and / or wirelessly interacting components, and / or logically interacting and / or logically interacting components.

[0090] With respect to the use of virtually any plural and / or singular terminology herein, those of skill in the art can convert from plural to singular and / or singular to plural as appropriate to the context and / or application. Various singular / plural permutations may be expressly set forth herein for clarity.

[0091] Those skilled in the art will generally understand that the terms used herein, and particularly in the appended claims (e.g., the body of the appended claims), are generally intended as "open" terms (e.g., the term "including" should be interpreted as "including, but not limited to," the term "having" should be interpreted as "having at least," the term "including" should be interpreted as "including, but not limited to," etc.). Moreover, those skilled in the art will understand that if a particular number of introduced claim recitations is intended, such intent will be expressly recited in the claim, and in the absence of such recitation, such intent does not exist. For example, as an aid to understanding, the appended claims may include the use of the introductory phrases "at least one" and "one or more" to introduce claim recitations. However, the use of such phrases should not be interpreted as implying that the introduction of a claim recitation by the indefinite article "a" or "an" limits any particular claim that includes such introduced claim recitation to embodiments that include only one such recitation. This is true even if the same claim includes the introductory phrase "one or more" or "at least one" and an indefinite article such as "a" or "an" (e.g., "a" and / or "an" should be construed to mean "at least one" or "one or more"). The same is true of the use of definite articles used to introduce claim recitations. Moreover, even if a particular number of claim recitations is explicitly recited, one of ordinary skill in the art will recognize that such recitation should be interpreted to mean at least the recited number (e.g., the recitation "two recitations" without other modifiers means at least two recitations or more than two recitations).Furthermore, in instances where idiomatic expressions similar to "at least one of A, B, and C, etc." are used, such syntax is generally intended in the sense that one of ordinary skill in the art would understand the idiomatic expressions (e.g., "a system having at least one of A, B, and C" includes, but is not limited to, systems having only A, only B, only C, both A and B, both A and C, both B and C, and / or all of A, B, and C, etc.). In instances where idiomatic expressions similar to "at least one of A, B, or C" are used, such syntax is generally intended in the sense that one of ordinary skill in the art would understand the idiomatic expressions (e.g., "a system having at least one of A, B, or C" includes, but is not limited to, systems having only A, only B, only C, both A and B, both A and C, both B and C, and / or all of A, B, and C, etc.). Virtually any disjunctive word and / or phrase presenting two or more alternative terms, whether in the specification, claims, or drawings, should be understood to contemplate the inclusion of either one of those terms, either both of those terms, or both terms. For example, the phrase "A or B" would be understood to include the possibilities of "A" or "B" or "A and B."

[0092] Moreover, when features or aspects of the disclosure are described in terms of a Markush group, one of skill in the art will recognize that the disclosure is also described in terms of any individual element or subgroup of elements of the Markush group.

[0093] As will be understood by those skilled in the art, for all purposes, including providing a written description, all ranges disclosed herein also encompass all possible subranges and combinations of subranges. Any range listed can be easily recognized as fully descriptive and allowing the same range to be divided into at least two, three, four, five, ten, etc. As a non-limiting example, each range discussed herein can be easily divided into a lower third, a middle third, an upper third, etc. As will also be understood by those skilled in the art, all phrases such as "up to," "at least," etc. refer to ranges that include the numbers listed and can then be broken down into subranges as described above. Finally, as will be understood by those skilled in the art, a range includes each individual element.

[0094] From the foregoing, various embodiments of the present disclosure have been described herein for purposes of illustration. Various modifications can be made without departing from the scope and spirit of the present disclosure. Thus, the various embodiments disclosed herein are not intended to be limiting, with the true scope and spirit being indicated by the following claims.

Claims

1. obtaining, by one or more computing devices, an input image including the object and a user identifier for the user; pre-processing, by the one or more computing devices, the input image to generate image feature data for the input image; pre-processing, by the one or more computing devices, user information corresponding to the user identifier to generate user characteristic data for the user, wherein the user information and the user characteristic data are stored in a user database; determining, by the one or more computing devices, user groups based on the user characteristic data to obtain group characteristic data for the user groups, the user groups and the group characteristic data being stored in the user database; performing, by the one or more computing devices, a search of an image database based on the image feature data and the group feature data to retrieve a first one or more images; and generating, by the one or more computing devices, first object information for the object based on image information of the first one or more images. method.

2. The method of claim 1 , wherein each of the image feature data, the user feature data, and the group feature data includes one or more feature vectors.

3. The method of claim 1 , wherein the user information includes at least one of demographic information of the user and behavioral information of the user.

4. Preprocessing the user information includes: encoding the user information to generate the user characteristic data; quantizing the generated user feature data; The method of claim 1.

5. The method of claim 1 , wherein determining the user group includes determining whether a difference between the user characteristic data and the group characteristic data is within a predetermined value.

6. performing, by the one or more computing devices, a search of the image database based on the image feature data to retrieve a second one or more images; and generating, by the one or more computing devices, second object information for the object based on image information of the second one or more images. The method of claim 1.

7. generating the first object information includes determining a confidence value for the first object information; The method further comprises: comparing the confidence value with a predetermined value; and in response to determining that the confidence value is higher than the predetermined value, selecting the first object information as object information for the object. The method according to claim 6.

8. Prior to acquiring the input image, grouping a plurality of users stored in the user database into a plurality of user groups including the user group based on user information of each user; extracting group characteristic data for each user group based on user information of one or more users in the same group; The method of claim 1.

9. 9. The method of claim 8, further comprising updating the user database at a predetermined frequency.

10. 2. The method of claim 1, further comprising: generating, by the one or more computing devices, a search query to search for one or more objects related to the object included in the input image based on the first object information.

11. at least one processor; at least one memory storing instructions that, when executed by the at least one processor, cause the at least one processor to perform operations, the operations including: obtaining an input image including an object and a user identifier for a user; pre-processing the input image to generate image feature data for the input image; pre-processing user information corresponding to the user identifier to generate user characteristic data for the user, the user information and the user characteristic data being stored in a user database; determining a user group based on the user characteristic data to obtain group characteristic data for the user group, the user group and the group characteristic data being stored in the user database; performing a search of an image database based on the image feature data and the group feature data to retrieve a first one or more images; generating first object information for the object based on image information of the first one or more images. Computing device.

12. 12. The computing device of claim 11, wherein determining the user group comprises determining whether a difference between the user characteristic data and the group characteristic data is within a predetermined value.

13. The generating the first object information includes determining a confidence value of the first object information, the operations further comprising: performing a search of the image database based on the image feature data to retrieve a second one or more images; generating, using the one or more computing devices, second object information for the object based on image information of the second one or more images; comparing the confidence value with a predetermined value; and in response to determining that the confidence value is higher than the predetermined value, selecting the first object information as object information for the object. The computing device of claim 11.

14. The operations may further include: Prior to acquiring the input image, grouping a plurality of users stored in the user database into a plurality of user groups including the user group based on user information of each user; extracting group characteristic data for each user group based on user information of one or more users in the same group; The computing device of claim 11.

15. A non-transitory computer-readable storage medium storing instructions executable by a computing device to cause the computing device to perform operations, the operations including: obtaining an input image including an object and a user identifier for a user; pre-processing the input image to generate image feature data for the input image; pre-processing user information corresponding to the user identifier to generate user characteristic data for the user, the user information and the user characteristic data being stored in a user database; determining a user group based on the user characteristic data to obtain group characteristic data for the user group, the user group and the group characteristic data being stored in the user database; performing a search of an image database based on the image feature data and the group feature data to retrieve a first one or more images; generating first object information for the object based on image information of the first one or more images. A non-transitory computer-readable storage medium.

16. 16. The non-transitory computer-readable storage medium of claim 15, wherein the user information includes at least one of demographic information of the user and behavioral information of the user.

17. Preprocessing the user information includes: encoding the user information to generate the user characteristic data; quantizing the generated user feature data; 16. The non-transitory computer readable storage medium of claim 15.

18. 16. The non-transitory computer-readable storage medium of claim 15, wherein determining the user group includes determining whether a difference between the user characteristic data and the group characteristic data is within a predetermined value.

19. The generating the first object information includes determining a confidence value of the first object information, the operations further comprising: performing a search of the image database based on the image feature data to retrieve a second one or more images; generating, using the one or more computing devices, second object information for the object based on image information of the second one or more images; comparing the confidence value with a predetermined value; and in response to determining that the confidence value is higher than the predetermined value, selecting the first object information as object information for the object.

16. The non-transitory computer readable storage medium of claim 15.

20. The act of: Prior to acquiring the input image, grouping a plurality of users stored in the user database into a plurality of user groups including the user group based on user information of each user; extracting group characteristic data for each user group based on user information of one or more users in the same group; 16. The non-transitory computer readable storage medium of claim 15.

Images