Method and device for obtaining category prediction model and feature extraction model

Abstract

The present application discloses a method for acquiring a category prediction model, including: acquiring a target object; acquiring candidate similar objects corresponding to the target object; determining candidate objects of interest that the user is interested in from the candidate similar objects; obtaining candidate category information to which the target object belongs based on the category information to which the candidate objects of interest belong; using the target object and the candidate category information to which the target object belongs as sample data to train a target category prediction model, wherein the target category prediction model is used to predict the category information to which the object to be retrieved belongs based on the object to be retrieved. The method acquires candidate category information to which the target object belongs based on candidate similar objects and candidate objects of interest corresponding to the target object, and uses the target object and the candidate category information to which the target object belongs to train a target category prediction model, which can improve the acquisition speed and accuracy of the category prediction model.

Description

Technical Field

[0001] The present application relates to image retrieval technology, and more specifically to a method, device, electronic device, and storage device for acquiring a category prediction model. The present application also relates to a method, device, electronic device, and storage device for acquiring a feature extraction model. The present application also relates to an object retrieval method, device, electronic device, and storage device. Background Art

[0002] With the continuous development of computer technology, image retrieval methods have been applied to various fields of people's lives, greatly facilitating people's lives. Image retrieval methods refer to methods that extract feature information of an image to be retrieved after obtaining it, and then obtain similar images similar to the image to be retrieved. For example, on an e-commerce website, a user can use an image containing product information as a retrieval image, and quickly select the product of interest from similar images corresponding to the retrieval image provided by the e-commerce website.

[0003] The current image retrieval method usually uses sample data with supervised information to first train a category prediction model for extracting category information of the object contained in the image, and a feature extraction model for extracting image feature information; then, the feature information of the image to be retrieved is obtained through the category prediction model and the feature extraction model; then, similar images corresponding to the image to be retrieved are obtained based on the feature information. The sample data used to train the category prediction model and the feature extraction model are either from a public sample data set or a sample data set collected by the user. That is, the user first collects the original data by himself, and then, by investing a lot of manpower and material resources to annotate the collected original data, thereby generating sample data with supervised information.

[0004] However, when actually applying image retrieval methods, each field usually has its own unique characteristics. For example, in the field of face recognition, the focus is usually on how to accurately extract facial feature information in an image, and retrieve similar images based on the extracted facial feature information; while in the field of e-commerce, the focus is usually on how to accurately extract feature information related to the product in the image, and retrieve similar images based on the extracted feature information related to the product. However, since public sample data is usually not targeted, when the category prediction model and feature extraction model trained according to the public sample data are used to perform image retrieval in various fields, the retrieval speed and accuracy are not good enough; and after obtaining sample data manually, the category prediction model and feature extraction model obtained by using the sample data to train can be used for targeted image retrieval according to the characteristics of each field. However, when obtaining sample data in this way and using it to train the category prediction model and feature extraction model, personnel are required to have targeted professional domain knowledge, which invisibly increases the difficulty of obtaining sample data; at the same time, manually annotating large amounts of raw data to obtain targeted sample data is a large workload, which wastes manpower and material resources, and will lead to slow acquisition of the category prediction model and feature extraction model, and the model cannot be put into use quickly; and if the error rate of the manually provided annotation information is high, it will also lead to inaccurate results of the obtained category prediction model and feature extraction model.

[0005] Therefore, the current image retrieval methods have the problem of not being able to easily and quickly obtain category prediction models and feature extraction models with high accuracy. Summary of the invention

[0006] The embodiment of the present application provides a method for obtaining a category prediction model to solve the problem that the existing image retrieval method cannot conveniently and quickly obtain a targeted and highly accurate category prediction model.

[0007] The present application embodiment provides a method for obtaining a category prediction model, including:

[0008] Acquire a target object; acquire candidate similar objects corresponding to the target object; determine candidate objects of interest of the user from the candidate similar objects; obtain candidate category information to which the target object belongs based on the category information to which the candidate objects of interest belong; use the target object and the candidate category information to which the target object belongs as sample data to train a target category prediction model, wherein the target category prediction model is used to predict the category information to which the object to be retrieved belongs based on the object to be retrieved.

[0009] Optionally, obtaining the candidate similar object corresponding to the target object includes: obtaining a first candidate similar object corresponding to the target object; obtaining user behavior data for the first candidate similar object; if the user behavior data for the first candidate similar object is obtained, using the first candidate similar object as the candidate similar object.

[0010] Optionally, it includes: if the user's behavior data for the first candidate similar object is not obtained, obtaining the candidate category information determined by the user, and obtaining the second candidate similar object corresponding to the target object based on the candidate category information, and using the second candidate similar object as the candidate similar object.

[0011] Optionally, determining the candidate objects of interest that the user is interested in from the candidate similar objects includes: acquiring behavior data of the user with respect to the candidate similar objects; and determining the candidate objects of interest that the user is interested in based on the behavior data.

[0012] Optionally, obtaining the candidate category information to which the target object belongs according to the category information to which the candidate object of attention belongs includes: obtaining the primary category information to which the candidate object of attention belongs, and using the primary category information to which the candidate object of attention belongs as the simple candidate category information to which the target object belongs.

[0013] Optionally, the method of obtaining the candidate category information to which the target object belongs based on the category information to which the candidate attention object belongs includes: obtaining the first-level category information to which the first candidate similar object belongs, using the first-level category information to which the first candidate similar object belongs as the negative candidate category information to which the target object belongs, and using the first-level category information to which the candidate attention object determined by the user among the candidate similar objects belongs as the difficult candidate category information to which the target object belongs.

[0014] Optionally, obtaining the candidate category information to which the target object belongs based on the category information to which the candidate object of attention belongs includes: obtaining leaf category information to which the candidate object of attention belongs, wherein the leaf category information is category information at the last level of the category tree corresponding to the candidate object of attention; obtaining virtual category information to which the target object belongs based on the leaf category information to which the candidate object of attention belongs, and using the virtual category information to which the target object belongs as the candidate category information to which the target object belongs, wherein the number of virtual category information is more than the number of first-level category information and no more than the number of leaf category information.

[0015] Optionally, the method of obtaining the virtual category information to which the target object belongs based on the leaf category information to which the candidate object of interest belongs includes: taking the leaf category information to which the candidate object of interest belongs as a node to obtain a leaf category information matrix; performing graph embedding processing on the leaf category information matrix to obtain a leaf category vector corresponding to the leaf category information matrix; and obtaining the virtual category information to which the target object belongs based on the leaf category vector.

[0016] Optionally, acquiring virtual category information to which the target object belongs according to the leaf category vector includes: performing cluster analysis on the leaf category vector to acquire virtual category information to which the target object belongs.

[0017] Optionally, taking the target object and the candidate category information to which the target object belongs as sample data to train and obtain the target category prediction model includes: taking the target object and the simple candidate category information to which the target object belongs as sample data to train and obtain the target category prediction model.

[0018] Optionally, taking the target object and the candidate category information to which the target object belongs as sample data to train a target category prediction model includes: taking the target object, the negative candidate category information to which the target object belongs, and the difficult candidate category information to which the target object belongs as sample data to train to obtain the target category prediction model.

[0019] Optionally, taking the target object and the candidate category information to which the target object belongs as sample data to train and obtain the target category prediction model includes: taking the target object and the virtual category information to which the target object belongs as sample data to train and obtain the target category prediction model.

[0020] Optionally, the target object and the candidate category information to which the target object belongs are used as sample data to train a target category prediction model, comprising: using at least two of the following sample data to train the target category prediction model: sample data consisting of the target object and simple candidate category information to which the target object belongs, sample data consisting of the target object and negative candidate category information to which the target object belongs and difficult candidate category information to which the target object belongs, and sample data consisting of the target object and virtual category information to which the target object belongs.

[0021] An embodiment of the present application also provides a method for obtaining a feature extraction model, including: obtaining a target object; obtaining candidate similar objects corresponding to the target object; determining candidate objects of interest that the user is interested in from the candidate similar objects; obtaining sample data based on the target object and the candidate objects of interest; and using the sample data to train a target feature extraction model, wherein the target feature extraction model is used to obtain feature information of an object to be retrieved based on the object to be retrieved.

[0022] Optionally, determining the candidate objects of interest that the user is interested in from the candidate similar objects includes: acquiring behavior data of the user with respect to the candidate similar objects; and determining the candidate objects of interest that the user is interested in based on the behavior data.

[0023] Optionally, obtaining sample data based on the target object and the candidate objects of interest includes: obtaining virtual feature information corresponding to the target object based on the candidate objects of interest; and obtaining sample data based on the target object and the virtual feature information corresponding to the target object.

[0024] Optionally, obtaining virtual feature information corresponding to the target object based on the candidate focus object includes: taking the candidate focus object as a node to obtain a candidate focus object matrix; performing graph embedding processing on the candidate focus object matrix to obtain a virtual feature vector corresponding to the candidate focus object; and obtaining virtual feature information corresponding to the target object based on the virtual feature vector of the candidate focus object.

[0025] Optionally, acquiring virtual feature information corresponding to the target object according to the virtual feature vector of the candidate object of interest includes: performing cluster analysis on the virtual feature vector of the candidate object of interest to acquire virtual feature information corresponding to the target object.

[0026] Optionally, obtaining sample data based on the target object and the candidate focus object includes: taking the candidate focus object as positive candidate labeling information corresponding to the target object, taking other objects in the candidate similar objects as negative candidate labeling information corresponding to the target object, and obtaining sample data based on the target object, the positive candidate labeling information and the negative candidate labeling information, wherein the other objects are objects in the candidate similar objects other than the candidate focus object.

[0027] Optionally, obtaining sample data based on the target object and the candidate objects of interest includes: obtaining positive sample sorting data corresponding to the target object, wherein the positive sample sorting data is data obtained after arranging the objects in the candidate similar objects in descending order according to their similarity to the target object; obtaining sample data based on the target object and the positive sample sorting data corresponding to the target object.

[0028] Optionally, it includes: using at least one of the following sample data to train a target feature extraction model: sample data consisting of the target object and virtual feature information corresponding to the target object, sample data consisting of the target object, positive candidate labeling information corresponding to the target object, and negative candidate labeling information corresponding to the target object, and sample data consisting of the target object and positive sample sorting data corresponding to the target object.

[0029] An embodiment of the present application also provides an object retrieval method, including: obtaining an object to be retrieved; inputting the object to be retrieved into a target category prediction model to obtain category information to which the object to be retrieved belongs, wherein the target category prediction model is a model obtained using the above-mentioned category prediction model acquisition method; inputting the object to be retrieved into a target feature extraction model corresponding to the category information to obtain feature information of the object to be retrieved, wherein the target feature extraction model is a model obtained using the above-mentioned feature extraction model acquisition method; based on the feature information, obtaining similar objects corresponding to the object to be retrieved.

[0030] Optionally, it also includes: if filtering instruction information for the similar objects is obtained, then: displaying the attribute information of the similar objects; obtaining target attribute information of the similar objects determined by the user; displaying similar objects corresponding to the target attribute information; if filtering instruction information for the similar objects is not obtained, directly displaying the similar objects.

[0031] An embodiment of the present application also provides a method for obtaining a commodity category prediction model, comprising: obtaining an original retrieval image containing a commodity object; obtaining a candidate similar image corresponding to the original retrieval image; determining a candidate focus image of interest to a user from the candidate similar images; obtaining candidate category information to which the commodity object in the original retrieval image belongs based on the category information to which the commodity object contained in the candidate focus image belongs; using the original retrieval image and the candidate category information to which the commodity object in the original retrieval image belongs as sample data, and training to obtain a target commodity category prediction model, wherein the target commodity category prediction model is used to predict the category information to which the commodity object in the image to be retrieved belongs based on the image to be retrieved.

[0032] An embodiment of the present application also provides a method for obtaining a product feature extraction model, comprising: obtaining an original retrieval image containing a product object; obtaining a candidate similar image corresponding to the original retrieval image; determining a candidate focus image of interest to a user from the candidate similar images; obtaining sample data based on the original retrieval image and the candidate focus image; and using the sample data to train a target product feature extraction model, wherein the target product feature extraction model is used to obtain feature information of the product object in the image to be retrieved based on the image to be retrieved.

[0033] An embodiment of the present application also provides an image retrieval method, comprising: obtaining an image to be retrieved that contains a commodity object; inputting the image to be retrieved into a target commodity category prediction model to obtain category information to which the commodity object in the image to be retrieved belongs, wherein the target commodity category prediction model is a model obtained using the above-mentioned commodity category prediction model acquisition method; inputting the image to be retrieved into a target commodity feature extraction model corresponding to the category information to obtain feature information of the commodity object in the image to be retrieved, wherein the target commodity feature extraction model is a model obtained using the above-mentioned commodity feature extraction model acquisition method; and obtaining a similar image corresponding to the image to be retrieved based on the feature information.

[0034] Optionally, the method further includes: if filtering instruction information for commodity objects contained in similar images is obtained, then: displaying attribute information of commodity objects contained in the similar images; obtaining target attribute information of commodity objects contained in the similar images determined by the user; and displaying similar images corresponding to the target attribute information; if filtering instruction information for commodity objects contained in similar images is not obtained, directly displaying the similar images;

[0035] If the accurate search instruction information input by the user is not obtained, the similar images are directly displayed.

[0036] Optionally, the attribute information includes a stock quantity unit of the commodity image contained in the similar image.

[0037] An embodiment of the present application also provides a device for acquiring a category prediction model, including: an object acquisition unit, used to acquire a target object; a similar object acquisition unit, used to acquire candidate similar objects corresponding to the target object; an attention object acquisition unit, used for a user to determine a candidate attention object of the user from the candidate similar objects; a category information acquisition unit, used to obtain candidate category information to which the target object belongs based on the category information to which the candidate attention object belongs; and a model training unit, used to train a target category prediction model using the target object and the candidate category information to which the target object belongs as sample data, wherein the target category prediction model is used to predict the category information to which the object to be retrieved belongs based on the object to be retrieved.

[0038] The present application also provides an electronic device, including:

[0039] processor;

[0040] A memory is used to store a program of a method for obtaining a category prediction model. After the device is powered on and the program of the method for obtaining a category prediction model is run by the processor, the following steps are performed:

[0041] Acquire a target object; acquire candidate similar objects corresponding to the target object; determine candidate objects of interest of the user from the candidate similar objects; obtain candidate category information to which the target object belongs based on the category information to which the candidate objects of interest belong; use the target object and the candidate category information to which the target object belongs as sample data to train a target category prediction model, wherein the target category prediction model is used to predict the category information to which the object to be retrieved belongs based on the object to be retrieved.

[0042] The embodiment of the present application further provides a storage device storing a program of a method for obtaining a category prediction model, wherein the program is executed by a processor to perform the following steps:

[0043] Acquire a target object; acquire candidate similar objects corresponding to the target object; determine candidate objects of interest of the user from the candidate similar objects; obtain candidate category information to which the target object belongs based on the category information to which the candidate objects of interest belong; use the target object and the candidate category information to which the target object belongs as sample data to train a target category prediction model, wherein the target category prediction model is used to predict the category information to which the object to be retrieved belongs based on the object to be retrieved.

[0044] An embodiment of the present application also provides an acquisition device of a feature extraction device, comprising: an object acquisition unit, used to acquire a target object; a similar object acquisition unit, used to acquire candidate similar objects corresponding to the target object; an attention object acquisition unit, used to determine candidate attention objects that a user pays attention to from the candidate similar objects; a sample data acquisition unit, used to acquire sample data based on the target object and the candidate attention objects; and a model training unit, used to train a target feature extraction model using the sample data, wherein the target feature extraction model is used to acquire feature information of an object to be retrieved based on the object to be retrieved.

[0045] The present application also provides an electronic device, including:

[0046] processor;

[0047] A memory is used to store a program of a method for acquiring a feature extraction model. After the device is powered on and the program of the method for acquiring a feature extraction model is run by the processor, the following steps are performed:

[0048] Acquire a target object; acquire candidate similar objects corresponding to the target object; determine candidate objects of interest of the user from the candidate similar objects; acquire sample data based on the target object and the candidate objects of interest; and use the sample data to train a target feature extraction model, wherein the target feature extraction model is used to acquire feature information of the object to be retrieved based on the object to be retrieved.

[0049] The embodiment of the present application further provides a storage device storing a program of a method for obtaining a feature extraction model, wherein the program is executed by a processor to perform the following steps:

[0050] Acquire a target object; acquire candidate similar objects corresponding to the target object; determine candidate objects of interest of the user from the candidate similar objects; acquire sample data based on the target object and the candidate objects of interest; and use the sample data to train a target feature extraction model, wherein the target feature extraction model is used to acquire feature information of the object to be retrieved based on the object to be retrieved.

[0051] An embodiment of the present application also provides an object retrieval device, including: an object acquisition unit, used to acquire an object to be retrieved; a category information acquisition unit, used to input the object to be retrieved into a target category prediction model, and acquire category information to which the object to be retrieved belongs, wherein the target category prediction model is a model acquired using the above-mentioned category prediction model acquisition method; a feature information acquisition unit, used to input the object to be retrieved into a target feature extraction model corresponding to the category information, and acquire feature information of the object to be retrieved, wherein the target feature extraction model is a model acquired using the above-mentioned feature extraction model acquisition method; a similar object acquisition unit, used to acquire similar objects corresponding to the object to be retrieved based on the feature information.

[0052] The present application also provides an electronic device, including:

[0053] processor;

[0054] The memory is used to store a program of the image retrieval method. After the device is powered on and the program of the image retrieval method is run by the processor, the following steps are performed:

[0055] Acquire the object to be retrieved; input the object to be retrieved into a target category prediction model to acquire category information to which the object to be retrieved belongs, wherein the target category prediction model is a model acquired using the above-mentioned category prediction model acquisition method; input the object to be retrieved into a target feature extraction model corresponding to the category information to acquire feature information of the object to be retrieved, wherein the target feature extraction model is a model acquired using the above-mentioned feature extraction model acquisition method; based on the feature information, acquire similar objects corresponding to the object to be retrieved.

[0056] The embodiment of the present application further provides a storage device storing a program of the image retrieval method, the program is executed by a processor to perform the following steps:

[0057] Acquire the object to be retrieved; input the object to be retrieved into a target category prediction model to acquire category information to which the object to be retrieved belongs, wherein the target category prediction model is a model acquired using the above-mentioned category prediction model acquisition method; input the object to be retrieved into a target feature extraction model corresponding to the category information to acquire feature information of the object to be retrieved, wherein the target feature extraction model is a model acquired using the above-mentioned feature extraction model acquisition method; based on the feature information, acquire similar objects corresponding to the object to be retrieved.

[0058] The embodiment of the present application also provides a device for obtaining a commodity category prediction model, including:

[0059] An original retrieval image acquisition unit is used to acquire an original retrieval image containing a commodity object; a similar image acquisition unit is used to acquire candidate similar images corresponding to the original retrieval image; an attention image acquisition unit is used to determine a candidate attention image that a user is paying attention to from the candidate similar images; a candidate category information acquisition unit is used to obtain candidate category information to which the commodity object in the original retrieval image belongs based on the category information to which the commodity object contained in the candidate attention image belongs; and a model acquisition unit is used to use the original retrieval image and the candidate category information to which the commodity object in the original retrieval image belongs as sample data to train a target commodity category prediction model, wherein the target commodity category prediction model is used to predict the category information to which the commodity object in the image to be retrieved belongs based on the image to be retrieved.

[0060] The present application also provides an electronic device, including:

[0061] processor;

[0062] The memory is used to store a program of a method for obtaining a commodity category prediction model. After the device is powered on and the program of the method for obtaining a commodity category prediction model is run by the processor, the following steps are performed:

[0063] Acquire an original search image containing a commodity object; acquire candidate similar images corresponding to the original search image; determine a candidate focus image of interest to a user from the candidate similar images; obtain candidate category information to which the commodity object in the original search image belongs based on the category information to which the commodity object contained in the candidate focus image belongs; use the original search image and the candidate category information to which the commodity object in the original search image belongs as sample data to train a target commodity category prediction model, wherein the target commodity category prediction model is used to predict the category information to which the commodity object in the image to be retrieved belongs based on the image to be retrieved.

[0064] The embodiment of the present application further provides a storage device storing a program of a method for obtaining a commodity category prediction model, wherein the program is executed by a processor to perform the following steps:

[0065] Acquire an original search image containing a commodity object; acquire candidate similar images corresponding to the original search image; determine a candidate focus image of interest to a user from the candidate similar images; obtain candidate category information to which the commodity object in the original search image belongs based on the category information to which the commodity object contained in the candidate focus image belongs; use the original search image and the candidate category information to which the commodity object in the original search image belongs as sample data to train a target commodity category prediction model, wherein the target commodity category prediction model is used to predict the category information to which the commodity object in the image to be retrieved belongs based on the image to be retrieved.

[0066] The embodiment of the present application also provides an acquisition device of a product feature extraction device, including: an original retrieval image acquisition unit, used to acquire an original retrieval image containing a product object; a similar image acquisition unit, used to acquire a candidate similar image corresponding to the original retrieval image; a focus image acquisition unit, used to determine a candidate focus image that a user focuses on from the candidate similar images; a sample data acquisition unit, used to acquire sample data based on the original retrieval image and the candidate focus image; a model acquisition unit, used to use the sample data to train a target product feature extraction model, wherein the target product feature extraction model is used to acquire feature information of the product object in the image to be retrieved based on the image to be retrieved.

[0067] The present application also provides an electronic device, including:

[0068] processor;

[0069] The memory is used to store a program of a method for obtaining a commodity feature extraction model. After the device is powered on and the program of the method for obtaining a commodity feature extraction model is run by the processor, the following steps are performed:

[0070] Acquire an original search image containing a commodity object; acquire candidate similar images corresponding to the original search image; determine a candidate focus image of interest to a user from the candidate similar images; acquire sample data based on the original search image and the candidate focus images; and use the sample data to train a target commodity feature extraction model, wherein the target commodity feature extraction model is used to acquire feature information of the commodity object in the image to be retrieved based on the image to be retrieved.

[0071] The embodiment of the present application further provides a storage device storing a program of a method for obtaining a commodity feature extraction model, wherein the program is executed by a processor to perform the following steps:

[0072] Acquire an original search image containing a commodity object; acquire candidate similar images corresponding to the original search image; determine a candidate focus image of interest to a user from the candidate similar images; acquire sample data based on the original search image and the candidate focus images; and use the sample data to train a target commodity feature extraction model, wherein the target commodity feature extraction model is used to acquire feature information of the commodity object in the image to be retrieved based on the image to be retrieved.

[0073] The present application also provides an image retrieval device, including:

[0074] A unit for acquiring an image to be retrieved, used for acquiring an image to be retrieved containing a commodity object; a category information acquisition unit, used for inputting the image to be retrieved into a target commodity category prediction model, and acquiring category information to which the commodity object in the image to be retrieved belongs, wherein the target commodity category prediction model is a model acquired using the above-mentioned commodity category prediction model acquisition method; a feature information acquisition unit, used for inputting the image to be retrieved into a target commodity feature extraction model corresponding to the category information, and acquiring feature information of the commodity object in the image to be retrieved, wherein the target commodity feature extraction model is a model acquired using the above-mentioned commodity feature extraction model acquisition method; a similar image acquisition unit, used for acquiring similar images corresponding to the image to be retrieved based on the feature information.

[0075] The present application also provides an electronic device, including:

[0076] processor;

[0077] The memory is used to store a program of the image retrieval method. After the device is powered on and the program of the image retrieval method is run by the processor, the following steps are performed:

[0078] Acquire an image to be retrieved that contains a commodity object; input the image to be retrieved into a target commodity category prediction model to obtain category information to which the commodity object in the image to be retrieved belongs, wherein the target commodity category prediction model is a model obtained using the above-mentioned commodity category prediction model acquisition method; input the image to be retrieved into a target commodity feature extraction model corresponding to the category information to obtain feature information of the commodity object in the image to be retrieved, wherein the target commodity feature extraction model is a model obtained using the above-mentioned commodity feature extraction model acquisition method; based on the feature information, acquire a similar image corresponding to the image to be retrieved.

[0079] The embodiment of the present application further provides a storage device storing a program of the image retrieval method, the program is executed by a processor to perform the following steps:

[0080] Acquire an image to be retrieved that contains a commodity object; input the image to be retrieved into a target commodity category prediction model to obtain category information to which the commodity object in the image to be retrieved belongs, wherein the target commodity category prediction model is a model obtained using the above-mentioned commodity category prediction model acquisition method; input the image to be retrieved into a target commodity feature extraction model corresponding to the category information to obtain feature information of the commodity object in the image to be retrieved, wherein the target commodity feature extraction model is a model obtained using the above-mentioned commodity feature extraction model acquisition method; based on the feature information, acquire a similar image corresponding to the image to be retrieved.

[0081] Compared with the prior art, this application has the following advantages:

[0082] The embodiment of the present application provides a method for obtaining a category prediction model, including: obtaining a target object; obtaining a candidate similar object corresponding to the target object; determining a candidate object of interest that a user is concerned about from the candidate similar objects; obtaining candidate category information to which the target object belongs according to the category information to which the candidate object of interest belongs; using the target object and the candidate category information to which the target object belongs as sample data, training to obtain a target category prediction model, wherein the target category prediction model is used to predict the category information to which the object to be retrieved belongs according to the object to be retrieved. Compared with the method of using a public sample data set or manually annotating the target object to obtain sample data for training a category prediction model, the method provided in the embodiment of the present application, after obtaining the candidate similar object corresponding to the target image, conveniently and quickly obtains the candidate category information by determining the candidate object of interest that the user is concerned about among the candidate similar objects, so that the sample data finally obtained is more targeted; thereafter, the target object and the obtained candidate category information are used as sample data to train the target category prediction model, which can improve the acquisition speed and accuracy of the category prediction model.

[0083] The embodiment of the present application provides a method for acquiring a feature extraction model, including: acquiring a target object; acquiring candidate similar objects corresponding to the target object; determining the candidate objects of interest that the user is paying attention to from the candidate similar objects; acquiring sample data based on the target object and the candidate objects of interest; and using the sample data to train a target feature extraction model, wherein the target feature extraction model is used to acquire feature information of the object to be retrieved based on the object to be retrieved. Compared to using a public sample data set or manually annotating the target object to acquire sample data for training a feature extraction model, the method provided in the embodiment of the present application, after acquiring the candidate similar objects corresponding to the target image, determines the candidate objects of interest that the user is paying attention to from the candidate similar objects, acquires sample data based on the acquired candidate objects of interest, and uses the acquired sample data to train a target feature extraction model, which can improve the acquisition speed and accuracy of the category prediction model.

[0084] The embodiment of the present application provides an object retrieval method, including: obtaining an object to be retrieved; inputting the object to be retrieved into a target category prediction model to obtain category information to which the object to be retrieved belongs, wherein the target category prediction model is a model obtained using the above-mentioned category prediction model acquisition method; inputting the object to be retrieved into a target feature extraction model corresponding to the category information to obtain feature information of the object to be retrieved, wherein the target feature extraction model is a model obtained using the above-mentioned feature extraction model acquisition method; according to the feature information, obtaining similar objects corresponding to the object to be retrieved. This method can obtain more accurate similar objects corresponding to the object to be retrieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0085] Figure 1 It is a schematic diagram of the application scenario provided by the first embodiment of the present application.

[0086] Figure 2 It is a flowchart of a method for obtaining a category prediction model provided in the first embodiment of the present application.

[0087] Figure 3 It is a schematic diagram of the process of obtaining the first sample data provided in the first embodiment of the present application.

[0088] Figure 4 It is a schematic diagram of the process of obtaining the second sample data provided in the first embodiment of the present application.

[0089] Figure 5 It is a schematic diagram of the process of obtaining the third sample data provided in the first embodiment of the present application.

[0090] Figure 6 This is a flowchart of a method for acquiring a feature extraction model provided in the second embodiment of the present application.

[0091] Figure 7 It is a schematic diagram of the process of obtaining the first sample data provided in the second embodiment of the present application.

[0092] Figure 8 It is a schematic diagram of the process of obtaining the second sample data provided in the second embodiment of the present application.

[0093] Fig. 9 It is a schematic diagram of the process of obtaining the third sample data provided in the second embodiment of the present application.

[0094] Fig.10 This is a flowchart of an object retrieval method provided in the third embodiment of the present application.

[0095] Fig.11 This is a flowchart of a method for obtaining a commodity category prediction model provided in the fourth embodiment of the present application.

[0096] Fig.12 This is a flowchart of a method for obtaining a commodity feature extraction model provided in the fifth embodiment of the present application.

[0097] Fig.13 This is a flowchart of an image retrieval method provided in the sixth embodiment of the present application.

[0098] Fig.14 It is a schematic diagram of a device for acquiring a category prediction model provided in the seventh embodiment of the present application.

[0099] Fig.15 It is a schematic diagram of an electronic device provided in the eighth embodiment of the present application.

[0100] Fig.16 It is a schematic diagram of a device for acquiring a feature extraction model provided in the tenth embodiment of the present application.

[0101] Fig.17 It is a schematic diagram of an image retrieval device provided in the thirteenth embodiment of the present application.

[0102] Fig.18 It is a schematic diagram of a device for acquiring a commodity category prediction model provided in the sixteenth embodiment of the present application.

[0103] Fig.19 It is a schematic diagram of a device for acquiring a commodity feature extraction model provided in the nineteenth embodiment of the present application.

[0104] Fig. 20 This is a schematic diagram of an image retrieval device provided in the twenty-second embodiment of the present application. DETAILED DESCRIPTION

[0105] Many specific details are described in the following description to facilitate a full understanding of the present application. However, the present application can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of the present application, so the present application is not limited by the specific implementation disclosed below.

[0106] In order to enable technical personnel in this field to better understand the present application scheme, the specific application scenarios of its embodiments are described in detail below based on the category prediction model acquisition method, feature extraction model acquisition method and object retrieval method provided in the present application. It should be noted that since the object retrieval method provided in the embodiment of the present application uses the target category prediction model obtained by the category prediction model acquisition method provided in the embodiment of the present application and the target feature extraction model obtained by the feature extraction model acquisition method provided in the embodiment of the present application as the main technical means for object retrieval, the three methods provided in the embodiment of the present application are combined to describe their application scenarios in detail.

[0107] like Figure 1 As shown, it is a schematic diagram of the application scenario provided by the first embodiment of the present application. When implementing the method for obtaining the category prediction model, the method for obtaining the feature extraction model, and the object retrieval method provided in the embodiment of the present application, the target category prediction model that can quickly and accurately obtain the category information of the object to be retrieved, and the target feature extraction model that can obtain the feature information of the object to be retrieved are usually trained in advance by the computing device; thereafter, in order to find similar objects corresponding to the object to be retrieved, when the user issues an instruction to the computing device to retrieve similar objects corresponding to the object to be retrieved, the computing device responds to the user's instruction, inputs the acquired object to be retrieved into the target category prediction model, and obtains the category information to which the object to be retrieved belongs; thereafter, the object to be retrieved is input into the target feature extraction model corresponding to the category information, and the feature information of the object to be retrieved is obtained; thereafter, the similar objects corresponding to the object to be retrieved are obtained through the feature information of the object to be retrieved; thereafter, the computing device presents the similar objects on the user interface for the user to view.

[0108] It should be noted that, in this embodiment, the computing device used to extract and train the target category prediction model and the target feature extraction model generally refers to the server-side computing device. Of course, with the continuous development of technology, it can also be other computing devices, which is not limited here. In addition, when implementing the object retrieval method provided in the embodiment of the present application, it can be applied to the scenario where the client computing device interacts with the server computing device. Of course, it can also be implemented separately on the client or on the server. Among them, the client computing device generally refers to a mobile terminal device, such as a mobile phone, a tablet computer, etc.; the server computing device generally refers to a server.

[0109] For example, in an e-commerce platform, the platform server computing device usually trains in advance to obtain a target commodity category prediction model that can quickly and accurately obtain the category information of the commodity object, and a target commodity feature extraction model that can obtain the feature information of the commodity object. When a user uploads an image to be retrieved containing a commodity object through a client computing device and issues a search instruction, the client computing device sends a request message to the server computing device to search for similar images of the image to be retrieved; after the server computing device obtains the request message, it obtains the image to be retrieved from the request message, and inputs the image to be retrieved into the pre-acquired target commodity category prediction model to obtain the category information of the commodity object contained in the image to be retrieved; then, the image to be retrieved is input into the target commodity feature extraction model corresponding to the category information to obtain the feature vector of the commodity object contained in the image to be retrieved; then, based on the feature vector, similar feature vector information similar to the feature vector is obtained; then, similar commodity information corresponding to the similar feature vector information is obtained; then, similar commodity images corresponding to the similar commodity information are displayed on the user interface for the user to view.

[0110] It should be noted that the above application scenario is only a specific embodiment of the category prediction model acquisition method, feature extraction model acquisition method and object retrieval method provided in this application. The purpose of providing the application scenario embodiment is to facilitate the understanding of the above method provided in the embodiment of the present application, and is not used to limit the above method provided in the embodiment of the present application.

[0111] In the following, the method for obtaining the category prediction model, the method for obtaining the feature extraction model and the object retrieval method provided in the present application are described in detail through different embodiments.

[0112] In view of the problem that it is not possible to conveniently and quickly obtain a category prediction model with high accuracy when using public sample data or manually annotating sample data for training category prediction models, the first embodiment of the present application provides a method for obtaining a category prediction model. The method automatically obtains different types of sample data for training to obtain a category prediction model by performing multi-level data mining on user behavior data, such as user click data. Figure 2-Figure 5 Provide explanation.

[0113] Step S201, obtaining a target object.

[0114] The target object refers to a carrier containing the information to be retrieved, wherein the carrier may be an image, or a video frame in a video resource, or of course, an audio resource. For example, the target object may be an image containing product information.

[0115] The target object is obtained from a data file stored in a computing device that provides a retrieval service, according to the corresponding retrieval field identifier, wherein the data file can be a user behavior log, that is, a log file that records the user's operation behavior on the target object and the candidate similar object corresponding to the target object when performing a retrieval; the candidate similar object refers to the candidate object similar to the target object that is presented to the user interface for the user to view after the computing device performs a corresponding retrieval according to the object to be retrieved. For example, in an e-commerce platform, the computing device usually generates a log record of the user's retrieval object and the user's operation behavior on the candidate similar object corresponding to the retrieval object, and saves the log record to the user behavior log file. Therefore, through this log file, the user's historical retrieval object and the user's operation behavior on the candidate similar object corresponding to the historical retrieval object can generally be obtained. Of course, the data file storing the target object can also be a data table; in this embodiment, for the convenience of description, the method for obtaining the category prediction model described in this embodiment is explained by taking the target object in image format obtained from the user behavior log stored in the e-commerce platform as an example.

[0116] Step S202: Acquire candidate similar objects corresponding to the target object.

[0117] The candidate similar objects are candidate objects similar to the target object acquired by the computing device. When the target object is an image to be retrieved containing a commodity object, the candidate similar objects are candidate images containing the same or similar commodity objects.

[0118] For example, in an e-commerce platform, for an image to be retrieved containing the word "shoulder bag", the platform usually provides multiple similar images that are the same or similar to the "shoulder bag" so that users can view the corresponding products through the similar images.

[0119] Usually, after the computing device searches according to the target object, it will provide the retrieved candidate similar objects corresponding to the target object to the user for viewing. However, the following situations generally exist: 1. The candidate similar objects include objects that the user is concerned about, that is, when the first-level category of the candidate similar objects provided by the computing device is the same as the category of the target object, the correlation between the candidate similar objects and the target object is relatively large; 2. The candidate similar objects do not include objects that the user is concerned about, that is, when the first-level category of the candidate similar objects provided by the computing device is different from the category of the target object, the correlation between the candidate similar objects and the target object is usually small. At this time, the user is usually required to manually switch the retrieval category, and the computing device will re-retrieve and provide secondary candidate similar objects according to the category selected by the user.

[0120] For the first case mentioned above, the acquisition of candidate similar objects corresponding to the target object includes: acquiring a first candidate similar object corresponding to the target object; acquiring the user's behavior data for the first candidate similar object; if the user's behavior data for the first candidate similar object is acquired, then the first candidate similar object is used as the candidate similar object. That is, when the computing device performs a search based on the target object, acquires the first candidate similar object corresponding to the target object, and provides the first candidate similar object to the user for viewing, if the first candidate similar object includes an object that the user is concerned about, the computing device will usually record the user's behavior data for the first candidate similar object. Therefore, if the user's behavior data for the first candidate similar object can be acquired in the user behavior log file stored in the computing device, the first candidate similar object can be directly regarded as the final candidate similar object.

[0121] like Figure 3 As shown, it is a schematic diagram of the process of obtaining the first sample data provided by the first embodiment of the present application. Figure 3 It can be seen from the content that when the computing device searches for the target object, the first candidate similar objects it provides include the objects that the user is concerned about, that is, there are objects similar to the target object. Therefore, the first candidate similar object can be directly used as a candidate similar object, and sample data can be obtained based on the obtained candidate similar objects.

[0122] For the second situation mentioned above, the acquisition of candidate similar objects corresponding to the target object includes: if the user's behavior data for the first candidate similar object is not obtained, then the information of the category to be selected determined by the user is obtained, and based on the information of the category to be selected, the second candidate similar object corresponding to the target object is obtained, and the second candidate similar object is used as the candidate similar object. That is, when the computing device performs a search based on the target object, obtains the first candidate similar object corresponding to the target object, and provides the first candidate similar object to the user for viewing, if the first candidate similar object does not include the object of interest to the user, the computing device usually does not record the user's behavior data for the first candidate similar object. Therefore, if the user's behavior data for the first candidate similar object is not obtained in the user behavior log file stored in the computing device, it means that the first candidate similar object is not related to the target object, and the user needs to provide information related to the target object as auxiliary information so that the computing device can perform a secondary search based on the auxiliary information. In this embodiment, the auxiliary information is the category information to be selected of the target object determined by the user.

[0123] like Figure 4 As shown, it is a schematic diagram of the process of obtaining the second sample data provided by the first embodiment of the present application. Figure 4It can be seen from the content that when the computing device searches for the target object, there is a possibility that the first candidate similar objects it provides do not include the object that the user is concerned about, that is, there is no object similar to the target object in the first candidate similar objects. This situation is most likely caused by inaccurate category of the target object predicted by the computing device. Therefore, the user is required to manually switch the candidate category, and the computing device performs a secondary search based on the candidate category determined by the user, and provides the second candidate similar objects obtained after the secondary search. At this time, the second candidate similar objects usually include the object that the user is concerned about. Therefore, the second candidate similar objects can be used as candidate similar objects for obtaining sample data.

[0124] Step S203: determining a candidate object of interest to be concerned by the user from the candidate similar objects.

[0125] The candidate object of interest is an object that exists in the candidate similar objects and is of interest to the user. The object is usually highly similar to or highly correlated with the target object.

[0126] The step of determining the candidate objects of interest that the user is interested in from the candidate similar objects comprises: acquiring the user's behavior data with respect to the candidate similar objects; and determining the candidate objects of interest that the user is interested in based on the behavior data.

[0127] The behavior data may be the user's click data, or the user's viewing time for a certain object among the candidate similar objects, wherein the user's click data refers to the user's click operation on a certain object among the candidate similar objects; the user's viewing time for a certain object among the candidate similar objects may be the residence time of the object in the user interface recorded by the computing device.

[0128] Since the computing device records the user's behavior data in the corresponding data file, after obtaining the target object and the candidate similar objects corresponding to the target object from the data file in the above steps S201 and S202, the candidate objects of interest of the user can be further obtained. Figure 3-Figure 5 As shown, the candidate objects of interest to the user may be determined based on the user's click data on the candidate similar objects.

[0129] In addition, the candidate similar objects and candidate focused objects in this embodiment are not limited to including only one object. For the same target object, the candidate similar objects obtained by different users and the candidate focused objects focused on by different users among the candidate similar objects are all different.

[0130] For example, for the target object s_img1, the candidate similar objects obtained by different users may be a part of the 1000 images d_img1-d_img1000. At the same time, the user may also pay attention to multiple candidate attention objects among the candidate similar objects. For example, the candidate similar images corresponding to user user1 are d_img1-d_img10, and its candidate attention objects are d_img8 and d_img10; while the candidate similar images corresponding to user user2 are d_img5-d_img15, and its candidate attention objects are d_img5, d_img8, d_img10; and so on. It can be seen that for the same target object, multiple candidate attention objects may be obtained, and different candidate attention objects have different degrees of attention from users. Here, it can be a priori considered that the candidate attention object with higher user attention has a relatively higher similarity with the target object. Therefore, data mining can be performed on the candidate attention objects of the same target object by multiple users to mine the most relevant fine-grained candidate category information of the target object.

[0131] Step S204, obtaining candidate category information to which the target object belongs according to the category information to which the candidate attention object belongs.

[0132] First of all, it should be noted that when classifying a target object, there is usually a category tree corresponding to the target object. In the category tree, there is the first-level category information at the root node, the leaf category information at the last level, and the category information at other levels between the first-level category information and the leaf category information. For example, for a commodity object, there is usually a relatively general first-level category, such as a bag; correspondingly, there is also a most fine-grained leaf category information, such as a lady's handbag.

[0133] In practice, it is impossible to effectively obtain the category information that is most relevant to the target object and covers a moderate range as the labeling information of the target object. Therefore, the category prediction model obtained by training the current sample data has category prediction errors. To address this problem, the method for obtaining the category prediction model provided in this embodiment obtains the target object, the candidate similar objects corresponding to the target object, and the candidate attention objects corresponding to the target object. After performing multi-level data mining processing on the above data, the category information most relevant to the target object is mined from the above data as the candidate category information, and the candidate category information is used as the labeling information of the target object, thereby obtaining sample data for training the category prediction model, which is described in detail below.

[0134] The obtaining of candidate category information to which the target object belongs based on the category information to which the candidate object of attention belongs includes: obtaining primary category information to which the candidate object of attention belongs, and using the primary category information to which the candidate object of attention belongs as simple candidate category information to which the target object belongs, wherein the simple candidate category information is category information that can be easily obtained and is most relevant to the category information to which the target object actually belongs.

[0135] like Figure 3 As shown, since the candidate objects of interest of the user can be obtained from the candidate similar objects corresponding to the target object through a single search, the primary category information to which the candidate objects of interest belong can be used as the simple candidate category information of the target object, and the simple candidate category information can be used as the annotation information of the target object to obtain the first sample data for training the target category prediction model. For example, [target object, package] can be used as the first sample data of this embodiment.

[0136] In addition, after obtaining a second candidate similar object corresponding to the target object based on the candidate category information determined by the user, and taking the second candidate similar object as the candidate similar object, the candidate category information to which the target object belongs is obtained based on the category information to which the candidate object of interest belongs, including: obtaining the first-level category information to which the first candidate similar object belongs, taking the first-level category information to which the first candidate similar object belongs as the negative candidate category information to which the target object belongs, and taking the first-level category information to which the candidate object of interest determined by the user among the candidate similar objects belongs as the difficult candidate category information to which the target object belongs, wherein the negative candidate category information refers to category information that is not in the same category as the category information to which the target object belongs, and the difficult candidate category information refers to category information that is easily confused and not easily predicted.

[0137] like Figure 4 As shown, when searching according to the target object, the user needs to switch the category information before obtaining the candidate focus object in the candidate similar objects corresponding to the target object. Therefore, it can be considered that the category prediction of the target object is difficult. Therefore, when annotating the target object, the first-level category information to which the primary candidate similar object belongs can be used as the negative candidate category information to which the target object belongs, and the first-level category information to which the candidate focus object determined in the secondary candidate similar objects belongs can be used as the difficult candidate category information to which the target object belongs, and the negative candidate category information and the difficult candidate category information can be used as the annotation information of the target object to obtain the second sample data for training the target category prediction model. For example, [target object, bottled beverage category, beauty category] can be used as the second sample data of this embodiment.

[0138] The method of obtaining the candidate category information to which the target object belongs based on the category information to which the candidate object of attention belongs includes: obtaining leaf category information to which the candidate object of attention belongs, wherein the leaf category information is category information at the last level of the category tree corresponding to the candidate object of attention; obtaining virtual category information to which the target object belongs based on the leaf category information to which the candidate object of attention belongs, and using the virtual category information to which the target object belongs as the candidate category information to which the target object belongs, wherein the number of virtual category information is more than the number of first-level category information and no more than the number of leaf category information.

[0139] The step of obtaining the virtual category information to which the target object belongs based on the leaf category information to which the candidate object of interest belongs includes: taking the leaf category information to which the candidate object of interest belongs as a node to obtain a leaf category information matrix; performing graph embedding processing on the leaf category information matrix to obtain a leaf category vector corresponding to the leaf category information matrix; and obtaining the virtual category information to which the target object belongs based on the leaf category vector. Wherein, the step of obtaining the virtual category information to which the target object belongs based on the leaf category vector includes: performing cluster analysis on the leaf category vector to obtain the virtual category information to which the target object belongs.

[0140] That is, based on the behavior data of multiple candidate attention objects for the same target object, mining processing is performed, and the leaf category information described by the multiple candidate attention objects is used as a node to form a leaf category information matrix, wherein the matrix elements corresponding to the row and column nodes in the leaf category information matrix are the number of candidate attention objects corresponding to the two types of leaf category information that are commonly followed by users. Afterwards, the leaf category information matrix is ​​subjected to graph embedding processing by deep random walk technology or other graph embedding algorithms to obtain the leaf category vector corresponding to the leaf category information matrix. Afterwards, the leaf category vector is subjected to cluster analysis to obtain several virtual category information that is most relevant to the target object and most concerned by users. Usually, the virtual category information is several virtual categories with the highest degree of user attention, so its number may not be high. For example, when the number of primary category information is 10 and the number of leaf category information is 10,000, the number of virtual category information may be only 30. In addition, when using the graph embedding algorithm to construct the leaf category vector, side information can also be introduced to enable it to obtain a better representation.

[0141] Step S205 , using the target object and the candidate category information to which the target object belongs as sample data, training a target category prediction model, wherein the target category prediction model is used to predict the category information to which the object to be retrieved belongs based on the object to be retrieved.

[0142] The using the target object and the candidate category information to which the target object belongs as sample data to train and obtain the target category prediction model includes: using the target object and the simple candidate category information to which the target object belongs as sample data to train and obtain the target category prediction model.

[0143] That is, the first sample data obtained above is input into the target category prediction model, the target category prediction model is trained in a supervised manner, and the difference between the category prediction result output by the target category prediction model and the labeling information in the sample data is continuously calculated through the loss function to adjust the parameters of the target category prediction model so that the target category prediction model converges.

[0144] The method of using the target object and the candidate category information to which the target object belongs as sample data to train and obtain the target category prediction model includes: using the target object, the negative candidate category information to which the target object belongs, and the difficult candidate category information to which the target object belongs as sample data to train and obtain the target category prediction model.

[0145] That is, the second sample data obtained above is input into the target category prediction model, the target category prediction model is trained in a supervised manner, and the difference between the category prediction result output by the target category prediction model and the labeling information in the sample data is continuously calculated through the loss function to adjust the parameters of the target category prediction model so that the target category prediction model converges.

[0146] The using the target object and the candidate category information to which the target object belongs as sample data to train and obtain the target category prediction model includes: using the target object and the virtual category information to which the target object belongs as sample data to train and obtain the target category prediction model.

[0147] That is, the third sample data obtained above is input into the target category prediction model, the target category prediction model is trained in a supervised manner, and the difference between the category prediction result output by the target category prediction model and the labeling information in the sample data is continuously calculated through the loss function to adjust the parameters of the target category prediction model so that the target category prediction model converges.

[0148] In addition, in order to further improve the accuracy of the prediction results of the target category prediction model, the target object and the candidate category information to which the target object belongs are used as sample data to train the target category prediction model, including: using at least two of the following sample data to train the target category prediction model: sample data consisting of the target object and simple candidate category information to which the target object belongs, sample data consisting of the target object and negative candidate category information to which the target object belongs and difficult candidate category information to which the target object belongs, and sample data consisting of the target object and virtual category information to which the target object belongs.

[0149] This embodiment provides a preferred implementation method, that is, using the above three sample data to jointly train the target category prediction model to obtain a target category prediction model containing multiple branches. In this way, the entire loss function of the target category prediction model can be expressed as: Cat =ζ virtual +αζ hart-aware +βζ pair , where ζ Cat Used to represent the loss function of the entire target category prediction model, ζ virtual is the loss function of the branch that uses the third sample data for category prediction in the target category prediction model, ζ hart-aware is the loss function of the branch that uses the second sample data to perform category prediction using the target category prediction model, ζ pair is the loss function of the branch that uses the first sample data to predict the category in the target category prediction model, and α and β are hyperparameters. The multi-branch target category prediction model includes three branches that predict the category based on the three sample data. When the target category prediction model obtains the input target object, the category information of the target object is predicted simultaneously through the three branches inside it, that is, the first predicted category information corresponding to the target object is obtained through the branch used for category prediction based on the first sample data, the second predicted category information corresponding to the target object is obtained through the branch used for category prediction based on the second sample data, and the third predicted category information corresponding to the target object is obtained through the branch used for category prediction based on the third sample data. After that, a comprehensive evaluation is performed through the weight values ​​corresponding to the three branches to obtain the target category information corresponding to the target object.

[0150] As described above, by performing multi-level data mining on the data files containing user behavior data corresponding to the target object, three types of sample data for training the target category prediction model can be obtained conveniently, quickly and specifically. Afterwards, the category prediction model is trained with these three types of sample data, which can improve the acquisition speed and accuracy of the category prediction model.

[0151] In summary, the method for obtaining a category prediction model provided in the first embodiment of the present application includes: obtaining a target object; obtaining a candidate similar object corresponding to the target object; determining a candidate object of interest that a user is concerned about from the candidate similar objects; obtaining the candidate category information to which the target object belongs according to the category information to which the candidate object of interest belongs; using the target object and the candidate category information to which the target object belongs as sample data, training to obtain a target category prediction model, wherein the target category prediction model is used to predict the category information to which the object to be retrieved belongs according to the object to be retrieved. Compared with the method of using a public sample data set or manually annotating the target object to obtain sample data for training a category prediction model, the method provided in the embodiment of the present application, after obtaining the candidate similar object corresponding to the target image, conveniently and quickly obtains the candidate category information by determining the candidate object of interest that the user is concerned about among the candidate similar objects, so that the sample data finally obtained is more targeted; thereafter, the target object and the obtained candidate category information are used as sample data to train the target category prediction model, which can improve the acquisition speed and accuracy of the category prediction model.

[0152] In the above description, a method for obtaining a category prediction model is provided. Corresponding to the above method for obtaining a category prediction model, in order to further improve the accuracy of object retrieval, this application also provides a method for obtaining a feature extraction model. Figure 6 As shown, it is a flowchart of a method for obtaining a feature extraction model provided in the second embodiment of the present application, wherein some steps have been described in detail in the above-mentioned first embodiment, so the description here is relatively simple. For relevant matters, please refer to some descriptions in a method for obtaining a category prediction model provided in the first embodiment of the present application. The processing process described below is only illustrative.

[0153] like Figure 6 As shown, it is a flow chart of a method for obtaining a feature extraction model provided in the second embodiment of the present application. Figure 6-Figure 9 Please explain.

[0154] Step S601, obtaining a target object.

[0155] Step S602: Acquire candidate similar objects corresponding to the target object.

[0156] The determining of the candidate objects of interest that the user is interested in from the candidate similar objects includes: acquiring the user's behavior data with respect to the candidate similar objects; and determining the candidate objects of interest that the user is interested in based on the behavior data.

[0157] Step S603: determining candidate objects of interest to be concerned by the user from the candidate similar objects.

[0158] Step S604: acquiring sample data according to the target object and the candidate objects of interest.

[0159] The acquiring of sample data according to the target object and the candidate objects of interest includes: acquiring virtual feature information corresponding to the target object according to the candidate objects of interest; and acquiring sample data according to the target object and the virtual feature information corresponding to the target object.

[0160] The method of obtaining virtual feature information corresponding to the target object based on the candidate attention object includes: taking the candidate attention object as a node to obtain a candidate attention object matrix; performing graph embedding processing on the candidate attention object matrix to obtain a virtual feature vector corresponding to the candidate attention object; and obtaining virtual feature information corresponding to the target object based on the virtual feature vector of the candidate attention object.

[0161] The acquiring virtual feature information corresponding to the target object according to the virtual feature vector of the candidate object of interest includes: performing cluster analysis on the virtual feature vector of the candidate object of interest to acquire the virtual feature information corresponding to the target object.

[0162] like Figure 7 As shown, it is a schematic diagram of the acquisition process of the first sample data provided by the second embodiment of the present application. Different from the processing process in the first embodiment described above, in which the leaf category information to which the candidate attention object belongs is used as a node, the leaf category information matrix is ​​obtained, and the virtual category information corresponding to the target object is further obtained according to the leaf category information matrix. In the second embodiment of the present application, the user's behavior data of multiple candidate attention objects for the same target object is mined and processed to obtain multiple virtual feature information corresponding to the target object. The processing process is: taking multiple candidate attention objects corresponding to the target object as nodes, the candidate attention object matrix is ​​obtained, wherein, in the candidate attention object matrix, the values ​​between the row and column nodes are the similarities of the two candidate attention objects. Afterwards, the candidate attention object matrix is ​​subjected to graph embedding processing by deep random walk technology or other graph embedding algorithms to obtain the virtual feature vector corresponding to the candidate attention object; then, by performing cluster analysis on the virtual feature vector, several virtual feature information that is most relevant to the target object and most concerned by the user is obtained; then, the obtained virtual feature information is used as the annotation information of the target object to obtain the first sample data of the second embodiment of the present application.

[0163] It should be noted that in order to improve the ability of the final target feature extraction model to extract visual features, it is necessary to increase the amount of virtual feature information as much as possible here to divide the candidate focus objects and target objects into a more fine-grained clustered feature space, and to use virtual feature information based on the cluster index that reflects the user's mutual click embedding.

[0164] In addition, the obtaining of sample data based on the target object and the candidate focus object includes: taking the candidate focus object as the positive candidate labeling information corresponding to the target object, taking other objects in the candidate similar objects as the negative candidate labeling information corresponding to the target object, and obtaining sample data based on the target object, the positive candidate labeling information and the negative candidate labeling information, wherein the other objects are objects in the candidate similar objects other than the candidate focus object.

[0165] like Figure 8 As shown, it is a schematic diagram of the acquisition process of the second sample data provided by the second embodiment of the present application. That is, usually among the candidate similar objects corresponding to the target object, the user only pays attention to the object that is most similar to the target object, that is, the candidate attention object. It can be a priori believed that the candidate attention object has the highest similarity with the target object, that is, the same visual feature information contained by the two is the most, therefore, the candidate attention object can be used as the positive candidate annotation information of the target object; at the same time, it can be a priori believed that the object that is not paid attention to by the user among the candidate similar objects corresponding to the target object has a lower similarity with the target object, that is, the same visual feature information contained by the two is less, therefore, this type of object can be used as the negative candidate annotation information of the target object; further, a triple sample data can be formed, the form of which is [target object, positive candidate annotation information, negative candidate annotation information], that is, the second sample data of the second embodiment of the present application.

[0166] Among them, when obtaining the second sample data from the candidate similar objects corresponding to the target object, in order to make the annotation information in the final sample data more accurate, a powerful similarity calculation model can be used to filter out objects in the candidate similar objects that obviously have no relevance to the target object, and then the second sample data is obtained from the filtered candidate similar objects, that is, a more streamlined target feature extraction model is obtained based on the teacher-student network architecture.

[0167] In addition, the obtaining of sample data based on the target object and the candidate objects of interest includes: obtaining positive sample sorting data corresponding to the target object, wherein the positive sample sorting data is data obtained after arranging the objects in the candidate similar objects in descending order according to their similarity to the target object; obtaining sample data based on the target object and the positive sample sorting data corresponding to the target object.

[0168] like Fig. 9 As shown, it is a schematic diagram of the process of obtaining the third sample data provided by the second embodiment of the present application. According to the above description, it can be seen that the process of obtaining the second sample data is relatively simple. However, in the second sample data, we directly use the objects in the candidate similar objects that are not followed by the user as the negative candidate annotation information of the target object. However, among the candidate similar objects, there is false negative candidate annotation information that the user has not followed but has a high similarity with the target object. This type of object can actually be used as positive candidate annotation information for the target object. In order to eliminate false negative candidate annotation information, in this embodiment, the third sample data can also be obtained, that is, a powerful similarity calculation model is used to calculate the similarity between all objects in the candidate similar objects and the target object, and the objects in the candidate similar objects are sorted in descending order according to the size of the similarity, and a positive sample sorting data corresponding to the target object is obtained, and the positive sample sorting data is used as the annotation information of the target object for training the target feature extraction model.

[0169] Step S905: Use the sample data to train and obtain a target feature extraction model, wherein the target feature extraction model is used to obtain feature information of the object to be retrieved based on the object to be retrieved.

[0170] The method of using the sample data to train a target feature extraction model comprises: using at least one of the following sample data to train a target feature extraction model: sample data consisting of the target object and virtual feature information corresponding to the target object, sample data consisting of the target object, positive candidate labeling information corresponding to the target object, and negative candidate labeling information corresponding to the target object, and sample data consisting of the target object and positive sample sorting data corresponding to the target object.

[0171] That is, after three types of sample data for training the target feature extraction model are obtained through the above steps, the target feature extraction model can be trained using any one or more of the sample data obtained above.

[0172] In summary, the second embodiment of the present application provides a method for acquiring a feature extraction model, including: acquiring a target object; acquiring candidate similar objects corresponding to the target object; determining candidate objects of interest that the user is concerned about from the candidate similar objects; acquiring sample data based on the target object and the candidate objects of interest; and using the sample data to train a target feature extraction model, wherein the target feature extraction model is used to acquire feature information of the object to be retrieved based on the object to be retrieved. Compared to using a public sample data set or manually annotating the target object to acquire sample data for training a feature extraction model, the method provided in the embodiment of the present application, after acquiring candidate similar objects corresponding to the target image, determines the candidate objects of interest that the user is concerned about among the candidate similar objects, acquires sample data based on the acquired candidate objects of interest, and uses the acquired sample data to train a target feature extraction model, which can improve the acquisition speed and accuracy of the category prediction model.

[0173] In the above description, a method for obtaining a category prediction model and a method for obtaining a feature extraction model are provided. Corresponding to the above method for obtaining a category prediction model and the method for obtaining a feature extraction model, the present application also provides an object retrieval method. Fig.10 As shown, it is a flowchart of an object retrieval method provided in the third embodiment of the present application, wherein some steps have been described in detail in the above-mentioned first and second embodiments, so the description here is relatively simple. For relevant matters, please refer to the partial description in the method for obtaining a category prediction model provided in the first embodiment of the present application and the method for obtaining a feature extraction model provided in the second embodiment of the present application. The processing process described below is only illustrative.

[0174] like Fig.10 As shown, it is a flowchart of an object retrieval method provided in the third embodiment of the present application. Fig.10 Please explain.

[0175] Step S1001, obtaining the object to be searched.

[0176] Step S1002: input the object to be searched into a target category prediction model to obtain category information to which the object to be searched belongs, wherein the target category prediction model is a model obtained using the method for obtaining a category prediction model provided in the first embodiment.

[0177] Step S1003: input the object to be retrieved into a target feature extraction model corresponding to the category information to obtain feature information of the object to be retrieved, wherein the target feature extraction model is a model obtained using the feature extraction model acquisition method provided in the second embodiment.

[0178] Step S1004: acquiring similar objects corresponding to the object to be retrieved according to the feature information.

[0179] In the third embodiment of the present application, after similar objects corresponding to the object to be retrieved are obtained through step S1004, in order to obtain similar objects that are more relevant to the object to be retrieved and more in line with user needs, it further includes: if filtering instruction information for the similar objects is obtained, then: displaying the attribute information of the similar objects; obtaining target attribute information of the similar objects determined by the user; displaying similar objects corresponding to the target attribute information; if filtering instruction information for the similar objects is not obtained, directly displaying the similar objects.

[0180] The filtering instruction information for the similar objects refers to instruction information for filtering similar objects acquired by the computing device. It should be noted that the filtering instruction information can be instruction information issued by the user or instruction information preset in the computing device.

[0181] The target attribute information refers to the attribute information that the user selects from the attribute information provided by the computing device and that attracts the user's attention.

[0182] That is, after the computing device obtains the similar objects corresponding to the object to be retrieved, the computing device first determines whether it can obtain the filtering instruction information for the similar objects. If such instruction information exists, the computing device first obtains the attribute information of the similar objects; then, the attribute information of the similar objects is displayed on the user interface for the user to select; then, the computing device obtains the target attribute information of the similar objects determined by the user; then, the computing device obtains the similar objects corresponding to the target attribute information from the obtained similar objects, and displays the similar objects corresponding to the target attribute information. In addition, if the computing device does not obtain the filtering instruction information for the similar objects, the obtained similar objects can be directly displayed on the user interface. In addition, it should be noted that the above is only a specific implementation method provided in the third embodiment of the present application. In specific implementation, similar objects that are more in line with user needs can also be provided to users in other ways according to actual needs, which will not be repeated here.

[0183] In the above description, a method for obtaining a category prediction model is provided. Corresponding to the above method for obtaining a category prediction model, the present application also provides a method for obtaining a commodity category prediction model. Fig.11As shown, it is a flowchart of a method for obtaining a commodity category prediction model provided in the fourth embodiment of the present application, wherein some steps have been described in detail in the above-mentioned first embodiment, so the description here is relatively simple. For relevant matters, please refer to some descriptions in the method for obtaining a category prediction model provided in the first embodiment of the present application. The processing process described below is only illustrative.

[0184] like Fig.11 As shown, it is a flowchart of a method for obtaining a commodity category prediction model provided in the fourth embodiment of the present application. Fig.11 Please explain.

[0185] Step S1101, obtaining an original search image containing a product object.

[0186] Step S1102: Acquire candidate similar images corresponding to the original search image.

[0187] Step S1103: determining a candidate focused image that the user focuses on from the candidate similar images.

[0188] Step S1104: obtaining candidate category information to which the commodity object in the original search image belongs according to the category information to which the commodity object contained in the candidate focus image belongs.

[0189] Step S1105, using the original search image and the candidate category information to which the commodity object in the original search image belongs as sample data, training a target commodity category prediction model, wherein the target commodity category prediction model is used to predict the category information to which the commodity object in the image to be searched belongs based on the image to be searched.

[0190] In the above description, a method for obtaining a feature extraction model is provided. Corresponding to the above method for obtaining a feature extraction model, the present application also provides a method for obtaining a commodity feature extraction model. Fig.12 As shown, it is a flowchart of a method for obtaining a commodity feature extraction model provided in the fifth embodiment of the present application, wherein some steps have been described in detail in the above-mentioned second embodiment, so the description here is relatively simple. For relevant matters, please refer to some descriptions in a method for obtaining a feature extraction model provided in the second embodiment of the present application. The processing process described below is only illustrative.

[0191] like Fig.12 As shown, it is a flowchart of a method for obtaining a commodity feature extraction model provided in the fifth embodiment of the present application. Fig.12 Please explain.

[0192] Step S1201, obtaining an original search image containing a product object.

[0193] Step S1202: Acquire candidate similar images corresponding to the original search image.

[0194] Step S1203: determining a candidate focused image that the user focuses on from the candidate similar images.

[0195] Step S1204: acquiring sample data according to the original search image and the candidate focus image.

[0196] Step S1205: Use the sample data to train a target product feature extraction model, wherein the target product feature extraction model is used to obtain feature information of the product object in the image to be retrieved based on the image to be retrieved.

[0197] In the above description, an object retrieval method is provided. Corresponding to the above object retrieval method, the present application also provides an image retrieval method. Fig.13 As shown, it is a flow chart of an image retrieval method provided in the sixth embodiment of the present application, wherein some steps have been described in detail in the above-mentioned third embodiment, so the description here is relatively simple. For relevant matters, please refer to some descriptions in an image retrieval method provided in the third embodiment of the present application. The processing process described below is only illustrative.

[0198] Step S1301, obtaining an image to be searched that contains a product object.

[0199] Step S1302: input the image to be retrieved into the target product category prediction model to obtain the category information to which the product object in the image to be retrieved belongs, wherein the target product category prediction model is a model obtained using the product category prediction model acquisition method provided in the fourth embodiment above.

[0200] Step S1303, the image to be retrieved is input into a target product feature extraction model corresponding to the category information to obtain feature information of the product object in the image to be retrieved, wherein the target product feature extraction model is a model obtained using the method for obtaining the product feature extraction model provided in the fifth embodiment above.

[0201] Step S1304: acquiring similar images corresponding to the image to be retrieved according to the feature information.

[0202] In the sixth embodiment of the present application, after similar images corresponding to the image to be retrieved are obtained through step S1304, in order to obtain similar images that are more relevant to the commodity objects contained in the image to be retrieved and more in line with user needs, it further includes: if filtering instruction information for similar images is obtained, then: display the attribute information of the commodity objects contained in the similar images; obtain the target attribute information of the commodity objects contained in the similar images determined by the user; display similar images corresponding to the target attribute information; if filtering instruction information for similar images is not obtained, directly display the similar images. It should be noted that the attribute information includes the inventory unit of the commodity images contained in the similar images.

[0203] The filtering instruction information for the commodity objects contained in the similar images refers to the instruction information for filtering the commodity objects contained in the similar images acquired by the computing device. It should be noted that the filtering instruction information can be instruction information issued by the user or instruction information pre-set in the computing device.

[0204] The SKU (Stock Keeping Unit) is a unit for measuring the inventory in and out of a commodity object. For example, the SKU for textiles can usually be expressed as: specification, color and style.

[0205] That is, after the computing device obtains the similar image corresponding to the image to be retrieved, the computing device first determines whether it can obtain the filtering instruction information for the commodity object contained in the similar image. If such instruction information exists, the computing device first obtains the attribute information of the commodity object contained in the similar image, for example, the specification, color and style of the commodity object, or the model, storage space size, etc. of the commodity object; then, the attribute information of the commodity object contained in the similar image is displayed on the user interface for the user to select; then, the computing device obtains the target attribute information of the commodity object contained in the similar image determined by the user; then, the computing device obtains the similar image corresponding to the target attribute information from the obtained similar image, and displays the similar image corresponding to the target attribute information. In addition, if the computing device does not obtain the filtering instruction information for the commodity object contained in the similar image, the obtained similar image can be directly displayed on the user interface. Of course, the above is only a specific implementation method provided in the sixth embodiment of the present application. In the specific implementation, it can also provide users with similar images that are more in line with user needs in other ways according to actual needs, which will not be repeated here.

[0206] In the above description, a method for obtaining a category prediction model is provided. Corresponding to the above method for obtaining a category prediction model, the present application also provides a device for obtaining a category prediction model. Fig.14 As shown, it is a schematic diagram of a device for obtaining a category prediction model provided in the seventh embodiment of the present application. Since the device embodiment is basically similar to the method embodiment, the description is relatively simple. For relevant parts, please refer to the partial description of the method embodiment. The device embodiment described below is only exemplary. The device for obtaining a category prediction model provided in the seventh embodiment of the present application includes the following parts:

[0207] The object acquisition unit 1401 is used to acquire a target object.

[0208] The similar object acquisition unit 1402 is configured to acquire candidate similar objects corresponding to the target object.

[0209] In the attention object acquisition unit 1403, the user determines a candidate attention object that the user pays attention to from the candidate similar objects.

[0210] The category information acquisition unit 1404 is used to obtain the candidate category information to which the target object belongs according to the category information to which the candidate attention object belongs.

[0211] The model training unit 1405 is used to train a target category prediction model using the target object and the candidate category information to which the target object belongs as sample data, wherein the target category prediction model is used to predict the category information to which the object to be retrieved belongs based on the object to be retrieved.

[0212] Corresponding to the method for obtaining a category prediction model provided in the first embodiment, the present application also provides an electronic device, see Fig.15 , which is a schematic diagram of an electronic device provided in the eighth embodiment of the present application. Since the electronic device embodiment is basically similar to the method embodiment, the description is relatively simple. For the relevant parts, please refer to the partial description of the method embodiment. The electronic device embodiment described below is only illustrative. The electronic device provided in the eighth embodiment of the present application includes:

[0213] Processor 1501;

[0214] The memory 1502 is used to store a program of a method for obtaining a category prediction model. After the device is powered on and the processor runs the program of the method for obtaining a category prediction model, the following steps are performed:

[0215] Get the target object;

[0216] Acquire candidate similar objects corresponding to the target object;

[0217] Determine a candidate object of interest that the user is interested in from the candidate similar objects;

[0218] Obtaining candidate category information to which the target object belongs according to the category information to which the candidate object of interest belongs;

[0219] The target object and the candidate category information to which the target object belongs are used as sample data to train a target category prediction model, wherein the target category prediction model is used to predict the category information to which the object to be retrieved belongs based on the object to be retrieved.

[0220] Corresponding to the method for obtaining a category prediction model provided in the first embodiment, the present application also provides a storage device. Since the storage device embodiment is basically similar to the method embodiment, the description is relatively simple. For relevant parts, please refer to the partial description of the method embodiment. The storage device embodiment described below is only exemplary. A ninth embodiment of the present application provides a storage device that stores a program for obtaining a category prediction model. The program is executed by a processor to perform the following steps:

[0221] Get the target object;

[0222] Acquire candidate similar objects corresponding to the target object;

[0223] Determine a candidate object of interest that the user is interested in from the candidate similar objects;

[0224] Obtaining candidate category information to which the target object belongs according to the category information to which the candidate object of interest belongs;

[0225] The target object and the candidate category information to which the target object belongs are used as sample data to train a target category prediction model, wherein the target category prediction model is used to predict the category information to which the object to be retrieved belongs based on the object to be retrieved.

[0226] In the above description, a method for obtaining a feature extraction model is provided. Corresponding to the above method for obtaining a feature extraction model, the present application also provides a device for obtaining a feature extraction model. Fig.16 As shown, it is a schematic diagram of a device for acquiring a feature extraction model provided in the tenth embodiment of the present application. Since the device embodiment is basically similar to the method embodiment, the description is relatively simple. For relevant parts, please refer to the partial description of the method embodiment. The device embodiment described below is only exemplary. A device for acquiring a feature extraction model provided in the tenth embodiment of the present application includes the following parts:

[0227] The object acquisition unit 1601 is used to acquire a target object.

[0228] The similar object acquisition unit 1602 is configured to acquire candidate similar objects corresponding to the target object.

[0229] The attention object acquisition unit 1603 is used to determine the candidate attention object that the user pays attention to from the candidate similar objects.

[0230] The sample data acquisition unit 1604 is used to acquire sample data according to the target object and the candidate objects of interest.

[0231] The model training unit 1605 is used to use the sample data to train a target feature extraction model, wherein the target feature extraction model is used to obtain feature information of the object to be retrieved based on the object to be retrieved.

[0232] Corresponding to the method for obtaining a feature extraction model provided in the second embodiment, the present application also provides an electronic device, whose structure is basically similar to the structure of the electronic device described in the eighth embodiment of the present application. Since the electronic device embodiment is basically similar to the method embodiment, the description is relatively simple. For the relevant parts, please refer to the partial description of the method embodiment. The electronic device embodiment described below is only exemplary. An electronic device provided in the eleventh embodiment of the present application includes:

[0233] processor;

[0234] A memory is used to store a program of a method for acquiring a feature extraction model. After the device is powered on and the program of the method for acquiring a feature extraction model is run by the processor, the following steps are performed:

[0235] Get the target object;

[0236] Acquire candidate similar objects corresponding to the target object;

[0237] Determine a candidate object of interest that the user is interested in from the candidate similar objects;

[0238] Acquire sample data according to the target object and the candidate objects of interest;

[0239] The sample data is used to train a target feature extraction model, wherein the target feature extraction model is used to obtain feature information of the object to be retrieved based on the object to be retrieved.

[0240] Corresponding to the method for obtaining a feature extraction model provided in the second embodiment, the present application also provides a storage device. Since the storage device embodiment is basically similar to the method embodiment, the description is relatively simple. For the relevant parts, please refer to the partial description of the method embodiment. The storage device embodiment described below is only illustrative. A storage device provided in the twelfth embodiment of the present application stores a program for obtaining a feature extraction model. The program is executed by a processor to perform the following steps:

[0241] Get the target object;

[0242] Acquire candidate similar objects corresponding to the target object;

[0243] Determine a candidate object of interest that the user is interested in from the candidate similar objects;

[0244] Acquire sample data according to the target object and the candidate objects of interest;

[0245] The sample data is used to train a target feature extraction model, wherein the target feature extraction model is used to obtain feature information of the object to be retrieved based on the object to be retrieved.

[0246] In the above description, an object retrieval method is provided. Corresponding to the above object retrieval method, the present application also provides an object retrieval device. Fig.17 As shown, it is a schematic diagram of an object retrieval device provided in the thirteenth embodiment of the present application. Since the device embodiment is basically similar to the method embodiment, the description is relatively simple. For the relevant parts, please refer to the partial description of the method embodiment. The device embodiment described below is only exemplary. The object retrieval device provided in the thirteenth embodiment of the present application includes the following parts:

[0247] The object acquisition unit 1701 is used to acquire the object to be retrieved.

[0248] The category information acquisition unit 1702 is used to input the object to be retrieved into the target category prediction model to obtain the category information to which the object to be retrieved belongs, wherein the target category prediction model is a model obtained using the category prediction model acquisition method provided in the first embodiment of the present application.

[0249] The feature information acquisition unit 1703 is used to input the object to be retrieved into the target feature extraction model corresponding to the category information to obtain the feature information of the object to be retrieved, wherein the target feature extraction model is a model obtained using the feature extraction model acquisition method provided in the second embodiment of the present application.

[0250] The similar object acquisition unit 1704 is used to acquire similar objects corresponding to the object to be retrieved according to the feature information.

[0251] Corresponding to the object retrieval method provided in the third embodiment, the present application also provides an electronic device, whose structure is basically similar to the structure of the electronic device described in the eighth embodiment of the present application. Since the electronic device embodiment is basically similar to the method embodiment, the description is relatively simple. For the relevant parts, please refer to the partial description of the method embodiment. The electronic device embodiment described below is only illustrative. The electronic device provided in the fourteenth embodiment of the present application includes:

[0252] processor;

[0253] The memory is used to store a program of the image retrieval method. After the device is powered on and the program of the image retrieval method is run by the processor, the following steps are performed:

[0254] Get the object to be retrieved;

[0255] Inputting the object to be searched into a target category prediction model to obtain category information to which the object to be searched belongs, wherein the target category prediction model is a model obtained using the method for obtaining a category prediction model provided in the first embodiment of the present application;

[0256] Inputting the object to be searched into a target feature extraction model corresponding to the category information to obtain feature information of the object to be searched, wherein the target feature extraction model is a model obtained using the feature extraction model acquisition method provided in the second embodiment of the present application;

[0257] According to the feature information, similar objects corresponding to the object to be retrieved are obtained.

[0258] Corresponding to the object retrieval method provided in the third embodiment, the present application also provides a storage device. Since the storage device embodiment is basically similar to the method embodiment, the description is relatively simple. For the relevant parts, please refer to the partial description of the method embodiment. The storage device embodiment described below is only illustrative. A storage device provided in the fifteenth embodiment of the present application stores a program of the object retrieval method. The program is executed by a processor to perform the following steps:

[0259] Get the object to be retrieved;

[0260] Inputting the object to be searched into a target category prediction model to obtain category information to which the object to be searched belongs, wherein the target category prediction model is a model obtained using the method for obtaining a category prediction model provided in the first embodiment of the present application;

[0261] Inputting the object to be searched into a target feature extraction model corresponding to the category information to obtain feature information of the object to be searched, wherein the target feature extraction model is a model obtained using the feature extraction model acquisition method provided in the first embodiment of the present application;

[0262] According to the feature information, similar objects corresponding to the object to be retrieved are obtained.

[0263] In the above description, a method for obtaining a commodity category prediction model is provided. Corresponding to the above method for obtaining a commodity category prediction model, the present application also provides a device for obtaining a commodity category prediction model. Fig.18As shown, it is a schematic diagram of a device for obtaining a commodity category prediction model provided in the sixteenth embodiment of the present application. Since the device embodiment is basically similar to the method embodiment, the description is relatively simple. For relevant parts, please refer to the partial description of the method embodiment. The device embodiment described below is only exemplary. The device for obtaining a commodity category prediction model provided in the sixteenth embodiment of the present application includes the following parts:

[0264] The original search image acquisition unit 1801 is used to acquire the original search image containing the commodity object.

[0265] The similar image acquisition unit 1802 is used to acquire candidate similar images corresponding to the original search image.

[0266] The attention image acquisition unit 1803 is used to determine the candidate attention image that the user pays attention to from the candidate similar images.

[0267] The candidate category information acquiring unit 1804 is configured to acquire the candidate category information to which the commodity object in the original search image belongs according to the category information to which the commodity object contained in the candidate focus image belongs.

[0268] The model acquisition unit 1805 is used to use the original search image and the candidate category information to which the commodity object in the original search image belongs as sample data to train and obtain a target commodity category prediction model, wherein the target commodity category prediction model is used to predict the category information to which the commodity object in the image to be searched belongs based on the image to be searched.

[0269] Corresponding to the method for obtaining a commodity category prediction model provided in the fourth embodiment, the present application also provides an electronic device, whose structure is basically similar to the structure of the electronic device described in the eighth embodiment of the present application. Since the electronic device embodiment is basically similar to the method embodiment, the description is relatively simple. For the relevant parts, please refer to the partial description of the method embodiment. The electronic device embodiment described below is only exemplary. The electronic device provided in the seventeenth embodiment of the present application includes:

[0270] processor;

[0271] The memory is used to store a program of a method for obtaining a commodity category prediction model. After the device is powered on and the program of the method for obtaining a commodity category prediction model is run by the processor, the following steps are performed:

[0272] Obtaining an original search image containing a product object;

[0273] Acquire a candidate similar image corresponding to the original search image;

[0274] Determining a candidate focus image that the user focuses on from the candidate similar images;

[0275] Obtaining candidate category information to which the commodity object in the original search image belongs according to the category information to which the commodity object contained in the candidate focus image belongs;

[0276] The original search image and the candidate category information to which the commodity object in the original search image belongs are used as sample data to train a target commodity category prediction model, wherein the target commodity category prediction model is used to predict the category information to which the commodity object in the image to be searched belongs based on the image to be searched.

[0277] Corresponding to the method for obtaining a commodity category prediction model provided in the fourth embodiment, the present application also provides a storage device. Since the storage device embodiment is basically similar to the method embodiment, the description is relatively simple. For relevant parts, please refer to the partial description of the method embodiment. The storage device embodiment described below is only exemplary. The eighteenth embodiment of the present application provides a storage device, which stores a program for obtaining a commodity category prediction model. The program is executed by a processor to perform the following steps:

[0278] Obtaining an original search image containing a product object;

[0279] Acquire a candidate similar image corresponding to the original search image;

[0280] Determining a candidate focus image that the user focuses on from the candidate similar images;

[0281] Obtaining candidate category information to which the commodity object in the original search image belongs according to the category information to which the commodity object contained in the candidate focus image belongs;

[0282] The original search image and the candidate category information to which the commodity object in the original search image belongs are used as sample data to train a target commodity category prediction model, wherein the target commodity category prediction model is used to predict the category information to which the commodity object in the image to be searched belongs based on the image to be searched.

[0283] In the above description, a method for obtaining a product feature extraction model is provided. Corresponding to the above method for obtaining a product feature extraction model, the present application also provides a device for obtaining a product feature extraction model. Fig.19As shown, it is a schematic diagram of a device for obtaining a commodity feature extraction model provided in the nineteenth embodiment of the present application. Since the device embodiment is basically similar to the method embodiment, the description is relatively simple. For relevant parts, please refer to the partial description of the method embodiment. The device embodiment described below is only exemplary. The device for obtaining a commodity feature extraction model provided in the nineteenth embodiment of the present application includes the following parts:

[0284] The original search image acquisition unit 1901 is used to acquire the original search image containing the commodity object.

[0285] The similar image acquisition unit 1902 is used to acquire candidate similar images corresponding to the original search image.

[0286] The attention image acquisition unit 1903 is used to determine the candidate attention image that the user pays attention to from the candidate similar images.

[0287] The sample data acquisition unit 1904 is used to acquire sample data according to the original search image and the candidate focus image.

[0288] The model acquisition unit 1905 is used to use the sample data to train a target product feature extraction model, wherein the target product feature extraction model is used to acquire feature information of a product object in the image to be retrieved based on the image to be retrieved.

[0289] Corresponding to the method for obtaining a commodity feature extraction model provided in the fifth embodiment, the present application also provides an electronic device, whose structure is basically similar to the structure of the electronic device described in the eighth embodiment of the present application. Since the electronic device embodiment is basically similar to the method embodiment, the description is relatively simple. For the relevant parts, please refer to the partial description of the method embodiment. The electronic device embodiment described below is only exemplary. An electronic device provided in the twentieth embodiment of the present application includes:

[0290] processor;

[0291] The memory is used to store a program of a method for obtaining a commodity feature extraction model. After the device is powered on and the program of the method for obtaining a commodity feature extraction model is run by the processor, the following steps are performed:

[0292] Obtaining an original search image containing a product object;

[0293] Acquire a candidate similar image corresponding to the original search image;

[0294] Determining a candidate focus image that the user focuses on from the candidate similar images;

[0295] Acquire sample data according to the original search image and the candidate focus image;

[0296] The sample data is used to train a target commodity feature extraction model, wherein the target commodity feature extraction model is used to obtain feature information of a commodity object in the image to be retrieved based on the image to be retrieved.

[0297] Corresponding to the method for obtaining a commodity feature extraction model provided in the fifth embodiment, the present application also provides a storage device. Since the storage device embodiment is basically similar to the method embodiment, the description is relatively simple. For relevant parts, please refer to the partial description of the method embodiment. The storage device embodiment described below is only illustrative. A storage device provided in the twenty-first embodiment of the present application stores a program for obtaining a commodity feature extraction model. The program is executed by a processor to perform the following steps:

[0298] Obtaining an original search image containing a product object;

[0299] Acquire a candidate similar image corresponding to the original search image;

[0300] Determining a candidate focus image that the user focuses on from the candidate similar images;

[0301] Acquire sample data according to the original search image and the candidate focus image;

[0302] The sample data is used to train a target commodity feature extraction model, wherein the target commodity feature extraction model is used to obtain feature information of a commodity object in the image to be retrieved based on the image to be retrieved.

[0303] In the above description, an image retrieval method is provided. Corresponding to the above object retrieval method, the present application also provides an image retrieval device. Fig. 20 As shown, it is a schematic diagram of an image retrieval device provided in the twenty-second embodiment of the present application. Since the device embodiment is basically similar to the method embodiment, the description is relatively simple. For relevant parts, please refer to the partial description of the method embodiment. The device embodiment described below is only exemplary. The image retrieval device provided in the twenty-second embodiment of the present application includes the following parts:

[0304] The to-be-searched image acquisition unit 2001 is used to acquire the to-be-searched image containing the commodity object.

[0305] The category information acquisition unit 2002 is used to input the image to be retrieved into the target commodity category prediction model to obtain the category information to which the commodity object in the image to be retrieved belongs, wherein the target commodity category prediction model is a model obtained using the commodity category prediction model acquisition method provided in the fourth embodiment above.

[0306] The feature information acquisition unit 2003 is used to input the image to be retrieved into the target product feature extraction model corresponding to the category information to obtain the feature information of the product object in the image to be retrieved, wherein the target product feature extraction model is a model obtained using the product feature extraction model acquisition method provided in the fifth embodiment above.

[0307] The similar image acquisition unit 2004 is used to acquire similar images corresponding to the image to be retrieved according to the feature information.

[0308] Corresponding to the image retrieval method provided in the sixth embodiment, the present application also provides an electronic device, whose structure is basically similar to the structure of the electronic device described in the eighth embodiment of the present application. Since the electronic device embodiment is basically similar to the method embodiment, the description is relatively simple. For the relevant parts, please refer to the partial description of the method embodiment. The electronic device embodiment described below is only exemplary. An electronic device provided in the twenty-third embodiment of the present application includes:

[0309] processor;

[0310] The memory is used to store a program of the image retrieval method. After the device is powered on and the program of the image retrieval method is run by the processor, the following steps are performed:

[0311] Obtain the image to be retrieved containing the product object;

[0312] Inputting the image to be retrieved into a target commodity category prediction model to obtain category information to which the commodity object in the image to be retrieved belongs, wherein the target commodity category prediction model is a model obtained using the commodity category prediction model acquisition method according to claim 23;

[0313] Inputting the image to be retrieved into a target product feature extraction model corresponding to the category information to obtain feature information of the product object in the image to be retrieved, wherein the target product feature extraction model is a model obtained using the method for obtaining a product feature extraction model according to claim 24;

[0314] According to the feature information, a similar image corresponding to the image to be retrieved is obtained.

[0315] Corresponding to the image retrieval method provided in the sixth embodiment, the present application also provides a storage device. Since the storage device embodiment is basically similar to the method embodiment, the description is relatively simple. For the relevant parts, please refer to the partial description of the method embodiment. The storage device embodiment described below is only illustrative. A storage device provided in the twenty-fourth embodiment of the present application stores a program of the image retrieval method. The program is executed by a processor to perform the following steps:

[0316] Obtain the image to be retrieved containing the product object;

[0317] Inputting the image to be searched into a target commodity category prediction model to obtain category information to which the commodity object in the image to be searched belongs, wherein the target commodity category prediction model is a model obtained by using the commodity category prediction model acquisition method provided in the fourth embodiment;

[0318] Inputting the image to be retrieved into a target product feature extraction model corresponding to the category information to obtain feature information of the product object in the image to be retrieved, wherein the target product feature extraction model is a model obtained by using the method for obtaining a product feature extraction model provided in the fifth embodiment;

[0319] According to the feature information, a similar image corresponding to the image to be retrieved is obtained.

[0320] Although the present application is disclosed as above in the form of a preferred embodiment, it is not intended to limit the present application. Any technical personnel in this field may make possible changes and modifications without departing from the spirit and scope of the present application. Therefore, the scope of protection of the present application shall be based on the scope defined by the claims of the present application.

[0321] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0322] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0323] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include non-transitory media such as modulated data signals and carrier waves.

[0324] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

Claims

1. A method for obtaining a category prediction model, characterized in that: include: Get the target object; Acquire candidate similar objects corresponding to the target object; Determine a candidate object of interest that the user is interested in from the candidate similar objects; Obtaining candidate category information to which the target object belongs according to the category information to which the candidate object of interest belongs; The target object and the candidate category information to which the target object belongs are used as sample data to train a target category prediction model, wherein the target category prediction model is used to predict the category information to which the object to be retrieved belongs according to the object to be retrieved; Among them, obtaining the candidate similar object corresponding to the target object includes: obtaining a first candidate similar object corresponding to the target object; obtaining the user's behavior data for the first candidate similar object; if the user's behavior data for the first candidate similar object is obtained, then using the first candidate similar object as the candidate similar object; if the user's behavior data for the first candidate similar object is not obtained, then obtaining the to-be-selected category information determined by the user, and based on the to-be-selected category information, obtaining a second candidate similar object corresponding to the target object, and using the second candidate similar object as the candidate similar object.

2. The method for obtaining a category prediction model according to claim 1, characterized in that: The determining of the candidate objects of interest to be concerned by the user from the candidate similar objects comprises: Obtaining user behavior data for the candidate similar objects; According to the behavior data, candidate attention objects of the user are determined.

3. The method for obtaining a category prediction model according to claim 2, characterized in that: The obtaining the candidate category information to which the target object belongs according to the category information to which the candidate object of interest belongs includes: The first-level category information to which the candidate object of attention belongs is obtained, and the first-level category information to which the candidate object of attention belongs is used as the simple candidate category information to which the target object belongs.

4. The method for obtaining a category prediction model according to claim 1, characterized in that: The obtaining the candidate category information to which the target object belongs according to the category information to which the candidate object of interest belongs includes: Obtain the first-level category information to which the first candidate similar object belongs, use the first-level category information to which the first candidate similar object belongs as the negative candidate category information to which the target object belongs, and use the first-level category information to which the candidate focus object determined by the user to be concerned among the candidate similar objects belongs as the difficult candidate category information to which the target object belongs.

5. The method for obtaining a category prediction model according to claim 1, characterized in that: The obtaining the candidate category information to which the target object belongs according to the category information to which the candidate object of interest belongs includes: Acquire leaf category information to which the candidate object of interest belongs, wherein the leaf category information is category information at the last level of the category tree corresponding to the candidate object of interest; According to the leaf category information to which the candidate object of interest belongs, the virtual category information to which the target object belongs is obtained, and the virtual category information to which the target object belongs is used as the candidate category information to which the target object belongs, wherein the number of the virtual category information is more than the number of the first-level category information and no more than the number of the leaf category information.

6. The method for obtaining a category prediction model according to claim 5, characterized in that: The acquiring the virtual category information to which the target object belongs according to the leaf category information to which the candidate object of interest belongs includes: Taking the leaf category information to which the candidate object of interest belongs as a node, obtaining a leaf category information matrix; Performing graph embedding processing on the leaf category information matrix to obtain a leaf category vector corresponding to the leaf category information matrix; According to the leaf category vector, the virtual category information to which the target object belongs is obtained.

7. The method for obtaining a category prediction model according to claim 6, characterized in that: The acquiring virtual category information to which the target object belongs according to the leaf category vector includes: Perform cluster analysis on the leaf category vectors to obtain virtual category information to which the target object belongs.

8. The method for obtaining a category prediction model according to claim 3, characterized in that: The step of taking the target object and the candidate category information to which the target object belongs as sample data and training to obtain a target category prediction model includes: The target object and the simple candidate category information to which the target object belongs are used as sample data to train and obtain the target category prediction model.

9. The method for obtaining a category prediction model according to claim 4, characterized in that: The step of taking the target object and the candidate category information to which the target object belongs as sample data and training to obtain a target category prediction model includes: The target object, the negative candidate category information to which the target object belongs, and the difficult candidate category information to which the target object belongs are used as sample data to train and obtain the target category prediction model.

10. The method for obtaining a category prediction model according to claim 5, characterized in that: The step of taking the target object and the candidate category information to which the target object belongs as sample data and training to obtain a target category prediction model includes: The target object and the virtual category information to which the target object belongs are used as sample data to train and obtain the target category prediction model.

11. The method for obtaining a category prediction model according to claim 1, characterized in that: The step of taking the target object and the candidate category information to which the target object belongs as sample data and training to obtain a target category prediction model includes: The target category prediction model is obtained by training using at least two of the following sample data: sample data consisting of the target object and simple candidate category information to which the target object belongs, sample data consisting of the target object and negative candidate category information to which the target object belongs and difficult candidate category information to which the target object belongs, and sample data consisting of the target object and virtual category information to which the target object belongs.

12. A method for acquiring a feature extraction model, characterized in that: include: Get the target object; Acquire candidate similar objects corresponding to the target object; Determine a candidate object of interest that the user is interested in from the candidate similar objects; Acquire sample data according to the target object and the candidate objects of interest; Using the sample data to train a target feature extraction model, wherein the target feature extraction model is used to obtain feature information of the object to be retrieved according to the object to be retrieved; Among them, obtaining the candidate similar object corresponding to the target object includes: obtaining a first candidate similar object corresponding to the target object; obtaining the user's behavior data for the first candidate similar object; if the user's behavior data for the first candidate similar object is obtained, then using the first candidate similar object as the candidate similar object; if the user's behavior data for the first candidate similar object is not obtained, then obtaining the to-be-selected category information determined by the user, and based on the to-be-selected category information, obtaining a second candidate similar object corresponding to the target object, and using the second candidate similar object as the candidate similar object.

13. The method for acquiring a feature extraction model according to claim 12, characterized in that: The determining of the candidate objects of interest to be concerned by the user from the candidate similar objects comprises: Obtaining user behavior data for the candidate similar objects; According to the behavior data, candidate attention objects of the user are determined.

14. The method for acquiring a feature extraction model according to claim 12, characterized in that: The acquiring of sample data according to the target object and the candidate objects of interest includes: According to the candidate object of interest, obtaining virtual feature information corresponding to the target object; Sample data is acquired according to the target object and virtual feature information corresponding to the target object.

15. The method for acquiring a feature extraction model according to claim 12, characterized in that: The acquiring virtual feature information corresponding to the target object according to the candidate object of interest includes: Taking the candidate attention objects as nodes, obtaining a candidate attention object matrix; Performing graph embedding processing on the candidate attention object matrix to obtain virtual feature vectors corresponding to the candidate attention objects; According to the virtual feature vector of the candidate object of interest, virtual feature information corresponding to the target object is obtained.

16. The method for acquiring a feature extraction model according to claim 15, characterized in that: The acquiring virtual feature information corresponding to the target object according to the virtual feature vector of the candidate object of interest includes: Perform cluster analysis on the virtual feature vectors of the candidate objects of interest to obtain virtual feature information corresponding to the target object.

17. The method for acquiring a feature extraction model according to claim 12, characterized in that: The acquiring of sample data according to the target object and the candidate objects of interest includes: The candidate focus object is used as the positive candidate annotation information corresponding to the target object, and other objects in the candidate similar objects are used as negative candidate annotation information corresponding to the target object. Sample data is obtained according to the target object, the positive candidate annotation information and the negative candidate annotation information, wherein the other objects are objects in the candidate similar objects other than the candidate focus object.

18. The method for acquiring a feature extraction model according to claim 12, characterized in that: The acquiring of sample data according to the target object and the candidate objects of interest includes: Acquire positive sample ranking data corresponding to the target object, wherein the positive sample ranking data is data obtained by arranging the objects in the candidate similar objects in descending order according to their similarity to the target object; The sample data is obtained according to the target object and the positive sample sorting data corresponding to the target object.

19. The method for acquiring a feature extraction model according to claim 12, characterized in that: The method of using the sample data to train and obtain a target feature extraction model comprises: The target feature extraction model is obtained by training using at least one of the following sample data: sample data consisting of the target object and virtual feature information corresponding to the target object, sample data consisting of the target object, positive candidate labeling information corresponding to the target object, and negative candidate labeling information corresponding to the target object, and sample data consisting of the target object and positive sample sorting data corresponding to the target object.

20. An object retrieval method, characterized in that: include: Get the object to be retrieved; Inputting the object to be searched into a target category prediction model to obtain category information to which the object to be searched belongs, wherein the target category prediction model is a model obtained using the method for obtaining a category prediction model according to any one of claims 1 to 11; Inputting the object to be searched into a target feature extraction model corresponding to the category information to obtain feature information of the object to be searched, wherein the target feature extraction model is a model obtained by using the feature extraction model acquisition method described in any one of claims 12 to 19; According to the feature information, similar objects corresponding to the object to be retrieved are obtained.

21. The object retrieval method according to claim 20, characterized in that: Also includes: If the filtering instruction information for the similar objects is obtained, then: displaying the attribute information of the similar objects; Acquire target attribute information of the similar object determined by the user; display similar objects corresponding to the target attribute information; If no filtering instruction information for the similar objects is obtained, the similar objects are directly displayed.

22. A method for obtaining a commodity category prediction model, characterized in that: include: Obtaining an original search image containing a product object; Acquire a candidate similar image corresponding to the original search image; Determining a candidate focus image that the user focuses on from the candidate similar images; Obtaining candidate category information to which the commodity object in the original search image belongs according to the category information to which the commodity object contained in the candidate focus image belongs; The original search image and the candidate category information to which the commodity object in the original search image belongs are used as sample data to train a target commodity category prediction model, wherein the target commodity category prediction model is used to predict the category information to which the commodity object in the to-be-searched image belongs according to the to-be-searched image; Among them, obtaining the candidate similarity image corresponding to the target object includes: obtaining a first candidate similarity image corresponding to the target object; obtaining the user's behavior data for the first candidate similarity image; if the user's behavior data for the first candidate similarity image is obtained, then using the first candidate similarity image as the candidate similarity image; if the user's behavior data for the first candidate similarity image is not obtained, then obtaining the to-be-selected category information determined by the user, and based on the to-be-selected category information, obtaining a second candidate similarity image corresponding to the target object, and using the second candidate similarity image as the candidate similarity image.

23. A method for obtaining a commodity feature extraction model, characterized in that: include: Obtaining an original search image containing a product object; Acquire a candidate similar image corresponding to the original search image; Determining a candidate focus image that the user focuses on from the candidate similar images; Acquire sample data according to the original search image and the candidate focus image; Using the sample data to train a target commodity feature extraction model, wherein the target commodity feature extraction model is used to obtain feature information of a commodity object in the image to be retrieved based on the image to be retrieved; Among them, obtaining the candidate similarity image corresponding to the target object includes: obtaining a first candidate similarity image corresponding to the target object; obtaining the user's behavior data for the first candidate similarity image; if the user's behavior data for the first candidate similarity image is obtained, then using the first candidate similarity image as the candidate similarity image; if the user's behavior data for the first candidate similarity image is not obtained, then obtaining the to-be-selected category information determined by the user, and based on the to-be-selected category information, obtaining a second candidate similarity image corresponding to the target object, and using the second candidate similarity image as the candidate similarity image.

24. An image retrieval method, characterized in that: include: Obtain the image to be retrieved containing the product object; Inputting the image to be retrieved into a target commodity category prediction model to obtain category information to which the commodity object in the image to be retrieved belongs, wherein the target commodity category prediction model is a model obtained using the commodity category prediction model acquisition method according to claim 19; Inputting the image to be retrieved into a target product feature extraction model corresponding to the category information to obtain feature information of the product object in the image to be retrieved, wherein the target product feature extraction model is a model obtained using the method for obtaining a product feature extraction model according to claim 23; According to the feature information, a similar image corresponding to the image to be retrieved is obtained.

25. The image retrieval method according to claim 24, characterized in that: Also includes: If filtering instruction information for the commodity objects contained in the similar images is obtained, then: displaying the attribute information of the commodity objects contained in the similar images; obtaining the target attribute information of the commodity objects contained in the similar images determined by the user; and displaying the similar images corresponding to the target attribute information; If no filtering instruction information for the commodity objects included in the similar images is obtained, the similar images are directly displayed.

26. The image retrieval method according to claim 25, characterized in that: The attribute information includes a stock quantity unit of the product image included in the similar image.

27. A device for acquiring a category prediction model, characterized in that: include: An object acquisition unit, used for acquiring a target object; A similar object acquisition unit, used to acquire candidate similar objects corresponding to the target object; an attention object acquisition unit, wherein the user determines a candidate attention object that the user pays attention to from the candidate similar objects; A category information acquisition unit, configured to obtain candidate category information to which the target object belongs according to the category information to which the candidate object of interest belongs; A model training unit, used to train a target category prediction model by taking the target object and the candidate category information to which the target object belongs as sample data, wherein the target category prediction model is used to predict the category information to which the object to be retrieved belongs according to the object to be retrieved; Among them, the similar object acquisition unit is used to obtain candidate similar objects corresponding to the target object through the following steps: obtaining a first candidate similar object corresponding to the target object; obtaining the user's behavior data for the first candidate similar object; if the user's behavior data for the first candidate similar object is obtained, then the first candidate similar object is used as the candidate similar object; if the user's behavior data for the first candidate similar object is not obtained, then obtaining the to-be-selected category information determined by the user, and based on the to-be-selected category information, obtaining a second candidate similar object corresponding to the target object, and using the second candidate similar object as the candidate similar object.

28. An electronic device, characterized in that: include: processor; A memory is used to store a program of a method for obtaining a category prediction model. After the device is powered on and the program of the method for obtaining a category prediction model is run by the processor, the following steps are performed: Get the target object; Acquire candidate similar objects corresponding to the target object; Determine a candidate object of interest that the user is interested in from the candidate similar objects; Obtaining candidate category information to which the target object belongs according to the category information to which the candidate object of interest belongs; The target object and the candidate category information to which the target object belongs are used as sample data to train a target category prediction model, wherein the target category prediction model is used to predict the category information to which the object to be retrieved belongs according to the object to be retrieved; Among them, obtaining the candidate similar object corresponding to the target object includes: obtaining a first candidate similar object corresponding to the target object; obtaining the user's behavior data for the first candidate similar object; if the user's behavior data for the first candidate similar object is obtained, then using the first candidate similar object as the candidate similar object; if the user's behavior data for the first candidate similar object is not obtained, then obtaining the to-be-selected category information determined by the user, and based on the to-be-selected category information, obtaining a second candidate similar object corresponding to the target object, and using the second candidate similar object as the candidate similar object.

29. A storage device, characterized in that: A program storing a method for obtaining a category prediction model is executed by a processor to perform the following steps: Get the target object; Acquire candidate similar objects corresponding to the target object; Determine a candidate object of interest that the user is interested in from the candidate similar objects; Obtaining candidate category information to which the target object belongs according to the category information to which the candidate object of interest belongs; The target object and the candidate category information to which the target object belongs are used as sample data to train a target category prediction model, wherein the target category prediction model is used to predict the category information to which the object to be retrieved belongs according to the object to be retrieved; Among them, obtaining the candidate similar object corresponding to the target object includes: obtaining a first candidate similar object corresponding to the target object; obtaining the user's behavior data for the first candidate similar object; if the user's behavior data for the first candidate similar object is obtained, then using the first candidate similar object as the candidate similar object; if the user's behavior data for the first candidate similar object is not obtained, then obtaining the to-be-selected category information determined by the user, and based on the to-be-selected category information, obtaining a second candidate similar object corresponding to the target object, and using the second candidate similar object as the candidate similar object.

30. A device for acquiring a feature extraction model, characterized in that: include: An object acquisition unit, used for acquiring a target object; A similar object acquisition unit, used to acquire candidate similar objects corresponding to the target object; An attention object acquisition unit, used to determine a candidate attention object that the user pays attention to from the candidate similar objects; A sample data acquisition unit, configured to acquire sample data according to the target object and the candidate objects of interest; A model training unit, used to train a target feature extraction model using the sample data, wherein the target feature extraction model is used to obtain feature information of the object to be retrieved according to the object to be retrieved; Among them, the similar object acquisition unit is used to carry out the following steps: obtain a first candidate similar object corresponding to the target object; obtain the user's behavior data for the first candidate similar object; if the user's behavior data for the first candidate similar object is obtained, then use the first candidate similar object as the candidate similar object; if the user's behavior data for the first candidate similar object is not obtained, then obtain the to-be-selected category information determined by the user, and based on the to-be-selected category information, obtain a second candidate similar object corresponding to the target object, and use the second candidate similar object as the candidate similar object.

31. An electronic device, characterized in that: include: processor; A memory is used to store a program of a method for acquiring a feature extraction model. After the device is powered on and the program of the method for acquiring a feature extraction model is run by the processor, the following steps are performed: Get the target object; Acquire candidate similar objects corresponding to the target object; Determine a candidate object of interest that the user is interested in from the candidate similar objects; Acquire sample data according to the target object and the candidate objects of interest; Using the sample data to train a target feature extraction model, wherein the target feature extraction model is used to obtain feature information of the object to be retrieved according to the object to be retrieved; Among them, obtaining the candidate similar object corresponding to the target object includes: obtaining a first candidate similar object corresponding to the target object; obtaining the user's behavior data for the first candidate similar object; if the user's behavior data for the first candidate similar object is obtained, then using the first candidate similar object as the candidate similar object; if the user's behavior data for the first candidate similar object is not obtained, then obtaining the to-be-selected category information determined by the user, and based on the to-be-selected category information, obtaining a second candidate similar object corresponding to the target object, and using the second candidate similar object as the candidate similar object.

32. A storage device, characterized in that: A program storing a method for obtaining a feature extraction model is executed by a processor to perform the following steps: Get the target object; Acquire candidate similar objects corresponding to the target object; Determine a candidate object of interest that the user is interested in from the candidate similar objects; Acquire sample data according to the target object and the candidate objects of interest; Using the sample data to train a target feature extraction model, wherein the target feature extraction model is used to obtain feature information of the object to be retrieved according to the object to be retrieved; Among them, obtaining the candidate similar object corresponding to the target object includes: obtaining a first candidate similar object corresponding to the target object; obtaining the user's behavior data for the first candidate similar object; if the user's behavior data for the first candidate similar object is obtained, then using the first candidate similar object as the candidate similar object; if the user's behavior data for the first candidate similar object is not obtained, then obtaining the to-be-selected category information determined by the user, and based on the to-be-selected category information, obtaining a second candidate similar object corresponding to the target object, and using the second candidate similar object as the candidate similar object.

33. An object retrieval device, characterized in that: include: An object acquisition unit, used for acquiring an object to be retrieved; A category information acquisition unit, used for inputting the object to be searched into a target category prediction model to acquire category information to which the object to be searched belongs, wherein the target category prediction model is a model acquired using the method for acquiring a category prediction model according to any one of claims 1 to 11; A feature information acquisition unit, used for inputting the object to be retrieved into a target feature extraction model corresponding to the category information to acquire feature information of the object to be retrieved, wherein the target feature extraction model is a model acquired using the feature extraction model acquisition method according to any one of claims 12 to 19; The similar object acquisition unit is used to acquire similar objects corresponding to the object to be retrieved according to the feature information.

34. An electronic device, characterized in that: include: processor; The memory is used to store a program of the image retrieval method. After the device is powered on and the program of the image retrieval method is run by the processor, the following steps are performed: Get the object to be retrieved; Inputting the object to be searched into a target category prediction model to obtain category information to which the object to be searched belongs, wherein the target category prediction model is a model obtained using the method for obtaining a category prediction model according to any one of claims 1 to 11; Inputting the object to be searched into a target feature extraction model corresponding to the category information to obtain feature information of the object to be searched, wherein the target feature extraction model is a model obtained by using the feature extraction model acquisition method described in any one of claims 12 to 19; According to the feature information, similar objects corresponding to the object to be retrieved are obtained.

35. A storage device, characterized in that: A program storing an image retrieval method is executed by a processor to perform the following steps: Get the object to be retrieved; Inputting the object to be searched into a target category prediction model to obtain category information to which the object to be searched belongs, wherein the target category prediction model is a model obtained using the method for obtaining a category prediction model according to any one of claims 1 to 11; Inputting the object to be searched into a target feature extraction model corresponding to the category information to obtain feature information of the object to be searched, wherein the target feature extraction model is a model obtained by using the feature extraction model acquisition method described in any one of claims 12 to 19; According to the feature information, similar objects corresponding to the object to be retrieved are obtained.

36. A device for obtaining a commodity category prediction model, characterized in that: include: An original search image acquisition unit, used to acquire an original search image containing a commodity object; A similar image acquisition unit, used to acquire a candidate similar image corresponding to the original search image; A focus image acquisition unit, used to determine a candidate focus image that the user focuses on from the candidate similar images; A candidate category information acquisition unit, configured to obtain candidate category information to which the commodity object in the original search image belongs according to the category information to which the commodity object contained in the candidate focus image belongs; a model acquisition unit, configured to use the original search image and the candidate category information to which the commodity object in the original search image belongs as sample data, and train a target commodity category prediction model, wherein the target commodity category prediction model is used to predict the category information to which the commodity object in the to-be-searched image belongs according to the to-be-searched image; Among them, obtaining the candidate similar object corresponding to the target object includes: obtaining a first candidate similar object corresponding to the target object; obtaining the user's behavior data for the first candidate similar object; if the user's behavior data for the first candidate similar object is obtained, then using the first candidate similar object as the candidate similar object; if the user's behavior data for the first candidate similar object is not obtained, then obtaining the to-be-selected category information determined by the user, and based on the to-be-selected category information, obtaining a second candidate similar object corresponding to the target object, and using the second candidate similar object as the candidate similar object.

37. An electronic device, characterized in that: include: processor; A memory is used to store a program of a method for obtaining a commodity category prediction model. After the device is powered on and the program of the method for obtaining a commodity category prediction model is run by the processor, the following steps are performed: Obtaining an original search image containing a product object; Acquire a candidate similar image corresponding to the original search image; Determining a candidate focus image that the user focuses on from the candidate similar images; Obtaining candidate category information to which the commodity object in the original search image belongs according to the category information to which the commodity object contained in the candidate focus image belongs; The original search image and the candidate category information to which the commodity object in the original search image belongs are used as sample data to train a target commodity category prediction model, wherein the target commodity category prediction model is used to predict the category information to which the commodity object in the to-be-searched image belongs according to the to-be-searched image; Among them, obtaining the candidate similar object corresponding to the target object includes: obtaining a first candidate similar object corresponding to the target object; obtaining the user's behavior data for the first candidate similar object; if the user's behavior data for the first candidate similar object is obtained, then using the first candidate similar object as the candidate similar object; if the user's behavior data for the first candidate similar object is not obtained, then obtaining the to-be-selected category information determined by the user, and based on the to-be-selected category information, obtaining a second candidate similar object corresponding to the target object, and using the second candidate similar object as the candidate similar object.

38. A storage device, characterized in that: A program storing a method for obtaining a commodity category prediction model is executed by a processor to perform the following steps: Obtaining an original search image containing a product object; Acquire a candidate similar image corresponding to the original search image; Determining a candidate focus image that the user focuses on from the candidate similar images; Obtaining candidate category information to which the commodity object in the original search image belongs according to the category information to which the commodity object contained in the candidate focus image belongs; The original search image and the candidate category information to which the commodity object in the original search image belongs are used as sample data to train a target commodity category prediction model, wherein the target commodity category prediction model is used to predict the category information to which the commodity object in the to-be-searched image belongs according to the to-be-searched image; Among them, obtaining the candidate similar object corresponding to the target object includes: obtaining a first candidate similar object corresponding to the target object; obtaining the user's behavior data for the first candidate similar object; if the user's behavior data for the first candidate similar object is obtained, then using the first candidate similar object as the candidate similar object; if the user's behavior data for the first candidate similar object is not obtained, then obtaining the to-be-selected category information determined by the user, and based on the to-be-selected category information, obtaining a second candidate similar object corresponding to the target object, and using the second candidate similar object as the candidate similar object.

39. A device for obtaining a commodity feature extraction model, characterized in that: include: An original search image acquisition unit, used to acquire an original search image containing a commodity object; A similar image acquisition unit, used to acquire a candidate similar image corresponding to the original search image; A focus image acquisition unit, used to determine a candidate focus image that the user focuses on from the candidate similar images; A sample data acquisition unit, configured to acquire sample data according to the original search image and the candidate focus image; A model acquisition unit, used to train a target commodity feature extraction model using the sample data, wherein the target commodity feature extraction model is used to acquire feature information of a commodity object in the image to be retrieved based on the image to be retrieved; Among them, the similar image acquisition unit is used to acquire the candidate similar image corresponding to the original retrieval image through the following steps: acquiring a first candidate similar object corresponding to the target object; acquiring the user's behavior data for the first candidate similar object; if the user's behavior data for the first candidate similar object is acquired, then using the first candidate similar object as the candidate similar object; if the user's behavior data for the first candidate similar object is not acquired, then acquiring the to-be-selected category information determined by the user, and based on the to-be-selected category information, acquiring a second candidate similar object corresponding to the target object, and using the second candidate similar object as the candidate similar object.

40. An electronic device, characterized in that: include: processor; The memory is used to store a program of a method for obtaining a commodity feature extraction model. After the device is powered on and the program of the method for obtaining a commodity feature extraction model is run by the processor, the following steps are performed: Obtaining an original search image containing a product object; Acquire a candidate similar image corresponding to the original search image; Determining a candidate focus image that the user focuses on from the candidate similar images; Acquire sample data according to the original search image and the candidate focus image; Using the sample data to train a target commodity feature extraction model, wherein the target commodity feature extraction model is used to obtain feature information of a commodity object in the image to be retrieved based on the image to be retrieved; Among them, obtaining the candidate similar object corresponding to the target object includes: obtaining a first candidate similar object corresponding to the target object; obtaining the user's behavior data for the first candidate similar object; if the user's behavior data for the first candidate similar object is obtained, then using the first candidate similar object as the candidate similar object; if the user's behavior data for the first candidate similar object is not obtained, then obtaining the to-be-selected category information determined by the user, and based on the to-be-selected category information, obtaining a second candidate similar object corresponding to the target object, and using the second candidate similar object as the candidate similar object.

41. A storage device, characterized in that: A program storing a method for obtaining a commodity feature extraction model is executed by a processor to perform the following steps: Obtaining an original search image containing a product object; Acquire a candidate similar image corresponding to the original search image; Determining a candidate focus image that the user focuses on from the candidate similar images; Acquire sample data according to the original search image and the candidate focus image; Using the sample data to train a target commodity feature extraction model, wherein the target commodity feature extraction model is used to obtain feature information of a commodity object in the image to be retrieved based on the image to be retrieved; Among them, obtaining the candidate similar object corresponding to the target object includes: obtaining a first candidate similar object corresponding to the target object; obtaining the user's behavior data for the first candidate similar object; if the user's behavior data for the first candidate similar object is obtained, then using the first candidate similar object as the candidate similar object; if the user's behavior data for the first candidate similar object is not obtained, then obtaining the to-be-selected category information determined by the user, and based on the to-be-selected category information, obtaining a second candidate similar object corresponding to the target object, and using the second candidate similar object as the candidate similar object.

42. An image retrieval device, characterized in that: include: A to-be-searched image acquisition unit, used to acquire the to-be-searched image containing the commodity object; a category information acquisition unit, configured to input the image to be retrieved into a target commodity category prediction model, and acquire category information to which the commodity object in the image to be retrieved belongs, wherein the target commodity category prediction model is a model acquired using the commodity category prediction model acquisition method according to claim 22; a feature information acquisition unit, used for inputting the image to be retrieved into a target product feature extraction model corresponding to the category information, and acquiring feature information of the product object in the image to be retrieved, wherein the target product feature extraction model is a model acquired using the method for acquiring a product feature extraction model according to claim 23; The similar image acquisition unit is used to acquire similar images corresponding to the image to be retrieved according to the feature information.

43. An electronic device, characterized in that: include: processor; The memory is used to store a program of the image retrieval method. After the device is powered on and the program of the image retrieval method is run by the processor, the following steps are performed: Obtain the image to be retrieved containing the product object; Inputting the image to be retrieved into a target commodity category prediction model to obtain category information to which the commodity object in the image to be retrieved belongs, wherein the target commodity category prediction model is a model obtained using the method for obtaining a commodity category prediction model according to claim 22; Inputting the image to be retrieved into a target product feature extraction model corresponding to the category information to obtain feature information of the product object in the image to be retrieved, wherein the target product feature extraction model is a model obtained using the method for obtaining a product feature extraction model according to claim 23; According to the feature information, a similar image corresponding to the image to be retrieved is obtained.

44. A storage device, characterized in that: A program storing an image retrieval method is executed by a processor to perform the following steps: Obtain the image to be retrieved containing the product object; Inputting the image to be retrieved into a target commodity category prediction model to obtain category information to which the commodity object in the image to be retrieved belongs, wherein the target commodity category prediction model is a model obtained using the method for obtaining a commodity category prediction model according to claim 22; Inputting the image to be retrieved into a target product feature extraction model corresponding to the category information to obtain feature information of the product object in the image to be retrieved, wherein the target product feature extraction model is a model obtained using the method for obtaining a product feature extraction model according to claim 23; According to the feature information, a similar image corresponding to the image to be retrieved is obtained.

Images