A method and device for searching similar items, and a computer-readable storage medium

Abstract

The embodiment of the present invention discloses a method and device for searching similar items. The method for searching similar items includes: receiving an image to be searched, and determining local attribute information corresponding to the image to be searched; using a preset image local attribute encoding model, encoding the image to be searched and the local attribute information, and determining the image local attribute encoding; using the image local attribute encoding, performing vector retrieval from an item library, and determining recommended items with similar local attributes. By adopting the above scheme, the commodity attributes and commodity image information are deeply integrated, so that different attributes extract key information of images in different regions, thereby enhancing the diversity of similar search results.

Description

Technical Field

[0001] The present invention relates to the field of image recognition, and in particular to a method and device for searching similar objects, and a computer-readable storage medium. Background Art

[0002] With the popularization of mobile Internet technology and devices, people may find their favorite items in various scenarios and arouse the desire to buy. Based on this demand of users, major e-commerce platforms have provided the product function of "photo purchase", which can intelligently retrieve the same or similar items through the photos of items uploaded by users.

[0003] Currently, when searching for identical or similar items, on the one hand, only items with similar main bodies can be identified, but items with similar local details cannot be identified, which cannot meet the needs of fine-grained local detail retrieval. On the other hand, when there are identical items in the item library, a large number of duplicate or similar items will be recommended, resulting in a lack of diversity in the recommendation results. Summary of the invention

[0004] The embodiments of the present invention provide a method and device for searching similar items, and a computer-readable storage medium, which can perform fine-grained similar item searches based on local attributes of images, thereby enhancing the diversity of similar search results.

[0005] The technical solution of the present invention is achieved in this way:

[0006] An embodiment of the present invention provides a method for searching for similar items, the method comprising:

[0007] Receiving an image to be searched, and determining local attribute information corresponding to the image to be searched;

[0008] Encoding the image to be searched and the local attribute information by using a preset image local attribute coding model to determine the image local attribute coding;

[0009] By using the local attribute encoding of the image, vector retrieval is performed from an item library to determine recommended items with similar local attributes.

[0010] In the above solution, the method further comprises: encoding the image to be searched and the local attribute information by using a preset image local attribute encoding model, and before determining the image local attribute encoding.

[0011] The original samples obtained are grouped according to attributes, attribute values ​​and categories to determine training samples; the training samples include positive and negative samples with a preset ratio;

[0012] Using the training samples to train the initial image coding model to obtain attribute prediction values;

[0013] The loss function is used to compare the error between the attribute prediction value and the actual value of the preset attribute corresponding to the training sample, and the model parameters are adjusted to achieve iterative training of the initial image coding model until the preset image local attribute coding model is obtained.

[0014] In the above scheme, the training sample includes a training sample image and a plurality of sample local attribute information;

[0015] The step of training the initial image coding model using the training samples to obtain the attribute prediction value includes:

[0016] Using the initial image coding model, the training sample image and the plurality of sample local attribute information are encoded to determine a training sample image vector encoding and a plurality of sample local attribute vector encodings; the plurality of sample local attribute information is the determined attribute information corresponding to the training sample image;

[0017] The training sample image vector code is taken as input, and the training sample image vector code is learned and concatenated using a multi-head attention mechanism to obtain a processed training sample image vector code.

[0018] Conditionally fusing the processed training sample image vector code, the training sample image vector code and the multiple sample local attribute vector codes to obtain multiple fused training sample image local attribute codes;

[0019] Performing dimensionality reduction processing on the fused local attribute codes of the multiple training sample images to obtain the reprocessed local attribute codes of the multiple training sample images;

[0020] The reprocessed multiple training sample image local attribute codes, the fused multiple training sample image local attribute codes and the multiple sample local attribute vector codes are fused again to obtain the attribute prediction value.

[0021] In the above scheme, the acquired original samples are grouped according to attributes, attribute values ​​and categories to determine training samples, including:

[0022] The original samples are bucketed according to attributes, attribute values ​​and categories to obtain four groups of sample sets; wherein the four groups of sample sets are a first sample set with the same attribute value, a second sample set with different attribute values ​​but the same attribute, a third sample set with different attribute values ​​and attributes but the same category, and a fourth sample set with different attribute values, attributes and categories;

[0023] In the first sample set, determining positive samples;

[0024] In the second sample set, determining a first negative sample;

[0025] In the third sample set, determining a second negative sample;

[0026] In the fourth sample set, determining a third negative sample;

[0027] A training sample is determined according to any negative sample between the first negative sample, the second negative sample, and the third negative sample in combination with the positive sample.

[0028] In the above solution, the local attribute information is local attribute information of multiple different attributes;

[0029] The method of encoding the image to be searched and the local attribute information by using a preset image local attribute encoding model to determine the image local attribute encoding includes:

[0030] Using the preset image local attribute coding model, the image to be searched and the plurality of local attribute information are respectively encoded to obtain an image vector code and a plurality of attribute vector codes;

[0031] The preset image local attribute coding model is used to fuse the image vector coding and a plurality of attribute vector codings to obtain a plurality of image local attributes.

[0032] In the above scheme, the preset image local attribute coding model is used to fuse the image vector coding and the multiple attribute vector codings to obtain multiple image local attribute codings, including:

[0033] Taking the image vector code as input, learning and concatenating the image vector code using a multi-head attention mechanism to obtain a processed image vector code;

[0034] Conditionally fusing the processed image vector code, the image code, and the plurality of attribute vector codes to obtain a plurality of fused image local attribute codes;

[0035] Performing dimensionality reduction processing on the plurality of image local attribute codes to obtain a plurality of image local attribute codes after reprocessing;

[0036] The reprocessed multiple image local attribute codes, the fused multiple image local attribute codes and the multiple attribute vector codes are fused again to obtain the multiple image local attribute codes.

[0037] In the above scheme, the conditional fusion of the processed image vector code, the image code and the multiple attribute vector codes to obtain the fused multiple image local attribute codes includes:

[0038] Summing the processed image vector code and the image vector code to determine the summed image vector code;

[0039] Obtaining the mean and variance of each row of elements in the summed image vector encoding;

[0040] Determine a normalized image vector code according to the mean, the variance and each element value of each row of the summed image vector code;

[0041] According to the normalized image vector code and the preset parameter vector, the normalized image vector code and the multiple attribute vector codes are conditionally fused to obtain the fused multiple image local attribute codes.

[0042] In the above solution, the local attribute information is local attribute information of multiple different attributes;

[0043] The method of utilizing the local attribute encoding of the image to perform vector retrieval from the item library to determine recommended items with similar local attributes includes:

[0044] By using the plurality of local attribute codes of the image and searching with the object library, an object similar to the plurality of attributes contained in the image to be searched is determined;

[0045] Among the multiple items with similar attributes, similar items of each attribute are determined according to the distance between the encoding vectors.

[0046] In the above scheme, before searching the item library by using the multiple local attribute codes of the image to determine the items similar to the multiple attributes contained in the image to be searched, the method further includes:

[0047] Acquire multiple object images and determine local attribute information corresponding to the object images;

[0048] Encoding the plurality of object images and the local attribute information corresponding to the plurality of object images by using the preset image local attribute encoding model to determine the local attribute encodings of the plurality of object images;

[0049] The item library is determined by using the local attribute codes of the multiple item images; wherein the item library includes the local attribute codes of the multiple item images and the identifiers of the multiple item images.

[0050] An embodiment of the present invention provides a similar item search device, the similar item search device comprising a receiving unit and a determining unit; wherein:

[0051] The receiving unit is used to receive the image to be searched uploaded by the user, and determine the local attribute information corresponding to the image to be searched;

[0052] The determination unit uses a preset image local attribute coding model to encode the image to be searched and the local attribute information to determine the image local attribute coding; and uses the image local attribute coding to perform vector retrieval from an item library to determine recommended items with similar local attributes.

[0053] An embodiment of the present invention provides a similar item search device, the similar item search device comprising:

[0054] A memory for storing executable instructions;

[0055] The processor is used to execute the executable instructions stored in the memory. When the executable instructions are executed, the processor executes the similar item search method.

[0056] An embodiment of the present invention provides a storage medium storing executable instructions. When the executable instructions are executed by one or more processors, the processors execute the similar item search method.

[0057] The embodiment of the present invention provides a method and device for searching similar items, which receives an image to be searched and determines the local attribute information corresponding to the image to be searched; uses a preset image local attribute encoding model to encode the image to be searched and the local attribute information to determine the image local attribute encoding; uses the image local attribute encoding to perform vector retrieval from an item library to determine recommended items with similar local attributes. The above scheme is used to deeply integrate the product attributes with the product image information, so that different attributes extract key information of different areas of the image, thereby enhancing the diversity of similar search results. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 A schematic diagram of a similar item search method provided by an embodiment of the present invention Figure 1 ;

[0059] Figure 2 A schematic diagram of a similar item search method provided by an embodiment of the present invention Figure 2 ;

[0060] Figure 3 A schematic diagram of a similar item search method provided by an embodiment of the present invention Figure 3 ;

[0061] Figure 4 A schematic diagram of a multi-modal attribute-aware encoder process according to an embodiment of the present invention;

[0062] Figure 5 A schematic diagram of a similar item search method provided by an embodiment of the present invention Figure 4 ;

[0063] Figure 6 A schematic diagram of the effect of a tree bucketing mechanism provided by an embodiment of the present invention Figure 1 ;

[0064] Figure 7 A schematic diagram of the effect of a tree bucketing mechanism provided by an embodiment of the present invention Figure 2 ;

[0065] Figure 8 A schematic diagram of the effect of an image local attribute coding model provided by an embodiment of the present invention;

[0066] Fig. 9 A schematic diagram of a similar item search method provided by an embodiment of the present invention Figure 5 ;

[0067] Fig.10 A schematic diagram of a similar item search method provided by an embodiment of the present invention Figure 6 ;

[0068] Fig.11 A schematic diagram of a conditional fusion module flow chart provided in an embodiment of the present invention;

[0069] Fig.12 A schematic diagram of a similar item search method provided by an embodiment of the present invention Figure 7 ;

[0070] Fig.13 A schematic diagram of a similar item search method provided by an embodiment of the present invention Figure 8 ;

[0071] Fig.14 A schematic diagram of an item retrieval effect provided by an embodiment of the present invention;

[0072] Fig.15 A schematic diagram of a similar item search method provided by an embodiment of the present invention Figure 9 ;

[0073] Fig.16 A schematic diagram of a similar item search method provided by an embodiment of the present invention Figure 10 ;

[0074] Fig.17 A schematic diagram of a training process of an image local attribute coding model provided by an embodiment of the present invention;

[0075] Fig.18A schematic diagram of a similar item search method provided by an embodiment of the present invention Figure 10 one;

[0076] Fig.19 A schematic diagram of a similar item search device provided by an embodiment of the present invention Figure 1 ;

[0077] Fig. 20 A schematic diagram of a similar item search device provided by an embodiment of the present invention Figure 2 . DETAILED DESCRIPTION

[0078] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the protection scope of the present invention.

[0079] In order to enable those skilled in the art to better understand the solution of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. Figure 1 A schematic diagram of a similar item search method provided by an embodiment of the present invention Figure 1 , will combine Figure 1 The steps shown are explained.

[0080] S101: Receive an image to be searched, and determine local attribute information corresponding to the image to be searched.

[0081] In the embodiment of the present invention, it is applicable to the scenario of receiving an image to be searched and acquiring attribute information corresponding to the image.

[0082] In an embodiment of the present invention, an image uploaded by a user for which a similar item needs to be found is received, and all attribute information contained in the image is determined. The image may be an image of an item used daily by ordinary people, such as clothing, household appliances, communication tools, etc., and the present invention does not limit this.

[0083] In the embodiment of the present invention, the image can be acquired by shooting with a camera of an electronic device (eg, a camera of a mobile phone or other storage media), and the present invention does not impose any limitation thereto.

[0084] S102: Encode the image to be searched and the local attribute information using a preset image local attribute encoding model to determine the image local attribute encoding.

[0085] In the embodiment of the present invention, it is applicable to a coding scenario of determining attributes contained in an image.

[0086] In the embodiment of the present invention, the trained image local attribute coding model is used to encode the image uploaded by the user and the attributes contained in the image to obtain the final image local attribute coding. The present invention does not limit the image local attribute coding model.

[0087] In the embodiment of the present invention, the local attribute code of the image is formed by fusing the image and the corresponding attribute in the image.

[0088] For example, if the image to be searched uploaded by the user is a T-shirt, and the attributes of the T-shirt are color, collar, pattern, material, etc., then the T-shirt and the attributes of the color, collar, pattern, material, etc. contained in the T-shirt are encoded separately using the trained image local attribute encoding model, and then the T-shirt code and the above-mentioned attribute codes are merged separately, and finally multiple image local attribute codes are obtained.

[0089] It can be understood that, in the embodiment of the present invention, the image uploaded by the user and the attributes contained in the image are encoded using the image local attribute encoding model to obtain the final image local attribute encoding, thereby improving the diversity of image search.

[0090] S103: Using the local attribute coding of the image, perform vector retrieval from the item library to determine recommended items with similar local attributes.

[0091] In the embodiment of the present invention, it is applicable to the scenario of item recommendation.

[0092] In the embodiment of the present invention, after the image local attribute code is obtained by using the image local attribute code model, the image local attribute code and the items in the item library are searched to finally obtain the required similar recommended items.

[0093] In an embodiment of the present invention, all images in the item library also need to be encoded using the image local attribute encoding model to obtain multiple item image local attribute encodings, so the final item library contains multiple item image local attribute encodings and identifications of the multiple item images.

[0094] For example, the image uploaded by the user contains four attributes: color, collar, pattern and material. After obtaining the local attribute code of the image, a vector search is performed based on the item library. The recommendation platform will list four blocks of recommendation results. The four blocks correspond to the four attributes of color, collar, pattern and material contained in the image uploaded by the user. The user can choose any one of the four blocks to select similar items as needed.

[0095] It can be understood that in an embodiment of the present invention, after the image local attribute coding model is used to obtain the image local attribute coding, the image local attribute coding and the items in the item library are retrieved, and finally similar product recommendations for each attribute of the uploaded image are obtained, thereby improving the diversity of product recommendations.

[0096] In some embodiments, see Figure 2 , Figure 2 This is a schematic diagram of a similar item search method provided by an embodiment of the present invention. Figure 2 , Figure 2 S201 to S203 shown are the training process of the image local attribute encoding model, which will be explained in conjunction with each step.

[0097] S201. Group the acquired original samples according to attributes, attribute values ​​and categories to determine training samples; the training samples include positive and negative samples with a preset ratio.

[0098] In some embodiments of the present invention, it is applicable to determine a scenario of training samples.

[0099] In some embodiments of the present invention, it is necessary to train the image local attribute coding model until the accuracy of the model meets the set threshold, and then use the trained image local attribute coding model to encode the image. Before training the model, the original samples are grouped using the attributes, attribute values ​​and categories of the image, and then the grouped samples are selected according to a preset ratio to form training samples.

[0100] It can be understood that in some embodiments of the present invention, the obtained original samples are grouped according to attributes, attribute values ​​and categories, and then the grouped samples are selected according to a preset ratio to form training samples. This selection method not only enables the model to have better execution results, but also reduces the model training time.

[0101] S202: Train the initial image coding model using training samples to obtain attribute prediction values.

[0102] In some embodiments of the present invention, it is applicable to scenarios where predicted values ​​are obtained.

[0103] In some embodiments of the present invention, training is performed using the training samples selected in S201 to obtain attribute prediction values.

[0104] It is understandable that in some embodiments of the present invention, the model is trained using selected training samples to obtain attribute prediction values, and the model parameters are continuously adjusted by comparing the predicted values ​​with actual values, thereby improving the accuracy of the model.

[0105] S203, using the loss function, performing error comparison between the attribute prediction value and the actual value of the preset attribute corresponding to the training sample, adjusting the model parameters, thereby implementing iterative training of the initial image coding model, until a preset image local attribute coding model is obtained.

[0106] In some embodiments of the present invention, the model is trained using training samples. After the predicted value is obtained, the predicted value is compared with the actual value of the training sample, and the error between the actual value and the predicted value is used to update the parameters of the encoding model. The model is continuously iteratively trained until an image local attribute encoding model with an accuracy rate that meets a certain threshold is obtained.

[0107] It is understandable that in some embodiments of the present invention, a loss function is used to perform an error comparison between the attribute prediction value and the actual value of the preset attribute corresponding to the training sample, and the model parameters are continuously adjusted to improve the accuracy of the model.

[0108] In some embodiments, see Figure 3 , Figure 3 This is a schematic diagram of a similar item search method provided by an embodiment of the present invention. Figure 3 , Figure 3 The illustrated S202 can be implemented through S2021 to S2025, which will be described in conjunction with each step.

[0109] S2021. Encode the training sample image and multiple sample local attribute information using the initial image coding model to determine the training sample image vector coding and multiple sample local attribute vector coding; the multiple sample local attribute information is the determined attribute information corresponding to the training sample image.

[0110] In some embodiments of the present invention, it is applicable to the scenario of obtaining attribute prediction values.

[0111] In some embodiments of the present invention, after obtaining the training samples, the training sample images and the local attribute information of the sample images are encoded respectively using the initial image coding model to obtain the sample image vector code and the sample local attribute vector code.

[0112] It can be understood that, using the initial image coding model, both the training samples and the local attribute information of the samples are encoded to obtain vector coding and sample local attribute vector coding. This allows the desired similar products to be selected from various attributes of the image when searching for similar products, thereby improving the diversity of the search.

[0113] S2022. Take the training sample image vector code as input, use the multi-head attention mechanism to learn and concatenate the training sample image vector code, and obtain the processed training sample image vector code.

[0114] In some embodiments of the present invention, it is applicable to a scenario of obtaining a plurality of sub-sample image vector codes.

[0115] In some embodiments of the present invention, after the sample image vector code is obtained, it is intercepted to obtain a plurality of sub-sample image vector codes.

[0116] For example, the training sample image vector encoding is used as input and input into a multimodal attribute-aware encoder (Transformer). The Encoder in the Transformer consists of N=6 identical layers, each of which consists of two sub-layers, namely a multi-head self-attention mechanism and a fully connected feed-forward network. Each sub-layer is added with residual connection and normalization, so the output of the sub-layer sub_layer_output can be expressed as the following formula (1):

[0117] sub_layer_output=LayerNorm(x+(SubLayer(x))) (1)

[0118] Figure 4 Based on the multimodal attribute-aware encoder (Transformer), the sample image is vector-encoded as the input of the Transformer structure. The multi-head attention mechanism (Multi-Head Attention) consists of multiple Self-Attentions. Taking the calculation process of a single Self-Attention as an example, the Self-Attention sublayer multiplies the sample image vector encoding by three trained matrices W of the same dimension. Q , W K and W V , we obtain the query, key, and value vectors respectively, and after obtaining the matrices Q, K, and V, we calculate the output of Self-Attention. The calculation formula (2) is as follows:

[0119]

[0120] Among them, d k It is the number of columns of the Q and K matrices, that is, the vector dimension.

[0121] The above calculation formula shows how the final output matrix is ​​calculated through Self-Attention. Multi-Head Attention contains multiple Self-Attention layers, so the input sample vector encoding is first passed to h different Self-Attention layers to calculate h output matrices, where h represents the number of heads. After obtaining h output matrices, Multi-Head Attention concatenates them together (Concat), and then passes them to a linear layer to obtain the final processed sample image vector encoding of Multi-Head Attention.

[0122] In some embodiments of the present invention, the dimension of the sample image vector encoding obtained after the Multi-Head Attention processing is the same as the dimension of the input sample image vector encoding.

[0123] It can be understood that in some embodiments of the present invention, the sample image vector is taken as input, and the sample image vector is learned and concatenated using a multi-head attention mechanism to obtain a processed sample image vector encoding, which helps the network capture richer features / information.

[0124] S2023, conditionally fuse the processed training sample image vector code, the training sample image vector code and multiple sample local attribute vector codes to obtain multiple fused training sample image local attribute codes.

[0125] In some embodiments of the present invention, after the processed training sample is obtained by the multi-head attention mechanism, it is encoded with the training sample image vector and multiple sample local attribute vectors as follows: Figure 4 The first conditional fusion layer in is used for fusion. The local attribute encoding of multiple training sample images after fusion is obtained.

[0126] In some embodiments of the present invention, the dimensions of the input sample image code and the processed sample image code are the same, so they can be added.

[0127] S2024, performing dimensionality reduction processing on the fused local attribute codes of the multiple training sample images to obtain the reprocessed local attribute codes of the multiple training sample images.

[0128] In some embodiments of the present invention, a conditional fusion module fuses the processed training sample image vector code, the training sample image vector code and multiple sample local attribute vector codes to obtain a fused sample image local attribute code, and uses the fused sample image local attribute code as the input of a feedforward neural network to perform dimensionality reduction processing to obtain a re-processed sample image local attribute code.

[0129] In some embodiments of the present invention, this step is as follows Figure 4 The feedforward neural network layer in is processed. The feedforward neural network layer is relatively simple, it is a two-layer fully connected layer, the activation function of the first layer is Relu, and the second layer does not apply the activation function. The corresponding formula (3) is as follows:

[0130] FFN(x)=ReLU(W 1 x+b 1 )W 2 +b 2 (3)

[0131] Where x represents the local attribute encoding of the fused training sample image; FFN(x) represents the output value of the feedforward neural network; W 1 and W 2 represents the weight of the network layer, b 1 and b 2 Represents the bias term.

[0132] In some embodiments of the present invention, the dimensions of the reprocessed local attribute code of the sample image obtained after being processed by the feedforward neural network and the processed local attribute code of the sample image input into the feedforward neural network are the same.

[0133] In some embodiments of the present invention, dimensionality reduction is because after the local attribute encoding of the fused sample image is obtained through the conditional fusion layer, it is necessary to project it into a larger space, in which the required information can be more conveniently extracted (using the Relu activation function), so it is finally necessary to project it back to the original space.

[0134] It can be understood that in some embodiments of the present invention, the fused local attribute encoding of the sample image is processed by a feedforward neural network, in which a nonlinear (Relu) activation function is introduced to transform the space of the attention output, thereby enhancing the expressive power of the model. This process also enables further learning of useful information of the local attribute encoding of the sample image.

[0135] S2025, fusing the reprocessed multiple training sample image local attribute codes, the fused multiple training sample image local attribute codes, and the multiple sample local attribute vector codes to obtain attribute prediction values.

[0136] In some embodiments of the present invention, the reprocessed multiple training sample image local attribute codes obtained after feedforward neural network processing, the fused multiple training sample image local attribute codes and the multiple local attribute vector codes are fused again to obtain attribute prediction values.

[0137] In some embodiments, see Figure 5 , Figure 5 This is a schematic diagram of a similar item search method provided by an embodiment of the present invention. Figure 4 , Figure 5 The illustrated S201 can be implemented through S2011 to S2016, which will be described in conjunction with each step.

[0138] S2011. Divide the original samples into buckets according to attributes, attribute values ​​and categories to obtain four sample sets; wherein the four sample sets are a first sample set with the same attribute values, a second sample set with different attribute values ​​but the same attributes, a third sample set with different attribute values ​​and attributes but the same categories, and a fourth sample set with different attribute values, attributes and categories.

[0139] In some embodiments of the present invention, in order to achieve better model execution results and reduce model training time, after a large number of samples are collected, the collected samples are bucketed using the attribute bucketing mechanism.

[0140] In some embodiments of the present invention, Figure 6 and Figure 7 As shown in the figure, the SKU is the unit of measurement for inventory in and out (buyers purchase, merchants purchase, suppliers prepare, and factories produce based on SKUs). That is to say, for a product, the specific inventory of goods can be determined based on the SKU, which can be in units of pieces, boxes, pallets, etc. It is most commonly used in clothing and footwear products. For example, in textiles, a SKU usually represents: specifications, colors, and styles.

[0141] In some embodiments of the present invention, Figure 7 As shown, samples with the same attribute value constitute the first sample set, samples with different attribute values ​​and the same attribute constitute the second sample set, samples with the same attribute value and attribute but different categories constitute the third sample set, and samples with different attribute values, attributes and categories constitute the fourth sample set. Finally, the training samples are determined by the first, second, third and fourth sample sets.

[0142] It is understandable that in some embodiments of the present invention, the collected samples are bucketed to obtain multiple sample sets, and training samples are determined from the obtained multiple sets. This approach enables the model to have better execution results and reduces the model training time.

[0143] S2012: Determine positive samples in the first sample set.

[0144] In some embodiments of the present invention, samples with the same attribute value constitute a first sample set, and samples are selected from the set and are called positive samples.

[0145] For example, Figure 8 As shown in the figure, the images in the figure are marked from top to bottom. The collar types of the first image and the second image are both round collars, so they are positive samples. The + or - encoded in the figure indicates whether the attributes between the samples are the same; loss is the loss function, and the model parameters are updated according to the loss function.

[0146] S2013. Determine a first negative sample in the second sample set.

[0147] In some embodiments of the present invention, samples with the same attribute value but different attributes constitute a second sample set, and samples are selected from the set and are referred to as first negative samples.

[0148] For example, Figure 8 As shown, the collar types of the first image and the third image are round collar and square collar respectively, so they are the first negative samples.

[0149] S2014. Determine a second negative sample in the third sample set.

[0150] In some embodiments of the present invention, samples with the same attribute values ​​and attributes but different categories constitute a third sample set, from which samples are selected and are referred to as second negative samples.

[0151] For example, Figure 8 As shown, the collar types of the first image and the fourth image are round collar and square collar respectively, and the thickness is different, so they are the second negative samples.

[0152] S2015. Determine a third negative sample in the fourth sample set.

[0153] In some embodiments of the present invention, samples with different attribute values ​​and attribute categories constitute a fourth sample set, and samples are selected from this set and are called third negative samples.

[0154] For example, Figure 8 As shown, the first image and the fifth image are clothes and shoes respectively, which are different in all aspects, so they are the third negative samples.

[0155] S2016. Determine a training sample based on any negative sample between the first negative sample, the second negative sample, and the third negative sample in combination with the positive sample.

[0156] In some embodiments of the present invention, the first, second, third and fourth negative samples are arbitrarily selected and combined with the positive samples to form training samples of the model.

[0157] In some embodiments, see Fig. 9 , Fig. 9 This is a schematic diagram of a similar item search method provided by an embodiment of the present invention. Figure 5 , Fig. 9 The illustrated S102 can be implemented through S1021 to S1022, which will be described in conjunction with each step.

[0158] S1021. Encode the image to be searched and multiple local attribute information respectively by using a preset image local attribute coding model to obtain an image vector code and multiple attribute vector codes.

[0159] In some embodiments of the present invention, the image local attribute coding model is trained until an image local attribute coding model with an accuracy rate that meets a preset threshold is obtained. The trained image local attribute coding model is used to encode the image to be searched uploaded by the user and the multiple local attribute information contained in the image, respectively, to obtain the image vector code and multiple attribute vector codes, and then perform subsequent processing.

[0160] S1022: Using a preset image local attribute coding model, the image vector coding and multiple attribute vector codings are merged to obtain multiple image local attribute codings.

[0161] In some embodiments of the present invention, the trained image local attribute coding model is used to encode the search image uploaded by the user, and the multiple attributes contained in the image are encoded at the same time. After the image vector code and multiple attribute vector codes are obtained, they are fused to obtain multiple image local attribute codes.

[0162] In some embodiments, see Fig.10 , Fig.10 This is a schematic diagram of a similar item search method provided by an embodiment of the present invention. Figure 6 , Fig.10 The illustrated S1022 can be implemented through S301 to S304 , which will be described in conjunction with each step.

[0163] S301. Take the image vector code as input, use the multi-head attention mechanism to learn and concatenate the image vector code, and obtain the processed image vector code.

[0164] In some embodiments of the present invention, the image to be searched uploaded by the user is encoded and used as the input of the transformer model, the image vector encoding is learned using the "multiple heads" of the multi-head attention mechanism layer in the model, and then the learned multiple image vector encodings are spliced ​​to obtain the processed image vector encoding.

[0165] In some embodiments of the present invention, this step is the same as S2022, and specific details may be referred to S2022.

[0166] It can be understood that in some embodiments of the present invention, the image vector encoding uploaded by the user is taken as input, and the multi-head attention mechanism is used to learn and splice it to obtain the processed image vector encoding, which helps the network capture richer features / information.

[0167] S302: Conditionally fuse the processed image vector code, the image code, and multiple attribute vector codes to obtain multiple fused image local attribute codes.

[0168] In some embodiments of the present invention, the attribute code is used as a conditional vector in the conditional fusion module together with the image vector code processed by the multi-head attention mechanism and the original image vector code as input, and is input into the conditional fusion layer for fusion to obtain the fused image local attribute code.

[0169] In some embodiments of the present invention, the dimensions of the input image code and the processed image code are the same, so they can be added.

[0170] In some embodiments of the present invention, Fig.11 It is a "conditional fusion module" introduced on the attribute-aware encoder (transformer) based on multimodality. When the conditional fusion module in the transformer model is used to fuse the image vector code and the attribute vector code, the conventional trained encoder is first used to encode the image and the multiple attributes contained in the image to obtain the image vector code and the attribute vector code. When the obtained image vector code and attribute vector code are input into the transformer model, the transformer model will usually embed the original input. The image vector code and the attribute vector code can be mapped to the required dimension by using a transformation matrix for matrix product.

[0171] S303 , performing dimensionality reduction processing on the multiple image local attribute codes to obtain multiple image local attribute codes after reprocessing.

[0172] In some embodiments of the present invention, a multi-head conditional fusion module fuses the processed image vector code, the image vector code and the local attribute vector code in a conditional fusion module to obtain a fused image local attribute code, and the fused image local attribute code is used as the input of a feedforward neural network to perform dimensionality reduction processing to obtain a re-processed image local attribute code.

[0173] In some embodiments of the present invention, this step is the same as S2024, and specific details may be referred to S2024.

[0174] S304: The reprocessed multiple image local attribute codes, the fused multiple image local attribute codes and the multiple attribute vector codes are fused again to obtain multiple image local attribute codes.

[0175] In some embodiments of the present invention, the processed multiple image local attribute codes obtained by the feedforward neural network layer, the fused image local attribute codes obtained by the conditional fusion module and the multiple attribute vector codes are fused to obtain the final multiple image local attribute codes.

[0176] The trained image local attribute coding model is used to encode the searchable image uploaded by the user, and the multiple attributes contained in the image are encoded at the same time. After obtaining the image vector coding and multiple attribute vector coding, the obtained image vector coding and multiple attribute vector coding are fused to obtain multiple image local attribute coding.

[0177] In some embodiments, see Fig.12 , Fig.12 This is a schematic diagram of a similar item search method provided by an embodiment of the present invention. Figure 7 , Fig.12 The illustrated S302 can be implemented through S3021 to S3024, which will be described in conjunction with each step.

[0178] S3021. Sum the processed image vector code and the image vector code to determine the summed image vector code.

[0179] In some embodiments of the present invention, the processed image vector code obtained by the multi-head attention mechanism layer and the original input image vector code are summed to obtain the summed image vector code.

[0180] In some embodiments of the present invention, the dimension of the processed image vector encoding obtained by the multi-head attention mechanism layer is the same as the dimension of the original input image vector encoding.

[0181] S3022. Obtain the mean and variance of each row of elements in the summed image vector encoding.

[0182] In some embodiments of the present invention, the processed image vector code and the image vector code are summed, and after the summed image vector code is determined, the mean and variance of the summed image vector code are obtained, such as Fig.11 μ and σ in , μ is the mean calculated from the elements of each row in the summed image vector encoding, and σ is the variance calculated from the elements of each row in the summed image vector encoding.

[0183] S3023. Determine the normalized image vector code according to the mean, variance and summed element values ​​of each row of the image vector code.

[0184] In some embodiments of the present invention, after obtaining the mean and variance of the values ​​of each row of elements in the summed image vector code, the normalized image vector code is obtained according to the existing formula in combination with the values ​​of each row of elements in the summed image vector code. The formula is as follows:

[0185]

[0186] Among them, x is each element of each row in the summed image vector encoding; μ is the mean calculated from the elements of each row; σ is the variance calculated from the elements of each row.

[0187] Each element of each row of the image code is subtracted from the obtained mean value to determine the operation result; then the operation result is divided by the standard deviation of each row to finally obtain the normalized image code.

[0188] S3024. Conditionally fuse the normalized image vector code and multiple attribute vector codes according to the normalized image vector code and the preset parameter vector to obtain multiple fused image local attribute codes.

[0189] In some embodiments of the present invention, after obtaining the normalized image vector encoding, the parameter vector obtained by training is combined to obtain the fused local attribute encoding of multiple images using the formula. Formula (5) is as follows:

[0190] z=γ s x norm +β s (5)

[0191] Among them, γ s and β s are the training parameters obtained by encoding the attribute vector through two fully connected network layers; z is the local attribute encoding of the fused image.

[0192] In some embodiments of the present invention, γ s and β s It is obtained by encoding attribute vectors and trainable network weights. After the conditional fusion module, it has the ability to represent fusion attributes and image information.

[0193] In some embodiments, see Fig.13 , Fig.13 This is a schematic diagram of a similar item search method provided by an embodiment of the present invention. Figure 8 , Fig.13 The illustrated S103 can be implemented through S1031 to S1032 , which will be described in conjunction with each step.

[0194] S1031. Using multiple image local attribute codes and searching with an item library, determine items that are similar to multiple attributes included in the image to be searched.

[0195] In some embodiments of the present invention, after obtaining multiple image local attribute codes of the image to be searched, the item library is used for retrieval, and different similar products are recommended according to different attributes.

[0196] In some embodiments of the present invention, the item library is constructed offline, and the items in the item library, like the images uploaded by the user, need to encode the items and the attributes contained in the items, and then merge them respectively.

[0197] In some embodiments of the present invention, Fig.14 As shown in the figure, the specific retrieval process is as follows: When the retrieval module is searching, the input of the request interface is the Base64 encoding of the image and the local attributes that need to be paid attention to, and the output is the list of local similar items and the corresponding similarity. When a request is received, the image and attributes are sent to the image encoder to obtain the local vector encoding of the image, and then a request is sent to the vector retrieval module through the load balancing module. In order to support large-scale vector indexing, the vector index library can be divided into shards to balance the data. When there is a request, each sub-library, such as sub-library 1 (index1), sub-library 2 (index2), sub-library 3 (index3)...sub-library n (indexn), is queried in parallel, and the returned results are merged to obtain the final result.

[0198] It can be understood that in some embodiments of the present invention, after the local attribute coding of the image is obtained using the image local attribute coding model, the local attribute coding and the items in the item library are retrieved, and finally similar product recommendations for each attribute of the uploaded image are obtained, thereby improving the diversity of product recommendations.

[0199] S1032. Among multiple items with similar attributes, determine similar items for each attribute based on the distance between the encoding vectors.

[0200] In some embodiments of the present invention, after obtaining objects similar to multiple attributes contained in the image to be searched according to the retrieval, similar objects of each attribute are obtained according to the distance between the encoding vectors.

[0201] Exemplarily, the image to be searched contains multiple attributes such as color, collar type and pattern. After obtaining items similar to the multiple attributes contained in the image to be searched, items with similar color to the image will be recommended based on the color coding of the image, and items with similar color to the image will be recommended based on the color coding of the image, and so on.

[0202] In some embodiments of the present invention, see Fig.15 , Fig.15 This is a schematic diagram of a similar item search method provided by an embodiment of the present invention. Figure 9 , Fig.15 The illustrated S1031 also includes S401 to S403 implementation, which will be described in conjunction with each step.

[0203] S401: Acquire multiple object images and determine local attribute information corresponding to the object images.

[0204] In some embodiments of the present invention, before the items to be searched are retrieved, an item library needs to be constructed offline.

[0205] In some embodiments of the present invention, items existing in the platform are acquired, an item library required for retrieval is jointly constructed, and all attributes contained in the multiple items are acquired.

[0206] S402: Encode multiple object images and local attribute information corresponding to the multiple object images using a preset image local attribute encoding model to determine local attribute encodings for the multiple object images.

[0207] In some embodiments of the present invention, the acquired objects are encoded using the trained image local attribute encoding, and the attributes contained in the acquired objects are encoded at the same time, and they are fused respectively to obtain the final multiple object image local attribute encodings.

[0208] S403: Determine an item library using the local attribute codes of the multiple item images; wherein the item library includes the local attribute codes of the multiple item images and the identifiers of the multiple item images.

[0209] In some embodiments of the present invention, an item library is constructed based on the acquired local attribute codes of multiple item images, and the item library contains identifiers of multiple item images corresponding to the local attributes of the multiple item images.

[0210] It is understandable that when users upload pictures to find similar products, the current method can only identify pictures with similar subjects, but not with similar local details, and cannot meet the needs of fine-grained local detail retrieval. When there are identical items in the item library, a large number of duplicate and similar items will be recommended, resulting in a large number of useless recommendations and a lack of diversity in the results. The present invention deeply integrates product attributes with product image information, so that different attributes extract key image information in different areas, thereby enhancing the diversity of search similarity results.

[0211] The embodiment of the present invention provides a method for searching similar items, and the optional flowchart is as follows: Fig.16 shown.

[0212] S1. User uploads photos.

[0213] S2. Process the image using an image local attribute coding model to obtain a plurality of image local attribute codes.

[0214] In some embodiments of the present invention, the image is processed using an image local attribute coding model to obtain N image local attribute codes, including:

[0215] S1.1. Use the tree bucketing mechanism to select training samples and train the initial image coding model.

[0216] In some embodiments of the present invention, positive samples with similar local styles and negative samples with dissimilar local styles are constructed. The collection of negative samples requires the targeted selection of some difficult samples to simulate normal data distribution, thereby enhancing the discrimination ability of the model. Figure 6 and Figure 7 The specific process is as follows: first, all samples are hierarchically bucketed according to attribute values, attributes, and categories, and then sample sampling is supported in three dimensions: attribute value + attribute + category, attribute + category, and category; during model training, for each sample, positive examples are sampled from sample groups with the same attribute value, and negative examples are sampled from sample groups with the same category + attribute and the same category. In addition, in order to increase the richness of sampling, global negative sampling is additionally performed, and finally a sample set with a positive-negative ratio of 1:3 is constructed.

[0217] In some embodiments of the present invention, the flowchart of the image local attribute encoding model training is as follows Fig.17 As shown in the figure, the process of training the initial image local attribute coding model using the selected training samples, after obtaining the training samples, the image is processed by the abnormal encoder (Xception Encoder) to obtain the image vector coding, and the attribute is embedded in the lookup (Embedding Lookup) to obtain the attribute vector coding. The obtained image vector coding and attribute vector coding are processed by the multi-modal attribute perception encoder (Transformer Encoder), and the parameters of the image local attribute coding model are adjusted by the contrast learning mechanism, and finally the image local attribute coding is output. The image contains multiple attributes, Fig.17 The conditional generation in can be seen as the selection of attributes.

[0218] S3. Retrieve items from the item library based on the local attribute codes of multiple images to obtain recommendation results.

[0219] In some embodiments of the present invention, the item library is constructed offline, and the item library in the platform is encoded using an image coding model to obtain multiple local attribute codes of item images, and the item library is constructed using the multiple local attribute codes of item images; wherein the item library contains multiple local attribute codes of item images and identifiers of the multiple item images.

[0220] It can be understood that the preset image local encoding model is used to encode the image to be searched and multiple local attribute information of the image uploaded by the user to obtain multiple image local attribute codes. Using the image local attribute codes, vector retrieval is performed from the item library to obtain similar recommended items. This method deeply integrates product attributes with product image information, so that different attributes extract key image information in different areas, enhancing the diversity of similar search results.

[0221] In some embodiments of the present invention, the optional flowchart may also be as follows Fig.18 shown.

[0222] The embodiment of the present invention provides a schematic diagram of a similar item search device structure. Fig.19 As stated, Fig.19 The schematic diagram of the structure of a similar item search device provided by an embodiment of the present invention includes: a receiving unit 1901 and a determining unit 1902; wherein:

[0223] The receiving unit 1901 is configured to receive an image to be searched and determine local attribute information corresponding to the image to be searched.

[0224] The determination unit 1902 is used to encode the image to be searched and the local attribute information using a preset image local attribute coding model to determine the image local attribute coding; and use the image local attribute coding to perform vector retrieval from the item library to determine recommended items with similar local attributes.

[0225] In some embodiments of the present invention, the determination unit 1902 is further used to group the acquired original samples according to attributes, attribute values ​​and categories to determine training samples; the training samples include positive and negative samples with a preset ratio; and the initial image coding model is trained using the training samples to obtain attribute prediction values; and a loss function is used to perform an error comparison between the attribute prediction values ​​and the actual values ​​of the preset attributes corresponding to the training samples, and the model parameters are adjusted, thereby realizing iterative training of the initial image coding model until the preset image local attribute coding model is obtained.

[0226] In some embodiments of the present invention, the determination unit 1902 is also used to use the initial image coding model to encode the training sample image and the multiple sample local attribute information to determine the training sample image vector code and the multiple sample local attribute vector codes; the multiple sample local attribute information is the determined attribute information corresponding to the training sample image; and the training sample image vector code is used as input, and the training sample image vector code is learned and spliced ​​using a multi-head attention mechanism to obtain a processed training sample image vector code; and the processed training sample image vector code, the training sample image vector code and the multiple sample local attribute vector codes are conditionally fused to obtain a fused multiple training sample image local attribute codes; and the fused multiple training sample image local attribute codes are subjected to dimensionality reduction processing to obtain a re-processed multiple training sample image local attribute codes; and the re-processed multiple training sample image local attribute codes, the fused multiple training sample image local attribute codes and the multiple sample local attribute vector codes are re-fused to obtain the attribute prediction value.

[0227] In some embodiments of the present invention, the determination unit 1902 is further used to bucket the original samples according to attributes, attribute values ​​and categories to obtain four groups of sample sets respectively; wherein the four groups of sample sets are respectively a first sample set with the same attribute values, a second sample set with different attribute values ​​but the same attributes, a third sample set with different attribute values ​​and attributes but the same categories, and a fourth sample set with different attribute values, attributes and categories; and in the first sample set, positive samples are determined; and in the second sample set, the first negative samples are determined; and in the third sample set, the second negative samples are determined; and in the fourth sample set, the third negative samples are determined; and in accordance with any negative samples between the first negative samples, the second negative samples and the third negative samples, in combination with the positive samples, the training samples of the initial image local attribute coding model are determined.

[0228] In some embodiments of the present invention, the determination unit 1902 is further used to use the preset image local attribute coding model to encode the image to be searched and the multiple local attribute information respectively to obtain an image vector code and multiple attribute vector codes; and to use the preset image local attribute coding model to fuse the image vector code and the multiple attribute vector codes to obtain multiple image local attribute codes.

[0229] In some embodiments of the present invention, the determination unit 1902 is also used to take the image vector code as input, use a multi-head attention mechanism to learn and concatenate the image vector code, and obtain a processed image vector code; and conditionally fuse the processed image vector code, the image code, and multiple attribute vector codes to obtain multiple fused image local attribute codes; and perform dimensionality reduction processing on the multiple image local attribute codes to obtain multiple reprocessed image local attribute codes; and fuse the multiple reprocessed image local attribute codes, the multiple fused image local attribute codes, and the multiple attribute vector codes to obtain the multiple image local attribute codes.

[0230] In some embodiments of the present invention, the determination unit 1902 is also used to sum the processed image vector code and the image vector code to determine the summed image vector code; and obtain the mean and variance of the elements in each row of the summed image vector code; and determine the normalized image vector code based on the mean, the variance and the values ​​of each element in each row of the summed image vector code; and conditionally fuse the normalized image vector code and the multiple attribute vector codes based on the normalized image vector code and a preset parameter vector to obtain the fused multiple image local attribute codes.

[0231] In some embodiments of the present invention, the determination unit 1902 is further used to determine items similar to the multiple attributes contained in the image to be searched by using the item library through multiple local attribute codes of the image; and to determine similar items for each attribute among the multiple items with similar attributes based on the distance between the encoding vectors.

[0232] In some embodiments of the present invention, the determination unit 1902 is further used to obtain multiple object images and determine local attribute information corresponding to the object images; and use the preset image local attribute coding model to encode the multiple object images and the local attribute information corresponding to the multiple object images to determine the local attribute codes of the multiple object images; and use the local attribute codes of the multiple object images to determine the object library; wherein the object library contains the local attribute codes of the multiple object images and the identifiers of the multiple object images.

[0233] Based on the method of the above embodiment, the embodiment of the present invention also provides a schematic diagram of the structure of a similar item search device, such as Fig. 20 As shown, Fig. 20A structural schematic diagram of a similar item search device provided in an embodiment of the present invention, the similar item search device comprises: a first processor 2001 and a first memory 2002; the first memory 2002 stores one or more programs executable by the first processor 2001, and when one or more programs are executed, any similar item search method of the above-described embodiments is executed by the first processor 2001.

[0234] An embodiment of the present invention provides a computer-readable storage medium storing one or more programs. The one or more programs can be executed by one or more processors. When the programs are executed by the processors, the similar item search method according to the embodiment of the present invention is implemented.

[0235] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) containing computer-usable program code.

[0236] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0237] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0238] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0239] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention.

Claims

1. A method for searching similar items, characterized in that: include: Receiving an image to be searched, and determining local attribute information corresponding to the image to be searched; Using a preset image local attribute coding model, the image to be searched and the local attribute information are encoded to determine the image local attribute coding; wherein the local attribute information is local attribute information of multiple different attributes; Using the local attribute encoding of the image, vector retrieval is performed from an item library to determine recommended items with similar local attributes; The step of encoding the image to be searched and the local attribute information by using a preset image local attribute encoding model to determine the image local attribute encoding includes: Using the preset image local attribute coding model, the image to be searched and the plurality of local attribute information are respectively encoded to obtain an image vector code and a plurality of attribute vector codes; The preset image local attribute coding model is used to fuse the image vector coding and a plurality of the attribute vector codings to obtain a plurality of image local attribute codings.

2. The method according to claim 1, characterized in that Before encoding the image to be searched and the local attribute information by using a preset image local attribute encoding model and determining the image local attribute encoding, the method further includes: The original samples obtained are grouped according to attributes, attribute values ​​and categories to determine training samples; the training samples include positive and negative samples with a preset ratio; Using the training samples to train the initial image coding model to obtain attribute prediction values; The loss function is used to compare the error between the attribute prediction value and the actual value of the preset attribute corresponding to the training sample, and the model parameters are adjusted to achieve iterative training of the initial image coding model until the preset image local attribute coding model is obtained.

3. The method according to claim 2, characterized in that The training sample includes a training sample image and a plurality of sample local attribute information; The step of training the initial image coding model using the training samples to obtain the attribute prediction value includes: Using the initial image coding model, the training sample image and the plurality of sample local attribute information are encoded to determine a training sample image vector encoding and a plurality of sample local attribute vector encodings; the plurality of sample local attribute information is the determined attribute information corresponding to the training sample image; Taking the training sample image vector code as input, learning and concatenating the training sample image vector code using a multi-head attention mechanism to obtain a processed training sample image vector code; Conditionally fusing the processed training sample image vector code, the training sample image vector code and the multiple sample local attribute vector codes to obtain multiple fused training sample image local attribute codes; Performing dimensionality reduction processing on the fused local attribute codes of the multiple training sample images to obtain the reprocessed local attribute codes of the multiple training sample images; The reprocessed multiple training sample image local attribute codes, the fused multiple training sample image local attribute codes and the multiple sample local attribute vector codes are fused again to obtain the attribute prediction value.

4. The method according to claim 2, characterized in that: The step of grouping the acquired original samples according to the attributes, attribute values ​​and categories to determine the training samples includes: The original samples are bucketed according to attributes, attribute values ​​and categories to obtain four groups of sample sets; wherein the four groups of sample sets are a first sample set with the same attribute value, a second sample set with different attribute values ​​but the same attribute, a third sample set with different attribute values ​​and attributes but the same category, and a fourth sample set with different attribute values, attributes and categories; In the first sample set, determining positive samples; In the second sample set, determining a first negative sample; In the third sample set, determining a second negative sample; In the fourth sample set, determining a third negative sample; According to any negative sample between the first negative sample, the second negative sample and the third negative sample, in combination with the positive sample, a training sample of the initial image local attribute encoding model is determined.

5. The method according to claim 1, characterized in that The method of using the preset image local attribute coding model to fuse the image vector coding and the multiple attribute vector codings to obtain multiple image local attribute codings includes: Taking the image vector code as input, learning and concatenating the image vector code using a multi-head attention mechanism to obtain a processed image vector code; Conditionally fusing the processed image vector code, the image code, and the plurality of attribute vector codes to obtain a plurality of fused image local attribute codes; Performing dimensionality reduction processing on the plurality of image local attribute codes to obtain a plurality of image local attribute codes after reprocessing; The reprocessed multiple image local attribute codes, the fused multiple image local attribute codes and the multiple attribute vector codes are fused again to obtain the multiple image local attribute codes.

6. The method according to claim 5, characterized in that The conditionally fusing the processed image vector code, the image code and the plurality of attribute vector codes to obtain a plurality of fused image local attribute codes comprises: Summing the processed image vector code and the image vector code to determine the summed image vector code; Obtaining the mean and variance of each row of elements in the summed image vector encoding; Determine a normalized image vector code according to the mean, the variance and each element value of each row of the summed image vector code; According to the normalized image vector code and the preset parameter vector, the normalized image vector code and the multiple attribute vector codes are conditionally fused to obtain the fused multiple image local attribute codes.

7. The method according to any one of claims 1 to 6, characterized in that: The local attribute information is local attribute information of multiple different attributes; The method of utilizing the local attribute encoding of the image to perform vector retrieval from the item library to determine recommended items with similar local attributes includes: By using the plurality of local attribute codes of the image and searching with the object library, an object similar to the plurality of attributes contained in the image to be searched is determined; Among the multiple items with similar attributes, similar items of each attribute are determined according to the distance between the encoding vectors.

8. The method according to claim 7, characterized in that Before searching the item library by using the plurality of image local attribute codes to determine items similar to the plurality of attributes included in the image to be searched, the method further includes: Acquire multiple object images and determine local attribute information corresponding to the object images; Encoding the plurality of object images and the local attribute information corresponding to the plurality of object images by using the preset image local attribute encoding model to determine the local attribute encodings of the plurality of object images; The item library is determined by using the local attribute codes of the multiple item images; wherein the item library includes the local attribute codes of the multiple item images and the identifiers of the multiple item images.

9. A similar item search device, characterized in that: The similar item search device comprises: A receiving unit, configured to receive an image to be searched uploaded by a user, and determine local attribute information corresponding to the image to be searched; A determination unit is used to encode the image to be searched and the local attribute information by using a preset image local attribute encoding model to determine the image local attribute encoding; wherein the local attribute information is local attribute information of multiple different attributes; and to perform vector retrieval from an item library by using the image local attribute encoding to determine recommended items with similar local attributes; The determination unit is specifically used to use the preset image local attribute coding model to encode the image to be searched and the multiple local attribute information respectively to obtain an image vector code and multiple attribute vector codes; and use the preset image local attribute coding model to fuse the image vector code and the multiple attribute vector codes to obtain multiple image local attribute codes.

10. A similar item search device, characterized in that: The similar item search device comprises: A memory for storing executable instructions; A processor, configured to implement the method according to any one of claims 1 to 7 when executing the executable instructions stored in the memory.

11. A computer-readable storage medium, characterized in that: Executable instructions are stored, which are used to cause a processor to execute and implement the method described in any one of claims 1 to 7.

Images