A multi-dimensional product information recommendation method based on conditional similarity semantic matching

Abstract

The present invention discloses a multi-dimensional commodity information recommendation method based on conditional similarity semantic matching, including a training data collection step, a model training step and a commodity recommendation step; firstly, the image data of relevant commodities are extracted on a shopping mall website and pre-processed and similarity-labeled to collect data, and after collecting sufficient commodity data, a global model is trained to obtain a global feature measurement, and then a local model of multiple dimensions is separated from the global model, each dimension corresponds to an attribute of a commodity, and finally commodity information is recommended by integrating multiple semantics. The method of the present invention does not need to make specific annotations on each dimension during implementation, but only needs to make similarity judgments to cope with the commodity information recommendation needs of users in multiple dimensions.

Description

Technical Field

[0001] The present invention relates to a multi-dimensional commodity information recommendation method based on conditional similarity semantic matching, and mainly relates to the technical fields of metric learning, image processing and image retrieval. Background Art

[0002] Over the years, product information recommendation algorithms have been a hot research topic in academia and industry. This is because this algorithm is closely related to people's daily lives, such as the current popular online shopping. How to recommend product information that users may be interested in based on the product information that users have already purchased, so that users can quickly find the product information they need, improve the customer's shopping experience and increase the store's sales, is the ultimate goal of this algorithm.

[0003] With the development of deep learning, product information recommendation algorithms have also ushered in a new wave. Although deep learning has greatly improved the success rate of information recommendation algorithms, existing information recommendation algorithms are hampered by the need for a large amount of labeled data for training, which requires a very high investment of time and manpower costs in the early stages. Summary of the invention

[0004] Purpose of the invention: In view of the problems and shortcomings in the prior art, the present invention provides a multi-dimensional product information recommendation method based on conditional similarity semantic matching, which does not require the annotation of relevant semantics for the training data, but only requires the provision of similarity relationships, which can greatly reduce the annotation cost. The model proposed in the method of the present invention maps the metric space extracted by the global model to multiple different subspaces, and these subspaces are used to explore the potential semantic associations between products, so there is no need to explicitly annotate semantics.

[0005] Technical solution: A multi-dimensional product information recommendation method based on conditional similarity semantic matching, including a training data collection step, a model training step and a product information recommendation step; in the training data collection process, first collect pictures of the product, then annotate the pictures with similarity attention to obtain training data; in the model training process, input the training data to train the global model and the local model, integrate the outputs of all local models to obtain the final result, compare the final result with the actual label, obtain the loss, and optimize the model; in the product information recommendation process, load the model, obtain the product pictures that the user was previously interested in, input the product pictures into the model, obtain the output results on all local models, and integrate all the outputs to obtain the final retrieval result.

[0006] The training data collection steps are specifically as follows:

[0007] Step 101, collecting pictures of products on product websites or platforms;

[0008] Step 102, pre-processing the collected images;

[0009] Step 103, marking similarity relationships on the preprocessed images;

[0010] Step 104: If the amount of product image data meets the training requirement, there is no need to continue collecting and obtain training data; if it does not meet the requirement, repeat steps 101 to 103.

[0011] The specific steps of model training are:

[0012] Step 201, initializing a deep neural network model;

[0013] Step 202, using all the training data to train a global model for the deep neural network model;

[0014] Step 203, training all local models;

[0015] Step 204, outputting similarity judgment results from all local models;

[0016] Step 205, synthesizing the outputs of all local models to obtain the final result;

[0017] Step 206, calculate the loss function and compare the final result with the actual label; the actual label refers to whether the triple is established, that is, the label used to judge the similarity relationship. For example, if the triple is established (that is, the distance between the similar sample and the anchor sample is less than the distance between the non-similar sample and the anchor sample), the label is recorded as 1, otherwise it is recorded as 0;

[0018] Step 207, optimizing the entire model by back propagation of the loss function; the entire model includes a global model and a local model;

[0019] Step 208, determine whether the entire model has converged, if so, go to step 209, otherwise go back to step 202;

[0020] Step 209, save the entire model M0.

[0021] The specific steps for product information recommendation are as follows:

[0022] Step 301, loading the entire model M0;

[0023] Step 302, obtaining product images that the user was previously interested in; the product images that the user was interested in refer to product image information that the user has clicked or purchased;

[0024] Step 303, inputting the product pictures that the user was previously interested in into the model;

[0025] Step 304, obtaining output results on all local models;

[0026] Step 305, synthesizing all output results to obtain the final search result;

[0027] Step 306: sort the final results and output them to the user.

[0028] The training workflow of the global model is as follows: (1) randomly initialize the global metric model parameters θ0; (2) collect training data through the data collection process, use the collected data and its labels to construct a data set, and sample several triplets (A, P, N) from the data set each time, where A is an anchor sample, P is a sample of the same class as A, and N is a sample of a different class from A; the label is a labeled similarity relationship; (3) optimize the global metric model so that the distance between A and P is within a threshold (m1) and the distance between A and N is greater than a threshold (m2). This goal is achieved through the global metric training objective function F0; (4) repeat the sampling in (2) and the optimization process in (3) until the training error converges.

[0029]

[0030] In the above formula, (A, P, N) ~ D means that the triples are sampled from the data distribution D, D refers to the data distribution satisfied by all data, and E represents the expectation of the distribution. dist(x, y) represents a function that measures the similarity of two vectors x and y. The smaller dist(x, y), the more similar x and y are.

[0031] The training workflow of the local model is as follows: the global model is mapped to different subspaces, which are different projections of the global space, and then a vector is learned for each subspace as its representation. For each training sample, the training sample refers to a triple composed of product images, and an additional neural network is used to learn the correlation between it and different subspaces, so as to distribute the sample to different subspaces according to different proportions. In each subspace, dist(x, y) is recalculated, and then the training objective function F0 is re-optimized. Specifically, the product images in the form of triples are spliced ​​into a vector, input into an additional neural network (mainly composed of a fully connected layer and a nonlinear activation layer), and an output vector is obtained, which can be considered as a metric representation of the triple. Then the cosine distance between this metric representation and different subspaces is calculated to calculate the similarity, and the similarity is used as a weight to distribute the triple to different subspaces. Repeat the sampling and optimization process until the training error converges. Sampling refers to the sampling of triplets (this is consistent with the global model part), and optimization refers to "recalculating dist(x, y) in each subspace, and then re-optimizing the training objective function F0."

[0032] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the multi-dimensional product information recommendation method based on conditional similarity semantic matching as described above is implemented.

[0033] A computer-readable storage medium stores a computer program for executing the multi-dimensional commodity information recommendation method based on conditional similarity semantic matching as described above.

[0034] Beneficial effect: Compared with the prior art, the multi-dimensional product information recommendation method based on conditional similarity semantic matching provided by the present invention can learn the similarity measurement under different semantic conditions without labeling the specific semantics, thereby providing recommendations for product information under multiple semantic dimensions. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 A flowchart of the training data collection process according to an embodiment of the present invention;

[0036] Figure 2 This is a model training workflow diagram of an embodiment of the present invention;

[0037] Figure 3 This is a flowchart of a product information recommendation process according to an embodiment of the present invention. DETAILED DESCRIPTION

[0038] The present invention is further explained below in conjunction with specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention. After reading the present invention, various equivalent forms of modifications to the present invention by those skilled in the art all fall within the scope defined by the claims attached to this application.

[0039] like Figure 1As shown, a certain number of images with similarity relationship annotations need to be collected during the training phase, which is the data collection process. Specifically, first initialize the mobile device, which usually includes hardware device inspection, program running environment inspection and other steps (step 101); then collect product images from websites or other channels (step 102); perform pre-processing such as denoising, resizing, and centering on the images (step 103); manually mark the similarity relationship of the training samples, and note that here the sample pairs in the form of triples (anchor samples, similar samples, and dissimilar samples) are marked; as for whether the judgment is similar, it depends on the specific downstream task requirements, and can be determined based on some specific objective standards, such as color, style, texture, shape, etc. In the absence of specific objective standards, it is determined by the subjective judgment of the standard setter (step 104); determine whether a preset number of training samples have been obtained (step 105), if sufficient, enter step 106, if not, then turn to step 102; send the labeled training samples to the model for training (step 106).

[0040] The workflow of the model training phase is as follows Figure 2 shown.

[0041] Specifically, the model is first initialized, mainly the construction and random initialization of the deep neural network (step 201).

[0042] Then, a global model is trained, that is, a global metric space is trained for all images. The global model used is the residual neural network Resnet50 (step 202).

[0043] The training workflow of the global model is as follows: (1) Randomly initialize the global metric model parameters θ0; (2) Collect training data through the data collection process, use the collected data and its labels to construct a dataset, and sample several triplets (A, P, N) from the dataset each time, where A is the anchor sample, P is the sample of the same class as A, and N is the sample of a different class from A; (3) Optimize the global metric model so that the distance between A and P is within a threshold (m1), and the distance between A and N is greater than a threshold (m2). This goal is achieved through the global metric training objective function F0; (4) Repeat the sampling in (2) and the optimization process in (3) until the training error converges.

[0044]

[0045] In the above formula, (A, P, N) ~ D means that the triples are sampled from the data distribution D, D refers to the data distribution satisfied by all data, and E represents the expectation of the distribution. dist(x, y) represents a function that measures the similarity of two vectors x and y. The smaller dist(x, y), the more similar x and y are.

[0046] Train all local models, that is, learn subspaces representing different semantics from the global space, where all local models are different projections of the global model (step 203). Calculate the distances between triplet images in different subspaces to obtain similarity judgment results. Specifically, the distances between two non-anchor samples and anchor samples are calculated respectively, and the similarity relationship is determined based on the comparison of distances (step 204). The outputs of all subspaces are integrated to obtain the final similarity judgment result. The integrated method here is weighted average, and the weight comes from the similarity calculation between the triplet and different subspaces (step 205). Compare with the actual label and calculate the loss function (step 206). Optimize the entire model through back propagation of the loss function (step 207). Determine whether the model converges (step 208). If so, go to step 209, otherwise go back to step 202 to continue training; finally, save the trained model (step 209).

[0047] The workflow of the product information recommendation stage is as follows: Figure 3 Specifically, first load the previously trained model (step 301); then obtain the product image that the target customer was interested in before (step 302); input the image into the model (step 303); calculate the distance between the image and other images on all local models (step 304); combine the distances on these different subspaces and output the final distance (step 305); sort the final results and output the most similar images (step 306); finally feed back the output results to the customer (step 307).

[0048] Obviously, those skilled in the art should understand that the various steps of the multi-dimensional commodity information recommendation method based on conditional similarity semantic matching of the above-mentioned embodiment of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. In this way, the embodiments of the present invention are not limited to any specific combination of hardware and software.

Claims

1. A multi-dimensional product information recommendation method based on conditional similarity semantic matching, characterized in that: The method includes a training data collection step, a model training step and a product information recommendation step. In the training data collection process, firstly, product images are collected, and then similarity relationships are annotated on the images to obtain training data. During the model training process, input training data to train the global model and local models, combine the outputs of all local models to get the final result, compare the final result with the actual label, get the loss, and optimize the model; during the product information recommendation process, load the model, obtain the product images that the user was interested in, input the product images into the model, get the output results on all local models, and select some results to feed back to the user; The model training steps are specifically as follows: Step 201, initializing a deep neural network model; Step 202, using all the training data to train a global model for the deep neural network model; Step 203, training all local models; Step 204, outputting similarity judgment results from all local models; Step 205, synthesizing the outputs of all local models to obtain the final result; Step 206, calculate the loss function and compare the final result with the actual label; Step 207, optimizing the entire model by back propagation of the loss function; the entire model includes a global model and a local model; Step 208, determine whether the entire model has converged, if so, go to step 209, otherwise go back to step 202; Step 209, saving the entire model M0; The specific steps for product information recommendation are as follows: Step 301, loading the entire model M0; Step 302, obtaining product images that the user was previously interested in; the product images that the user was interested in refer to product image information that the user has clicked or purchased; Step 303, inputting the product pictures that the user was previously interested in into the model; Step 304, obtaining output results on all local models; Step 305, synthesizing all output results to obtain the final search result; Step 306: sort the final results and output them to the user.

2. The multi-dimensional product information recommendation method based on conditional similarity semantic matching according to claim 1, characterized in that: The training data collection steps are specifically as follows: Step 101, collect pictures of products; Step 102, pre-processing the collected images; Step 103, marking similarity relationships on the preprocessed images; Step 104: If the amount of product image data meets the training requirement, there is no need to continue collecting and obtain training data; if it does not meet the requirement, repeat steps 101 to 103.

3. The multi-dimensional product information recommendation method based on conditional similarity semantic matching according to claim 1, characterized in that: The training workflow of the global model is as follows: (1) randomly initialize the global metric model parameters; (2) collect training data through the data collection process, use the collected data and its labels to construct a dataset, and sample several triplets (A, P, N) from the dataset each time, where A is the anchor sample, P is a sample of the same class as A, and N is a sample of a different class from A; (3) optimize the global metric model so that the distance between A and P is within a threshold (m1) and the distance between A and N is greater than a threshold (m2). This goal is achieved through the global metric training objective function F0; (4) repeat the sampling in (2) and the optimization process in (3) until the training error converges; ， In the above formula, (A, P, N)~D indicates that the triple is sampled from the data distribution D, D refers to the data distribution satisfied by all data, and E represents the expectation of the distribution; dist(x, y) represents a function that measures the similarity of two vectors x and y. The smaller dist(x, y) is, the more similar x and y are.

4. The multi-dimensional product information recommendation method based on conditional similarity semantic matching according to claim 1, characterized in that: The training workflow of the local model is as follows: mapping the global model to different subspaces, where the different subspaces are different projections of the global space, and then learning a vector for each subspace as its representation; For each training sample, the product images in triple form are concatenated into a vector and input into a neural network to obtain an output vector, which is a metric representation of the triple. The cosine distance between the metric representation and different subspaces is then calculated to calculate the similarity, and the similarity is used as a weight to assign the triple to different subspaces. The sampling and optimization process is repeated until the training error converges.

5. A computer device, characterized in that: The computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, a multi-dimensional product information recommendation method based on conditional similarity semantic matching as described in any one of claims 1 to 4 is implemented.

6. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program for executing the multi-dimensional product information recommendation method based on conditional similarity semantic matching as described in any one of claims 1 to 4.

Images