Method and device for predicting similar items and training model

Abstract

The invention discloses a method and a device for predicting similar items and training a model. The method for training a predicted similar item model comprises the following steps of acquiring the word bag expression and the vector expression of items, wherein the vector expression of the item is related to the attribute dimension value of the item and a word vector; obtaining the vector expression of an item pair based on the distance characteristic of the item pair, wherein the distance characteristic of the item pair is related to the word bag expression and the vector expression of the item; and combining the vector expression of a similar item pair and the vector expression of a randomly-sampled item pair, training a classification model. In the invention, through the vector expression of the item, the vector expression of the item pair is acquired and the classification model is trained; and the input data of the trained classification model is compressed several times so as toimprove the speed of the trained classification model and the accuracy of a training result. The method is suitable for various items, such as a commodity, a cargo, a product and the like.

Description

Technical field [0001] The present invention relates to the field of computers, in particular to the field of machine learning, and in particular to a method and device for predicting similar items and training their models. Background technique [0002] Similar item pairs are two items that are similar in appearance, price, and use in dozens of dimensions. Among them, different dimensions affect similarity to different degrees, and different categories of the same dimension affect similarity to different degrees. [0003] At present, traditional natural language processing often uses a bag-of-words model to quantify a piece of text that describes a thing, but for computers, this method is difficult to accurately calculate the semantic and grammatical similarity of two words and articles, and the data of the bag-of-words model is sparse The curse of sex and dimensionality will lead to a decrease in the performance of the model. In recent years, shallow neural networks have been gr...

AI Technical Summary

Problems solved by technology

[0003] Currently, traditional natural language processing often uses the bag-of-words model to quantify a piece of text describing things, but for computers, this method is difficult to accurately calculate the semantic and grammatical similarity between two words and articles, and the data of the bag-of-words model is sparse The curse of sex and dimensionality can lead to poor performance of the model

In recent years, shallow neural networks have been gradually used to learn low-dimensional and continuous word vectors directly from a large amount of text data. Word vectors can effectively express the semantics and grammar of a word, but they still have certain limitations for classifying items to determine whether two items are similar. Limitations, one means that the item word vector still needs high latitude, and most of the dimensions are useless, and the really useful features are hidden in those few dimensions, resulting in a large loss of training time, and the judgment results are not informative , and the correlation between the second item pair is relatively weak, for example, it will determine that the two product description texts of "long skirt" and "short skirt" are products with the same similarity

Images