Attribute category representation method and apparatus, terminal device, and storage medium

Abstract

The application provides a representation method and device of an attribute category, terminal equipment and a storage medium. Attribute category data to be identified is obtained, the attribute category data to be identified comprises at least one attribute category; the encoding of each attribute category is found from a pre-configured attribute category encoding table; the encoding of each attribute category is input into a pre-established word vector table to obtain a representation vector of each attribute category; the word vector table is learned from a word vector embedding model, and the word vector embedding model is pre-trained on an Embedding module based on a self-attention mechanism method by using a masked attribute category encoding sequence sample. The method can quickly and effectively convert the input attribute category encoding into a vector by using the pre-established word vector table; and the converted output vector is low-dimensional and dense, which can greatly reduce vector operation and storage space.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and in particular to an attribute category representation method and device, a terminal device and a storage medium. BACKGROUND

[0002] With the continuous development of science and technology, the Internet has penetrated into all aspects of people's life, and therefore a large amount of Internet data has been generated. The Internet data often contains a large amount of category data (i.e. attribute category), such as gender, region, hobby, color, etc. Before being used in many application scenarios for modeling, the category data often needs to be converted into a numerical type, for example, the category data is converted into a vector value, i.e. an attribute category is represented by a set of vector values.

[0003] At present, the one-hot encoding method is mainly used for vectorization representation of the category data. However, the vector converted by the method is very sparse, which causes huge overheads for model training and storage. Moreover, the vector converted by the method cannot represent the similarity relationship of similar attribute categories. SUMMARY

[0004] Therefore, the present application provides an attribute category representation method, device, terminal device and storage medium to overcome the problem that the vector converted by the vectorization representation method of the category data in the prior art is very sparse, which causes huge overheads for model training and storage, and the vector converted by the method cannot represent the similarity relationship of similar attribute categories.

[0005] In a first aspect, the present application provides an attribute category representation method, which comprises:

[0006] obtaining attribute category data to be identified, wherein the attribute category data to be identified comprises at least one attribute category;

[0007] finding the encoding of each attribute category from a pre-configured attribute category encoding table;

[0008] finding the representation vector of each attribute category from a pre-established word vector table, wherein the word vector table is learned from a word vector embedding model, and the word vector embedding model is trained on an Embedding module based on a self-attention mechanism method by using a mask-processed attribute category encoding sequence sample.

[0009] In a second aspect, the present application provides an attribute category representation device, which comprises:

[0010] an attribute category data to be identified obtaining module, configured to obtain attribute category data to be identified, wherein the attribute category data to be identified comprises at least one attribute category;

[0011] The encoding searching module is configured to search the encoding of each attribute category from a pre-configured attribute category encoding table.

[0012] The vector obtaining module is configured to search the representation vector of each attribute category from a pre-established word vector table, wherein the word vector table is learned from a word vector embedding model, and the word vector embedding model is trained based on a self-attention mechanism method by using the attribute category encoding sequence sample processed by the mask.

[0013] In a third aspect, an embodiment of the present application provides a terminal device, comprising: a memory; one or more processors coupled to the memory; and one or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the one or more processors, and the one or more application programs are configured to perform the attribute category representation method provided in the first aspect.

[0014] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, and the computer readable storage medium stores program codes, and the program codes can be invoked by a processor to execute the attribute category representation method provided in the first aspect.

[0015] The attribute category representation method, the device, the terminal device and the storage medium provided in the embodiments of the present application first acquire attribute category data to be recognized, wherein the attribute category data to be recognized comprises at least one attribute category; then search the encoding of each attribute category from a pre-configured attribute category encoding table; and then search the representation vector of each attribute category from a pre-established word vector table, wherein the word vector table is learned from a word vector embedding model, and the word vector embedding model is trained based on a self-attention mechanism method by using the attribute category encoding sequence sample processed by the mask.

[0016] The attribute category representation method provided in the embodiments of the present application can quickly and effectively search the vector corresponding to the encoding of the attribute category by using the pre-established word vector table; and the word vector embedding model is obtained by training the Embedding module, so that the vector output by the conversion is low-dimensional and dense, which can greatly reduce the vector operation and storage space, and the similarity of different attribute categories can be calculated by using the vector representation. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor based on the provided drawings.

[0018] Figure 1 The application scenario diagram of the attribute category representation method provided by the embodiment of the present application is shown.

[0019] Figure 2 The flowchart of the method provided by an embodiment of the present application is shown.

[0020] Figure 3 The network structure diagram used when training the word vector embedding model provided by an embodiment of the present application is shown.

[0021] Figure 4 The structure diagram of the attribute category representation device provided by an embodiment of the present application is shown.

[0022] Figure 5 The structure diagram of the terminal device provided by an embodiment of the present application is shown.

[0023] Figure 6 The structure diagram of the computer readable storage medium provided by an embodiment of the present application is shown. DETAILED DESCRIPTION

[0024] The technical solutions in the embodiments of the present application will be described clearly and completely below. Obviously, the described embodiments are only some of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0025] In order to more clearly illustrate the present application, the following will specifically describe the attribute category representation method, device, terminal device and computer readable storage medium provided by the present application in combination with the drawings.

[0026] Please refer to Figure 1 , Figure 1A schematic diagram of an application scenario of the representation method of the attribute category provided in the embodiments of the present application is shown, and the application scenario includes a terminal device 100 provided in the embodiments of the present application. The terminal device 100 can be various electronic devices (such as the structural diagrams of 102, 104, 106, and 108) having a display screen, including but not limited to a smart phone and a computer device. The computer device can be at least one of a desktop computer, a portable computer, a laptop computer, a tablet computer, and the like. The terminal device 100 can store a pre-established word vector table and a preconfigured attribute category coding table. When it is necessary to represent attribute category data to be identified, the pre-established word vector table and the preconfigured attribute category coding table are called to perform the representation method of the attribute category, so as to obtain a representation vector of the attribute category.

[0027] The terminal device 100 can be used to represent one of a plurality of terminal devices, and the embodiments of the present application are only used as an example. Those skilled in the art can know that the number of the terminal devices can be more or less. For example, the terminal devices can be only a few, or the terminal devices can be dozens or hundreds, or more, and the number and type of the terminal devices are not limited in the embodiments of the present application. The terminal device 100 can be used to perform the representation method of the attribute category provided in the embodiments of the present application.

[0028] In an optional implementation, the application scenario includes a server in addition to the terminal device 100 provided in the embodiments of the present application, and a network is arranged between the server and the terminal device. The network is used as a medium to provide a communication link between the terminal device and the server. The network can include various connection types, such as a wired, wireless communication link, or a fiber cable, and the like.

[0029] It should be understood that the number of terminal devices, networks, and servers is only illustrative. According to the implementation needs, there can be any number of terminal devices, networks, and servers. For example, the server can be a server cluster composed of multiple servers. The terminal device interacts with the server through the network to receive or send messages, and the like. The server can be a server providing various services. The server can be used to perform the steps of the representation method of the attribute category provided in the embodiments of the present application. In addition, the terminal device can perform part of the steps in the terminal device and part of the steps in the server when performing the representation method of the attribute category provided in the embodiments of the present application, which is not limited herein.

[0030] In related data in the Internet field, there are often category data, such as gender (including male and female), region, hobby, and the like. These data need to be converted into numerical type before implementing algorithm modeling of many application scenarios, and it is particularly common to represent the category data by vectorization, that is, to represent each category by a set of numerical values.

[0031] The category type data involved in the e-commerce field is particularly so, in order to facilitate understanding, taking the e-commerce field as an example, the category type data, especially the multi-value category type data, is described. For example, when describing clothing goods, the attributes it contains include material, color, style, etc.; among them, the material includes acetate, chemical fiber, gauze, fiber, cotton, fine sulfur cotton, cotton blend, cotton fabric, cotton spandex, polyester, corduroy, coral velvet, artificial leather, real leather, lace, linen, wool, duck down, down, silk, etc. tens of thousands of categories.

[0032] The color includes red, Chinese red, light red, dark red, blue, lake blue, peacock blue, pink, light pink, beige, brown, light brown, dark brown, champagne, khaki, gray, etc. up to thousands of categories.

[0033] The style includes all-match, office, formal, casual, home, retro, classic, elegant, romantic, noble, bohemian, national, fashion, outdoor, simple, cute, two-dimensional, fresh, sweet, etc. tens of thousands of categories.

[0034] In addition, the clothing attribute also has collar type, skirt type, sleeve type, clothes version, pants type, heel type, boot type, etc., which also has multiple category attribute characteristics.

[0035] The above attributes are important dimensions for describing clothing goods and often need to be used as input features for some application scenario algorithm models, including but not limited to: similar goods recommendation, user interested goods recommendation, goods sales estimation, and goods advertisement click rate estimation. When using category type data in the algorithm model of the application scenario, the category type data often needs to be vectorized first. Currently, the common practice of category type data vectorization is to first encode each attribute category in the category type data with a number, and then use one-hot encoding to get the vector representation corresponding to each attribute category. For example: for the "material" attribute, assuming it contains 30,000 categories, it can be encoded according to the order number of "acetate, chemical fiber, gauze, etc." categories to get the following corresponding relationship: 1 represents acetate, 2 represents chemical fiber, and 3 represents gauze. Then, one-hot encoding is used to get their vector representation: [1, 0, 0, 0, …, 0] represents acetate, [0, 1, 0, 0, …, 0] represents chemical fiber, and [0, 0, 1, 0, …, 0] represents gauze. Each vector is 30,000-dimensional, except for the 1 shown, the others are omitted. For the i-th category, the vectorization example is as follows:

[0036]

[0037] This vectorization method has two prominent problems: (1) It suffers from the curse of dimensionality. The vector is very sparse, which will cause huge overhead to the later model training and storage process, and may even affect the selection of the algorithm model, its generalization ability and convergence. The above-mentioned single "material" attribute requires a 30,000-dimensional vector. (2) The similarity between vectors cannot be measured because each vector is orthogonal. No matter what common similarity measurement algorithms such as Euclidean distance or cosine distance are used, the similarity between any two vectors is 0. The similarity between "red" and "Chinese red" in the above-mentioned "color" attribute is the same as that between "red" and "blue", which is obviously not in line with common sense.

[0038] Based on this, this application provides a method for representing attribute categories. Please refer to... Figure 2 , Figure 2 This illustration shows a flowchart of a method for representing attribute categories provided in an embodiment of this application, demonstrating how this method can be applied to... Figure 1 Taking a terminal device as an example, the explanation includes the following steps:

[0039] Step S110: Obtain the attribute category data to be identified.

[0040] The attribute type data to be identified includes at least one attribute category.

[0041] The attribute type data to be identified can be any data formed by attribute categories that need to be represented by vectors. For example, the attribute type data to be identified could be clothing with the "material" attribute category being acetate fabric.

[0042] The attribute type data to be identified can be one or more attribute categories; for example, when the attribute type data to be identified is "material" attribute category of "clothes made of acetate fabric", then the attribute type data to be identified only has one attribute category: "material" attribute category of "acetate fabric"; when the attribute type data to be identified is "clothes made of acetate fabric" attribute category and "color" attribute category of "red", then the attribute type data to be identified has two attribute categories: "material" attribute category of "acetate fabric" and "color" attribute category of "red".

[0043] It should be noted that the attribute type data to be identified is different for different fields, objects or application scenarios. Specifically, it can be determined according to the attributes of the object to be identified. For example, when the object to be identified is clothing, then the attribute type data is determined according to clothing; when the object to be identified is food, then the attribute type data is determined according to food.

[0044] Step S120: Find the code for each attribute category from the pre-configured attribute category coding table.

[0045] The attribute category coding table is a data table formed by coding different attribute categories of an object (e.g. an article (such as a clothing article), food, etc.). In this embodiment, simple Arabic numerals are generally used to code different attribute categories. The coding method can be various, and the coding of each attribute category is different.

[0046] Optionally, when coding different attribute categories, the attribute categories can be coded according to the order of appearance of the attribute categories in a certain environment or use scenario, or the frequency of appearance. Alternatively, the attribute categories can be coded according to the order of the names of the attribute categories in a dictionary.

[0047] To facilitate understanding, a detailed embodiment is given. Taking clothing article data as an example, all attribute categories (i.e. all categories of all attributes) of the clothing can be coded as id according to the order of appearance, to obtain an attribute category id coding table as follows: {0: "mask symbol <mask>1: "A attribute | category A_1", 2: "A attribute | category A_2", …, N: "Z attribute | category Z_1", N+1: "Z attribute | category Z_2", …}. For example, the attribute category of the above-mentioned clothing goods can be coded as: {1: "material | acetate", 2: "material | chemical fiber", …, 30001: "color | red", 30002: "color | Chinese red", …}.

[0048] After the attribute category coding table is pre-configured, when the attribute category data to be recognized is obtained, the coding of each attribute category can be obtained by querying the pre-configured attribute category coding table.

[0049] In step S130, the representation vector of each attribute category is found from the pre-established word vector table; wherein the word vector table is learned from a word vector embedding model, and the word vector embedding model is trained based on a self-attention mechanism method by using the masked attribute category coding sequence sample.

[0050] Wherein, the model training is to give the input vector and the target output value, then input the input vector into one or more network structures or functions to obtain the actual output value, and calculate the bias according to the target output value and the actual output value, and judge whether the bias is within the allowable range; if it is within the allowable range, the training is ended and the related parameters are fixed; if it is not within the allowable range, some parameters in the network structure or function are adjusted constantly until the bias is within the allowable range or a certain ending condition is reached, the training is ended and the related parameters are fixed, and finally the trained model can be obtained according to the fixed related parameters.

[0051] In the embodiment, the word vector table is actually a parameter learned when the Embedding module is learned and trained (i.e. the parameter of learning and training the Embedding module), which is the vector representation of the attribute category coding. Wherein, the masked attribute category coding sequence sample and the self-attention mechanism method are used when learning and training the parameter of the Embedding module.

[0052] Embedding is a way to convert discrete variables into continuous vector representation, which not only reduces the spatial dimension of discrete variables, but also meaningfully represents the variable. The key to training Embedding is to define a training task, which is to define or establish the masked attribute category (i.e. <mask>corresponding attribute category) and adopts a self-attention mechanism method to train the Embedding.

[0053] The Embedding module is composed of an Embedding layer, and in the embodiment of the application, one Embedding layer can be used to construct the Embedding module.

[0054] After the parameter learning of the Embedding module is completed, the Embedding module is saved as a word vector table. In actual application, for input attribute category data to be recognized, the pre-configured attribute category code table is first searched to obtain the code (i.e., id) of each attribute category, and then the word vector table is searched to obtain the vector corresponding to each attribute category id. For example, for attribute category data of the "material" attribute with a value of "acetic acid cloth", the vector representation process of the material attribute is as follows: from the attribute category code table, it is found that the id of the "material | acetic acid cloth" attribute category is 1, and the vector with the index number 1 from the word vector table is the vector corresponding to it.

[0055] The attribute category representation method provided in the embodiment of the application first obtains attribute category data to be recognized, wherein the attribute category data to be recognized includes at least one attribute category; then the code of each attribute category is found from a pre-configured attribute category code table; and then the code of each attribute category is input to a pre-established word vector table to obtain the representation vector of each attribute category; wherein the word vector table is formed by training an Embedding module based on a self-attention mechanism method using a masked attribute category code sequence sample.

[0056] The attribute category representation method provided in the embodiment of the application can quickly and effectively find the vector corresponding to the code of the attribute category using the pre-established word vector table; and the word vector embedding mode is obtained by training the Embedding module, so the vector output by the conversion has low dimension and high density, which can greatly reduce vector operation and storage space, and the similarity of different attribute categories can be calculated through vector representation.

[0057] Further, a specific implementation of establishing a word vector table is given, and the details are as follows:

[0058] In one embodiment, the method for establishing a word vector table includes:

[0059] S1: Obtain an attribute category code sequence sample.

[0060] Specifically, a large number of attribute category code sequences need to be prepared to form an attribute category code sequence sample. Each attribute category code sequence includes the codes of multiple attribute categories.

[0061] In one embodiment, in the step S1, the attribute category code sequence sample is obtained, including: obtaining attribute category data samples; wherein the attribute category data samples include a plurality of attribute category data, each attribute category data includes a plurality of attributes, each attribute includes a plurality of categories; encoding each attribute category in each attribute category data; splitting the attribute category data after encoding each attribute category to form a plurality of attribute category code sequences with equal length; and forming the attribute category code sequence sample according to the plurality of attribute category code sequences.

[0062] The attribute category data sample is constructed by a plurality of attribute category data, each attribute category data includes a plurality of attributes, each attribute includes a plurality of categories, i.e. each attribute category data includes a plurality of attribute categories. Each attribute category is encoded, and then the attribute category data after encoding is split to form a plurality of attribute category code sequences with equal length. Optionally, the Cartesian product can be used to split the attribute category data.

[0063] To facilitate understanding, a detailed embodiment is given. Taking the clothing commodity data as an example, the forming process of the attribute type sequence sample is as follows: first, a sufficient number of attribute category data (i.e. attribute category data samples) are obtained, each attribute category data contains a plurality of attributes, each attribute contains a plurality of categories, i.e. each attribute category data contains a plurality of attribute categories. Second, all the categories of all the attributes are encoded according to the order of appearance as id to obtain an attribute category id encoding table, for example: {1: "A attribute | category A_1", 2: "A attribute | category A_2", …, N: "Z attribute | category Z_1", N+1: "Z attribute | category Z_2", …}. Then, each attribute (or attribute feature) of each attribute category data has some category values, and the attribute category encoding table is looked up to convert each attribute into its corresponding code (i.e. id), for example, the category value of the "material" attribute is "acetate cloth", and the value of acetate cloth is 1, i.e. "material | acetate cloth", which is represented by 1.

[0064] Since one attribute often corresponds to multiple categories, the attribute corresponds to multiple values. For example, a piece of leather down jacket, its "material" attribute includes leather and down jacket, at this time each value can be regarded as a case, using Cartesian product method to split can form multiple attribute category coding sequence. For example, there is an attribute category data A attribute has 2 categories {A1 and A2}, B attribute has 3 categories {B1, B2 and B3}, then the attribute category data is split to produce (A1, B1) (A1, B2) (A1, B3) (A2, B1) (A2, B2) (A2, B3) 6 attribute category coding sequence, wherein A1, A2, B1, B2 and B3 in attribute category coding sequence respectively refer to the coding of attribute category. Using this way can make each attribute category data of multi-category attribute is 1 category, the input attribute category coding sequence before vectorization is equal length.

[0065] It should be noted that in actual application, an attribute category coding sequence can include the coding of one attribute type, the coding of two attribute types, or the coding of many attribute types, which can be determined according to the number of attribute categories included in an attribute category data. In addition, an attribute category data can be split into one or more attribute category coding sequences, and the specific number is determined according to the number of attributes and the number of categories included in an attribute category data.

[0066] S2: randomly selecting one or more attribute category encodings from the attribute category coding sequence of the attribute category coding sequence sample for mask processing to obtain a masked attribute category coding sequence sample.

[0067] Wherein, after obtaining the attribute category coding sequence sample (i.e. multiple attribute category coding sequences), the attribute category coding sequence can be masked. The specific process is: randomly selecting one or more attribute category encodings from the attribute category coding sequence, and replacing these selected index category encodings with a specific number (such as 0). For example: randomly selecting one id from an attribute category coding sequence to mask.

[0068] In addition, in order to obtain a larger number of masked attribute category coding sequences, when masking an attribute category coding sequence, the attribute type coding sequence can be copied first, and then different masking is performed on the multiple same attribute category coding sequences formed by copying, so as to obtain multiple masked attribute category coding sequences.

[0069] A detailed embodiment is given for the convenience of understanding. The actual operation of the mask processing is: randomly extracting m ids from one or more attribute category encoding sequences to form an id sequence, copying the id sequence into m parts, and then replacing the ith id in the ith part of the id sequence with a specific number 0 to form m mask attribute category encoding sequences. For example, an attribute category encoding sequence (A1, B2, C1, D4), randomly extracting (A1, D4) as the two ids to be masked; the first id "A1" in (A1, D4) is replaced with 0 in the first copied (A1, B2, C1, D4) to obtain (0, B2, C1, D4); the second id "D4" in (A1, D4) is replaced with 0 in the second copied (A1, B2, C1, D4) to obtain (A1, B2, C1, 0).

[0070] If the clothing commodity data is taken as an example, it is assumed that the category value of the "material" attribute of a certain attribute category is "acetate cloth", the category value of the "color" attribute is "Chinese red", the category value of the "style" attribute is "national", and the values of other attributes are omitted. If the "style" attribute is masked when randomly masking, the obtained mask attribute category encoding sequence is [1, 30002, 0,...].

[0071] S3: inputting the mask attribute category encoding sequence sample into the Embedding module to output the attribute category vector.

[0072] Please refer to Figure 3 The Embedding module can be composed of an Embedding layer, as shown in FIG. 2. The specific process of inputting the mask attribute category encoding sequence sample into the Embedding module to output the attribute category vector is: mapping each input attribute category id into a specific Embedding vector xi of the same dimension through an Embedding lookup table. Assuming that the category of all attribute categories is vocab_size and the dimension of the Embedding vector xi is embedding_dim, any attribute category id can be converted into a low-dimensional dense vector of embedding_dim after passing through the Embedding layer, and this vector is the representation vector of each attribute category required in the embodiment of the application, that is, the attribute category vector. The parameter [vocab_size, embedding_dim] of the Embedding layer is the parameter to be learned and trained by the subsequent model.

[0073] Regarding `embedding_dim`, the size of the corpus and vocabulary need to be considered. Different dimensions can be chosen based on the actual effect and application scenario, but generally, the dimension of `embedding_dim` is much smaller than the number of categories in the multi-category attribute itself. In this embodiment, `embedding_dim` can be 500-dimensional. The "material" attribute has 30,000 categories. If one-hot encoding is used for the categories of the material attribute, a sparse vector with up to 30,000 dimensions is obtained. However, if an embedding lookup table encoding is used, only 500 dimensions are required, a difference of more than 60 times. It can be seen that the dimension of the vector representation can be significantly reduced.

[0074] S4: Input the attribute category vector into the self-attention module to obtain the attention vector.

[0075] The output X = [x1, x2, ..., x] from the Embedding layer i ,…,x N ], where x i The embedding vector corresponding to the i-th attribute is used as input. A multi-head self-attention module is adopted to select key information related to the current attribute category from all input information, which effectively alleviates the problem of information overload and improves the training efficiency of the neural network.

[0076] The self-attention module can be a multi-head self-attention module or a spatial attention module.

[0077] In one embodiment, please refer to Figure 3 As shown, there are one or more self-attention modules; when there are multiple self-attention modules, each self-attention module is connected sequentially; the self-attention module includes a multi-head self-attention layer and an additional layer, the additional layer including one or more of a fully connected layer, a batch normalization layer, or a Dropout layer; in S4: the attribute category vector is input into the self-attention module to obtain the attention vector, including: inputting the attribute category vector into the multi-head self-attention layer to obtain the attention features; inputting the attention features into the additional layer for spatial transformation, normalization processing, or hiding processing to obtain the attention vector.

[0078] Specifically, the computation process of the multi-head attention model is as follows: for each input xi in any input sequence X, it is first linearly mapped to three different spaces to obtain three different vectors: query vector qi, key vector ki, and value vector vi.

[0079] Q = W q X

[0080] K = W k X

[0081] V = W v X

[0082] where W q W k W v is the parameter matrix of linear mapping, Q = [q1, q2, …, q i , …, q N ] is a matrix composed of query vectors qi, K = [k1, k2, …, k i , …, k N ] is a matrix composed of key vectors ki, V = [v1, v2, …, v i , …, v N ] is a matrix composed of query vectors vi.

[0083] Then, the significance weighted vector of each query vector qi in the query matrix Q and all key-value pair vectors is calculated, that is, the Attention learned by the query from the context. Generally, multiple self-attention integration methods are used, that is, multiple heads, to obtain the features of X in multiple different representation spaces. The specific expression is:

[0084]

[0085] MultiHead(Q, K, V) = Concat(head1, head2, …, head h )W o

[0086] where head i = Attention(QW i Q , KW i K , VW i V )

[0087] In addition, a fully connected layer is connected after the multi-head self-attention layer to further process and learn the attention features in different spaces. Optionally, the multi-head self-attention layer can be connected in a residual manner to improve the problem of gradient disappearance and model degradation. Optionally, the self-attention module (i.e., Block) composed of the multi-head self-attention layer or other operations (i.e., additional layers) can be stacked to obtain higher-order features.

[0088] In one embodiment, in the step S3 of inputting the attribute category vector into the self-attention module, the attribute category vector is input into a batch normalization layer for normalization processing, and the normalized attribute category vector is input into the self-attention module.

[0089] Specifically, when or before inputting the attribute category vector output by the Embedding layer into the self-attention module, the attribute category vector can be input into a batch normalization layer (BN layer in Figure 3 , i.e., a BN layer in Batch Normalization), and a normalization method of Batch Normalization is adopted because the output of the Embedding layer of each sample is a tensor of [N, embedding_dim], where N is the number of attributes of the multi-category attribute. Optionally, the batch size is set in a range of 16-256, and the attribute category vector output by the Embedding layer can be normalized by using Batch Normalization. This method effectively improves the efficiency of model training and optimization; at the same time, it is a kind of implicit regularization method, which can improve the generalization ability of the model.

[0090] In an optional implementation, the normalized attribute category vector is input into the self-attention module, including: inputting the normalized attribute category vector into a Dropout layer for hidden processing; and inputting the hidden processed attribute category vector into the self-attention module.

[0091] Optionally, after normalizing the attribute category vector, it can also be input into a Dropout layer, and a Dropout method is adopted to improve the generalization ability of the model and reduce the risk of overfitting.

[0092] S5: inputting the attention vector into a classifier to obtain a probability of attribute category coding.

[0093] After obtaining the attention vector, the attention vector can be input into a classifier (i.e., a fully connected layer and a softmax activation layer in Figure 3 ), and the probability p of each attribute category of the model dictionary being selected is predicted by the softmax classifier.

[0094]

[0095]

[0096] wherein c represents the id of the attribute category, and "." represents dot multiplication; is the id of the attribute category with the highest probability predicted by the model.

[0097] S6: adjusting parameters of the Embedding module based on the probability of attribute category coding until convergence to obtain a word vector embedding model.

[0098] Specifically, after obtaining the probability of the attribute type coding, the actual coding of the attribute category (i.e., the coding obtained by model prediction) can be determined, and then the actual predicted coding is compared with the target coding (i.e., the coding determined by the pre-configured attribute category coding table) to calculate the deviation and determine whether the bias is within the allowable range; if it is within the allowable range, the training is ended and the parameters of the Embedding module are fixed; if it is not within the allowable range, the parameters of the Embedding module are adjusted until the bias is within the allowable range or a certain ending condition is reached, the training is ended and the parameters of the Embedding module are fixed, and the word vector embedding model is obtained; finally, the word vector table can be obtained according to the fixed parameters of the Embedding module.

[0099] Optionally, the cross-entropy loss function and the AdamW optimizer can be used, and the parameters of the Embedding module are adjusted based on the probability of the attribute category coding to train the model to convergence, thereby obtaining the word vector embedding model.

[0100] It should be understood that, although Figure 2 The steps in the flowchart of the method are displayed in sequence according to the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other orders. Moreover Figure 2 At least part of the steps in the method can include multiple sub-steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily sequential, but can be alternately executed with other steps or at least part of the sub-steps or stages of other steps.

[0101] The above embodiment of the present application discloses a method for representing an attribute category, and the above method of the present application can be implemented in various forms of devices, and therefore the present application also discloses an attribute category representation device corresponding to the above method, and the following specific embodiments are given for detailed description.

[0102] Please refer to Figure 4 The attribute category representation device disclosed in the embodiment of the present application mainly includes:

[0103] The to-be-identified data acquisition module 410 is configured to acquire attribute category data to be identified, wherein the attribute type data to be identified includes at least one attribute category.

[0104] The coding lookup module 420 is configured to look up the coding of each attribute category from a pre-configured attribute category coding table.

[0105] The vector searching module 430 is configured to search for a representation vector of each attribute category from a pre-established word vector table, wherein the word vector table is learned from a word vector embedding model, and the word vector embedding model is trained based on a self-attention mechanism method by using the attribute category coding sequence sample after mask processing.

[0106] In one embodiment, the apparatus further comprises:

[0107] The sequence sample obtaining module is configured to obtain the attribute category coding sequence sample.

[0108] The mask processing module is configured to randomly select one or more attribute category encodings from the attribute category coding sequence of the attribute category coding sequence sample for mask processing to obtain a mask attribute category coding sequence sample.

[0109] The attribute category vector output module is configured to input the mask attribute category coding sequence sample into the Embedding module to correspondingly output the attribute category vector.

[0110] The attention vector obtaining module is configured to input the attribute category vector into the self-attention module to obtain the attention vector.

[0111] The probability obtaining module is configured to input the attention vector into the classifier to obtain the probability of the attribute category coding.

[0112] The word vector embedding model obtaining module is configured to adjust parameters of the Embedding module based on the probability of the attribute category coding until convergence to obtain the word vector embedding model.

[0113] In one embodiment, the attention vector obtaining module is configured to input the attribute category vector into a batch normalization layer for normalization processing, and input the normalized attribute category vector into the self-attention module.

[0114] In one embodiment, the attention vector obtaining module is configured to input the normalized attribute category vector into a Dropout layer for hidden processing, and input the hidden attribute category vector into the self-attention module.

[0115] In one embodiment, the self-attention module is one or more; when the self-attention module is multiple, each self-attention module is connected in sequence; the self-attention module comprises a multi-head self-attention layer and an additional layer, the additional layer comprises one or more of a full connection layer, a batch normalization layer or a Dropout layer; the attention vector obtaining module is configured to input the attribute category vector into the multi-head self-attention layer to obtain an attention feature, and input the attention feature into the additional layer for spatial conversion, normalization processing or hidden processing to obtain the attention vector.

[0116] In an embodiment, the word vector embedding model obtaining module is configured to employ a cross-entropy loss function and an AdamW optimizer, and adjust parameters of the Embedding module based on probabilities of the attribute category encodings.

[0117] In an embodiment, the sequence sample obtaining module is configured to obtain attribute category data samples, wherein each attribute category data sample comprises a plurality of attribute category data, each attribute category data comprises a plurality of attributes, each attribute comprises a plurality of categories, encode each attribute category in each attribute category data, split each attribute category data after attribute category encoding to form a plurality of attribute category encoding sequences of equal length, and form attribute category encoding sequence samples according to the plurality of attribute category encoding sequences.

[0118] The specific limitations of the attribute category representation device can be referred to the limitations of the method described above, which will not be repeated here. Each module in the device can be realized by software, hardware and their combination in whole or in part. Each module described above can be embedded in or independent of the processor in the terminal device in hardware form, or can be stored in the memory in the terminal device in software form, so as to be called and executed by the processor to perform the operations corresponding to each module.

[0119] Please refer to Figure 5 , Figure 5 which shows a structural block diagram of a terminal device provided by an embodiment of the present application. The terminal device 50 can be a computer device. The terminal device 50 in the present application can include one or more of the following components: a processor 52, a memory 54, and one or more application programs, wherein the one or more application programs can be stored in the memory 54 and configured to be executed by the one or more processors 52, and the one or more application programs are configured to execute the method described in the above method embodiment applied to the terminal device, and can also be configured to execute the method described in the above method embodiment applied to the attribute category representation.

[0120] The processor 52 can include one or more processing cores. The processor 52 connects various parts within the terminal device 50 by various interfaces and lines, and performs various functions of the terminal device 50 and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 54, and calling data stored in the memory 54. Alternatively, the processor 52 can be implemented in at least one of a hardware form of a digital signal processing (DSP), a field-programmable gate array (FPGA), a programmable logic array (PLA). The processor 52 can be integrated with a combination of one or more of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. Among them, the CPU mainly processes operating systems, user interfaces, and application programs; the GPU is responsible for rendering and drawing display content; and the modem is used for processing wireless communication. It can be understood that the above-mentioned modem can also not be integrated into the processor 52, but can be realized by a separate communication chip.

[0121] The memory 54 can include a random access memory (RAM) and can also include a read-only memory (ROM). The memory 54 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 54 can include a program storage area and a data storage area, wherein the program storage area can store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing each method embodiment described below, etc. The data storage area can also store data created by the terminal device 50 in use, etc.

[0122] Those skilled in the art can understand that the above-mentioned structures shown in the figures are only block diagrams of part of the structures related to the schemes of the present application, and do not constitute a limitation on the terminal device to which the schemes of the present application are applied. The specific terminal device can include more or fewer components than those shown in the figures, or combine certain components, or have a different component arrangement. Figure 5 The above-mentioned structures shown in the figures are only block diagrams of part of the structures related to the schemes of the present application, and do not constitute a limitation on the terminal device to which the schemes of the present application are applied. The specific terminal device can include more or fewer components than those shown in the figures, or combine certain components, or have a different component arrangement.

[0123] In summary, the terminal device provided by the embodiments of the present application is used to implement the representation method of the corresponding attribute category in the above-mentioned method embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be described here.

[0124] Please refer to Figure 6 Fig. 6 shows a structural block diagram of a computer-readable storage medium according to an embodiment of the present application. The computer-readable storage medium 60 stores program codes, which can be invoked by a processor to execute the method described in the embodiments of the representation method of the attribute category, or can be invoked by the processor to execute the method described in the embodiments of the representation method of the attribute category.

[0125] The computer-readable storage medium 60 can be an electronic memory such as a flash memory, an EEPROM (Electrically Erasable Programmable Read-Only Memory), an EPROM, a hard disk, or a ROM. Alternatively, the computer-readable storage medium 60 comprises a non-transitory computer-readable storage medium. The computer-readable storage medium 60 has a storage space for the program codes 62 to execute any of the method steps described above. These program codes can be read from or written to one or more computer program products. The program codes 62 can be compressed in a suitable form, for example.

[0126] In the description of the present specification, the description referring to the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" etc. means that the specific feature, structure, material or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present application. The illustrative expressions of the above terms in the present specification do not necessarily refer to the same embodiment or example. Moreover, the specific feature, structure, material or characteristic described can be combined in any suitable manner in one or more embodiments or examples. Furthermore, the person skilled in the art can combine and combine the different embodiments or examples described in the present specification and the features of the different embodiments or examples, without contradiction.

[0127] The above description of disclosed embodiments enables a person skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.< / mask> < / mask>

Claims

1. A method for representing attribute categories, characterized in that, The method includes: Obtain attribute category data to be identified, wherein the attribute category data to be identified includes at least one attribute category; the attribute type data to be identified is data formed by any attribute category that needs to be represented by a vector. The attribute category data to be identified is different for different fields, objects or application scenarios. When the object to be identified is clothing, the attribute category data includes material, color and style. Find the code for each attribute category from the pre-configured attribute category code table; The representation vector of each attribute category is found from a pre-established word vector table; wherein the word vector table is learned from a word embedding model, and the word embedding model is trained on the Embedding module using masked attribute category encoding sequence samples and a self-attention mechanism. The training method for the word vector embedding model includes: Obtain attribute category encoded sequence samples; One or more attribute category codes are randomly selected from the attribute category code sequences of the attribute category code sequence sample and masked to obtain a masked attribute category code sequence sample; The mask attribute category encoding sequence sample is input into the Embedding module to output the corresponding attribute category vector; The attribute category vector is input into the self-attention module to obtain the attention vector; The attention vector is input into the classifier to obtain the probability of the attribute category encoding; The parameters of the Embedding module are adjusted based on the probability of the attribute category encoding until convergence, so as to obtain the word vector embedding model. The process of obtaining attribute category encoded sequence samples includes: Obtain attribute category data samples; wherein, the attribute category data samples include multiple attribute category data, each attribute category data includes multiple attributes, and each attribute includes multiple categories; Encode each attribute category in the attribute category data; The attribute category data after attribute category encoding is split to form multiple attribute category encoding sequences of equal length; The attribute category encoding sequence sample is formed based on multiple attribute category encoding sequences.

2. The method according to claim 1, characterized in that, The step of inputting the attribute category vector into the self-attention module includes: The attribute category vector is input into the batch standardization layer for normalization processing; The normalized attribute category vector is input into the self-attention module.

3. The method according to claim 2, characterized in that, The step of inputting the normalized attribute category vector into the self-attention module includes: The normalized attribute category vector is input into the Dropout layer for hiding. The hidden attribute category vector is input into the self-attention module.

4. The method according to any one of claims 2-3, characterized in that, The self-attention module is one or more; when there are multiple self-attention modules, each self-attention module is connected sequentially; the self-attention module includes a multi-head self-attention layer and an additional layer, the additional layer including one or more of a fully connected layer, a batch normalization layer or a Dropout layer; The step of inputting the attribute category vector into the self-attention module to obtain the attention vector includes: The attribute category vector is input into the multi-head self-attention layer to obtain attention features; The attention features are input into the additional layer for spatial transformation, normalization, or hiding to obtain the attention vector.

5. The method according to any one of claims 2-3, characterized in that, The adjustment of the parameters of the Embedding module based on the probability of the attribute category encoding includes: The parameters of the Embedding module are adjusted based on the probability of the attribute category encoding, using the cross-entropy loss function and the AdamW optimizer.

6. A device for representing attribute categories, characterized in that, The device includes: The data to be identified module is used to acquire attribute category data to be identified, wherein the attribute category data to be identified includes at least one attribute category; the attribute type data to be identified is data formed by any attribute category that needs to be represented by a vector. The attribute category data to be identified is different for different fields, objects or application scenarios. When the object to be identified is clothing, the attribute category data includes material, color and style. The encoding lookup module is used to look up the encoding of each attribute category from a pre-configured attribute category encoding table; The vector acquisition module is used to find the representation vector of each attribute category from a pre-established word vector table; wherein the word vector table is learned from a word vector embedding model, and the word vector embedding model is trained on the Embedding module using masked attribute category encoding sequence samples and a self-attention mechanism. The training method for the word vector embedding model includes: Obtain attribute category encoded sequence samples; One or more attribute category codes are randomly selected from the attribute category code sequences of the attribute category code sequence sample and masked to obtain a masked attribute category code sequence sample; The mask attribute category encoding sequence sample is input into the Embedding module to output the corresponding attribute category vector; The attribute category vector is input into the self-attention module to obtain the attention vector; The attention vector is input into the classifier to obtain the probability of the attribute category encoding; The parameters of the Embedding module are adjusted based on the probability of the attribute category encoding until convergence, so as to obtain the word vector embedding model. The process of obtaining attribute category encoded sequence samples includes: Obtain attribute category data samples; wherein, the attribute category data samples include multiple attribute category data, each attribute category data includes multiple attributes, and each attribute includes multiple categories; Encode each attribute category in the attribute category data; The attribute category data after attribute category encoding is split to form multiple attribute category encoding sequences of equal length; The attribute category encoding sequence sample is formed based on multiple attribute category encoding sequences.

7. A terminal device, characterized in that, include: Memory; One or more processors are coupled to the memory; One or more applications, wherein the one or more applications are stored in memory and configured to be executed by one or more processors, and the one or more applications are configured to perform the method as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium contains program code that can be invoked by a processor to execute the method as described in any one of claims 1-5.

Images