Attribute recognition model training method, training device and electronic equipment

Abstract

Embodiments of the present disclosure disclose a training method of an attribute recognition model, a training device and an electronic device. A specific implementation of the training method comprises: inputting an image of a sample item and attribute information set into an initial model, wherein the initial model comprises a static recognition sub-model and a dynamic recognition sub-model; obtaining an attribute prediction value to which the sample item belongs based on the image of the sample item and the static recognition sub-model; obtaining an attribute prediction value associated with the sample item based on the image of the sample item and the dynamic recognition sub-model; outputting predicted attribute information of the sample item according to the two attribute prediction values, and determining whether the initial model is trained based on the predicted attribute information. The implementation is related to artificial intelligence technology, which not only considers image dimension information, but also considers the co-occurrence relationship between attributes. In this way, the prediction accuracy of item attribute recognition can be improved, thereby helping to improve the access amount and order amount of item information.

Description

Technical Field

[0001] The embodiments disclosed herein relate to the field of artificial intelligence technology, and specifically to training methods, training devices, and electronic devices for attribute recognition models. Background Technology

[0002] On e-commerce platforms, item attributes are the most basic information about a product, such as the collar type, length, and pant length of clothing. This attribute information helps users quickly understand the product's characteristics. Accurate item attributes contribute to increasing the actual number of orders.

[0003] However, the inventors discovered that relevant attribute recognition methods typically analyze images of items uploaded by merchants to identify item attribute information. This type of method often only considers the information within the image itself, addressing the issue of item attribute recognition solely from the image dimension. This limited information analysis affects the accuracy of the recognition results.

[0004] The information disclosed in this background section is only intended to enhance the understanding of the background of the inventive concept, and therefore may contain information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] The summary portion of this disclosure is intended to provide a brief overview of the concepts, which will be described in detail in the detailed description portion. This summary portion is not intended to identify key or essential features of the claimed technical solutions, nor is it intended to limit the scope of the claimed technical solutions.

[0006] Some embodiments of this disclosure provide training methods, training devices, electronic devices, computer-readable media, and computer program products for attribute recognition models to solve one or more of the technical problems mentioned in the background section above.

[0007] In a first aspect, some embodiments of this disclosure provide a training method for an attribute recognition model, comprising: inputting an image of a sample item and an attribute information set into an initial model, wherein the initial model includes a static recognition sub-model and a dynamic recognition sub-model; obtaining a predicted attribute value of the sample item based on the image of the sample item and the static recognition sub-model, wherein the static recognition sub-model is used to determine the attribute information of the item based on the image information of the item image; obtaining a predicted attribute value associated with the sample item based on the image of the sample item and the dynamic recognition sub-model, wherein the dynamic recognition sub-model is used to determine the attribute information associated with the item based on the feature information of the item image; outputting predicted attribute information of the sample item based on the predicted attribute value of the sample item and the predicted attribute value associated with the sample item; and determining whether the initial model has been trained successfully based on the predicted attribute information.

[0008] In some embodiments, inputting the image and attribute information set of the sample item into the initial model includes: inputting the image of the sample item into the initial model and extracting the feature map of the sample item; inputting the attribute information set into the initial model and extracting the semantic features of each attribute information in the attribute information set to obtain an attribute feature vector set.

[0009] In some embodiments, based on the image of the sample item and a dynamic recognition sub-model, the predicted attribute values ​​associated with the sample item are obtained, including: generating a set of feature vectors for the sample item according to the feature map of the sample item; obtaining a dynamic feature vector based on the set of feature vectors, the set of attribute feature vectors, and the dynamic convolutional network in the dynamic recognition sub-model, wherein the dynamic recognition sub-model includes a dynamic convolutional network and a second classifier; inputting the dynamic feature vector into the second classifier and outputting the predicted attribute values ​​associated with the sample item.

[0010] In some embodiments, generating a set of feature vectors for a sample item based on its feature map includes: processing the feature map through a convolutional layer to obtain a new feature map, wherein the number of feature nodes in the new feature map is the same as the number of attribute feature nodes; processing the feature map through a linear classification layer to obtain a category-related attention matrix; and multiplying the new feature map and the attention matrix by matrix multiplication to obtain a set of node feature vectors for the sample item.

[0011] In some embodiments, the dynamic convolutional network is obtained by: concatenating the set of node feature vectors with the set of attribute feature vectors to obtain a static input node feature set; calculating the co-occurrence relationship between attribute feature nodes to obtain a first adjacency matrix, and constructing a static convolutional network; inputting the static input node feature set into the static convolutional network to obtain a new set of node feature vectors; processing the new set of node feature vectors through a pooling layer to obtain a global feature vector; concatenating the global feature vector with the new set of node feature vectors to obtain a concatenated feature vector set; and performing a convolution transformation on the concatenated feature vector set to obtain a second adjacency matrix, thus constructing a dynamic convolutional network.

[0012] In some embodiments, dynamic feature vectors are obtained based on a set of feature vectors, a set of attribute feature vectors, and a dynamic convolutional network in a dynamic recognition sub-model. This includes inputting a new set of node feature vectors into the dynamic convolutional network to obtain dynamic feature vectors.

[0013] In some embodiments, based on the image of the sample item and a static recognition sub-model, the predicted attribute value of the sample item is obtained, including: inputting the feature map of the sample item into the processing layer of the static recognition sub-model to obtain the static feature vector of the sample item, wherein the static recognition sub-model includes a processing layer and a first classifier, and the processing layer includes a linear layer; inputting the static feature vector into the first classifier and outputting the predicted attribute value of the sample item.

[0014] In some embodiments, the method outputs predicted attribute information of a sample item based on the predicted attribute value of the sample item and the predicted attribute value associated with the sample item, and determines whether the initial model has been trained based on the predicted attribute information, including: taking a weighted average of the predicted attribute value of the sample item and the predicted attribute value associated with the sample item, and using the calculation result as the predicted attribute information of the sample item; determining the loss value between the predicted attribute information of the sample item and the label attribute information; and determining that the initial model has been trained successfully in response to the determination that the loss value is less than a preset value, and determining the trained initial model as the attribute information recognition model.

[0015] In some embodiments, the method further includes: receiving a displayed item image and pre-configured attribute information to be identified, inputting the attribute information recognition model, and outputting the predicted attribute information of the item indicated by the displayed item image; and displaying the displayed item image and the corresponding predicted attribute information in the application page.

[0016] Secondly, some embodiments of this disclosure provide a training apparatus for an attribute recognition model, comprising: an input unit configured to input an image of a sample item and an attribute information set into an initial model, wherein the initial model includes a static recognition sub-model and a dynamic recognition sub-model; a first attribute recognition unit configured to obtain a predicted attribute value of the sample item based on the image of the sample item and the static recognition sub-model, wherein the static recognition sub-model is used to determine the attribute information of the item based on the image information of the item image; a second attribute recognition unit configured to obtain a predicted attribute value associated with the sample item based on the image of the sample item and the dynamic recognition sub-model, wherein the dynamic recognition sub-model is used to determine the attribute information associated with the item based on the feature information of the item image; and an output unit configured to output predicted attribute information of the sample item based on the predicted attribute value of the sample item and the predicted attribute value associated with the sample item, and to determine whether the initial model has been trained successfully based on the predicted attribute information.

[0017] In some embodiments, the input unit is further configured to input an image of the sample item into the initial model to extract a feature map of the sample item; input an attribute information set into the initial model to extract the semantic features of each attribute information in the attribute information set to obtain an attribute feature vector set.

[0018] In some embodiments, the second attribute recognition unit includes: a generation subunit configured to generate a set of feature vectors for the sample item based on the feature map of the sample item; a dynamic feature generation subunit configured to obtain a dynamic feature vector based on the set of feature vectors, the set of attribute feature vectors, and a dynamic convolutional network in the dynamic recognition submodel, wherein the dynamic recognition submodel includes a dynamic convolutional network and a second classifier; and a second attribute output subunit configured to input the dynamic feature vectors into the second classifier and output a predicted attribute value associated with the sample item.

[0019] In some embodiments, the generating subunit is further configured to process the feature map through a convolutional layer to obtain a new feature map, wherein the number of feature nodes in the new feature map is the same as the number of attribute feature nodes; process the feature map through a linear classification layer to obtain a category-related attention matrix; and multiply the new feature map and the attention matrix by matrix multiplication to obtain a set of node feature vectors of the sample item.

[0020] In some embodiments, the second attribute recognition unit further includes a dynamic convolutional network generation subunit, configured to concatenate the node feature vector set and the attribute feature vector set to obtain a static input node feature set; statistically analyze the co-occurrence relationships between attribute feature nodes to obtain a first adjacency matrix, and construct a static convolutional network; input the static input node feature set into the static convolutional network to obtain a new node feature vector set; process the new node feature vector set through a pooling layer to obtain a global feature vector; concatenate the global feature vector with the new node feature vector set to obtain a concatenated feature vector set; and perform a convolution transformation on the concatenated feature vector set to obtain a second adjacency matrix, and construct a dynamic convolutional network.

[0021] In some embodiments, the dynamic feature generation subunit is further configured to input a new set of node feature vectors into a dynamic convolutional network to obtain dynamic feature vectors.

[0022] In some embodiments, the first attribute recognition unit is further configured to input the feature map of the sample item into the processing layer in the static recognition sub-model to obtain the static feature vector of the sample item, wherein the static recognition sub-model includes a processing layer and a first classifier, and the processing layer includes a linear layer; the static feature vector is input into the first classifier to output the attribute prediction value to which the sample item belongs.

[0023] In some embodiments, the output unit is further configured to perform a weighted average of the predicted attribute values ​​of the sample item and the predicted attribute values ​​associated with the sample item, and use the calculation result as the predicted attribute information of the sample item; determine the loss value between the predicted attribute information and the label attribute information of the sample item; and in response to determining that the loss value is less than a preset value, determine that the initial model training is complete, and determine the trained initial model as the attribute information recognition model.

[0024] In some embodiments, the training device further includes a display unit configured to receive an image of a displayed item and pre-configured attribute information to be identified, input the attribute information recognition model, output the predicted attribute information of the item indicated by the image of the displayed item, and display the image of the displayed item and the corresponding predicted attribute information in an application page.

[0025] Thirdly, some embodiments of this disclosure provide an electronic device, including: one or more processors; and a storage device having one or more programs stored thereon, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the training method described in any of the implementations of the first aspect above.

[0026] Fourthly, some embodiments of this disclosure provide a computer-readable medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the training method described in any of the implementations of the first aspect above.

[0027] Fifthly, some embodiments of this disclosure provide a computer program product, including a computer program that, when executed by a processor, implements the training method described in any of the implementations of the first aspect above.

[0028] The various embodiments disclosed above have the following beneficial effects: the training methods for attribute recognition models in some embodiments of this disclosure can improve the accuracy of attribute recognition, thereby increasing the number of visits and orders for item information. Specifically, related attribute recognition methods typically analyze item images uploaded by merchants to identify item attribute information. Such attribute recognition methods often only consider the information in the image itself, i.e., they only address the identification of item attribute information from the image dimension. This singular information analysis affects the accuracy of the recognition results.

[0029] Based on this, the training method of the attribute recognition model in some embodiments of this disclosure can analyze the image information of an item through a static recognition sub-model to obtain the predicted attribute value of the item. Additionally, through a dynamic recognition sub-model, the co-occurrence relationship between attribute information can be analyzed to obtain the predicted attribute value associated with the item. Based on the two predicted attribute values, the final predicted attribute information of the item is obtained. This approach considers both image dimensional information and the relationship between attributes, improving the accuracy of attribute recognition results. This, in turn, increases the access volume and order volume of item information in applications. Attached Figure Description

[0030] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and elements are not necessarily drawn to scale.

[0031] Figure 1 This is a flowchart of some embodiments of the training method for the attribute recognition model disclosed herein;

[0032] Figure 2 These are flowcharts of some other embodiments of the training method for the attribute recognition model disclosed herein;

[0033] Figure 3 These are flowcharts of some embodiments for generating dynamic convolutional networks;

[0034] Figure 4 These are schematic diagrams illustrating the structure of some embodiments of the training apparatus for the attribute recognition model disclosed herein;

[0035] Figure 5 This is a schematic diagram of the structure of a processor suitable for implementing some embodiments of the present disclosure. Detailed Implementation

[0036] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0037] It should also be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings. Unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other.

[0038] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0039] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0040] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0041] This disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0042] Figure 1 A flow 100 illustrating some embodiments of a training method for an attribute recognition model according to the present disclosure is shown. The method includes the following steps:

[0043] Step 101: Input the image and attribute information set of the sample items into the initial model.

[0044] In some embodiments, the execution entity (e.g., a server) of the attribute recognition model training method can acquire images and attribute information sets of sample items via wired or wireless connections. The images can be any images representing information about the sample items, such as color or black-and-white images. The image format is not limited here, as long as it can be recognized by the execution entity. The attribute information set can contain various information representing item attributes, such as category information (long / short sleeves, tops, spring / autumn, summer, couples, etc.), size information, etc.

[0045] In some embodiments, the executing entity can input the image and attribute information set of the sample item into the initial model. Here, to facilitate subsequent analysis, the executing entity can input the image of the sample item into the initial model to extract the feature map of the sample item. Alternatively, the attribute information set can be input into the initial model to extract the semantic features of each attribute in the attribute information set, resulting in a set of attribute feature vectors.

[0046] As an example, such as Figure 2 As shown, the initial model can include commonly used backbone networks to extract features from images, resulting in feature maps. For example, the backbone network can be at least one of the following: Residual Network (ResNet), Convolutional Neural Network (Visual Geometry Group, VGG), Lightweight Network (MobileNet), etc. Here, the feature map from the last network layer can be extracted. Furthermore, the initial model can also include a semantic feature extraction network to extract semantic features of each attribute in the attribute information set. As an example, the semantic feature extraction network can include at least one of the following methods: word embedding method (GloVe, Global Vector), text feature extraction method (one-hot encoding), etc.

[0047] In some embodiments, the initial model may include a static recognition sub-model and a dynamic recognition sub-model. The static recognition sub-model can be used to determine the attribute information of an item based on image information from an item image. The dynamic recognition sub-model can be used to determine the attribute information associated with an item based on feature information from an item image. In other words, the static recognition sub-model can determine the attribute label of an item based on information from the image itself, i.e., from the image dimension. The dynamic recognition sub-model can determine the attribute information associated with an item based on feature information contained in the image.

[0048] Step 102: Based on the image and static recognition sub-model of the sample item, obtain the predicted value of the attribute to which the sample item belongs.

[0049] In some embodiments, the executing entity can obtain the predicted attribute value of the sample item based on the image of the sample item and a static recognition sub-model. As an example, the static recognition sub-model may include a processing layer and a first classifier. The processing layer may be a linear layer. Figure 2 As shown on the left, the executing entity can input the feature map of the sample item into the processing layer of the static recognition sub-model to obtain the static feature vector F of the sample item. Then, the static feature vector can be input into the first classifier C1 to output the predicted attribute value of the sample item. It can be understood that the attribute of the sample item is usually an attribute from the aforementioned attribute information set. Here, the previously obtained set of attribute feature vectors can be pre-input into the first classifier, or it can be input together with the static feature vector.

[0050] In some embodiments, a pure labeling approach can also be used to identify the attribute information of the sample item. In this case, the static recognition sub-model can output at least one attribute information of the sample item based on the input feature map.

[0051] It's important to note that the static recognition sub-model here typically identifies and determines item attribute information from the image dimension. That is, based on the information contained in the image, multiple attribute tags belonging to the item can be identified. For example, for an image of a short-sleeved shirt, the static recognition sub-model can identify its attributes as short-sleeved and T-shirt. However, this recognition method usually only addresses the problem of product attribute information identification from the perspective of multi-tag recognition. It only considers image-dimensional information and does not consider the co-occurrence relationships between attributes. This often lacks information dimension, affecting the accuracy of recognition.

[0052] It's understandable that item attributes are not only related to the item's image, but also to the relationships between attributes themselves. For example, sleeve length (short sleeve) is related to the applicable season; clothing style and collar type are related to the applicable scenario. Therefore, for item attribute recognition, it's necessary to consider not only the item's image information but also the co-occurrence relationships between attributes. For instance, a graph convolutional network branch can be introduced into the original recognition model to learn the co-occurrence relationships between attributes, thereby recognizing product attributes.

[0053] Alternatively, the processing layer in the static recognition sub-model here can also adopt a static convolutional network structure, such as... Figure 3 The static convolutional network described in [the text] should be noted. While static convolutional networks consider the relationships between attributes, these relationships are typically static and based on pre-observed statistical results, independent of the input graph. In other words, these static relationships are often fixed once training is complete, regardless of the input graph. For example, an image of a long-sleeved shirt is statistically likely related to spring and autumn. That is, long sleeves are associated with spring and autumn. However, in practical applications, the relationships between attributes should be dynamic, and different input graphs should correspond to different attribute relationships. As an example, sun-protective clothing or very thin long sleeves (such as long-sleeved shirts) are typically suitable for summer.

[0054] Step 103: Based on the image of the sample item and the dynamic recognition sub-model, obtain the attribute prediction value associated with the sample item.

[0055] In some embodiments, based on the image of the sample item and a dynamic recognition sub-model, the executing agent can obtain predicted attribute values ​​associated with the sample item. Here, the dynamic recognition sub-model can determine the dynamic attribute information associated with the item based on different item images. That is, the attribute information associated with different input images is often different. In other words, the relationship between attributes can change depending on the different input images.

[0056] In some embodiments, the dynamic recognition sub-model may include a dynamic convolutional network and a second classifier. Here, using an initial model, the executing agent first generates a set of feature vectors for the sample item based on its feature map. Then, based on the feature vector set, the attribute feature vector set, and the dynamic convolutional network in the dynamic recognition sub-model, a dynamic feature vector can be obtained. Afterward, the dynamic feature vector can be input into the second classifier to output the predicted attribute values ​​associated with the sample item. See also... Figure 2 The relevant descriptions in the embodiments will not be repeated here. It is understood that the attributes associated with the sample items here are also the attributes in the aforementioned attribute information set.

[0057] Optionally, the dynamic recognition sub-model can also be trained based on an attribute information relationship table (graph). Here, the attribute information relationship table (graph) can be used to characterize the co-occurrence relationships between various attribute information. In some embodiments, the attribute information relationship table may also include co-occurrence conditions, which represent the scenarios, times, and other conditions under which the co-occurrence relationship occurs between attribute information. For example, the co-occurrence relationship between long sleeves and summer often occurs within a certain time period (such as June to September). In this way, the time set by the merchant (such as the applicable season of the item) can be used as auxiliary information to determine the special co-occurrence relationship between two attribute information.

[0058] Step 104: Based on the predicted attribute values ​​of the sample item and the predicted attribute values ​​associated with the sample item, output the predicted attribute information of the sample item, and determine whether the initial model has been trained successfully based on the predicted attribute information.

[0059] In some embodiments, based on the predicted attribute value of the sample item obtained in step 102 and the predicted attribute value associated with the sample item obtained in step 103, the executing entity can output the predicted attribute information of the sample item. Furthermore, it can determine whether the initial model has been successfully trained based on the predicted attribute information. As an example, the executing entity can use the larger of the two predicted values ​​as the predicted attribute information.

[0060] In some embodiments, to further improve the accuracy of the prediction results, the executing entity can perform a weighted average of the predicted values ​​of the attribute to which the sample item belongs and the predicted values ​​of the attributes associated with the sample item. The calculation result can then be used as the predicted attribute information of the sample item. Additionally, as... Figure 2 As shown, the executing entity can use a loss function to determine the loss value between the predicted attribute information and the label attribute information of the sample item. Based on the comparison between this loss value and a preset value, it can determine whether the initial model has been successfully trained. The label attribute information here can be obtained by collecting and accumulating a large amount of item information in the application. It is understood that in this embodiment, the training sample data can come from information filled in by merchants online, so no additional manual annotation is required.

[0061] Here, if the loss value is determined to be not less than the preset value, it indicates that training is not yet complete. At this point, gradient backpropagation can be used to adjust and optimize the relevant parameters in the initial model. The adjusted initial model can then be used as the initial model to continue the above training process. If the loss value is determined to be less than the preset value, it indicates that the initial model training is complete. At this point, the trained initial model can be designated as the attribute information recognition model.

[0062] Based on the above description, the training method of the attribute recognition model in some embodiments of this disclosure can analyze the image information of an item using a static recognition sub-model to obtain the predicted attribute value of the sample item. Additionally, a dynamic recognition sub-model can analyze the co-occurrence relationship between attribute information to obtain the predicted attribute value associated with the sample item. Based on the predicted attribute value of the sample item and the predicted attribute value associated with the sample item, the final predicted attribute information of the item is obtained. This approach considers both image dimensional information and the relationship between attributes, thereby improving the accuracy of the attribute recognition results.

[0063] Understandably, the main image of a product is often uploaded by the merchant to describe the item. It's usually the most direct description and can be considered a highly reliable source of information. Item attributes are often reflected in the main image. Therefore, accurately identifying item attributes can increase the number of visits and orders for item information within the application.

[0064] In some application scenarios, merchants can upload product images to the application. Based on the pre-configured attribute information to be identified by the application platform, the attribute information recognition model obtained in this embodiment can directly output the identified product attribute information. This reduces or eliminates the need for merchants to input attribute information, thus improving the efficiency of uploading and displaying product information. Simultaneously, it improves the accuracy of attribute information recognition, which helps increase the number of visits to product information and consequently boosts the application's order volume.

[0065] As an example, the executing entity can receive an image of a displayed item and pre-configured attribute information to be identified, input them into an attribute information recognition model, and then output the predicted attribute information of the item indicated by the displayed item image. The executing entity can then display the item image and the corresponding predicted attribute information on the application page.

[0066] The following is for reference. Figure 2 The diagram illustrates flow 200 of another embodiment of a training method for an attribute recognition model according to the present disclosure. The method includes the following steps:

[0067] like Figure 2 As shown, firstly, the feature map X is processed through a convolutional layer to obtain a new feature map. The number of feature nodes in the new feature map is the same as the number of attribute feature nodes. Here, the number of attribute feature nodes is typically the number of attribute feature vectors in the aforementioned set of attribute feature vectors.

[0068] Next, the feature map is processed through a linear classification layer to obtain an attention matrix related to the category.

[0069] Then, the new feature map and the attention matrix are multiplied together to obtain the set of node feature vectors V of the sample items.

[0070] Continuing, based on the node feature vector set V, the attribute feature vector set, and the dynamic convolutional network, the dynamic feature vector of the sample item can be obtained.

[0071] Finally, after the dynamic feature vector passes through the second classifier C2, the predicted attribute values ​​associated with the sample items can be obtained.

[0072] It should be noted that the dynamic convolutional network here can be achieved through... Figure 3 The process shown in step 300 is obtained. Specifically,

[0073] First, the attribute feature vector set L is concatenated with the node feature vector set V to obtain the static input node feature set V'.

[0074] Next, by analyzing the co-occurrence relationships between statistical attribute feature nodes, the first adjacency matrix As can be obtained. Based on the first adjacency matrix As, a static convolutional network can be constructed.

[0075] Continuing, by inputting the static input node feature set V' into the static convolutional network, a new set of node feature vectors H can be obtained.

[0076] Then, the new set of node feature vectors H is processed through a pooling layer to obtain the global feature vector.

[0077] Furthermore, the global feature vector is concatenated with the new set of node feature vectors H to obtain the concatenated feature vector set.

[0078] Next, the concatenated feature vector set is subjected to a convolutional transformation to obtain the second adjacency matrix Ad. Based on the second adjacency matrix Ad, a dynamic convolutional network can be constructed. It is understandable that, unlike a static convolutional network, the adjacency matrix and node feature vectors here are often related to the input graph. Thus, the predicted attribute values ​​associated with items output by the dynamic recognition sub-model are usually related to the input graph. In other words, different input graphs often result in different predicted attribute values ​​associated with items.

[0079] Finally, by inputting the new set of node feature vectors H into the dynamic convolutional network, dynamic feature vectors can be obtained.

[0080] The training method for the attribute recognition model disclosed in this embodiment further enriches and improves the generation process of the dynamic recognition sub-model. During the generation process, not only the relationships between feature nodes in the feature graph but also the co-occurrence relationships between attribute nodes are considered. Both the node parameters and edge parameters of the graph are analyzed. This allows the dynamic recognition sub-model to identify the associated attribute information of an item based on the input item image. Furthermore, the relationships between the identified attribute information dynamically change with different item images. This improves the overall accuracy of the model's recognition results.

[0081] Further reference Figure 4 As a response to the above Figures 1 to 3 The present disclosure provides some embodiments of a training apparatus for an attribute recognition model, illustrating the implementation of the training method shown. These training apparatus embodiments are related to... Figures 1 to 3 The training methods illustrated correspond to specific examples. This training device can be applied to various electronic devices.

[0082] like Figure 4 As shown, the training apparatus 400 for the attribute recognition model in some embodiments may include: an input unit 401 configured to input the image and attribute information set of the sample item into an initial model, wherein the initial model includes a static recognition sub-model and a dynamic recognition sub-model; a first attribute recognition unit 402 configured to obtain the attribute prediction value of the sample item based on the image of the sample item and the static recognition sub-model, wherein the static recognition sub-model is used to determine the attribute information of the item based on the image information of the item image; a second attribute recognition unit 403 configured to obtain the attribute prediction value associated with the sample item based on the image of the sample item and the dynamic recognition sub-model, wherein the dynamic recognition sub-model is used to determine the attribute information associated with the item based on the feature information of the item image; and an output unit 404 configured to output the predicted attribute information of the sample item based on the attribute prediction value of the sample item and the attribute prediction value associated with the sample item, and to determine whether the initial model has been trained successfully based on the predicted attribute information.

[0083] In some embodiments, the input unit 401 may be further configured to input an image of a sample item into an initial model to extract a feature map of the sample item; input an attribute information set into the initial model to extract the semantic features of each attribute information in the attribute information set to obtain an attribute feature vector set.

[0084] In some embodiments, the second attribute recognition unit 403 may include: a generation subunit (not shown in the figure) configured to generate a set of feature vectors for the sample item based on the feature map of the sample item; a dynamic feature generation subunit (not shown in the figure) configured to obtain a dynamic feature vector based on the set of feature vectors, the set of attribute feature vectors, and a dynamic convolutional network in the dynamic recognition submodel, wherein the dynamic recognition submodel includes a dynamic convolutional network and a second classifier; and a second attribute output subunit (not shown in the figure) configured to input the dynamic feature vectors into the second classifier and output the attribute prediction value associated with the sample item.

[0085] In some embodiments, the generating subunit may be further configured to process the feature map through a convolutional layer to obtain a new feature map, wherein the number of feature nodes in the new feature map is the same as the number of attribute feature nodes; process the feature map through a linear classification layer to obtain a category-related attention matrix; and multiply the new feature map and the attention matrix by matrix multiplication to obtain a set of node feature vectors of the sample item.

[0086] In some embodiments, the second attribute recognition unit 403 may further include a dynamic convolutional network generation subunit (not shown in the figure), configured to concatenate the set of node feature vectors with the set of attribute feature vectors to obtain a static input node feature set; statistically analyze the co-occurrence relationships between attribute feature nodes to obtain a first adjacency matrix, and construct a static convolutional network; input the static input node feature set into the static convolutional network to obtain a new set of node feature vectors; process the new set of node feature vectors through a pooling layer to obtain a global feature vector; concatenate the global feature vector with the new set of node feature vectors to obtain a concatenated feature vector set; and perform a convolution transformation on the concatenated feature vector set to obtain a second adjacency matrix, and construct a dynamic convolutional network.

[0087] In some embodiments, the dynamic feature generation subunit can be further configured to input a new set of node feature vectors into a dynamic convolutional network to obtain dynamic feature vectors.

[0088] In some embodiments, the first attribute recognition unit 402 may be further configured to input the feature map of the sample item into the processing layer in the static recognition sub-model to obtain the static feature vector of the sample item, wherein the static recognition sub-model includes a processing layer and a first classifier, and the processing layer includes a linear layer; the static feature vector is input into the first classifier, and the predicted value of the attribute to which the sample item belongs is output.

[0089] In some embodiments, the output unit 404 may be further configured to perform a weighted average of the predicted attribute values ​​of the sample item and the predicted attribute values ​​associated with the sample item, and use the calculation result as the predicted attribute information of the sample item; determine the loss value between the predicted attribute information and the label attribute information of the sample item; and in response to determining that the loss value is less than a preset value, determine that the initial model training is complete, and determine the trained initial model as the attribute information recognition model.

[0090] In some embodiments, the training device 400 may further include a display unit (not shown in the figure), configured to receive a display item image and pre-configured attribute information to be identified, input the attribute information recognition model, output the predicted attribute information of the item indicated by the display item image, and display the display item image and the corresponding predicted attribute information in the application page.

[0091] It is understandable that the units described in the training device 400 are related to the reference. Figures 1 to 3 The steps in the described training method correspond to each other. Therefore, the operations, features, and beneficial effects described above for the method also apply to the training device 400 and the units contained therein, and will not be repeated here.

[0092] The following is for reference. Figure 5 It shows a schematic diagram of the structure of an electronic device 500 suitable for implementing some embodiments of the present disclosure. Figure 5 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of this disclosure.

[0093] like Figure 5 As shown, the electronic device 500 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage device 508 into a random access memory (RAM) 503. The RAM 503 also stores various programs and data required for the operation of the electronic device 500. The processing unit 501, ROM 502, and RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0094] Typically, the following devices can be connected to I / O interface 505: input devices 506 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 507 including, for example, displays, speakers, vibrators, etc.; storage devices 508 including, for example, magnetic tapes, hard disks, etc.; and communication devices 509. Communication device 509 allows electronic device 500 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 5 An electronic device 500 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively. Figure 5 Each box shown can represent a device or multiple devices as needed.

[0095] In particular, according to some embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, some embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication device 509, or installed from storage device 508, or installed from ROM 502. When the computer program is executed by processing device 501, it performs the functions defined in the methods of some embodiments of this disclosure.

[0096] It should be noted that, in some embodiments of this disclosure, the computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In some embodiments of this disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In some embodiments of this disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0097] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0098] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: input a set of images and attribute information of sample items into an initial model, wherein the initial model includes a static recognition sub-model and a dynamic recognition sub-model; obtain predicted attribute values ​​for the sample items based on the images of the sample items and the static recognition sub-model, wherein the static recognition sub-model is used to determine the attribute information to which the item belongs based on the image information of the item image; obtain predicted attribute values ​​associated with the sample items based on the images of the sample items and the dynamic recognition sub-model, wherein the dynamic recognition sub-model is used to determine the attribute information associated with the item based on the feature information of the item image; output predicted attribute information of the sample items based on the predicted attribute values ​​to which the sample items belong and the predicted attribute values ​​associated with the sample items; and determine whether the initial model has been trained successfully based on the predicted attribute information.

[0099] Furthermore, computer program code for performing operations of some embodiments of this disclosure can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages—such as Java, Smalltalk, and C++—and conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0100] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0101] The units described in some embodiments of this disclosure can be implemented in software or hardware. The described units can also be housed in a processor; for example, a processor may be described as including an input unit, a first attribute recognition unit, a second attribute recognition unit, and an output unit. The names of these units do not necessarily limit the specific unit; for example, an input unit may also be described as "a unit that inputs an image and attribute information set of a sample item into an initial model."

[0102] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0103] Some embodiments of this disclosure also provide a computer program product, including a computer program that, when executed by a processor, implements the training method for any of the attribute recognition models described above.

[0104] The above description is merely a selection of preferred embodiments of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in the embodiments of this disclosure.

Claims

1. A training method for an attribute recognition model, comprising: The image and attribute information set of the sample item are input into the initial model to obtain the feature map and attribute feature vector set of the sample item. The initial model includes a static recognition sub-model and a dynamic recognition sub-model. Based on the image of the sample item and the static recognition sub-model, the predicted attribute value of the sample item is obtained, wherein the static recognition sub-model is used to determine the attribute information of the item based on the image information of the item image; Based on the image of the sample item and the dynamic recognition sub-model, predicted attribute values ​​associated with the sample item are obtained. The dynamic recognition sub-model is used to determine the attribute information associated with the item based on the feature information of the item image. The dynamic recognition sub-model includes a dynamic convolutional network and a second classifier. The dynamic convolutional network is obtained in the following way: The set of node feature vectors is concatenated with the set of attribute feature vectors to obtain a static input node feature set, which is generated based on the feature map of the sample item. The co-occurrence relationship between statistical attribute feature nodes is used to obtain the first adjacency matrix, and a static convolutional network is constructed. The static input node feature set is input into the static convolutional network to obtain a new set of node feature vectors; The new set of node feature vectors is processed by a pooling layer to obtain a global feature vector; The global feature vector is concatenated with the new set of node feature vectors to obtain a concatenated feature vector set; The concatenated feature vector set is subjected to a convolutional transformation to obtain a second adjacency matrix, and a dynamic convolutional network is constructed. Based on the predicted attribute values ​​of the sample item and the predicted attribute values ​​associated with the sample item, the predicted attribute information of the sample item is output, and based on the predicted attribute information, it is determined whether the initial model has been trained successfully.

2. The method according to claim 1, wherein, The step of inputting the image and attribute information set of the sample items into the initial model includes: The image of the sample item is input into the initial model to extract the feature map of the sample item; The attribute information set is input into the initial model, and the semantic features of each attribute information in the attribute information set are extracted to obtain the attribute feature vector set.

3. The method according to claim 2, wherein, The step of obtaining attribute prediction values ​​associated with the sample item based on the image of the sample item and the dynamic recognition sub-model includes: Based on the feature map of the sample item, generate a set of node feature vectors for the sample item; Based on the set of feature vectors, the set of attribute feature vectors, and the dynamic convolutional network in the dynamic recognition sub-model, a dynamic feature vector is obtained, wherein the dynamic recognition sub-model includes a dynamic convolutional network and a second classifier; The dynamic feature vector is input into the second classifier, and the predicted attribute values ​​associated with the sample item are output.

4. The method according to claim 3, wherein, The step of generating a set of node feature vectors for the sample item based on its feature map includes: The feature map is processed through a convolutional layer to obtain a new feature map, wherein the number of feature nodes in the new feature map is the same as the number of attribute feature nodes; The feature map is processed through a linear classification layer to obtain a category-related attention matrix; The new feature map and the attention matrix are multiplied together to obtain the set of node feature vectors of the sample item.

5. The method according to claim 1, wherein, The process of obtaining dynamic feature vectors based on the feature vector set, the attribute feature vector set, and the dynamic convolutional network in the dynamic recognition sub-model includes: The new set of node feature vectors is input into the dynamic convolutional network to obtain dynamic feature vectors.

6. The method according to claim 2, wherein, The step of obtaining the predicted attribute value of the sample item based on the image of the sample item and the static recognition sub-model includes: The feature map of the sample item is input into the processing layer of the static recognition sub-model to obtain the static feature vector of the sample item. The static recognition sub-model includes a processing layer and a first classifier. The processing layer includes a linear layer. The static feature vector is input into the first classifier, and the predicted attribute value of the sample item is output.

7. The method according to any one of claims 1-6, wherein, The step of outputting predicted attribute information of the sample item based on the predicted attribute value of the sample item and the predicted attribute value associated with the sample item, and determining whether the initial model has been trained successfully based on the predicted attribute information, includes: The predicted values ​​of the attributes to which the sample item belongs and the predicted values ​​of the attributes associated with the sample item are weighted and averaged, and the calculation result is used as the predicted attribute information of the sample item. Determine the loss value between the predicted attribute information and the label attribute information of the sample item; In response to determining that the loss value is less than a preset value, the initial model training is determined to be complete, and the trained initial model is determined as the attribute information recognition model.

8. The method according to claim 7, wherein, The method further includes: The system receives an image of a displayed item and pre-configured attribute information to be identified, inputs the attribute information recognition model, and outputs the predicted attribute information of the item indicated by the displayed item image. The application page displays the image of the displayed item and its corresponding predicted attribute information.

9. A training device for an attribute recognition model, comprising: The input unit is configured to input the image and attribute information set of the sample item into the initial model to obtain the feature map and attribute feature vector set of the sample item, wherein the initial model includes a static recognition sub-model and a dynamic recognition sub-model; The first attribute recognition unit is configured to obtain the attribute prediction value of the sample item based on the image of the sample item and the static recognition sub-model, wherein the static recognition sub-model is used to determine the attribute information of the item based on the image information of the item image; The second attribute recognition unit is configured to obtain attribute prediction values ​​associated with the sample item based on the image of the sample item and the dynamic recognition sub-model. The dynamic recognition sub-model is used to determine attribute information associated with the item based on the feature information of the item image. The dynamic recognition sub-model includes a dynamic convolutional network and a second classifier. The dynamic convolutional network is obtained by: concatenating the node feature vector set with the attribute feature vector set to obtain a static input node feature set, which is generated based on the feature map of the sample item; statistically analyzing the co-occurrence relationships between attribute feature nodes to obtain a first adjacency matrix, and constructing a static convolutional network; inputting the static input node feature set into the static convolutional network to obtain a new node feature vector set; processing the new node feature vector set through a pooling layer to obtain a global feature vector; concatenating the global feature vector with the new node feature vector set to obtain a concatenated feature vector set; and performing a convolution transformation on the concatenated feature vector set to obtain a second adjacency matrix, thus constructing the dynamic convolutional network. The output unit is configured to output predicted attribute information of the sample item based on the predicted attribute value of the sample item and the predicted attribute value associated with the sample item, and to determine whether the initial model has been trained successfully based on the predicted attribute information.

10. An electronic device, comprising: One or more processors; Storage device, on which one or more programs are stored, When the one or more programs are executed by the one or more processors, the one or more processors implement the training method as described in any one of claims 1-8.

11. A computer-readable medium having a computer program stored thereon, wherein, When the computer program is executed by the processor, it implements the training method as described in any one of claims 1-8.

12. A computer program product comprising a computer program that, when executed by a processor, implements the training method according to any one of claims 1-8.

Images