Commodity attribute extraction method and device, equipment, storage medium and program product

Abstract

The invention relates to a commodity attribute extraction method and device, equipment, a storage medium and a program product. The method comprises the following steps: acquiring multi-modal commodity information; the multi-modal commodity information comprises text information and image information; the multi-modal commodity information is processed, and a multi-modal information vector is obtained; performing extraction processing on the multi-modal information vector by utilizing an attribute extraction model to obtain a plurality of commodity attributes; wherein the attribute extraction model is pre-trained based on commodity attributes in a relational database. By adopting the method and the device, the automation level and generalization ability of commodity attribute extraction can be improved, and the manual dependence is reduced. And in addition, multi-modal conjoint analysis of commodity information is supported, and the adaptability to complex scenes is improved.

Description

Technical Field

[0001] The present application relates to the technical field of information processing, and in particular, to a method, apparatus, device, storage medium, and program product for extracting product attributes. Background Art

[0002] With the development of the Internet, transaction data is getting larger and the product categories in e-commerce platforms are becoming more diverse. To better classify and recommend products, it is necessary to extract product attributes.

[0003] Currently, rules templates written manually are often used to extract product attributes. The rules templates include keywords, attribute extraction rules, etc.

[0004] However, the above method relies on manual labor and has limited adaptability to complex scenarios. Summary of the Invention

[0005] Based on this, in view of the above technical problems, it is necessary to provide a method, apparatus, device, storage medium, and program product for extracting product attributes, which can improve the automation level and generalization ability of product attribute extraction, reduce the dependence on manual labor, support the joint analysis of multi-modal product information, and improve the adaptability to complex scenarios.

[0006] In a first aspect, the present application provides a method for extracting product attributes, the method comprising:

[0007] Obtain multi-modal product information; the multi-modal product information includes text information and image information;

[0008] Process the multi-modal product information to obtain a multi-modal information vector;

[0009] Use an attribute extraction model to perform extraction processing on the multi-modal information vector to obtain multiple product attributes; wherein, the attribute extraction model is pre-trained based on product attributes in a relational database.

[0010] In one embodiment, the multi-modal information vector includes a text information vector and an image information vector. Processing the multi-modal product information to obtain a multi-modal information vector includes the following steps:

[0011] Encode the text information to obtain a text information vector;

[0012] Extract the image information to obtain an image information vector.

[0013] In one embodiment, using an attribute extraction model to perform extraction processing on the multi-modal information vector to obtain multiple product attributes includes:

[0014] Input the text information vector and the image information vector into the attribute extraction model to obtain multiple product attributes output by the attribute extraction model.

[0015] In one embodiment, the method further includes:

[0016] Obtain a training sample set by using a pre-set product database and data augmentation techniques; the training sample set includes multiple training samples, the annotations and confidence levels of each training sample;

[0017] Perform staged training based on the training samples and annotations with different confidence levels in the training sample set to obtain an attribute extraction model; wherein, a mixed loss function is used in each training stage.

[0018] In one embodiment, the method further includes:

[0019] For any product, construct a relational network based on a pre-set multiple product attributes, wherein the relational network takes the product as the core, takes each product attribute as a node, and uses edges to represent the association relationships between product attributes;

[0020] Store the relational network in a relational database.

[0021] In one embodiment, the method further includes at least one of the following:

[0022] Perform attribute alignment on the relational database by using an external knowledge graph;

[0023] Update the relational database by using the product attributes output by the attribute extraction model and the feedback information for the product attributes;

[0024] Perform correction processing on the conflicting product attributes in the relational database by using a voting mechanism and an external knowledge source.

[0025] In a second aspect, the present application further provides an extraction device for product attributes, and the device includes:

[0026] An information acquisition module, configured to acquire multimodal product information; the multimodal product information includes text information and image information;

[0027] An information processing module, configured to process the multimodal product information to obtain a multimodal information vector;

[0028] An attribute extraction module, configured to perform extraction processing on the multimodal information vector by using an attribute extraction model to obtain multiple product attributes; wherein, the attribute extraction model is pre-trained based on the product attributes in the relational database.

[0029] In a third aspect, the present application further provides a computer device, including a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0030] Obtain multi-modal product information; the multi-modal product information includes text information and image information;

[0031] Process the multi-modal product information to obtain a multi-modal information vector;

[0032] Use an attribute extraction model to perform extraction processing on the multi-modal information vector to obtain multiple product attributes; wherein, the attribute extraction model is pre-trained based on product attributes in a relational database.

[0033] In a fourth aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0034] Obtain multi-modal product information; the multi-modal product information includes text information and image information;

[0035] Process the multi-modal product information to obtain a multi-modal information vector;

[0036] Use an attribute extraction model to perform extraction processing on the multi-modal information vector to obtain multiple product attributes; wherein, the attribute extraction model is pre-trained based on product attributes in a relational database.

[0037] In a fifth aspect, the present application further provides a computer program product, including a computer program. When the computer program is executed by a processor, the following steps are implemented:

[0038] Obtain multi-modal product information; the multi-modal product information includes text information and image information;

[0039] Process the multi-modal product information to obtain a multi-modal information vector;

[0040] Use an attribute extraction model to perform extraction processing on the multi-modal information vector to obtain multiple product attributes; wherein, the attribute extraction model is pre-trained based on product attributes in a relational database.

[0041] The above-mentioned method, device, equipment, storage medium and program product for extracting commodity attributes obtain multi-modal commodity information; process the multi-modal commodity information to obtain multi-modal information vectors; and use an attribute extraction model to perform extraction processing on the multi-modal information vectors to obtain multiple commodity attributes. In the embodiments of the present application, commodity information can be automatically obtained, and attribute extraction can be automatically performed according to the commodity information. The automation level and generalization ability of commodity attribute extraction are relatively high, reducing the dependence on manual labor. Moreover, it supports the joint analysis of multi-modal commodity information, improving the adaptability to complex scenarios. Brief Description of the Drawings

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following will briefly introduce the drawings required for use in the description of the embodiments or related technologies. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0043] Figure 1 It is an application environment diagram of the method for extracting commodity attributes in an embodiment;

[0044] Figure 2 It is a schematic flowchart of the method for extracting commodity attributes in an embodiment;

[0045] Figure 3 It is a schematic flowchart of the steps for training an attribute extraction model in an embodiment;

[0046] Figure 4 It is a schematic flowchart of the steps for storing commodity attributes in a relational database in an embodiment;

[0047] Figure 5 It is a structural block diagram of the device for extracting commodity attributes in an embodiment;

[0048] Figure 6 It is an internal structure diagram of a computer device in an embodiment. Detailed Embodiments

[0049] In order to make the purpose, technical solutions and advantages of the present application clearer, the following will further describe the present application in detail with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0050] The method for extracting commodity attributes provided by the embodiments of the present application can be applied to, for example Figure 1In the application environment shown. The application environment may include a server 101, and the server 101 may communicate with other servers and terminals through a network. The data storage system may store the data that the server 101 needs to process, such as multi-modal commodity information, multi-modal information vectors, attribute extraction models, and relational databases, etc. The data storage system may be integrated on the server 101, or may be placed on the cloud or other network servers. The server 101 may construct a relational network with commodities as the core and store the relational network in the relational database; the server 101 may also train an attribute extraction model based on the commodity attributes in the relational database; in actual applications, the server 101 may obtain multi-modal commodity information from the network and perform attribute extraction based on the multi-modal commodity information to obtain commodity attributes. The server 101 may be implemented by an independent server or a server cluster composed of multiple servers.

[0051] In an exemplary embodiment, as Figure 2 shown, a method for extracting commodity attributes is provided. Taking the server in Figure 1 as an example, the method includes the following steps:

[0052] Step 201, obtain multi-modal commodity information.

[0053] Among them, the multi-modal commodity information includes text information and image information. The text information includes the description text of the commodity, for example, the brand, model, color, size, etc. of the commodity. The image information includes the images of the commodity, for example, the front image, side image, top view image, sectional view, internal image, etc. of the commodity. The table information includes the tabular form description of the commodity.

[0054] The server may crawl various modal commodity information from the network through a crawler software, or may obtain various modal commodity information from a website or other servers through an API (Application Programming Interface).

[0055] It should be noted that the modality of the commodity information and the way of obtaining the commodity information are not limited to the above examples. In actual applications, other modalities and other obtaining methods may also be adopted.

[0056] Step 202, process the multi-modal commodity information to obtain a multi-modal information vector.

[0057] Different processing methods are used for product information in different modalities to obtain information vectors for each modality. For example, context dependencies and correspondences between different languages are extracted from text information to obtain the corresponding information vector for the text information. Text, symbols, etc. are extracted from image information to obtain the corresponding information vector for the image information. The associations between text, rows, and columns are extracted from table information to obtain the corresponding information vector for the table information.

[0058] Step 203: Use the attribute extraction model to extract the multimodal information vector to obtain multiple product attributes.

[0059] Product attributes refer to the various elements used to describe a product's characteristics and features. They depict and reflect the product's essence, helping consumers understand the specific details of the product and facilitating product management and classification. Product attributes include basic attributes, material attributes, functional attributes, appearance attributes, and other attributes.

[0060] Basic attributes may include product name, product code, brand, specifications, model, etc.

[0061] Material attributes can include raw materials and ingredients. Raw materials refer to the primary substances that make up a product, such as wood, metal, or plastic. Ingredients detail the various chemical substances or components contained in a product. This information is particularly important for products like food and cosmetics, as it helps consumers understand whether a product meets their needs and health requirements.

[0062] Functional attributes include usage functions and performance parameters. Usage functions are the core features of a product that can meet certain consumer needs, such as a mobile phone's call, photo taking, and internet access functions, or a washing machine's washing and dehydration functions. Performance parameters are used to quantify and describe the functional performance of a product, such as a mobile phone's processor performance, camera pixel count, and air conditioner cooling capacity. These parameters can help consumers compare the performance of different products.

[0063] Appearance attributes include color, shape, pattern, etc.

[0064] Other attributes include production date, shelf life, origin, price, etc.

[0065] The attribute extraction model is pre-trained based on the product attributes in the relational database. Various product attributes are pre-stored in the relational database, and a deep learning model is trained for these product attributes to obtain the attribute extraction model. For example, by training a deep learning model for basic attributes, a model for extracting basic attributes can be obtained; by training a deep learning model for functional attributes, a model for extracting functional attributes can be obtained. It should be noted that the attribute extraction model can be dedicated to extracting a certain type of attribute or can extract multiple attributes. For example, the attribute extraction model can extract basic attributes such as product name, brand, specification, model, etc., and appearance attributes such as color, pattern, etc. In practical applications, the attributes that the model can extract can be set during the training process.

[0066] After the server obtains the multi-modal information vector, it inputs the multi-modal information vector into the attribute extraction model. The attribute extraction model can analyze the correlation relationship between the information vectors of different modalities, and perform extraction processing based on the information vectors of different modalities and the correlation relationship, and then output multiple product attributes.

[0067] In the above embodiment, multi-modal product information is obtained; the multi-modal product information is processed to obtain a multi-modal information vector; the multi-modal information vector is processed by using the attribute extraction model to obtain multiple product attributes. The embodiment of the present application can automatically obtain product information, automatically perform attribute extraction according to the product information, and has a relatively high automation level and generalization ability for product attribute extraction, reducing the dependence on manual labor. Moreover, it supports the joint analysis of multi-modal product information, improving the adaptability to complex scenarios.

[0068] In an exemplary embodiment, the multi-modal information vector includes a text information vector and an image information vector. The above-mentioned "processing the multi-modal product information to obtain a multi-modal information vector" includes the following items:

[0069] 1) Perform encoding processing on the text information to obtain a text information vector.

[0070] First, preprocess the text information to obtain the preprocessed text information. The preprocessing can include: removing redundant symbols in the text information, correcting spelling mistakes in the text information, unifying the unit format in the text information, and so on.

[0071] Input the preprocessed text information into a pre-trained encoding model for semantic encoding processing, perform context encoding on the text, and obtain the text information vector output by the encoding model. Among them, the encoding model can include an attention mechanism module, and the attention mechanism module can capture long-distance dependence relationships, thereby improving the accuracy of the text information vector.

[0072] It should be noted that the method for obtaining the text information vector is not limited to the above examples. In actual applications, other methods can also be used.

[0073] 2) Extract and process the image information to obtain an image information vector.

[0074] First, preprocess the image information to obtain the preprocessed image information. The preprocessing can include: grayscale processing, noise reduction processing, binarization processing, skew correction processing, etc. The preprocessing can improve the image quality and prepare for subsequent character segmentation and recognition.

[0075] Encode the preprocessed image information to obtain an image information vector. The goal of image encoding is to convert the original pixels into high-dimensional semantic feature vectors, which is usually achieved through the following methods:

[0076] Encoder based on Transformer.

[0077] Vision Transformer (ViT): Cut the image into fixed-size patches, and capture global dependencies through the self-attention mechanism.

[0078] Patch Embedding: Flatten each image patch and map it to an embedding vector.

[0079] Position encoding: Add position information to preserve the spatial relationship.

[0080] Multi-layer Transformer encoder: Extract context-related features through self-attention and feed-forward networks.

[0081] In some embodiments, an associated model that has been pre-trained can also be used to align the image and text, map the image features and text features to a unified semantic space, and achieve cross-model attribute association.

[0082] In the above embodiments, the text information is encoded to obtain a text information vector; the image information is extracted and processed to obtain an image information vector. The embodiments of the present application process the text information and image information respectively, which is convenient for the subsequent attribute extraction model to perform attribute extraction, thereby improving the accuracy of attribute recognition in complex scenarios.

[0083] In an exemplary embodiment, "using an attribute extraction model to extract and process the multi-modal information vectors to obtain multiple product attributes" in the above embodiments can include: inputting the text information vector and the image information vector into the attribute extraction model to obtain multiple product attributes output by the attribute extraction model.

[0084] The multi-modal product information includes text information and image information. The information vector obtained by processing the multi-modal product information may include a text information vector and an image information vector.

[0085] The server inputs the text information vector and the image information vector into an attribute extraction model. The attribute extraction model analyzes each information vector respectively, analyzes the association relationship between multiple information vectors, and then outputs product attributes according to the pre-set attributes.

[0086] For example, input the text information vector and the image information vector corresponding to product A into the attribute extraction model. The attribute extraction model outputs that the name of product A is X, the brand is Y, the specifications include L1 and W1, the color is white, and the usage functions include calling, taking pictures, and surfing the Internet. Input the text information vector and the image information vector corresponding to product B into the attribute extraction model. The attribute extraction model outputs that the name of product B is M, the brand is N, the specifications include L2 and W2, the color is black, and the usage functions include calling, taking pictures, and surfing the Internet.

[0087] The above attribute extraction model can adopt a neural network model, such as a convolutional neural network, a recurrent neural network, etc. The embodiments of the present application do not limit this.

[0088] In the above embodiments, the text information vector and the image information vector are input into the attribute extraction model to obtain multiple product attributes output by the attribute extraction model. The embodiments of the present application use the attribute extraction model for extraction processing, which not only has high extraction efficiency, but also does not rely on manually writing rule templates and can adapt to complex scenarios.

[0089] In an exemplary embodiment, the attribute extraction model can adopt a large model. Compared with ordinary deep learning models or machine learning models, the large model has the characteristics of a large number of parameters, high requirements for data and computing volume, the ability to process multi-modal data, high consumption, high power cost and heat dissipation cost, and high local deployment cost. Based on the above characteristics, when the attribute extraction model adopts a large model, there are special settings for both model training and deployment.

[0090] As Figure 3 shown, the embodiments of the present application may further include the steps of model training:

[0091] Step 301, obtain a training sample set by using a pre-set product database and data augmentation technology.

[0092] Among them, the training sample set includes multiple training samples, the attribute annotations and confidence levels of each training sample. For example, the training sample set may include multiple texts describing products, each text having a corresponding attribute annotation and confidence level; the training sample set may also include multiple product images, each product image having a corresponding attribute annotation and confidence level; the training sample set may also include multiple tables, each table having a corresponding attribute annotation and confidence level. Among them, the confidence level is used to characterize the accuracy of the attribute annotation corresponding to the text, image or table. It can be understood that the higher the confidence level, the higher the accuracy of the attribute annotation, and the lower the confidence level, the lower the accuracy of the attribute annotation.

[0093] For merchants or manufacturers, the relevant data of each product is stored in the product database. Therefore, a large amount of product data is stored in the product database.

[0094] The server screens a preset number of text data, image data and table data from the product database, and respectively performs data cleaning and data screening processing on the text data, image data and table data to obtain the processed text data, image data and table data.

[0095] The server uses data augmentation techniques to perform expansion processing on the processed text data, image data and table data to obtain the expanded text data, image data and table data. Attribute annotations and confidence levels are obtained for the expanded text data, image data and table data to obtain a training sample set.

[0096] In some embodiments, using data augmentation techniques to perform expansion processing on the processed text data may include: translating the text data in one language into text data in another language, and then translating the translated text data back into the original language text data, and the text data translated back into the original language is the expanded data. Or, randomly select some words or tokens in the text data, replace them with special mask tokens, and then use a pre-established data augmentation model to predict the masked content according to the context information, and the predicted content is the expanded data.

[0097] In some embodiments, using data augmentation techniques to perform expansion processing on the processed image data may include: using a pre-established image conversion model to convert the image data into text data, and then using AI (Artificial Intelligence) to generate image data according to the converted text data, and the generated image data is the expanded data.

[0098] It should be noted that the data augmentation technique is not limited to the above examples. In practical applications, other methods can also be used for expansion processing to enhance the diversity of training samples.

[0099] In some embodiments, the data stored in the commodity database is semi-structured data. Compared with structured data, it can enrich the training samples and reduce the dependence on manually labeled data.

[0100] Step 302: Perform staged training based on the training samples and annotations with different confidence levels in the training sample set to obtain an attribute extraction model.

[0101] Among them, a hybrid loss function is adopted in each training stage.

[0102] To reduce the difficulty of model training, staged training can be carried out according to the complexity of the training samples. The training process may include: using the training samples with high confidence levels and attribute annotations to train the model to obtain a standby extraction model; then, using the training samples with low confidence levels and attribute annotations to train the standby extraction model to obtain an attribute extraction model. In this way, high-confidence training samples are used in the early stage, and noisy data is gradually introduced in the later stage, which can improve the robustness of the model.

[0103] A hybrid loss function can be adopted in each training stage. In some embodiments, the hybrid loss function may include a cross-entropy loss function and a contrastive loss function.

[0104] For example, the process of using the training samples with high confidence levels and attribute annotations to train the model includes: inputting the training samples with high confidence levels into the original model to obtain the training results output by the original model; using the cross-entropy loss function to calculate the loss between the training results and the attribute annotations to obtain a first loss value, and using the contrastive loss function to calculate the loss between the training results and the attribute annotations to obtain a second loss value; adjusting the model parameters according to the first loss value and the second loss value.

[0105] In the process of adjusting the model parameters, a gradient balancing algorithm can be adopted to balance or adjust the gradient information to improve the convergence speed of model training.

[0106] It should be noted that the staged training can be divided into two or more stages of training, and the hybrid loss function is not limited to the above examples.

[0107] In some embodiments, model training can adopt distributed training: First, deploy the hardware environment, such as setting up multiple computer devices, each equipped with multiple GPUs (Graphics Processing Units). Different computer devices need to be connected through a high-speed network (such as Ethernet) to ensure the efficiency of data transmission. Then, install a deep learning framework, such as the DeepSeed framework, on each computer device. Initialize the distributed environment, and to achieve multi-machine multi-GPU training, divide the training sample set into different processes. Next, package the model to enable parallel training of the model on multiple GPUs. For single-node multi-GPU training, multiple processes can be started; for multi-node multi-GPU sequences, a startup script needs to be run on each node. During the training process, record key information, such as loss values, accuracy, etc., for subsequent analysis and debugging. And use tools to monitor the GPU usage to ensure that all GPUs are working properly. Further, pay attention to data consistency to ensure that the data on all computer devices is consistent, which can be achieved using shared storage or data synchronization tools; pay attention to network stability. Distributed training has high requirements for network stability, and it is necessary to ensure a stable network connection to avoid data transmission interruptions.

[0108] In some embodiments, mixed-precision training, such as FP16, can be adopted. This is because FP16 occupies less memory, has a low data transmission bandwidth requirement, and has a fast calculation speed on the GPU, which can significantly reduce the training time. Especially when dealing with large-scale data and complex models, it can accelerate convergence.

[0109] It can be understood that various methods, such as staged training, distributed training, and mixed-precision training, can reduce the impact of large model parameters, high data and computing requirements, as well as high consumption, high power cost, and high heat dissipation cost on model training, thereby reducing the model training difficulty and improving the model training efficiency.

[0110] In some embodiments, perform compression processing on the attribute extraction model to obtain a compressed attribute extraction model.

[0111] After training the attribute extraction model, the knowledge distillation technique can be used to distill the attribute extraction model into a lightweight model.

[0112] In some embodiments, when it is necessary to deploy the attribute extraction model to a mobile device, further compression processing can be performed on the attribute extraction model through quantization (INT8 quantization) and pruning techniques.

[0113] It can be understood that compression processing, pruning techniques, etc. can reduce the local deployment cost of large models and help with the wide application of large models.

[0114] In some embodiments, the model is quickly migrated through domain adaptation technology, so that the model can be applicable to different product categories (such as electronic products, clothing), enhancing the cross-domain adaptation ability of the attribute extraction method.

[0115] In the above embodiments, a training sample set is obtained by using a pre-set product database and data augmentation technology; based on the training samples and annotations with different confidence levels in the training sample set, phased training is performed to obtain an attribute extraction model. In the embodiments of the present application, a training sample set is obtained through data augmentation technology, which can enrich the diversity of training samples. Through phased training, the model training difficulty can be reduced, and the model robustness can be improved. Further, through distributed training, the model processing can be accelerated, and the model training speed can be improved; by compressing the model, the model can be made lightweight, thus facilitating deployment on mobile devices.

[0116] In an exemplary embodiment, as Figure 4 shown, the embodiments of the present application may further include the following steps:

[0117] Step 401, for any product, a relational network is constructed based on a plurality of pre-set product attributes.

[0118] Among them, the relational network takes the product as the core, takes each product attribute as a node, and uses edges to represent the association relationships between product attributes.

[0119] For a certain product, a plurality of product attributes can be pre-set. In this way, the server takes the product as the core, and then takes the product attributes as nodes to construct a relational network.

[0120] For example, for product A, the pre-set basic attributes include name, brand, and specification, the appearance attributes include color, and the function attributes include usage function and function parameters. Taking product A as the core, taking the basic attributes, material attributes, appearance attributes, and function attributes as the child nodes of the core, taking name, brand, and specification as the child nodes of the basic attributes, taking color as the child node of the appearance attributes, and taking usage function and function parameters as the child nodes of the function attributes, a relational network of product A is constructed.

[0121] It can be understood that for the same type of product, the structure of the relational network is similar. For example, the basic attributes, material attributes, appearance attributes, and function attributes are all taken as the child nodes of the core, the name, brand, and specification are taken as the child nodes of the basic attributes, the color is taken as the child node of the appearance attributes, and the usage function and function parameters are taken as the child nodes of the function attributes. In this case, an attribute extraction model can be trained based on this type of product, and the product attributes output by the attribute extraction model are restricted to the attributes of the above child nodes.

[0122] Step 402, store the relational network in a relational database.

[0123] Store the relational networks corresponding to each commodity in a relational database.

[0124] In the above embodiments, for any commodity, a relational network is constructed based on a plurality of preset commodity attributes; the relational network is stored in a relational database. By constructing a heterogeneous network in the embodiments of the present application, a relational database can be used to store commodity attributes, reflecting the association relationships between the attributes; moreover, it also provides a basis for the training of the attribute extraction model.

[0125] In an exemplary embodiment, the embodiments of the present application can also perform at least one of the following:

[0126] 1) Align the attributes in the relational database using an external knowledge graph.

[0127] The server periodically pulls commodity data from the external knowledge graph, and counts the proportion of each commodity attribute under each category according to the pulled commodity data; then, determines the commodity attributes to be extracted according to the proportion. For example, determines the commodity attribute with the highest proportion as the commodity attribute to be extracted.

[0128] Compare the values of the commodity attributes to be extracted with the commodity attributes in the relational database, and update the relational network in the relational database.

[0129] For example, the nodes of the relational network of commodity A include commodity attributes a, b, and c. If the server determines that the commodity attribute to be extracted is d according to the pulled commodity data, then update the relational network so that the nodes of the relational network include commodity attributes a, b, c, and d.

[0130] 2) Update and process the relational database using the commodity attributes output by the attribute extraction model and the feedback information for the commodity attributes.

[0131] During the process of using the attribute extraction model, the feedback information of the user for the commodity attributes output by the attribute extraction model can be obtained. The server can update and process the relational network in the relational database based on the feedback information.

[0132] For example, for commodity A, the commodity attributes output by the attribute extraction model include that the color is white, and the feedback information input by the user for this commodity attribute is that the color is pearl white. The server updates the color sub-node in the relational network of commodity A from white to pearl white.

[0133] 3) Use a voting mechanism and an external knowledge source to correct the conflicting commodity attributes in the relational database.

[0134] In the case of conflicting product attributes in a relational database, the server can use a voting mechanism to determine the correct product attributes; it can also pull product data from an external knowledge source and determine the correct product attributes based on the pulled product data; it can also vote on the conflicting product attributes based on the pulled product data and determine the correct product attributes according to the voting results.

[0135] For example, for product A in the relational database, the colors are white and pearl white. A voting mechanism can be used to determine which is the correct color, or the correct color can be determined according to the product data from the external knowledge source. It is also possible to vote on the two conflicting colors based on the colors pulled from the external knowledge source to determine which is the correct color.

[0136] After that, the relational network in the relational database is corrected according to the correct product attributes.

[0137] In the above embodiments, the relational database is aligned with attributes by using an external knowledge graph; the relational database is updated by using the product attributes output by the attribute extraction model and the feedback information for the product attributes; the conflicting product attributes in the relational database are corrected by using a voting mechanism and an external knowledge source. By aligning the knowledge graph, updating and correcting the relational database, the embodiments of the present application can achieve the automatic discovery of new product attributes and the real-time expansion of the relational database.

[0138] In an exemplary embodiment, a method for extracting product attributes is provided. Taking the server in Figure 1 as an example, the method includes the following steps:

[0139] Step 1, for any product, a relational network is constructed based on a plurality of preset product attributes.

[0140] Among them, the relational network takes the product as the core, each product attribute as a node, and the edges represent the association relationships between the product attributes;

[0141] Step 2, store the relational network in the relational database.

[0142] Step 3, a training sample set is obtained by using a preset product database and data enhancement technology.

[0143] Among them, the training sample set includes multiple training samples, the annotations and confidence levels of each training sample.

[0144] Step 4, based on the training samples with different confidence levels and the annotations in the training sample set, perform staged training to obtain an initial extraction model.

[0145] Among them, a hybrid loss function is used in each training stage.

[0146] Step 5: Compress the initial extraction model to obtain an attribute extraction model.

[0147] Step 6: Obtain multi-modal product information.

[0148] Step 7: Encode the text information to obtain a text information vector; extract the image information to obtain an image information vector.

[0149] Step 8: Input the text information vector and the image information vector into the attribute extraction model to obtain multiple product attributes output by the attribute extraction model.

[0150] Step 9: Align the attributes of the relational database using an external knowledge graph; update the relational database using the product attributes output by the attribute extraction model and the feedback information for the product attributes; correct the conflicting product attributes in the relational database using a voting mechanism and an external knowledge source.

[0151] Obtain multi-modal product information; process the multi-modal product information to obtain a multi-modal information vector; use the attribute extraction model to extract the multi-modal information vector to obtain multiple product attributes. In the embodiments of the present application, product information can be automatically obtained, and attribute extraction can be automatically performed based on the product information. The automation level and generalization ability of product attribute extraction are relatively high, reducing manual dependence. Moreover, it supports the joint analysis of multi-modal product information, improving the adaptability to complex scenarios.

[0152] It should be understood that although the steps in the flowcharts involved in the above-described embodiments are shown in sequence according to the arrows, these steps do not necessarily have to be executed in the order indicated by the arrows. Unless otherwise clearly stated in this article, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, at least some of the steps in the flowcharts involved in the above-described embodiments may include multiple steps or multiple stages. These steps or stages do not necessarily have to be executed at the same time, but can be executed at different times. The execution order of these steps or stages does not necessarily have to be sequential, but can be executed alternately or in turn with at least some of the steps or stages in other steps or other steps.

[0153] Based on the same inventive concept, the present application also provides a device for extracting product attributes for implementing the aforementioned method for extracting product attributes. The solution provided by this device is similar to the solution described in the aforementioned method. Therefore, the specific limitations of the following embodiments of the device for extracting one or more product attributes can be found in the aforementioned limitations of the method for extracting product attributes, and will not be repeated here.

[0154] In an exemplary embodiment, Figure 5 As shown, a device for extracting commodity attributes is provided, comprising:

[0155] The information acquisition module 501 is used to acquire multimodal product information; the multimodal product information includes text information and image information;

[0156] An information processing module 502 is configured to process the multimodal product information to obtain a multimodal information vector;

[0157] The attribute extraction module 503 is used to extract the multimodal information vector using an attribute extraction model to obtain multiple product attributes; wherein the attribute extraction model is pre-trained based on product attributes in a relational database.

[0158] In one embodiment, the multimodal information vector includes a text information vector and an image information vector. The information processing module 502 is specifically used to encode the text information to obtain the text information vector; and to extract the image information to obtain the image information vector.

[0159] In one embodiment, the attribute extraction module 503 is specifically configured to input the text information vector and the image information vector into the attribute extraction model to obtain a plurality of product attributes output by the attribute extraction model.

[0160] In one embodiment, the apparatus further comprises:

[0161] The sample acquisition module is used to obtain a training sample set using a pre-set product database and data enhancement technology; the training sample set includes multiple training samples, labels and confidence levels of each training sample;

[0162] The model training module is used to perform phased training based on training samples and annotations with different confidence levels in the training sample set to obtain an attribute extraction model; wherein, a hybrid loss function is used in each training stage.

[0163] In one embodiment, the apparatus further comprises:

[0164] The heterogeneous graph construction module is used to construct a relational network for any commodity based on multiple preset commodity attributes. In the relational network, the commodity is the core, each commodity attribute is a node, and the edges represent the association relationships between commodity attributes;

[0165] The storage module is used to store the relational network in a relational database.

[0166] In one embodiment, the device further includes at least one of the following:

[0167] The attribute alignment module is used to perform attribute alignment on the relational database by using an external knowledge graph;

[0168] The update processing module is used to perform update processing on the relational database by using the commodity attributes output by the attribute extraction model and the feedback information for the commodity attributes;

[0169] The correction processing module is used to correct the conflicting commodity attributes in the relational database by using a voting mechanism and an external knowledge source.

[0170] Each module in the above-mentioned commodity attribute extraction device can be implemented in whole or in part by software, hardware, and their combination. The above-mentioned modules can be embedded in the processor of the computer device in hardware form or be independent of it, or can be stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to the above-mentioned modules.

[0171] In an exemplary embodiment, a computer device is provided. The computer device can be a server, and its internal structure diagram can be as Figure 6 shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O), and a communication interface. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store XX data. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals through a network connection. When the computer program is executed by the processor, it implements a commodity attribute extraction method.

[0172] Those skilled in the art can understand, Figure 6The structure shown is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0173] In an exemplary embodiment, a computer device is provided, including a memory and a processor. A computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0174] Obtain multi-modal product information; the multi-modal product information includes text information and image information;

[0175] Process the multi-modal product information to obtain a multi-modal information vector;

[0176] Use an attribute extraction model to perform extraction processing on the multi-modal information vector to obtain multiple product attributes; among them, the attribute extraction model is pre-trained based on the product attributes in the relational database.

[0177] In one embodiment, the multi-modal information vector includes a text information vector and an image information vector. When the processor executes the computer program, the following steps are also implemented:

[0178] Perform encoding processing on the text information to obtain a text information vector;

[0179] Perform extraction processing on the image information to obtain an image information vector.

[0180] In one embodiment, when the processor executes the computer program, the following steps are also implemented:

[0181] Input the text information vector and the image information vector into the attribute extraction model to obtain multiple product attributes output by the attribute extraction model.

[0182] In one embodiment, when the processor executes the computer program, the following steps are also implemented:

[0183] Use a pre-set product database and data augmentation technology to obtain a training sample set; the training sample set includes multiple training samples, the annotations and confidence levels of each training sample;

[0184] Perform staged training based on the training samples and annotations with different confidence levels in the training sample set to obtain an attribute extraction model; among them, a mixed loss function is used in each training stage.

[0185] In one embodiment, when the processor executes the computer program, the following steps are also implemented:

[0186] For any commodity, a relational network is constructed based on multiple preset commodity attributes. Among them, the relational network takes the commodity as the core, each commodity attribute as a node, and the edges represent the association relationships between commodity attributes;

[0187] Store the relational network in a relational database.

[0188] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0189] Align the attributes of the relational database by using an external knowledge graph;

[0190] Update the relational database by using the commodity attributes output by the attribute extraction model and the feedback information for the commodity attributes;

[0191] Correct the conflicting commodity attributes in the relational database by using a voting mechanism and an external knowledge source.

[0192] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0193] Obtain multi-modal commodity information; the multi-modal commodity information includes text information and image information;

[0194] Process the multi-modal commodity information to obtain a multi-modal information vector;

[0195] Extract the multi-modal information vector by using an attribute extraction model to obtain multiple commodity attributes; among them, the attribute extraction model is pre-trained based on the commodity attributes in the relational database.

[0196] In one embodiment, the multi-modal information vector includes a text information vector and an image information vector. When the computer program is executed by a processor, the following steps are further implemented:

[0197] Encode the text information to obtain a text information vector;

[0198] Extract the image information to obtain an image information vector.

[0199] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0200] Input the text information vector and the image information vector into the attribute extraction model to obtain multiple commodity attributes output by the attribute extraction model.

[0201] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0202] Obtain a training sample set by using a pre-set product database and data augmentation techniques; the training sample set includes multiple training samples, the annotations and confidence levels of each training sample;

[0203] Perform staged training based on the training samples and annotations with different confidence levels in the training sample set to obtain an attribute extraction model; among them, a mixed loss function is used in each training stage.

[0204] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0205] For any product, construct a relational network based on multiple pre-set product attributes, where the relational network takes the product as the core, each product attribute as a node, and the edges represent the association relationships between product attributes;

[0206] Store the relational network in a relational database.

[0207] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0208] Align the attributes of the relational database by using an external knowledge graph;

[0209] Update the relational database by using the product attributes output by the attribute extraction model and the feedback information for the product attributes;

[0210] Correct the conflicting product attributes in the relational database by using a voting mechanism and an external knowledge source.

[0211] In one embodiment, a computer program product is provided, including a computer program, which when executed by a processor implements the following steps:

[0212] Obtain multi-modal product information; the multi-modal product information includes text information and image information;

[0213] Process the multi-modal product information to obtain a multi-modal information vector;

[0214] Extract the multi-modal information vector by using the attribute extraction model to obtain multiple product attributes; among them, the attribute extraction model is pre-trained based on the product attributes in the relational database.

[0215] In one embodiment, the multi-modal information vector includes a text information vector and an image information vector. When the computer program is executed by a processor, the following steps are further implemented:

[0216] Encode the text information to obtain a text information vector;

[0217] Extract and process the image information to obtain an image information vector.

[0218] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0219] Input the text information vector and the image information vector into an attribute extraction model to obtain multiple product attributes output by the attribute extraction model.

[0220] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0221] Use a pre-set product database and data augmentation technology to obtain a training sample set; the training sample set includes multiple training samples, the annotations and confidence levels of each training sample;

[0222] Based on the training samples and annotations with different confidence levels in the training sample set, conduct staged training to obtain an attribute extraction model; among them, a mixed loss function is used in each training stage.

[0223] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0224] For any product, construct a relational network based on a pre-set multiple product attributes, where the relational network takes the product as the core, each product attribute as a node, and the edges represent the association relationships between the product attributes;

[0225] Store the relational network in a relational database.

[0226] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0227] Use an external knowledge graph to perform attribute alignment on the relational database;

[0228] Use the product attributes output by the attribute extraction model and the feedback information for the product attributes to perform update processing on the relational database;

[0229] Use a voting mechanism and an external knowledge source to correct the conflicting product attributes in the relational database.

[0230] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data that have been authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with relevant regulations.

[0231] Those of ordinary skill in the art can understand that all or part of the processes in the above-described embodiment methods can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memories. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in this application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in this application can be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, data processing logics based on quantum computing, etc., without limitation.

[0232] The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0233] The above-described embodiments only represent several implementation manners of this application. The description is relatively specific and detailed, but it should not be construed as a limitation on the patent scope of this application. It should be noted that for those of ordinary skill in the art, without departing from the concept of this application, several modifications and improvements can still be made, and these all belong to the protection scope of this application. Therefore, the protection scope of this application should be subject to the appended claims.

Claims

1. A method for extracting commodity attributes, characterized in that, The method includes: Obtaining multi-modal product information; the multi-modal product information includes text information and image information; Processing the multi-modal product information to obtain a multi-modal information vector; Using an attribute extraction model to perform extraction processing on the multi-modal information vector to obtain multiple product attributes; wherein, the attribute extraction model is pre-trained based on product attributes in a relational database.

2. The method according to claim 1, wherein The multi-modal information vector includes a text information vector and an image information vector. The processing of the multi-modal product information to obtain a multi-modal information vector includes the following items: Encoding the text information to obtain a text information vector; Performing extraction processing on the image information to obtain an image information vector.

3. The method according to claim 2, wherein The using of the attribute extraction model to perform extraction processing on the multi-modal information vector to obtain multiple product attributes includes: Inputting the text information vector and the image information vector into the attribute extraction model to obtain the multiple product attributes output by the attribute extraction model.

4. The method according to any one of claims 1 to 3, characterized in that, The method further includes: Using a pre-set product database and data augmentation technology to obtain a training sample set; the training sample set includes multiple training samples, the annotations and confidence levels of each training sample; Performing staged training based on the training samples with different confidence levels and annotations in the training sample set to obtain the attribute extraction model; wherein, a mixed loss function is used in each training stage.

5. The method according to any one of claims 1-3, characterized in that, The method further includes: For any product, constructing a relational network based on a pre-set multiple product attributes, wherein the relational network takes the product as the core, takes each product attribute as a node, and uses edges to represent the association relationships between the product attributes; Storing the relational network into the relational database.

6. The method according to claim 5, wherein The method further includes at least one of the following: Using an external knowledge graph to perform attribute alignment on the relational database; Using the product attributes output by the attribute extraction model and the feedback information for the product attributes to perform update processing on the relational database; Using a voting mechanism and an external knowledge source to correct the conflicting product attributes in the relational database.

7. An extraction device for commodity attributes, characterized in that, The device includes: An information acquisition module for obtaining multi-modal product information; the multi-modal product information includes text information and image information; An information processing module for processing the multi-modal product information to obtain a multi-modal information vector; An attribute extraction module for using an attribute extraction model to perform extraction processing on the multi-modal information vector to obtain multiple product attributes; wherein, the attribute extraction model is pre-trained based on product attributes in a relational database.

8. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When this computer program is executed by the processor, it implements the steps of the method according to any one of claims 1 to 6.

Images