Article matching method and device

Abstract

The invention discloses an article matching method and device, and relates to the technical field of artificial intelligence. A specific embodiment of the method comprises the steps of determining an initial associated article of a to-be-matched article from an article pool; for a target attribute except the category attribute in the plurality of attributes of the to-be-matched article, obtaining a target extension rule for the target attribute; under the condition that the target extension rule indicates that the extension attribute value different from the original attribute value of the to-be-matched article in the target attribute exists, the original attribute value of the to-be-matched article in the target attribute and the extension attribute value form an attribute value set of the to-be-matched article in the target attribute; and detecting the initial associated article by using the original attribute value of the to-be-matched article in the category attribute and the attribute value set to obtain a target associated article of the to-be-matched article. According to the embodiment, high-precision and sufficient target associated articles can be obtained for the to-be-matched articles.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an object matching method and device. Background Art

[0002] In the purchase scenario of certain items, users want to know various specific data of other related items when they already own an item, so that they can choose more suitable items. Service providers want to know other items related to their own items for market analysis. This requires matching specific items based on an item pool containing a large number of items. In the prior art, item matching is generally performed by extracting and comparing image or text features, with low accuracy. In addition, in actual scenarios, users generally need to obtain a certain number of related items to achieve business purposes, but the prior art can only calculate a small number of related items in consideration of accuracy requirements, which is difficult to meet the needs. Summary of the invention

[0003] In view of this, an embodiment of the present invention provides an object matching method and device, which can obtain high-precision and sufficient target related objects for the objects to be matched.

[0004] To achieve the above objective, according to one aspect of the present invention, a method for matching items is provided.

[0005] The item matching method of the embodiment of the present invention includes: determining multiple initial associated items of the to-be-matched item from a preset item pool; for any target attribute other than the category attribute among the multiple attributes of the to-be-matched item, obtaining a target extension rule for the target attribute that is pre-configured for the original attribute value of the to-be-matched item in the category attribute; when the target extension rule indicates that there is an extended attribute value that is different from the original attribute value of the to-be-matched item in the target attribute, combining the original attribute value of the to-be-matched item in the target attribute and the extended attribute value to form an attribute value set of the to-be-matched item in the target attribute; and detecting the initial associated items using the original attribute value of the to-be-matched item in the category attribute and the attribute value set to obtain the target associated items of the to-be-matched item.

[0006] Optionally, the extension rule for any target attribute contains at least one attribute value of the target attribute contained in the item pool; and the method further includes: when the target extension rule only contains the original attribute value of the to-be-matched item in the corresponding target attribute, forming an attribute value set of the to-be-matched item in the target attribute with the original attribute value; when the target extension rule contains the original attribute value of the to-be-matched item in the corresponding target attribute and other attribute values, determining the other attribute values ​​as the extended attribute values.

[0007] Optionally, the method of detecting the initial associated item by using the original attribute value of the item to be matched in the category attribute and the attribute value set to obtain the target associated item of the item to be matched includes: for any initial associated item, if the initial associated item and the item to be matched have the same original attribute value in the category attribute, and the original attribute value of the initial associated item in each target attribute belongs to the attribute value set of the item to be matched in the corresponding target attribute, determining the initial associated item as the target associated item of the item to be matched.

[0008] Optionally, determining multiple initial associated items of the to-be-matched item from a preset item pool includes: acquiring image information and text information of the to-be-matched item; inputting the image information into a pre-trained image processing model to obtain foreground coordinate data and item category data in the image information, and combining the foreground coordinate data and the item category data into image features of the to-be-matched item; inputting the text information into a pre-trained text processing model to obtain text features of the to-be-matched item; fusing the image features and the text features into multimodal features of the to-be-matched item, combining the image features, the text features and the multimodal features into comprehensive features of the to-be-matched item, and using the comprehensive features to determine the initial associated items from the item pool.

[0009] Optionally, determining the initial associated items from the item pool using the comprehensive features includes: calculating the similarity between the item to be matched and each item in the item pool according to the comprehensive features of the item to be matched and the comprehensive features of each item in the item pool, and determining the items in the item pool having a similarity greater than a preset threshold as the initial associated items.

[0010] Optionally, the text processing model further outputs the original attribute values ​​of the to-be-matched item in the multiple attributes based on the input text information; and the method further includes: after obtaining the category data to which the item belongs and the original attribute values ​​of the to-be-matched item in the category attributes, determining whether the category data to which the item belongs and the original attribute values ​​of the to-be-matched item in the category attributes are consistent; if they are consistent, executing the step of obtaining the target extension rule for the target attribute that is pre-configured for the original attribute values ​​of the to-be-matched item in the category attributes.

[0011] Optionally, the image processing model includes: a convolutional neural network, a candidate region network, a region of interest pooling layer and a classification layer connected in sequence; wherein the convolutional neural network outputs primary image features based on the input image information; the candidate region network outputs bounding box position data of the object to be matched based on the input initial image features; the region of interest pooling layer outputs the foreground coordinate data based on the input bounding box position data; and the classification layer outputs the category data of the object based on the input foreground coordinate data.

[0012] To achieve the above objective, according to another aspect of the present invention, an object matching device is provided.

[0013] The item matching device of the embodiment of the present invention may include: a coarse-grained matching unit, used to determine multiple initial associated items of the item to be matched from a preset item pool; a rule acquisition unit, used to: for any target attribute other than the category attribute among the multiple attributes of the item to be matched, obtain a target extension rule for the target attribute that is pre-configured for the original attribute value of the item to be matched in the category attribute; an attribute extension unit, used to: when the target extension rule indicates that there is an extended attribute value different from the original attribute value of the item to be matched in the target attribute, combine the original attribute value of the item to be matched in the target attribute and the extended attribute value to form an attribute value set of the item to be matched in the target attribute; and a fine-grained matching unit, used to detect the initial associated items using the original attribute value of the item to be matched in the category attribute and the attribute value set to obtain the target associated items of the item to be matched.

[0014] To achieve the above objective, according to another aspect of the present invention, an electronic device is provided.

[0015] An electronic device of the present invention includes: one or more processors; a storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the item matching method provided by the present invention.

[0016] To achieve the above objective, according to yet another aspect of the present invention, a computer-readable storage medium is provided.

[0017] A computer-readable storage medium of the present invention stores a computer program, which implements the object matching method provided by the present invention when executed by a processor.

[0018] According to the technical solution of the present invention, the embodiments of the above invention have the following advantages or beneficial effects:

[0019] The object matching process is divided into a coarse-grained association process and a fine-grained association process. In the coarse-grained association process, the image features and text features of the objects to be matched are extracted based on the machine learning model and fused into multimodal features. Thereafter, the multimodal features are combined with the image features and text features into comprehensive features as the final representative features of the objects to be matched to calculate the similarity with the objects in the object pool, thereby obtaining the initial associated objects of the objects to be matched, which improves the accuracy compared to the existing technology that uses image features or text features alone. In addition, the embodiment of the present invention designs an image processing model with a special structure to output the above image features. The special structure includes a convolutional neural network, a candidate region network, an area of ​​interest pooling layer, and a classification layer connected in sequence, thereby further improving the feature extraction accuracy and the final matching accuracy. In addition, in order to perform further screening on the above initial associated items to achieve higher matching accuracy while ensuring that the target associated items finally screened out reach a certain number to meet business needs, the embodiment of the present invention first outputs the original attribute values ​​of the to-be-matched items in each attribute through a text processing model used to output the above text features, and then reasonably expands the original attribute values ​​of each attribute through the expansion rules pre-set for the category attribute values ​​to obtain an attribute value set that is strongly correlated with the original attribute value and contains more elements, and finally uses the attribute value set to screen the initial associated items, thereby obtaining more target associated items to meet business needs while ensuring high-precision matching.

[0020] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with the specific implementation manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings are used to better understand the present invention and do not constitute an improper limitation of the present invention.

[0022] Figure 1 Schematic diagram of the main steps of the object matching method in an embodiment of the present invention;

[0023] Figure 2 is a structural schematic diagram of an image processing model according to an embodiment of the present invention;

[0024] Figure 3 Schematic diagram of text feature extraction and attribute value extraction in an embodiment of the present invention;

[0025] Figure 4 is a schematic diagram of attribute value expansion in an embodiment of the present invention;

[0026] Figure 5 is a schematic diagram of specific execution steps of the object matching method in an embodiment of the present invention;

[0027] Figure 6 is a schematic diagram of components of an object matching device in an embodiment of the present invention;

[0028] Figure 7 is an exemplary system architecture diagram in which embodiments of the present invention may be applied;

[0029] Figure 8 It is a schematic diagram of the structure of an electronic device used to implement the object matching method in the embodiment of the present invention. DETAILED DESCRIPTION

[0030] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0031] It should be pointed out that the embodiments of the present invention and the technical features therein may be combined with each other without conflict.

[0032] Figure 1 Schematic diagram of main steps of the object matching method according to an embodiment of the present invention.

[0033] like Figure 1 As shown, the item matching method of the embodiment of the present invention can be executed by a server, and the specific execution steps are as follows:

[0034] Step S101: determining a plurality of initially associated items of an item to be matched from a preset item pool.

[0035] In the embodiment of the present invention, the server can determine the initial associated items of the to-be-matched items based on the image information and text information of the to-be-matched items. The initial associated items refer to the items that are initially screened and are associated with the initial associated items. The initial associated items will be further screened to obtain target associated items that are more associated with the to-be-matched items. The above initial associated items and target associated items can be accurately defined through the following steps.

[0036] In practical applications, the server can first use the identification of the object to be matched to obtain the image information of the object to be matched from the image library as the data source, and obtain the text information of the object to be matched from the text library as the data source. Then, the server inputs the image information into the pre-trained image processing model to obtain the foreground coordinate data and the category data of the object in the image information. The above foreground coordinate data is used to characterize the exact position of the object to be matched in the image, and the above category data of the object to be matched is used to characterize the specific category of the object to be matched (category refers to the type of item under the preset level, such as clothing, food, daily necessities, etc.), that is, the original attribute value of the category attribute of the object to be matched. The original attribute value in this article refers to the inherent attribute value of the object, which is relative to the extended attribute value to be explained below. The extended attribute value is an attribute value related to the corresponding original attribute value and can be used as a screening benchmark for the target associated object. Thereafter, the server combines the above foreground coordinate data and the above category data of the object to be matched into the image feature of the object to be matched.

[0037] In particular, the embodiment of the present invention designs a special structure of the image processing model to improve the accuracy of image feature extraction, see Figure 2 . Specifically, the image processing model may include: a convolutional neural network, a candidate region network (i.e., a Region Proposal Network - RPN network), a region of interest pooling layer (i.e., a Region of Interest - ROI pooling layer) and a classification layer connected in sequence. Among them, the convolutional neural network outputs the primary features of the image based on the input image information of the object to be matched, and the candidate region network outputs the bounding box position data of the object to be matched based on the input initial features of the image, i.e., the BBOX coordinate data. The region of interest pooling layer outputs the foreground coordinate data based on the input bounding box position data, and the classification layer outputs the category data to which the object belongs based on the input foreground coordinate data. In specific applications, the primary features of the image output by the convolutional neural network can enter the region candidate network and the region of interest pooling layer in parallel, and then the above bounding box position data and foreground coordinate data are obtained based on the interaction and calculation result sharing between the region candidate network and the region of interest pooling layer. The above classification layer can also perform regression tasks according to business needs.

[0038] Independently, the server inputs the text information of the to-be-matched object into a pre-trained text processing model to obtain the text features of the to-be-matched object. After obtaining the image features and text features of the to-be-matched object, the server fuses the above image features and the above text features into the multimodal features of the to-be-matched object, and then combines the image features, text features and multimodal features of the to-be-matched object into the comprehensive features of the to-be-matched object, and finally uses the above comprehensive features to determine the initial associated items of the to-be-matched object from the object pool.

[0039] In a specific scenario, the above item pool contains a large number of items including the items to be matched, and the method for determining the above initial associated items can be as follows. The server calculates the similarity between the items to be matched and the items in the item pool based on the comprehensive features of the items to be matched and the comprehensive features of each item in the item pool, and determines the items in the item pool with a similarity greater than a preset threshold as the above initial associated items, and each initial associated item can form an initial associated item pool. The above similarity calculation can be performed by various methods such as cosine similarity, vector inner product, Euclidean distance, and Pearson correlation coefficient. In actual applications, the server may not calculate the above comprehensive features, but calculate the similarity between the items to be matched and the items in the item pool from three channels: image features, text features, and multimodal features, and determine the above initial associated items based on a preset threshold comparison mechanism for the three similarities. The coarse-grained item matching process based on the above feature fusion method can improve the matching accuracy compared to the traditional matching method that uses image features or text features alone.

[0040] As a preferred solution, the above text processing model can further output the original attribute values ​​of the items to be matched in multiple attributes based on the input text information. Exemplarily, the above attributes are important attributes preset according to business needs, which may include: category attributes, specification attributes, weight attributes, capacity attributes, price attributes, etc. The attributes other than the category attributes in the multiple attributes are referred to as target attributes. The attribute value expansion will be performed on the target attributes, and the initial associated items will be screened based on the attribute value set of the target attributes formed by the attribute value expansion. The above steps of text processing can be found in Figure 3 In practical applications, the server may also input text information into other machine learning models other than text processing models to obtain original attribute values ​​of multiple attributes of the items to be matched.

[0041] Preferably, after the server obtains the above-mentioned category data of the item from the image information and the original attribute value of the category attribute of the item to be matched from the text information, it can perform verification by judging whether the two are consistent to find data errors that may exist in the initial stage of item matching. Specifically, the server judges whether the category data of the item output by the image processing model is consistent with the original attribute value of the category attribute of the item to be matched output by the text processing model; if they are consistent, the following steps are executed; if they are inconsistent, an alarm is issued to request manual intervention, and the following steps are executed after the problem is resolved.

[0042] Step S102: For any target attribute other than the category attribute among the multiple attributes of the to-be-matched object, a target expansion rule for the target attribute pre-configured for the original attribute value of the category attribute of the to-be-matched object is obtained.

[0043] The following will further screen each initial associated item to obtain target associated items that have higher relevance and similarity to the item to be matched, see Figure 4 . First, for any target attribute other than the category attribute among the multiple attributes of the item to be matched, the server obtains the target extension rule for the target attribute, which is pre-configured for the original attribute value of the item to be matched in the category attribute, from the pre-built rule engine. In other words, each extension rule in the rule engine is formulated according to different specific category attribute values ​​(i.e., the original attribute value of the item in the category attribute). For example, corresponding extension rules are formulated for the clothing category, and corresponding extension rules are formulated for the food category. In addition, each extension rule also corresponds to a target attribute. For example, an extension rule formulated for the food category corresponds to the weight attribute. In actual application, the extension rule is extended for the attribute value of the weight attribute.

[0044] In an embodiment of the present invention, the above extension rules have two forms based on the corresponding attribute values ​​of specific categories and a certain target attribute. The first is that the extension rule contains an attribute value traversal set of the corresponding target attribute, and the attribute value traversal set has at least one attribute value contained in the above item pool. "Contained in the above item pool" means that each attribute value in the attribute value traversal set appears in the items in the item pool, and there is at least one item in the item pool that has this attribute value in the attribute value traversal set. This setting helps to improve the efficiency of subsequent initial associated item screening. For example, in a certain extension rule corresponding to the weight attribute formulated for the food category, the attribute value traversal set is (100g, 200g, 300g). When executing the following attribute value extension, the server uses the attribute values ​​in the corresponding attribute value traversal set except the original attribute values ​​of the to-be-matched item in the corresponding target attribute as extended attribute values.

[0045] Step S103: when the target extension rule indicates that there is an extended attribute value different from the original attribute value of the target attribute of the to-be-matched item, the original attribute value and the extended attribute value of the target attribute of the to-be-matched item are combined into an attribute value set of the target attribute of the to-be-matched item.

[0046] Specifically, the above attribute value expansion steps in the first form are as follows. In the case where the target expansion rule indicates that there is an extended attribute value different from the original attribute value of the target attribute of the item to be matched (at this time, the attribute value traversal set contains other attribute values ​​other than the original attribute value of the target attribute of the item to be matched, namely, the extended attribute value), the server forms the attribute value set of the target attribute of the item to be matched with the original attribute value and the extended attribute value of the target attribute. In the case where the target expansion rule only contains the original attribute value of the corresponding target attribute of the item to be matched (at this time, the attribute value traversal set only contains the original attribute value of the target attribute of the item to be matched, and does not contain other attribute values), the original attribute value is used to form the attribute value set of the target attribute of the item to be matched. The above attribute value set contains various attribute values ​​that can meet the current business needs. Later, it can be detected whether the relevant original attribute values ​​of each initially associated item are included in the attribute value set to determine the degree of association between the initially associated item and the item to be matched.

[0047] In the second form of extension rules, the extension rules correspond to a specific attribute value of the target attribute and an extended attribute value of the target attribute on the basis of corresponding to the attribute value of the specific category and a certain target attribute. For example, in an extension rule corresponding to the weight attribute formulated for the food category, two extended attribute values ​​of 200g and 300g are configured for the specific attribute value of 100g. In this form, when the server determines that the target extension rule indicates that there is an extended attribute value different from the original attribute value of the target attribute of the item to be matched, the original attribute value and the extended attribute value of the target attribute of the item to be matched are combined into an attribute value set of the target attribute of the item to be matched; when the server determines that the target extension rule only contains the original attribute value of the corresponding target attribute of the item to be matched, the original attribute value is used to form an attribute value set of the target attribute of the item to be matched.

[0048] Step S104: Detecting initial associated items using the original attribute values ​​and attribute value sets of the category attributes of the to-be-matched item to obtain target associated items of the to-be-matched item.

[0049] Specifically, the server performs the following calculations based on the rule engine. For any initial associated item, the server determines whether the initial associated item and the item to be matched have the same original attribute value in the category attribute, and whether the original attribute value of the initial associated item in each target attribute belongs to the attribute value set of the item to be matched in the corresponding target attribute. If both judgment results are "yes", the initial associated item is determined as the target associated item of the item to be matched. If the server determines that the initial associated item and the item to be matched have different original attribute values ​​in the category attribute (indicating different categories), or the original attribute value of the initial associated item in any target attribute does not belong to the attribute value set of the item to be matched in the corresponding target attribute, the initial associated item is eliminated. In actual applications, the server can evaluate the extended attribute value or attribute value set output by the rule engine, and optimize the extended rules in the rule engine according to the evaluation results.

[0050] In the above technical scheme, in order to perform further screening on the above initial associated items to achieve higher matching accuracy while ensuring that the target associated items finally screened out reach a certain number to meet business needs, the embodiment of the present invention first outputs the original attribute values ​​of the to-be-matched items in each attribute through a text processing model used to output the above text features, and then reasonably expands the original attribute values ​​of each attribute through the expansion rules pre-set for the category attribute values ​​to obtain an attribute value set that is strongly correlated with the original attribute value and contains more elements, and finally uses the attribute value set to screen the initial associated items, thereby obtaining more target associated items to meet business needs while ensuring high-precision matching.

[0051] The following is an example of a specific embodiment of the present invention, where the items to be matched are automotive supplies. The item matching solution for automotive parts can enable consumers to obtain more related item information for optimal selection, and can enable service providers to conduct market analysis based on a variety of similar items. It is understood that the matching of automotive supplies is only an example and does not constitute any limitation to the technical solution of the present invention.

[0052] First, the server obtains the identifier of the object to be matched, and uses the identifier to query from the image library and text library to obtain the image information and text information of the object to be matched. Then, the server uses the aforementioned image processing model to obtain the image features of the object to be matched, and uses the aforementioned text processing model to obtain the text features and original attribute values ​​of the object to be matched. In order to achieve dimensional alignment of image features and text features, linear transformation or straightening operations can be performed on the image features and text features. The convolutional neural network in the image processing model can use Faster-Rcnn, and the text processing model can use BERT (Bidirectional Encoder Representations from Transformers). BERT can be fine-tuned through the training set and the test set before use. The training samples in the training set are text information of automotive products, such as "original automatic transmission oil CVT FE continuously variable transmission oil 4L", and the label can be the original attribute value of each attribute in the text (including category attributes and target attributes), such as "transmission oil" of the category attribute, "CVT FE" of the specification attribute, and "4L" of the capacity attribute. The above labels are obtained by manual annotation.

[0053] The model training hyperparameters can be set as follows: set the number of model sample training (num_train_epochs) to 4, the batch training data size (train_batch_size) to 32, the maximum text length (max_seq_length) to 128, and the learning rate (learning_rate) to 5e-5. During the model training process, the F1 value, accuracy (precision), and recall (recall) evaluated on the test set on the model are used to evaluate the model effect, and finally the model with the best training results is saved.

[0054] After that, the server can fuse the image features and text features of the to-be-matched object into multimodal features, and then fuse the multimodal features with the image features and text features into comprehensive features. After that, the server calculates the similarity based on the comprehensive features of the to-be-matched object and the objects in the object pool, and determines the initial associated objects based on the similarity threshold comparison.

[0055] The following is a screening of the initial associated items. First, the server obtains the extension rules pre-configured in the rule engine, which correspond to the category attribute value and a certain target attribute of the item to be matched. The extension method of the extension rule relative to the original attribute value can be determined based on business needs, which can be upward expansion (i.e., expansion to a higher level) or downward expansion (i.e., expansion to a lower level). For example, the upward expansion method is that the original attribute value of the grade attribute of the motor oil category is level three, and the extended attribute values ​​indicated by the extension rule are level two and level one (in this example, the motor oil grades are level one, level two, and level three from high to low). For example, the downward expansion method is that the original attribute value of the electrode material attribute of the battery category is an inert electrode, and the extended attribute value indicated by the extension rule is an active electrode. Thereafter, the server determines the attribute value set of the target attribute of the item to be matched based on the corresponding extension rule, and detects each initial associated item based on the original attribute value of the category attribute of the item to be matched and the above attribute value set, and finally obtains the target associated item of the item to be matched.

[0056] According to the technical solution of this embodiment, on the basis of optimizing the traditional matching based on image features and text features alone, a filtering matching strategy of feature fusion and attribute extraction extension is added to make the matching results more accurate and meet more practical application scenarios.

[0057] It should be noted that the collection, collection, updating, analysis, processing, use, transmission, storage and other aspects of user personal information involved in the technical solution of the present invention are in compliance with the provisions of relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. Necessary measures are taken for user personal information to prevent illegal access to user personal information data and maintain the security of user personal information, network security and national security.

[0058] For the above-mentioned method embodiments, for the convenience of description, they are expressed as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the order of the actions described, and some steps can actually be performed in other orders or simultaneously. In addition, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily necessary to implement the present invention.

[0059] In order to better implement the above-mentioned solution of the embodiment of the present invention, relevant devices for implementing the above-mentioned solution are also provided below.

[0060] See also Figure 6 As shown, the object matching device 600 provided by the embodiment of the present invention may include: a coarse-grained matching unit 601 , a rule acquisition unit 602 , an attribute extension unit 603 and a fine-grained matching unit 604 .

[0061] Among them, the coarse-grained matching unit 601 can be used to determine multiple initial associated items of the to-be-matched item from a preset item pool; the rule acquisition unit 602 can be used to: for any target attribute other than the category attribute among the multiple attributes of the to-be-matched item, obtain a target extension rule for the target attribute that is pre-configured for the original attribute value of the to-be-matched item in the category attribute; the attribute extension unit 603 can be used to: when the target extension rule indicates that there is an extended attribute value different from the original attribute value of the to-be-matched item in the target attribute, combine the original attribute value of the to-be-matched item in the target attribute and the extended attribute value to form an attribute value set of the to-be-matched item in the target attribute; the fine-grained matching unit 604 can be used to detect the initial associated items using the original attribute value of the to-be-matched item in the category attribute and the attribute value set to obtain the target associated items of the to-be-matched item.

[0062] In an embodiment of the present invention, the extension rule for any target attribute contains at least one attribute value of the target attribute contained in the item pool; and the attribute extension unit 603 can be further used to: when the target extension rule only contains the original attribute value of the to-be-matched item in the corresponding target attribute, form the attribute value set of the to-be-matched item in the target attribute with the original attribute value; when the target extension rule contains the original attribute value of the to-be-matched item in the corresponding target attribute and other attribute values, determine the other attribute values ​​as the extended attribute values.

[0063] In a specific application, the fine-grained matching unit 604 can be further used to: for any initial associated item, when the initial associated item and the item to be matched have the same original attribute value in the category attribute, and the original attribute value of the initial associated item in each target attribute belongs to the attribute value set of the item to be matched in the corresponding target attribute, determine the initial associated item as the target associated item of the item to be matched.

[0064] In practical applications, the coarse-grained matching unit 601 can be further used to: obtain image information and text information of the object to be matched; input the image information into a pre-trained image processing model to obtain foreground coordinate data and item category data in the image information, and combine the foreground coordinate data and the item category data into image features of the object to be matched; input the text information into a pre-trained text processing model to obtain text features of the object to be matched; fuse the image features and the text features into multimodal features of the object to be matched, combine the image features, the text features and the multimodal features into comprehensive features of the object to be matched, and use the comprehensive features to determine the initial associated items from the item pool.

[0065] As a preferred solution, the coarse-grained matching unit 601 may be further used to: calculate the similarity between the item to be matched and each item in the item pool according to the comprehensive features of the item to be matched and the comprehensive features of each item in the item pool, and determine the items in the item pool whose similarity is greater than a preset threshold as the initial associated items.

[0066] Preferably, the text processing model may further output the original attribute values ​​of the to-be-matched item in the multiple attributes based on the input text information; and the device 600 may further include: a verification unit, used to: after obtaining the category data to which the item belongs and the original attribute values ​​of the to-be-matched item in the category attributes, determine whether the category data to which the item belongs and the original attribute values ​​of the to-be-matched item in the category attributes are consistent; if they are consistent, execute the step of obtaining the target extension rule for the target attribute that is pre-configured for the original attribute values ​​of the to-be-matched item in the category attributes.

[0067] In addition, in an embodiment of the present invention, the image processing model includes: a convolutional neural network, a candidate region network, a region of interest pooling layer and a classification layer connected in sequence; wherein the convolutional neural network outputs primary image features based on the input image information; the candidate region network outputs the bounding box position data of the object to be matched based on the input initial image features; the region of interest pooling layer outputs the foreground coordinate data based on the input bounding box position data; and the classification layer outputs the category data to which the object belongs based on the input foreground coordinate data.

[0068] According to the technical solution of the embodiment of the present invention, the feature extraction accuracy and completeness are improved through feature fusion, and ultimately the item matching accuracy is improved. In addition, the fine-grained screening method based on attribute extension obtains more target-related items to meet business needs while ensuring high-precision matching.

[0069] Figure 7 An exemplary system architecture 700 to which the object matching method or the object matching apparatus according to the embodiment of the present invention can be applied is shown.

[0070] like Figure 7 As shown, the system architecture 700 may include terminal devices 701, 702, 703, a network 704 and a server 705 (this architecture is only an example, and the components included in the specific architecture may be adjusted according to the specific application). The network 704 is used to provide a medium for a communication link between the terminal devices 701, 702, 703 and the server 705. The network 704 may include various connection types, such as wired, wireless communication links or optical fiber cables.

[0071] The user can use the terminal devices 701, 702, 703 to interact with the server 705 through the network 704 to receive or send messages, etc. Various client applications can be installed on the terminal devices 701, 702, 703, such as item matching applications (only as an example).

[0072] The terminal devices 701 , 702 , and 703 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers.

[0073] The server 705 may be a server that provides various services, such as a backend server (for example only) that provides support for item matching applications operated by users using the terminal devices 701, 702, and 703. The backend server may process the received item matching request and feed back the processing result (for example, the matched target related item - for example only) to the terminal devices 701, 702, and 703.

[0074] It should be noted that the item matching method provided in the embodiment of the present invention is generally executed by the server 705 , and accordingly, the item matching device is generally disposed in the server 705 .

[0075] It should be understood that Figure 7 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0076] The present invention also provides an electronic device. The electronic device of an embodiment of the present invention includes: one or more processors; a storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the item matching method provided by the present invention.

[0077] Reference below Figure 8 , which shows a schematic diagram of the structure of a computer system 800 of an electronic device suitable for implementing an embodiment of the present invention. Figure 8 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0078] like Figure 8As shown, the computer system 800 includes a central processing unit (CPU) 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage part 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the computer system 800 are also stored. The CPU 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0079] The following components are connected to the I / O interface 805: an input section 806 including a keyboard, a mouse, etc.; an output section 807 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 808 including a hard disk, etc.; and a communication section 809 including a network interface card such as a LAN card, a modem, etc. The communication section 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the I / O interface 805 as needed. A removable medium 811, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 810 as needed so that a computer program read therefrom is installed into the storage section 808 as needed.

[0080] In particular, according to the embodiments disclosed in the present invention, the process described in the main step diagram above can be implemented as a computer software program. For example, an embodiment of the present invention includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the main step diagram. In the above embodiment, the computer program can be downloaded and installed from the network through the communication part 809, and / or installed from the removable medium 811. When the computer program is executed by the central processing unit 801, the above functions defined in the system of the present invention are executed.

[0081] It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer readable signal media may also be any computer readable medium other than computer readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0082] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the above-mentioned module, program segment or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0083] The units involved in the embodiments of the present invention may be implemented by software or hardware. The units described may also be set in a processor, for example, it may be described as: a processor includes a coarse-grained matching unit, a rule acquisition unit, an attribute extension unit, and a fine-grained matching unit. The names of these units do not constitute a limitation on the units themselves in some cases, for example, the coarse-grained matching unit may also be described as "a unit that provides initial associated items to the fine-grained matching unit".

[0084] As another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiment; or may exist independently and not be assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by the device, the steps executed by the device include: determining multiple initial associated items of the to-be-matched item from a preset item pool; for any target attribute other than the category attribute among the multiple attributes of the to-be-matched item, obtaining a target extension rule for the target attribute that is pre-configured for the original attribute value of the to-be-matched item in the category attribute; when the target extension rule indicates that there is an extended attribute value different from the original attribute value of the to-be-matched item in the target attribute, the original attribute value of the to-be-matched item in the target attribute and the extended attribute value are combined into a set of attribute values ​​of the to-be-matched item in the target attribute; using the original attribute value of the to-be-matched item in the category attribute and the attribute value set to detect the initial associated item, so as to obtain the target associated item of the to-be-matched item.

[0085] In the technical solution of the embodiment of the present invention, the feature extraction accuracy and completeness are improved through feature fusion, which ultimately improves the item matching accuracy. In addition, the fine-grained screening method based on attribute extension obtains more target-related items to meet business needs while ensuring high-precision matching.

[0086] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An object matching method, characterized in that: include: Determine multiple initial associated items of the to-be-matched item from a preset item pool; For any target attribute other than the category attribute among the multiple attributes of the to-be-matched object, obtaining a target expansion rule for the target attribute that is pre-configured for the original attribute value of the to-be-matched object in the category attribute; In the case where the target extension rule indicates that there is an extended attribute value different from the original attribute value of the to-be-matched item at the target attribute, the original attribute value of the to-be-matched item at the target attribute and the extended attribute value are combined into an attribute value set of the to-be-matched item at the target attribute; The initial associated object is detected using the original attribute value of the to-be-matched object in the category attribute and the attribute value set to obtain the target associated object of the to-be-matched object.

2. The method according to claim 1, characterized in that: The expansion rule for any target attribute contains at least one attribute value of the target attribute contained in the item pool; and the method further comprises: In the case where the target expansion rule only contains the original attribute value of the to-be-matched object at the corresponding target attribute, the attribute value set of the to-be-matched object at the target attribute is formed by the original attribute value; In the case where the target extension rule contains the original attribute value of the to-be-matched object at the corresponding target attribute and other attribute values, the other attribute value is determined as the extended attribute value.

3. The method according to claim 1, characterized in that The detecting the initial associated object by using the original attribute value of the to-be-matched object in the category attribute and the attribute value set to obtain the target associated object of the to-be-matched object includes: For any initial associated item, when the initial associated item and the item to be matched have the same original attribute value in the category attribute, and the original attribute value of the initial associated item in each target attribute belongs to the attribute value set of the item to be matched in the corresponding target attribute, the initial associated item is determined as the target associated item of the item to be matched.

4. The method according to claim 1, characterized in that The step of determining a plurality of initial associated items of the to-be-matched item from the preset item pool includes: Acquire image information and text information of the object to be matched; Input the image information into a pre-trained image processing model to obtain foreground coordinate data and item category data in the image information, and combine the foreground coordinate data and item category data as image features of the item to be matched; Inputting the text information into a pre-trained text processing model to obtain text features of the object to be matched; The image features and the text features are fused into multimodal features of the to-be-matched object, the image features, the text features and the multimodal features are combined into comprehensive features of the to-be-matched object, and the initial associated object is determined from the object pool using the comprehensive features.

5. The method according to claim 4, characterized in that The determining the initial associated items from the item pool by using the comprehensive features includes: The similarity between the to-be-matched object and each object in the object pool is calculated according to the comprehensive characteristics of the to-be-matched object and the comprehensive characteristics of each object in the object pool, and the objects in the object pool having a similarity greater than a preset threshold are determined as the initial associated objects.

6. The method according to claim 4, characterized in that The text processing model further outputs the original attribute values ​​of the to-be-matched item in the multiple attributes based on the input text information; and the method further comprises: After obtaining the category data to which the item belongs and the original attribute value of the item to be matched in the category attribute, determining whether the category data to which the item belongs and the original attribute value of the item to be matched in the category attribute are consistent; If they are consistent, the step of obtaining the target expansion rule for the target attribute, which is pre-configured for the original attribute value of the to-be-matched item in the category attribute, is performed.

7. The method according to claim 4, characterized in that The image processing model includes: a convolutional neural network, a candidate region network, a region of interest pooling layer and a classification layer connected in sequence; wherein, The convolutional neural network outputs primary image features based on the input image information; The candidate region network outputs the bounding box position data of the to-be-matched object based on the input initial features of the image; The region of interest pooling layer outputs the foreground coordinate data based on the input bounding box position data; The classification layer outputs the category data to which the object belongs based on the input foreground coordinate data.

8. An object matching device, characterized in that: include: A coarse-grained matching unit, used for determining a plurality of initial associated items of the to-be-matched item from a preset item pool; A rule acquisition unit, configured to: for any target attribute other than the category attribute among the multiple attributes of the to-be-matched object, acquire a target expansion rule for the target attribute that is pre-configured for the original attribute value of the to-be-matched object in the category attribute; an attribute extension unit, configured to: when the target extension rule indicates that there is an extended attribute value different from the original attribute value of the to-be-matched item at the target attribute, combine the original attribute value of the to-be-matched item at the target attribute and the extended attribute value to form an attribute value set of the to-be-matched item at the target attribute; The fine-grained matching unit is used to detect the initial associated items by using the original attribute value of the to-be-matched item in the category attribute and the attribute value set to obtain the target associated items of the to-be-matched item.

9. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images