Model training, item matching method and device, electronic equipment and storage medium

Abstract

Embodiments of the present application disclose a model training method, an article matching method, a device, an electronic device and a storage medium. The model training method comprises: taking a training image and a training text of a training article as a set of training samples, training an original feature extraction model based on the obtained multiple sets of training samples to obtain an intermediate feature extraction model; taking a test image and a test text of a test article as a set of test samples, inputting the obtained multiple sets of test samples into the intermediate feature extraction model to obtain a first matching degree set; inputting the multiple sets of training samples into the intermediate feature extraction model to obtain a second matching degree set; screening the multiple sets of training samples based on the first matching degree set and the second matching degree set, and training the original feature extraction model based on the screened training samples to obtain a target feature extraction model. The technical solution of the embodiments of the present application can improve the accuracy of article matching.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of computer application, and particularly to a model training method, an item matching method, a device, an electronic device and a storage medium. BACKGROUND

[0002] Item matching technology has a wide range of application scenarios in e-commerce platforms, such as item price comparison, same item identification, and photo shopping. As can be seen, the accuracy of item matching is crucial for e-commerce platforms.

[0003] In the process of implementing the present application, the inventors have found that the existing item matching technology cannot achieve good matching accuracy and needs to be improved. SUMMARY

[0004] Embodiments of the present application provide a model training method, an item matching method, a device, an electronic device and a storage medium to improve the accuracy of item matching.

[0005] According to an aspect of the present application, a model training method is provided, which can include:

[0006] The training image and the training text of the training item are taken as a set of training samples, and the original feature extraction model is trained based on the obtained multiple sets of training samples to obtain an intermediate feature extraction model;

[0007] The inspection image and the inspection text of the inspection item are taken as a set of inspection samples, and the multiple sets of inspection samples obtained are input into the intermediate feature extraction model to obtain a first matching degree set;

[0008] The multiple sets of training samples are input into the intermediate feature extraction model to obtain a second matching degree set;

[0009] Based on the first matching degree set and the second matching degree set, the multiple sets of training samples are screened, and the original feature extraction model is trained based on the screened training samples to obtain a target feature extraction model;

[0010] The first matching degree set is used to represent the matching degree between each inspection image and each inspection text in the multiple sets of inspection samples, and the second matching degree set is used to represent the matching degree between the training image and the training text within each set of training samples in the multiple sets of training samples.

[0011] According to another aspect of the present application, an item matching method is provided, which can include:

[0012] Obtaining the matching information of the to-be-matched item, and the target feature extraction model obtained by training according to the model training method provided in any embodiment of the present application;

[0013] input the to-be-matched information to the target feature extraction model to obtain to-be-matched features, wherein the to-be-matched features include to-be-matched image features and / or to-be-matched text features;

[0014] match the to-be-matched features with candidate features of at least one candidate item respectively to determine a target item matched with the to-be-matched item from the at least one candidate item, wherein the candidate features include candidate image features and / or candidate text features.

[0015] According to another aspect of the present application, a model training device can include:

[0016] a coarse training module configured to take training images and training texts of training items as a plurality of sets of training samples, and train an original feature extraction model based on the obtained sets of training samples to obtain an intermediate feature extraction model;

[0017] a first input module configured to take inspection images and inspection texts of inspection items as a plurality of sets of inspection samples, and input the obtained sets of inspection samples to the intermediate feature extraction model to obtain a first matching degree set;

[0018] a second input module configured to input the sets of training samples to the intermediate feature extraction model to obtain a second matching degree set;

[0019] a fine training module configured to filter the sets of training samples based on the first matching degree set and the second matching degree set, and train the original feature extraction model based on the filtered sets of training samples to obtain a target feature extraction model;

[0020] wherein the first matching degree set is used to represent matching degrees between each inspection image and each inspection text in the sets of inspection samples, and the second matching degree set is used to represent matching degrees between training images and training texts in each set of training samples.

[0021] According to another aspect of the present application, a model training device can include:

[0022] a model obtaining module configured to obtain to-be-matched information of a to-be-matched item, and a target feature extraction model trained according to the model training method provided in any of the embodiments of the present application;

[0023] a model application module configured to input the to-be-matched information to the target feature extraction model to obtain to-be-matched features, wherein the to-be-matched features include to-be-matched image features and / or to-be-matched text features;

[0024] The item matching module is configured to match the to-be-matched feature with candidate features of at least one candidate item respectively, to determine a target item matched with the to-be-matched item from the at least one candidate item, wherein the candidate features include candidate image features and / or candidate text features.

[0025] According to another aspect of the present application, there is provided an electronic device, which can include:

[0026] at least one processor; and

[0027] a memory communicatively connected to the at least one processor; wherein

[0028] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to cause the at least one processor to implement the model training method or the item matching method provided by any of the embodiments of the present application when executed.

[0029] According to another aspect of the present application, there is provided a computer readable storage medium having stored thereon computer instructions for causing a processor to implement the model training method or the item matching method provided by any of the embodiments of the present application when executed.

[0030] The technical solution of the embodiments of the present application trains the training image and the training text of the training item as a group of training samples, and trains the original feature extraction model based on the obtained multiple groups of training samples to obtain an intermediate feature extraction model. Then, the inspection image and the inspection text of the inspection item are taken as a group of inspection samples, and the obtained multiple groups of inspection samples are input into the intermediate feature extraction model to obtain a first matching degree set. In addition, the multiple groups of training samples are input into the intermediate feature extraction model to obtain a second matching degree set. Further, based on the first matching degree set and the second matching degree set, the multiple groups of training samples are screened, and the original feature extraction model is trained based on the screened training samples to obtain a target feature extraction model applied in the subsequent item matching process. The above technical solution can effectively avoid the influence of low-quality training samples on the model effect through the cooperation of the coarse-fine two-round training and the automatic screening of high-quality training samples, thereby improving the effectiveness of item representation and ensuring the accuracy of the subsequent application model in item matching.

[0031] It should be understood that the content described in this section is not intended to identify key or important features of the embodiments of the present application, nor is it used to limit the scope of the present application. Other features of the present application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0032] In order to make the technical solutions in the embodiments of the present application clearer, the following will briefly introduce the drawings needed in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative effort based on these drawings.

[0033] Figure 1 is a flow chart of a model training method according to an embodiment of the present application;

[0034] Figure 2 is a flow chart of another model training method according to an embodiment of the present application;

[0035] Figure 3 is a flow chart of another model training method according to an embodiment of the present application;

[0036] Figure 4 is a schematic diagram of a model training example in another model training method according to an embodiment of the present application;

[0037] Figure 5 is a flow chart of an article matching method according to an embodiment of the present application;

[0038] Figure 6 is a schematic diagram of a model application example in an article matching method according to an embodiment of the present application;

[0039] Figure 7 is a structural block diagram of a model training device according to an embodiment of the present application;

[0040] Figure 8 is a structural block diagram of an article matching device according to an embodiment of the present application;

[0041] Figure 9 is a structural schematic diagram of an electronic device implementing the model training method or the article matching method according to an embodiment of the present application. DETAILED DESCRIPTION

[0042] In order to make the technical solutions in the embodiments of the present application clearer, the following will briefly introduce the drawings needed in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative effort based on these drawings.

[0043] It is to be understood that the terms "first", "second", and the like used in the description and the claims of the present application as well as the above-described drawings do not necessarily have to connote any ordinal, sequential or chronological order, but are merely used for the purpose of nomenclature. It is to be understood that the terms "target", "original" and the like are used in the same sense as the terms "first", "second" and the like, and are not repeated here. Furthermore, the terms "comprising", "having", "including", and the like, as well as any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, product, or apparatus that comprises, has, includes, or the like, a list of steps or units, does not necessarily comprise, have, include, or the like, only those steps or units expressly listed, but can include other steps or units not expressly listed or inherent to such process, method, product, or apparatus.

[0044] Figure 1 is a flowchart of a model training method provided by an embodiment of the present application. The embodiment is applicable to the case of training a target feature extraction model that can be used to extract multi-modal features, and is particularly applicable to the case of obtaining a target feature extraction model for extracting multi-modal features by means of coarse-fine two-round multi-modal training and low-quality training sample automatic screening technology. The method can be executed by a model training device provided by an embodiment of the present application. The device can be implemented in the form of software and / or hardware. The device can be integrated on an electronic device. The electronic device can be various user terminals or servers.

[0045] Referring to Figure 1 , the method of the embodiment specifically comprises the following steps:

[0046] S110, training images and training texts of training items are taken as a set of training samples, and an original feature extraction model is trained based on the obtained multiple sets of training samples to obtain an intermediate feature extraction model.

[0047] The training items can be understood as items used for model training, and on this basis, they can also be understood as items used for model verification. The training images can be understood as images used to represent the training items, i.e., images containing the training items, especially images taking the training items as the subject. The training texts can be understood as texts used to represent the training items, such as item titles or item attributes, etc. The training images and the training texts are taken as a set of training samples, from which multiple sets of training samples can be obtained, including training images and training texts of multiple training items.

[0048] The original feature extraction model can be understood as a model to be trained for implementing the function of multi-modal feature extraction, specifically a model for implementing image feature extraction and text feature extraction. Therefore, the multi-modal here can be understood as an image model and a text model. The original feature extraction model is trained based on multiple sets of training samples to obtain an intermediate feature extraction model. This round of training can be referred to as coarse training.

[0049] It should be noted that in actual applications, there can be one or more low-quality training samples in the multiple sets of training samples. The low-quality training sample can be understood as a training sample in which the training image and / or the training text cannot accurately represent the training item. The above situation often occurs on e-commerce platforms, because the training image and the training text are usually edited by the merchant for the training item, and there can be editing errors. According to practical experience, these low-quality training samples will directly affect the model effect of the intermediate feature extraction model trained therefrom, thereby affecting the accuracy of item matching when the model is applied to item matching.

[0050] To solve the above problem, the intermediate feature extraction model can be tested by the following steps, and then based on the obtained test results, high-quality training samples are selected from the multiple sets of training samples, that is, training samples in which the training image and the training text can accurately represent the training item are selected, and then model training can be performed based on the selected high-quality training samples, thereby obtaining a target feature extraction model capable of extracting multi-modal features that effectively represent the training item. Specifically,

[0051] S120, the test image and the test text of the test item are taken as a set of test samples, and the obtained multiple sets of test samples are input into the intermediate feature extraction model to obtain a first matching degree set.

[0052] The first matching degree set is used to represent the matching degree between each test image and each test text in the multiple sets of test samples.

[0053] The test article can be understood as an article used to test the model effect of the intermediate feature extraction model. The test image can be understood as an image used to represent the test article, i.e., an image containing the test article, especially an image taking the test article as the subject. The test text can be understood as text used to represent the test article, which can be, for example, an article title or an article attribute, etc. The test image and the test text are taken as a set of test samples, and thus a plurality of sets of test samples can be obtained, which include test images and test texts of a plurality of test articles. In actual application, optionally, to ensure the accuracy of the training sample screening, the test image and the test text in the test sample can accurately represent the test article; and further optionally, for a plurality of test articles and a plurality of training articles, there can be partially overlapping articles or completely different articles, which can be set according to actual conditions and is not limited here.

[0054] The plurality of sets of test samples are input into the intermediate feature extraction model, so that a first matching degree set can be obtained according to the output result of the intermediate feature extraction model, i.e., the feature extraction result for the plurality of sets of test samples.

[0055] It should be noted that the first matching degree set can be used to represent the matching degree between each test image and each test text in the plurality of sets of test samples. For example, the plurality of sets of test samples can include test sample 1, test sample 2 and test sample 3, the test sample 1 can include a test image 1 and a test text 1 of a test article 1, the test sample 2 can include a test image 2 and a test text 2 of a test article 2, and the test sample 3 can include a test image 3 and a test text 3 of a test article 3. Then, the six first matching degrees in the first matching degree set can be used to represent the matching degree between the test image 1 and the test text 1, the matching degree between the test image 1 and the test text 2, the matching degree between the test image 1 and the test text 3, the matching degree between the test image 2 and the test text 2, the matching degree between the test image 2 and the test text 3, and the matching degree between the test image 3 and the test text 3, respectively.

[0056] In S130, the plurality of sets of training samples are input into the intermediate feature extraction model to obtain a second matching degree set.

[0057] The second matching degree set is used to represent the matching degree between the training image and the training text in each set of training samples in the plurality of sets of training samples.

[0058] In order to screen high-quality training samples from the plurality of sets of training samples, the plurality of sets of training samples can also be input into the intermediate feature extraction model, so that a second matching degree set can be obtained according to the output result of the intermediate feature extraction model, i.e., the feature extraction result for the plurality of sets of training samples.

[0059] It should be noted that the second matching degree set can be used to represent the matching degree between the training image and the training text in each of the plurality of training samples. For example, similar to the above example, the plurality of training samples can include training sample 1, training sample 2 and training sample 3, training sample 1 includes training image 1 and training text 1 of training item 1, training sample 2 includes training image 2 and training text 2 of training item 2, and training sample 3 includes training image 3 and training text 3 of training item 3, then the three first matching degrees in the second matching degree set can be used to represent the matching degree between training image 1 and training text 1, the matching degree between training image 2 and training text 2, and the matching degree between training image 3 and training text 3, respectively.

[0060] S140, based on the first matching degree set and the second matching degree set, screening the plurality of training samples, and training the original feature extraction model based on the screened training samples to obtain the target feature extraction model.

[0061] According to the above description, the first matching degree set can represent the matching degree between the matched (i.e. corresponding to the same test item) test image and test text, and the matching degree between the unmatched (i.e. not corresponding to the same test item) test image and test text. By comparing the two, the basic level of the matching degree between the matched test image and test text can be obtained.

[0062] On this basis, since the second matching degree set can reflect the matching degree between the matched training image and training text, by combining the first matching degree set and the second matching degree set, high-quality training samples can be selected from the plurality of training samples, for example, training samples with a matching degree between the training image and the training text above the basic level are selected as high-quality training samples.

[0063] Further, based on the screened training samples, the original feature extraction model is retrained to obtain a target feature extraction model that can extract effective multi-modal features representing the training items, thereby ensuring the accuracy of the subsequent application model in matching items. This round of training can be called fine-tuning.

[0064] The technical solution of this invention involves using training images and training text of training items as a set of training samples, and training an original feature extraction model based on the obtained multiple sets of training samples to obtain an intermediate feature extraction model. Then, testing images and testing text of testing items are used as a set of testing samples, and inputting these multiple sets of testing samples into the intermediate feature extraction model to obtain a first matching degree set. Furthermore, multiple sets of training samples are input into the intermediate feature extraction model to obtain a second matching degree set. Further, based on the first and second matching degree sets, multiple sets of training samples are selected, and the original feature extraction model is trained based on the selected training samples to obtain the final target feature extraction model applied in the item matching process. This technical solution, through the cooperation of coarse and fine training rounds and the automatic selection of high-quality training samples, effectively avoids the impact of low-quality training samples on the model's performance, thereby improving the effectiveness of item representation and ensuring the accuracy of subsequent item matching applications.

[0065] An alternative technical solution, the above model training method, further includes:

[0066] For each target category among the multiple item categories associated with the item library, sample the category items in the item library that are located under the target category to obtain sampled items, so as to select multiple sets of training samples corresponding to training items from all the obtained sampled items.

[0067] Specifically, for all items stored in the item library belonging to multiple item categories, the target category can be understood as the item category that is currently active among these item categories. In practical applications, optionally, the item category to which the active items belong among all items can be used as the target category. The number of target categories can be one, two, or more, depending on the actual situation, and no specific limitation is made here.

[0068] For each target category, samples are taken from all items belonging to that target category, resulting in one or more sampled items, thus obtaining sampled items for different target categories. Further, multiple sets of training samples are selected from the obtained sampled items, each corresponding to a training sample. For example, a predetermined proportion of training items can be selected from the obtained sampled items to construct training samples. Further, optionally, sampled items other than training items can be used as test items to construct test samples.

[0069] The above technical solution, by sampling items across all target categories to construct training samples for model training, can basically cover the training samples of most item categories, making the target feature extraction model trained in this way have good versatility. Moreover, this versatility also means that when new categories are added in the future, there is no need to retrain the model. The target feature extraction model trained above can still effectively represent the multimodal features of items in the new categories, with good scalability, and can meet actual business needs.

[0070] Figure 2 This is a flowchart of another model training method provided in this embodiment of the invention. This embodiment is based on and optimized from the above-described technical solutions. In this embodiment, optionally, multiple sets of training samples are screened based on a first matching degree set and a second matching degree set, including: determining a matching degree screening threshold based on each first matching degree in the first matching degree set; and screening multiple sets of training samples based on the matching degree screening threshold and the second matching degree corresponding to each set of training samples in the second matching degree set. The explanations of terms that are the same as or corresponding to those in the above embodiments are not repeated here.

[0071] See Figure 2 The method in this embodiment may specifically include the following steps:

[0072] S210. Take the training images and training text of the training items as a set of training samples, and train the original feature extraction model based on the obtained multiple sets of training samples to obtain the intermediate feature extraction model.

[0073] S220. Take the inspection images and inspection text of the inspected items as a set of inspection samples, and input the obtained multiple sets of inspection samples into the intermediate feature extraction model to obtain the first matching degree set.

[0074] The first matching degree set is used to characterize the matching degree between each test image and each test text in multiple test samples.

[0075] S230. Input multiple sets of training samples into the intermediate feature extraction model to obtain the second matching degree set.

[0076] The second matching degree set is used to characterize the matching degree between the training image and the training text in each group of training samples.

[0077] S240. Based on each first matching degree in the first matching degree set, determine the matching degree filtering threshold.

[0078] As explained above, based on the first matching degree set, the basic level of matching degree between the matched test image and the test text can be obtained. This basic level is called the matching degree screening threshold, which is the threshold used to screen multiple sets of training samples from the perspective of matching degree.

[0079] S250. Based on the matching degree screening threshold and the second matching degree corresponding to each group of training samples in the second matching degree set, multiple groups of training samples are screened.

[0080] The second matching degree can be used to characterize the matching degree between training images and training text for the same training item. Therefore, multiple sets of training samples can be filtered based on the matching degree screening threshold, i.e., each second matching degree in the second matching degree set. In practical applications, optionally, for the second matching degree corresponding to each set of training samples in the second matching degree set, if the second matching degree is greater than the matching degree screening threshold, the training sample can be retained to realize the screening process of multiple sets of training samples. That is to say, if the second matching degree is greater than the matching degree screening threshold, it means that the training image and training text in the training sample corresponding to the second matching degree are likely to correspond to the same training item, that is, the training image and training text can accurately characterize the training item, so the training sample is a high-quality training sample and can be retained.

[0081] S260. Based on the selected training samples, the original feature extraction model is trained to obtain the target feature extraction model.

[0082] The technical solution of this invention determines the matching degree screening threshold through a first matching degree set, and then screens multiple sets of training samples based on the matching degree screening threshold and a second matching degree set, thereby achieving the effect of accurate screening of high-quality training samples.

[0083] Based on this, an optional technical solution is to determine the matching degree filtering threshold based on each first matching degree in the first matching degree set, including:

[0084] For each first matching degree in the first matching degree set, the matching ground value corresponding to the first matching degree is obtained. The matching ground value can be used to characterize the similarity between the test item represented by the test image corresponding to the first matching degree and the test item represented by the test text.

[0085] Based on each first matching degree in the first matching degree set, and the matching truth value corresponding to each first matching degree in the first matching degree set, the matching degree filtering threshold is obtained.

[0086] Specifically, for each first matching degree in the first matching degree set, the true value of the matching degree can be used to characterize whether the inspection item represented by the inspection image and the inspection item represented by the inspection text are the same inspection item. To better understand this concept, let's continue with the above example. For instance, the true value of the matching degree corresponding to the first matching degree of inspection image 1 and inspection text 1 is 1, because the inspection items represented by inspection image 1 and inspection text 1 are both inspection item 1; as another example, the true value of the matching degree corresponding to the first matching degree of inspection image 1 and inspection text 2 is 0, because the inspection item represented by inspection image 1 is inspection item 1 and the inspection item represented by inspection text 2 is inspection item 2, and the two are not the same inspection item.

[0087] Typically, the first matching degree between inspection images and inspection texts corresponding to different inspection items is lower than the first matching degree between inspection images and inspection texts corresponding to the same inspection item. Therefore, the matching degree screening threshold can be obtained based on each first matching degree in the first matching degree set, and the matching truth value corresponding to each of these first matching degrees. The above technical solution achieves accurate determination of the matching degree screening threshold.

[0088] Based on this, optionally, according to each first matching degree in the first matching degree set, and the matching truth value corresponding to each first matching degree in the first matching degree set, the matching degree filtering threshold is obtained, including:

[0089] Sort the first matching degrees in the first matching degree set to obtain the sorting result;

[0090] Based on the true match values ​​corresponding to each first matching degree in the ranking results, calculate the cumulative accuracy corresponding to each first matching degree in the ranking results.

[0091] Based on the pre-set cumulative accuracy threshold and cumulative accuracy, the matching degree filtering threshold is obtained from each first matching degree in the sorting results.

[0092] Generally, a higher first match degree indicates a greater likelihood that the corresponding test image and test text represent the same test item, meaning a higher probability that its corresponding true match value is true. Therefore, the first match degrees in the set can be sorted, and then the cumulative accuracy corresponding to each first match degree can be calculated based on its true match value in the sorting results. Then, a match degree filtering threshold is obtained from these first match degrees based on a pre-set cumulative accuracy threshold. For example, when the first match degrees are sorted in descending order, the cumulative accuracy can be calculated from top to bottom, and the first match degree corresponding to the first cumulative accuracy value less than the cumulative accuracy threshold is used as the match degree filtering threshold.

[0093] Figure 3 This is a flowchart of another model training method provided in this embodiment of the invention. This embodiment is based on the above-described technical solutions and optimized. In this embodiment, optionally, the original feature extraction model includes an original image feature extraction network and an original text feature extraction network. The original feature extraction model is trained based on multiple sets of training samples to obtain an intermediate feature extraction model, including: inputting each training image from the multiple sets of training samples into the original image feature extraction network, and inputting each training text into the original text feature extraction network; using contrastive learning loss, calculating the loss on the output results of the original image feature extraction network and the output results of the original text feature extraction network, and adjusting the network parameters in the original feature extraction model according to the obtained loss calculation results to obtain the intermediate feature extraction model. The explanations of terms that are the same as or corresponding to those in the above embodiments are not repeated here.

[0094] See Figure 3 The method in this embodiment may specifically include the following steps:

[0095] S310. Obtain the original feature extraction model, which includes the original image feature extraction network and the original text feature extraction network.

[0096] The original image feature extraction network can be understood as a network used to extract image features. Similarly, the original text feature extraction network can be understood as a network used to extract text features. The two feature extraction networks are two branches of the original feature extraction model.

[0097] S320. Take the training images and training text of the training items as a set of training samples to obtain multiple sets of training samples.

[0098] S330. Input each training image from multiple training samples into the original image feature extraction network, and input each training text into the original text feature extraction network.

[0099] In this process, each training image from multiple training samples is input into the original image feature extraction network, so that the original image feature extraction network can extract features from these training images respectively.

[0100] Similarly, each training text from multiple training samples is input into the original text feature extraction network to extract features from these training texts.

[0101] S340. Using contrastive learning loss, calculate the loss on the output of the original image feature extraction network and the output of the original text feature extraction network, and adjust the network parameters in the original feature extraction model according to the obtained loss calculation results to obtain the intermediate feature extraction model.

[0102] In this embodiment of the invention, self-supervised training is performed on the original feature extraction model. Therefore, contrastive learning loss can be used to calculate the loss on the outputs of the two feature extraction networks to adjust their network parameters, thereby training an intermediate feature extraction model. In practical applications, the contrastive learning loss can optionally be implemented using InfoLoss, InfoNCE, MOCO, or SimCLR, etc., without specific limitations.

[0103] S350. Take the inspection images and inspection text of the inspected items as a set of inspection samples, and input the obtained multiple sets of inspection samples into the intermediate feature extraction model to obtain the first matching degree set.

[0104] The first matching degree set is used to characterize the matching degree between each test image and each test text in multiple test samples.

[0105] S360. Input multiple sets of training samples into the intermediate feature extraction model to obtain the second matching degree set.

[0106] The second matching degree set is used to characterize the matching degree between the training image and the training text in each group of training samples.

[0107] S370. Based on the first matching degree set and the second matching degree set, multiple sets of training samples are selected, and the original feature extraction model is trained based on the selected training samples to obtain the target feature extraction model.

[0108] The technical solution of this invention uses existing training images and training text to self-supervised train the original feature extraction model. This eliminates the need to rely on labeled data during model training, thereby reducing model training costs and improving model training efficiency. Moreover, it directly uses the overall representation of the training image to represent the training item, that is, it directly extracts the overall features of the training image, rather than first detecting the main area where the training item is located in the training image and then using the main area to represent the training item for model training. This solves the problem of being unable to match and search due to incorrect subject detection.

[0109] Based on this, an optional technical solution is that the intermediate feature extraction model includes an intermediate image feature extraction network and an intermediate text feature extraction network;

[0110] The obtained multiple sets of test samples are input into the intermediate feature extraction model to obtain the first matching degree set, which includes:

[0111] For each test image in the multiple sets of test samples obtained, the test image is input into the intermediate image feature extraction network to obtain the test image features;

[0112] For each test text in multiple test samples, the test text is input into the intermediate text feature extraction network to obtain the test text features;

[0113] For each obtained test image feature and each test text feature, calculate the first matching degree between the test image feature and the test text feature;

[0114] Based on the calculated first matching degree, a first matching degree set is obtained.

[0115] The intermediate image feature extraction network can be understood as a network obtained by training the original image feature extraction network with multiple sets of training samples. Based on the intermediate image feature extraction network, each test image in multiple sets of test samples is processed to obtain the test image features corresponding to each test image.

[0116] Similarly, the intermediate text feature extraction network can be understood as a network obtained by training the original text feature extraction network with multiple sets of training samples. Based on the intermediate text feature extraction network, each test text in multiple sets of test samples is processed separately to obtain the test text features corresponding to each test text.

[0117] Cross - calculate the matching degrees for each inspected image feature and each inspected text feature to obtain multiple first matching degrees. Exemplarily, continuing with the above example of six first matching degrees, calculate the first matching degree between the inspected image feature of inspected image 1 and the inspected text feature of inspected text 1, the first matching degree between the inspected image feature of inspected image 1 and the inspected text feature of inspected text 2, the first matching degree between the inspected image feature of inspected image 1 and the inspected text feature of inspected text 3, the first matching degree between the inspected image feature of inspected image 2 and the inspected text feature of inspected text 2, the first matching degree between the inspected image feature of inspected image 2 and the inspected text feature of inspected text 3, and the first matching degree between the inspected image feature of inspected image 3 and the inspected text feature of inspected text 3. Further, obtain a first matching degree set based on the multiple first matching degrees obtained above.

[0118] To better understand each of the above model training schemes as a whole, the following will provide an exemplary illustration with specific examples. Exemplarily, refer to Figure 4 :

[0119] 1) Sampling of commodity multi - modal data:

[0120] Randomly sample n commodities in each third - level category of all active commodities in the commodity library. In practical applications, for third - level categories with fewer than n commodities, all their commodities can be sampled. For each sampled commodity, extract the main commodity image and the commodity title of this commodity respectively, and form a multi - modal data pair with them.

[0121] At this point, assume that a total of N multi - modal data pairs (i.e., samples) are obtained. Select a preset proportion p of the samples (0 < p < 1) as the training sample set, and the remaining samples as the test sample set. Denote the training sample set as T, and its number as N T ; and denote the test sample set as V, and its number as N V .

[0122] 2) Multi - modal self - supervised rough training:

[0123] Input the main commodity images and commodity titles of the training sample set T into the Figure 4 image feature extraction branch and text feature extraction branch in it, and use contrastive learning Loss to perform multi - modal self - supervised pre - training based on two - stream. In this example, the Wenlan two - stream multi - modal training framework is adopted.

[0124] 3) Statistics of matching degree screening thresholds:

[0125] The multimodal model (i.e., the intermediate feature extraction model mentioned above) M obtained through coarse training is used to perform cross-calculation of the matching degree between the main product image and the product title in the test sample set V. The specific steps are as follows: i: Extract the image feature vector of each product's main product image in the test sample set V through the image feature extraction branch of the multimodal model M, totaling N. V Let F be a vector. img ;

[0126] ii: For each product title in the test sample set V, extract text feature vectors using the text feature extraction branch of the multimodal model M, totaling N. V Let F be a vector. txt ;

[0127] iii: Calculate the first similarity between any image feature vector and any text feature vector, and label the value (i.e., the true match), totaling N. v *N v The calculation formula is as follows:

[0128]

[0129]

[0130] Where m and n represent the m-th and n-th items in the test sample set V, respectively;

[0131] Iv: For all matching results (i.e., the second matching degree set mentioned above), sort them in descending order of the second matching degree, and calculate the cumulative accuracy from top to bottom. The calculation formula is as follows:

[0132]

[0133] IIV: Select the position k of the first Accuracy less than p (set to 0.95 in this example), and use the Similarity at position k as the matching degree filtering threshold, denoted as s.

[0134] 4) Selection of training sample set T:

[0135] i: Calculate the first matching degree between each product main image and its corresponding product title in the training sample set T in the manner of step 3);

[0136] ii: Select the training samples in the training sample set T whose first matching degree is greater than s, and form a new training sample set T'.

[0137] 5) Multimodal self-supervised fine training:

[0138] Following the method in step 2), multimodal self-supervised training is performed using the new training sample set T' to obtain a new multimodal model M'.

[0139] Figure 5 This is a flowchart of an item matching method provided in an embodiment of the present invention. This embodiment is applicable to item matching situations. The method can be executed by the item matching device provided in this embodiment of the present invention. The device can be implemented by software and / or hardware, and can be integrated into an electronic device, which can be various user terminals or servers.

[0140] See Figure 5 The method of this invention specifically includes the following steps:

[0141] S410. Obtain the matching information of the item to be matched, and the target feature extraction model trained according to the model training method provided in any embodiment of the present invention.

[0142] Here, "item to be matched" can be understood as an item to be matched, specifically, an item to be matched among at least one candidate item. "Matching information" can be understood as detailed information about the item to be matched, specifically it can be an image to be matched and / or text to be matched, without specific limitations. The matching information and the target feature extraction model trained according to any of the above technical solutions are then obtained.

[0143] S420. Input the information to be matched into the target feature extraction model to obtain the features to be matched, wherein the features to be matched include the image features to be matched and / or the text features to be matched.

[0144] In this process, the information to be matched is input into the target feature extraction model to obtain the features to be matched. In practical applications, optionally, when the information to be matched is an image to be matched, the features to be matched can be the features of the image to be matched; alternatively, when the information to be matched is text to be matched, the features to be matched can be the features of the text to be matched; alternatively, when the information to be matched is both an image to be matched and text to be matched, the features to be matched can be both the features of the image to be matched and the features of the text to be matched.

[0145] S430. The feature to be matched is matched with the candidate features of at least one candidate item to determine the target item that matches the item to be matched from at least one candidate item, wherein the candidate features include candidate image features and / or candidate text features.

[0146] Specifically, for each of the at least one candidate items, the candidate features of that candidate item can be obtained by feature extraction from the candidate information of that candidate item, which may include candidate image features and / or candidate text features. It should be noted that the type of information to be matched and the type of candidate information can be exactly the same, partially the same, or completely different. This is because the training images and training text have been aligned during model training. Therefore, during model application, it is possible to match candidate images with the image to be matched, match candidate text with the image to be matched, match candidate images with the text to be matched, match candidate text with the text to be matched, and match candidate images and candidate text with the image to be matched and the text to be matched.

[0147] The feature to be matched is matched with the candidate features of at least one candidate item to determine the target item that matches the feature to be matched from at least one candidate item.

[0148] The technical solution of this invention involves inputting the matching information of the item to be matched into a target feature extraction model to obtain the matching features; then, the matching features are matched with the candidate features of at least one candidate item to determine the target item that matches the item to be matched from at least one candidate item. This technical solution improves the accuracy of item matching by applying a target feature extraction model that can effectively represent items to process the matching information.

[0149] An optional technical solution, the above-mentioned item matching method, may further include: for each of at least one candidate item, obtaining candidate information of the candidate item, and inputting the candidate information into a target feature extraction model to obtain candidate features of the candidate item. By processing the information to be matched and the candidate information through the same feature extraction model, the consistency of the matching benchmark can be guaranteed when matching the obtained features to be matched with the candidate features.

[0150] To better understand the various item matching schemes described above, specific examples are provided below for illustration. For examples, see [link to example]. Figure 6 :

[0151] 1) Extracting and storing product main image features from the database:

[0152] All product main images in the product main image base are input into the image feature extraction branch of the multimodal model M', and the obtained image features are stored in the product feature base.

[0153] 2) Image feature extraction and matching of the products to be matched:

[0154] The main image of the product to be matched is input into the image feature extraction branch of the multimodal model M', and the obtained image features are matched and searched against the image features in the product feature base database. In practical applications, when the number of image features in the product feature base database is small, a full search can be used; when the number of image features in the product feature base database is large, various vector retrieval optimization methods can be used.

[0155] 3) Matching results returned:

[0156] Based on actual needs, return the product information corresponding to the Top N matching results with the highest matching degree.

[0157] Figure 7 This is a structural block diagram of a model training apparatus provided in an embodiment of the present invention. This apparatus is used to execute the model training method provided in any of the above embodiments. This apparatus and the model training methods of the above embodiments belong to the same inventive concept. Details not described in detail in the embodiments of the model training apparatus can be found in the embodiments of the above model training methods. See also... Figure 7 The device may specifically include: a coarse training module 510, a first input module 520, a second input module 530, and a fine training module 540.

[0158] The coarse training module 510 is used to take the training images and training text of the training items as a set of training samples, and train the original feature extraction model based on the obtained multiple sets of training samples to obtain the intermediate feature extraction model.

[0159] The first input module 520 is used to take the inspection image and inspection text of the inspected item as a set of inspection samples, and input the obtained multiple sets of inspection samples into the intermediate feature extraction model to obtain the first matching degree set;

[0160] The second input module 530 is used to input multiple sets of training samples into the intermediate feature extraction model to obtain the second matching degree set;

[0161] The fine training module 540 is used to select multiple sets of training samples based on the first matching degree set and the second matching degree set, and to train the original feature extraction model based on the selected training samples to obtain the target feature extraction model.

[0162] The first matching degree set is used to characterize the matching degree between each test image and each test text in multiple test samples, and the second matching degree set is used to characterize the matching degree between the training image and the training text in each training sample in multiple training samples.

[0163] Optional, fine training module 540, including:

[0164] The matching degree filtering threshold determination submodule is used to determine the matching degree filtering threshold based on each first matching degree in the first matching degree set;

[0165] The training sample filtering submodule is used to filter multiple training samples based on the matching degree filtering threshold and the second matching degree corresponding to each training sample in the second matching degree set.

[0166] Based on this, an optional matching degree filtering threshold determination submodule includes:

[0167] The matching truth value acquisition unit is used to obtain the matching truth value corresponding to each first matching degree in the first matching degree set. The matching truth value can be used to characterize the similarity between the test item represented by the test image and the test item represented by the test text corresponding to the first matching degree.

[0168] The matching degree filtering threshold obtaining unit is used to obtain the matching degree filtering threshold based on each first matching degree in the first matching degree set, and the matching truth value corresponding to each first matching degree in the first matching degree set.

[0169] Based on this, optionally, the matching degree filtering threshold is used to obtain the unit, specifically for:

[0170] Sort the first matching degrees in the first matching degree set to obtain the sorting result;

[0171] Based on the true match values ​​corresponding to each first matching degree in the ranking results, calculate the cumulative accuracy corresponding to each first matching degree in the ranking results.

[0172] Based on the pre-set cumulative accuracy threshold and cumulative accuracy, the matching degree filtering threshold is obtained from each first matching degree in the sorting results.

[0173] Another optional training sample selection submodule is specifically used for:

[0174] For each training sample, the training sample is retained if the second matching degree is greater than the matching degree screening threshold, thus realizing the screening process of multiple training samples.

[0175] Optionally, the original feature extraction model may include an original image feature extraction network and an original text feature extraction network. The coarse training module 510 may include:

[0176] The training text input submodule is used to input each training image from the multiple sets of training samples into the original image feature extraction network, and to input each training text into the original text feature extraction network.

[0177] The coarse training submodule is used to calculate the loss of the output results of the original image feature extraction network and the original text feature extraction network, and adjust the network parameters in the original feature extraction model according to the obtained loss calculation results to obtain the intermediate feature extraction model.

[0178] Based on this, optionally, the intermediate feature extraction model may include an intermediate image feature extraction network and an intermediate text feature extraction network, and the first input module 520 may include:

[0179] The test image input submodule is used to input the test image into the intermediate image feature extraction network for each test image in the multiple sets of test samples to obtain the test image features;

[0180] The test text input submodule is used to input the test text into the intermediate text feature extraction network for each test text in multiple test samples to obtain the test text features;

[0181] The first matching degree calculation submodule is used to calculate the first matching degree between the test image features and the test text features for each obtained test image feature and each test text feature;

[0182] The first matching degree set acquisition submodule is used to obtain the first matching degree set based on the calculated multiple first matching degrees.

[0183] Optionally, the above-mentioned model training device further includes:

[0184] The item sampling module is used to sample items in the item library that are located under the target category for each of the multiple item categories associated with the item library, so as to obtain sampled items, and select multiple sets of training samples corresponding to the training items from all the obtained sampled items.

[0185] The model training apparatus provided in this embodiment of the invention uses a coarse training module to take training images and training text of training items as a set of training samples, and trains an original feature extraction model based on the obtained multiple sets of training samples to obtain an intermediate feature extraction model. Then, through a first input module, it takes inspection images and inspection text of inspection items as a set of inspection samples, and inputs the obtained multiple sets of inspection samples into the intermediate feature extraction model to obtain a first matching degree set. Furthermore, through a second input module, it inputs multiple sets of training samples into the intermediate feature extraction model to obtain a second matching degree set. Further, through a fine training module, it selects multiple sets of training samples based on the first and second matching degree sets, and trains the original feature extraction model based on the selected training samples to obtain the target feature extraction model applied in the item matching process. This apparatus, through the cooperation of coarse and fine training rounds and the automatic selection of high-quality training samples, can effectively avoid the influence of low-quality training samples on the model's performance, thereby improving the effectiveness of item representation and ensuring the accuracy of subsequent item matching applications.

[0186] The model training apparatus provided in this embodiment of the invention can execute the model training method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0187] It is worth noting that in the embodiments of the above-mentioned model training device, the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the scope of protection of the present invention.

[0188] Figure 8 This is a structural block diagram of an item matching device provided in an embodiment of the present invention. This device is used to execute the item matching method provided in any of the above embodiments. This device and the item matching methods of the above embodiments belong to the same inventive concept. Details not described in detail in the embodiments of the item matching device can be found in the embodiments of the above item matching methods. See also... Figure 8 The device may specifically include: a model acquisition module 610, a model application module 620, and an item matching module 630.

[0189] The model acquisition module 610 is used to acquire the matching information of the item to be matched, and the target feature extraction model trained according to the model training method provided in any embodiment of the present invention.

[0190] The model application module 620 is used to input the information to be matched into the target feature extraction model to obtain the features to be matched, wherein the features to be matched include the image features to be matched and / or the text features to be matched.

[0191] The item matching module 630 is used to match the feature to be matched with the candidate features of at least one candidate item, respectively, to determine the target item that matches the item to be matched from the at least one candidate item, wherein the candidate features include candidate image features and / or candidate text features.

[0192] Optionally, the above-mentioned item matching device further includes:

[0193] The candidate feature acquisition module is used to acquire candidate information for each candidate item in at least one candidate item, and input the candidate information into the target feature extraction model to obtain candidate features of the candidate item.

[0194] The item matching device provided in this embodiment of the invention uses a model acquisition module to input the matching information of the item to be matched into a target feature extraction model to obtain matching features; a model application module to input the matching information of the item to be matched into the target feature extraction model to obtain matching features; and an item matching module to match the matching features with candidate features of at least one candidate item to determine the target item that matches the item to be matched from at least one candidate item. This device improves the accuracy of item matching by applying a target feature extraction model that can effectively represent items to process the matching information.

[0195] The item matching device provided in the embodiments of the present invention can execute the item matching method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the method execution.

[0196] It is worth noting that in the embodiments of the above-mentioned item matching device, the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be realized; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the scope of protection of the present invention.

[0197] Figure 9 A schematic diagram of an electronic device 10 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0198] likeFigure 9 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded into the RAM 13 from storage unit 18. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0199] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0200] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as model training methods or item matching methods.

[0201] In some embodiments, the model training method or item matching method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the model training method or item matching method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the model training method or item matching method by any other suitable means (e.g., by means of firmware).

[0202] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0203] Computer programs used to implement the methods of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The computer programs can be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0204] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0205] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0206] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0207] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0208] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0209] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A model training method, characterized in that, include: The training images and training text of the training items are used as a set of training samples, and the original feature extraction model is trained based on the obtained multiple sets of training samples to obtain the intermediate feature extraction model. The inspection images and inspection text of the inspected items are used as a set of inspection samples, and the resulting multiple sets of inspection samples are input into the intermediate feature extraction model to obtain the first matching degree set; The multiple sets of training samples are input into the intermediate feature extraction model to obtain the second matching degree set; Based on the first matching degree set and the second matching degree set, the multiple sets of training samples are selected, and based on the selected training samples, the original feature extraction model is trained to obtain the target feature extraction model; Wherein, the first matching degree set is used to characterize the matching degree between each test image and each test text in the multiple sets of test samples, and the second matching degree set is used to characterize the matching degree between the training image and the training text in each set of training samples in the multiple sets of training samples.

2. The method according to claim 1, characterized in that, The step of filtering the multiple sets of training samples based on the first matching degree set and the second matching degree set includes: Based on each first matching degree in the first matching degree set, determine the matching degree filtering threshold; Based on the matching degree screening threshold and the second matching degree corresponding to each group of training samples in the second matching degree set, the multiple groups of training samples are screened.

3. The method according to claim 2, characterized in that, The step of determining the matching degree filtering threshold based on each first matching degree in the first matching degree set includes: For each first matching degree in the first matching degree set, a matching truth value corresponding to the first matching degree is obtained, wherein the matching truth value is used to characterize the similarity between the inspection item represented by the inspection image and the inspection item represented by the inspection text corresponding to the first matching degree. Based on each first matching degree in the first matching degree set, and the matching truth value corresponding to each first matching degree in the first matching degree set, the matching degree filtering threshold is obtained.

4. The method according to claim 3, characterized in that, The step of obtaining the matching degree filtering threshold based on each first matching degree in the first matching degree set and the matching truth value corresponding to each first matching degree in the first matching degree set includes: Sort the first matching degrees in the first matching degree set to obtain the sorting result; Based on the true match values ​​corresponding to each first matching degree in the ranking results, calculate the cumulative accuracy corresponding to each first matching degree in the ranking results; Based on the pre-set cumulative accuracy threshold and the cumulative accuracy, a matching degree filtering threshold is obtained from each first matching degree in the sorting results.

5. The method according to claim 2, characterized in that, Based on the matching degree screening threshold, and the second matching degree corresponding to each group of training samples in the second matching degree set, the multiple groups of training samples are screened, including: For each training sample in the second matching degree set, if the second matching degree is greater than the matching degree screening threshold, the training sample is retained to achieve the screening process of the multiple training samples.

6. The method according to claim 1, characterized in that, The original feature extraction model includes an original image feature extraction network and an original text feature extraction network. The intermediate feature extraction model is obtained by training the original feature extraction model based on multiple sets of training samples, including: Each training image from the multiple sets of training samples is input into the original image feature extraction network, and each training text is input into the original text feature extraction network. Using contrastive learning loss, the output results of the original image feature extraction network and the original text feature extraction network are subjected to loss calculation, and the network parameters in the original feature extraction model are adjusted according to the obtained loss calculation results to obtain an intermediate feature extraction model.

7. The method according to claim 6, characterized in that, The intermediate feature extraction model includes an intermediate image feature extraction network and an intermediate text feature extraction network. The process of inputting multiple sets of test samples into the intermediate feature extraction model to obtain a first matching degree set includes: For each test image in the multiple sets of test samples obtained, the test image is input into the intermediate image feature extraction network to obtain test image features; For each test text in the multiple sets of test samples, the test text is input into the intermediate text feature extraction network to obtain the test text features; For each obtained test image feature and each test text feature, calculate the first matching degree between the test image feature and the test text feature; Based on the calculated first matching degree, a first matching degree set is obtained.

8. The method according to claim 1, characterized in that, Also includes: For each target category among multiple item categories associated with the item library, the category items in the item library that are located under the target category are sampled to obtain sampled items, so as to select the training items corresponding to the multiple sets of training samples from all the obtained sampled items.

9. An item matching method, characterized in that, include: Obtain matching information for the items to be matched, and a target feature extraction model trained according to the model training method of any one of claims 1-8; The information to be matched is input into the target feature extraction model to obtain the features to be matched, wherein the features to be matched include image features to be matched and / or text features to be matched; The feature to be matched is matched with the candidate features of at least one candidate item to determine the target item that matches the feature to be matched from the at least one candidate item, wherein the candidate features include candidate image features and / or candidate text features.

10. The method according to claim 9, characterized in that, Also includes: For each of the at least one candidate items, candidate information of the candidate item is obtained, and the candidate information is input into the target feature extraction model to obtain candidate features of the candidate item.

11. A model training device, characterized in that, include: The coarse training module is used to take the training images and training text of the training items as a set of training samples, and train the original feature extraction model based on the obtained multiple sets of training samples to obtain the intermediate feature extraction model. The first input module is used to take the inspection image and inspection text of the inspected item as a set of inspection samples, and input the obtained multiple sets of inspection samples into the intermediate feature extraction model to obtain the first matching degree set; The second input module is used to input the multiple sets of training samples into the intermediate feature extraction model to obtain the second matching degree set; The fine training module is used to filter the multiple sets of training samples based on the first matching degree set and the second matching degree set, and to train the original feature extraction model based on the selected training samples to obtain the target feature extraction model. Wherein, the first matching degree set is used to characterize the matching degree between each test image and each test text in the multiple sets of test samples, and the second matching degree set is used to characterize the matching degree between the training image and the training text in each set of training samples in the multiple sets of training samples.

12. An item matching device, characterized in that, include: The model acquisition module is used to acquire the matching information of the item to be matched, and the target feature extraction model trained according to the model training method of any one of claims 1-8. The model application module is used to input the information to be matched into the target feature extraction model to obtain the features to be matched, wherein the features to be matched include image features to be matched and / or text features to be matched; An item matching module is used to match the feature to be matched with the candidate features of at least one candidate item, so as to determine the target item that matches the item to be matched from the at least one candidate item, wherein the candidate features include candidate image features and / or candidate text features.

13. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor to cause the at least one processor to perform the model training method as described in any one of claims 1-8, or the item matching method as described in claim 9 or 10.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the model training method as described in any one of claims 1-8, or the item matching method as described in claim 9 or 10.

Images