Label determination method and device, model training method and device

Abstract

The embodiments of this specification provide a label determination method and device, and a model training method and device, wherein the label determination method includes receiving an image to be predicted; inputting the image to be predicted into a first machine learning model to obtain an initial label of the image to be predicted and an initial prediction probability of the initial label; inputting the initial prediction probability of the initial label into a second machine learning model to obtain a target prediction probability of the initial label; and determining a target label of the image to be predicted based on the target prediction probability of the initial label; specifically, the label determination method inputs an initial prediction result of the image to be predicted obtained by the first machine learning model into the second machine learning model for secondary correction, and by updating the initial prediction result by the second machine learning model, the recognition accuracy of the image label corresponding to the image to be predicted can be greatly improved.

Description

Technical Field

[0001] The embodiments of this specification relate to the field of computer technology, and in particular to a label determination method. One or more embodiments of this specification also relate to a model training method, a label determination device, a model training device, a computing device, and a computer-readable storage medium. Background Art

[0002] In C2C (Consumer to Consumer, e-commerce platform) e-commerce platforms, ordinary sellers have not received professional training similar to that of professional sellers on B2C (Business-to-Consumer, commercial retail) platforms. When publishing products, sellers need to upload product pictures, which contain rich product information. However, the product information filled in by sellers through manual editing is limited, and they often fill in the wrong or miss the product information, resulting in the inability of search engines to effectively recall the products. Currently, with the rapid development of artificial intelligence algorithms, label recognition algorithms can be designed to intelligently identify product description labels, automatically add detailed and accurate product information to products, simplify the product release process, and promote product search and recall through rich structured information. However, the existing label recognition algorithms cover a limited number of product labels, have low accuracy, and even have contradictory labels, which are of limited help to the final product search and recall.

[0003] Therefore, there is an urgent need to provide a tag determination method that can improve the recognition accuracy of commercial tags based on the association relationship between tags. Summary of the invention

[0004] In view of this, the present specification provides a label determination method. One or more embodiments of the present specification also relate to a model training method, a label determination device, a model training device, a computing device, and a computer-readable storage medium to solve the technical defects existing in the prior art.

[0005] According to a first aspect of an embodiment of this specification, a tag determination method is provided, including:

[0006] Receiving an image to be predicted;

[0007] Inputting the image to be predicted into a first machine learning model to obtain an initial label of the image to be predicted and an initial prediction probability of the initial label;

[0008] Inputting the initial predicted probability of the initial label into a second machine learning model to obtain a target predicted probability of the initial label;

[0009] Determine the target label of the image to be predicted based on the target prediction probability of the initial label,

[0010] The first machine learning model is suitable for labeling the image to be predicted, and outputting an initial label of the image to be predicted and an initial prediction probability of the initial label.

[0011] The second machine learning model is suitable for adjusting the initial prediction probability of the initial label and outputting the adjusted target prediction probability of the initial label.

[0012] According to a second aspect of an embodiment of this specification, a model training method is provided, including:

[0013] Acquire a sample image training set, wherein the sample image training set includes sample images, sample labels corresponding to the sample images, and true prediction probabilities of the sample labels;

[0014] Inputting the sample image into a first machine learning model to obtain a sample label of the sample image and an initial prediction probability of the sample label;

[0015] Building a second machine learning model based on the sample labels and the co-occurrence probabilities between the sample labels;

[0016] Adjusting the initial prediction probability of the sample label according to the co-occurrence probability between the sample labels in the second machine learning model to obtain the target prediction probability of the sample label;

[0017] The first machine learning model and the second machine learning model are obtained by training based on the target prediction probability and the true prediction probability of the sample label.

[0018] According to a third aspect of an embodiment of this specification, a tag determination method is provided, including:

[0019] Displaying an image input interface to the user based on a call request from the user;

[0020] receiving an image to be predicted input by the user based on the image input interface;

[0021] Inputting the image to be predicted into a first machine learning model to obtain an initial label of the image to be predicted and an initial prediction probability of the initial label;

[0022] Inputting the initial predicted probability of the initial label into a second machine learning model to obtain a target predicted probability of the initial label;

[0023] The target label of the image to be predicted is determined based on the target prediction probability of the initial label and returned to the user.

[0024] According to a fourth aspect of the embodiments of this specification, a tag determination method is provided, including:

[0025] Receiving a call request sent by a user, wherein the call request carries an image to be predicted;

[0026] Inputting the image to be predicted into a first machine learning model to obtain an initial label of the image to be predicted and an initial prediction probability of the initial label;

[0027] Inputting the initial predicted probability of the initial label into a second machine learning model to obtain a target predicted probability of the initial label;

[0028] The target label of the image to be predicted is determined based on the target prediction probability of the initial label and returned to the user.

[0029] According to a fifth aspect of the embodiments of this specification, a model training method is provided, including:

[0030] Displaying an image input interface to the user based on a call request from the user;

[0031] Receiving a sample image training set input by the user based on the image input interface, wherein the sample image training set includes sample images, sample labels corresponding to the sample images, and true prediction probabilities of the sample labels;

[0032] Inputting the sample image into a first machine learning model to obtain a sample label of the sample image and an initial prediction probability of the sample label;

[0033] Building a second machine learning model based on the sample labels and the co-occurrence probabilities between the sample labels;

[0034] Adjusting the initial prediction probability of the sample label according to the co-occurrence probability between the sample labels in the second machine learning model to obtain the target prediction probability of the sample label;

[0035] The first machine learning model and the second machine learning model are trained based on the target prediction probability and the true prediction probability of the sample label and returned to the user.

[0036] According to a sixth aspect of the embodiments of this specification, a model training method is provided, including:

[0037] Receive a call request sent by a user, wherein the call request carries a sample image training set, wherein the sample image training set includes a sample image, a sample label corresponding to the sample image, and a true prediction probability of the sample label;

[0038] Inputting the sample image into a first machine learning model to obtain a sample label of the sample image and an initial prediction probability of the sample label;

[0039] Building a second machine learning model based on the sample labels and the co-occurrence probabilities between the sample labels;

[0040] Adjusting the initial prediction probability of the sample label according to the co-occurrence probability between the sample labels in the second machine learning model to obtain the target prediction probability of the sample label;

[0041] The first machine learning model and the second machine learning model are trained based on the target prediction probability and the true prediction probability of the sample label and returned to the user.

[0042] According to a seventh aspect of the embodiments of this specification, a commodity identification method is provided, including:

[0043] Receiving a product image, and determining a target label of the product in the product image based on the label recognition method;

[0044] Receiving a user's search request for a product, wherein the search request carries description information of the product;

[0045] Matching the description information of the product with the target tag to determine the target tag that matches the description information of the product;

[0046] The commodity corresponding to the target tag matching the description information of the commodity is determined as a target commodity, and the target commodity is sent to the user.

[0047] According to an eighth aspect of the embodiments of this specification, a label determination device is provided, including:

[0048] A first receiving module is configured to receive an image to be predicted;

[0049] A first probability acquisition module is configured to input the image to be predicted into a first machine learning model to obtain an initial label of the image to be predicted and an initial prediction probability of the initial label;

[0050] A second probability acquisition module is configured to input the initial prediction probability of the initial label into a second machine learning model to obtain a target prediction probability of the initial label;

[0051] A first label determination module is configured to determine a target label of the image to be predicted based on a target prediction probability of the initial label,

[0052] The first machine learning model is suitable for labeling the image to be predicted, and outputting an initial label of the image to be predicted and an initial prediction probability of the initial label.

[0053] The second machine learning model is suitable for adjusting the initial prediction probability of the initial label and outputting the adjusted target prediction probability of the initial label.

[0054] According to a ninth aspect of the embodiments of this specification, a label determination device is provided, including:

[0055] A first interface display module, configured to display an image input interface for the user based on a call request from the user;

[0056] A second receiving module is configured to receive the image to be predicted input by the user based on the image input interface;

[0057] A third probability acquisition module is configured to input the image to be predicted into the first machine learning model to obtain an initial label of the image to be predicted and an initial prediction probability of the initial label;

[0058] a fourth probability acquisition module, configured to input the initial prediction probability of the initial label into a second machine learning model to obtain a target prediction probability of the initial label;

[0059] The second label determination module is configured to determine the target label of the image to be predicted based on the target prediction probability of the initial label and return it to the user.

[0060] According to a tenth aspect of the embodiments of this specification, a label determination device is provided, including:

[0061] A third receiving module is configured to receive a call request sent by a user, wherein the call request carries an image to be predicted;

[0062] a fifth probability acquisition module, configured to input the image to be predicted into the first machine learning model to obtain an initial label of the image to be predicted and an initial prediction probability of the initial label;

[0063] a sixth probability acquisition module, configured to input the initial prediction probability of the initial label into a second machine learning model to obtain a target prediction probability of the initial label;

[0064] The third label determination module is configured to determine the target label of the image to be predicted based on the target prediction probability of the initial label and return it to the user.

[0065] According to an eleventh aspect of the embodiments of this specification, a model training device is provided, including:

[0066] A first sample acquisition module is configured to acquire a sample image training set, wherein the sample image training set includes sample images, sample labels corresponding to the sample images, and true prediction probabilities of the sample labels;

[0067] a seventh probability acquisition module, configured to input the sample image into the first machine learning model to obtain a sample label of the sample image and an initial prediction probability of the sample label;

[0068] A first model building module is configured to build a second machine learning model based on the sample labels and the co-occurrence probability between the sample labels;

[0069] an eighth probability acquisition module, configured to adjust the initial prediction probability of the sample label according to the co-occurrence probability between the sample labels in the second machine learning model to obtain a target prediction probability of the sample label;

[0070] The first model training module is configured to obtain the first machine learning model and the second machine learning model through training based on the target prediction probability and the true prediction probability of the sample label.

[0071] According to a twelfth aspect of the embodiments of this specification, a model training device is provided, including:

[0072] A second interface display module is configured to display an image input interface for the user based on a call request from the user;

[0073] a fourth receiving module, configured to receive a sample image training set input by the user based on the image input interface, wherein the sample image training set includes sample images, sample labels corresponding to the sample images, and true prediction probabilities of the sample labels;

[0074] a ninth probability acquisition module, configured to input the sample image into the first machine learning model to obtain a sample label of the sample image and an initial prediction probability of the sample label;

[0075] A second model building module is configured to build a second machine learning model based on the sample labels and the co-occurrence probability between the sample labels;

[0076] a tenth probability obtaining module, configured to adjust the initial prediction probability of the sample label according to the co-occurrence probability between the sample labels in the second machine learning model to obtain a target prediction probability of the sample label;

[0077] The second model training module is configured to obtain the first machine learning model and the second machine learning model based on the target prediction probability and the true prediction probability of the sample label and return them to the user.

[0078] According to a thirteenth aspect of the embodiments of this specification, a model training device is provided, including:

[0079] a fifth receiving module, configured to receive a call request sent by a user, wherein the call request carries a sample image training set, wherein the sample image training set includes a sample image, a sample label corresponding to the sample image, and a true prediction probability of the sample label;

[0080] an eleventh probability acquisition module, configured to input the sample image into a first machine learning model to obtain a sample label of the sample image and an initial predicted probability of the sample label;

[0081] A third model building module is configured to build a second machine learning model based on the sample labels and the co-occurrence probability between the sample labels;

[0082] a twelfth probability obtaining module, configured to adjust the initial prediction probability of the sample label according to the co-occurrence probability between the sample labels in the second machine learning model to obtain a target prediction probability of the sample label;

[0083] The third model training module is configured to obtain the first machine learning model and the second machine learning model based on the target prediction probability and the true prediction probability of the sample label and return them to the user.

[0084] According to a fourteenth aspect of the embodiments of this specification, there is provided a commodity identification device, including:

[0085] A product labeling module is configured to receive a product image and determine a target label of the product in the product image based on the label recognition method;

[0086] A search request receiving module, configured to receive a user's search request for a product, wherein the search request carries description information of the product;

[0087] a tag matching module, configured to match the description information of the commodity with the target tag, and determine a target tag that matches the description information of the commodity;

[0088] The target product determination module is configured to determine the product corresponding to the target tag matching the description information of the product as the target product, and send the target product to the user.

[0089] According to a fifteenth aspect of the embodiments of this specification, a computing device is provided, including:

[0090] Memory and processor;

[0091] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the instructions are executed by the processor, the steps of the label determination method, the steps of the model training method, or the steps of the commodity identification method are implemented.

[0092] According to the sixteenth aspect of the embodiments of this specification, a computer-readable storage medium is provided, which stores computer-executable instructions, which, when executed by a processor, implement the steps of the label determination method, the steps of the model training method, or the steps of the product identification method.

[0093] One embodiment of the present specification implements a label determination method and device, wherein the label determination method includes receiving an image to be predicted; inputting the image to be predicted into a first machine learning model to obtain an initial label of the image to be predicted and an initial prediction probability of the initial label; inputting the initial prediction probability of the initial label into a second machine learning model to obtain a target prediction probability of the initial label; determining a target label of the image to be predicted based on the target prediction probability of the initial label; specifically, the label determination method inputs an initial prediction result of the image to be predicted obtained by the first machine learning model into the second machine learning model for secondary correction, and by updating the initial prediction result by the second machine learning model, the recognition accuracy of the image label corresponding to the image to be predicted can be greatly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0094] Figure 1 This is an example diagram of a specific application scenario of a tag determination method provided by an embodiment of this specification;

[0095] Figure 2 is a flow chart of a first model training method provided by an embodiment of this specification;

[0096] Figure 3 is a schematic diagram of a sample label matrix in a model training method provided by an embodiment of this specification;

[0097] Figure 4 It is a schematic diagram of a specific processing process of a model training method provided by an embodiment of this specification;

[0098] Figure 5 is a flow chart of a first tag determination method provided by an embodiment of this specification;

[0099] Figure 6is a flow chart of a second tag determination method provided by an embodiment of this specification;

[0100] Figure 7 is a flow chart of a third tag determination method provided by an embodiment of this specification;

[0101] Figure 8 is a flow chart of a second model training method provided by an embodiment of this specification;

[0102] Fig. 9 is a flowchart of a third model training method provided by an embodiment of this specification;

[0103] Fig.10 is a structural schematic diagram of a first tag determination device provided by an embodiment of this specification;

[0104] Fig.11 is a structural schematic diagram of a second tag determination device provided by an embodiment of this specification;

[0105] Fig.12 is a schematic structural diagram of a third tag determination device provided by an embodiment of this specification;

[0106] Fig.13 It is a structural schematic diagram of a first model training device provided by an embodiment of this specification;

[0107] Fig.14 is a structural schematic diagram of a second model training device provided by an embodiment of this specification;

[0108] Fig.15 is a structural schematic diagram of a third model training device provided by an embodiment of this specification;

[0109] Fig.16 is a flow chart of a commodity identification method provided by an embodiment of this specification;

[0110] Fig.17 It is a structural schematic diagram of a commodity identification device provided by an embodiment of this specification;

[0111] Fig.18 It is a structural block diagram of a computing device provided by an embodiment of this specification. DETAILED DESCRIPTION

[0112] Many specific details are described in the following description to facilitate a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of this specification, so this specification is not limited to the specific implementation disclosed below.

[0113] The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms of "a", "said" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0114] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0115] First, the terms involved in one or more embodiments of this specification are explained.

[0116] C2C e-commerce platform: that is, Consumer To Consumer e-commerce platform, provides an online trading platform for buyers and sellers through e-commerce websites, so that sellers can post information about items to be sold on it, and buyers can choose to purchase from it. At the same time, in order to facilitate transactions between buyers and sellers, a series of supporting services required for transactions are provided.

[0117] B2C: (Business-to-Consumer, commercial retail), B2C means that enterprises provide consumers with a new shopping environment through the Internet - online stores, and consumers can shop online through this B2C platform.

[0118] Message Passing Networks: Message passing networks are used to model the relationship between multiple variables and are used for tasks such as graph embedding learning and multi-label recognition.

[0119] In this specification, a label determination method is provided. One or more embodiments of this specification simultaneously relate to a model training method, a label determination device, a model training device, a computing device, and a computer-readable storage medium, which are described in detail one by one in the following embodiments.

[0120] See also Figure 1 , Figure 1An example diagram showing a specific application scenario of a tag determination method provided by an embodiment of this specification.

[0121] Figure 1 The application scenario includes terminals and servers. Specifically, the user sends image a to the server through the terminal. After receiving image a, the server inputs image a into the pre-trained convolutional neural network model to obtain the initial label of image a and the initial prediction probability of each initial label (ψ(x) i ), and then the initial predicted probability of the initial label of image a is input into the pre-trained message passing network model to obtain the predicted probability (Q(x)) after correcting the initial predicted probability of the initial label of image a. i ), and finally, the corrected prediction probabilities are sorted in descending order, and a preset number of initial labels are taken as target labels for image a based on project requirements, wherein the preset number can be set according to actual project requirements, for example, to 20 or 30.

[0122] In practical applications, when there are a large number of labels, the ambiguity between labels will become more and more serious, and even mutually exclusive labels may occur. For example, when the label "old age" appears, the label "high heels" appears. It is obvious that the elderly and high heels are not very compatible. These two labels are two contradictory labels. Therefore, the label recognition method of the embodiment of this specification introduces a label network to model the label relationship, and uses the constructed message passing network model to iteratively update the initial prediction results of the label. Combined with the basic distinguishing ability of the image label of the basic label model (i.e., the convolutional neural network model), the recognition accuracy of the label recognition algorithm can be greatly improved.

[0123] See also Figure 2 , Figure 2 A flowchart of a first model training method provided according to an embodiment of the present specification is shown, which specifically includes the following steps.

[0124] Step 202: Obtain a sample image training set, wherein the sample image training set includes sample images, sample labels corresponding to the sample images, and true prediction probabilities of the sample labels.

[0125] Among them, the sample images include but are not limited to network images downloaded through a smart terminal, images taken by a shooting terminal, or video frame images formed after video segmentation, etc., wherein the preset segmentation method can be segmentation by frame or segmentation by time length; and the content of the specific sample images can include but is not limited to objects, scenery or people, etc.

[0126] In specific implementation, each sample image in the sample image training set corresponds to at least one sample label and the true prediction probability of each sample label. For example, sample image 1 corresponds to sample label a1, sample label a2, and sample label a3, where the true prediction probability of sample label a1 is 0.6, the true prediction probability of sample label a2 is 0.5, and the true prediction probability of sample label a3 is 0.7. It can be seen that each sample image can correspond to multiple sample labels, and each sample label corresponds to a true prediction probability.

[0127] Step 204: Input the sample image into a first machine learning model to obtain a sample label of the sample image and an initial prediction probability of the sample label.

[0128] The first machine learning model includes but is not limited to an image tag recognition model, and the second machine learning model includes but is not limited to a message passing network model.

[0129] The first machine learning model inputs the sample image and outputs a sample label corresponding to the sample image and an initial prediction probability of the sample label.

[0130] The second machine learning model inputs the initial prediction probability of the sample label and outputs the target prediction probability of the sample label.

[0131] Specifically, after obtaining the sample image training set, each sample image is input into the image label recognition model to obtain a sample label corresponding to each sample image and an initial prediction probability corresponding to each sample label.

[0132] In the embodiments of the present specification, when performing model training, the basic distinguishing ability of the image labels of the basic label model (i.e., the image label recognition model) is first used to obtain the initial prediction probability of the sample label of each sample image. Subsequently, the initial prediction probability of the sample label is corrected by the second machine learning model to obtain the target prediction probability of the sample label. The first machine learning model and the second machine learning model can be trained more accurately based on the target prediction probability.

[0133] Step 206: Construct a second machine learning model based on the sample labels and the co-occurrence probabilities between the sample labels.

[0134] Among them, the second machine learning model is a message passing network model. In actual applications, after obtaining the sample image training set, the second machine learning model will be constructed based on the sample labels of all sample images and the co-occurrence probability between each sample label and other sample labels.

[0135] In a specific implementation, constructing a second machine learning model based on the sample labels and the co-occurrence probability between the sample labels includes:

[0136] Constructing a sample label matrix based on the sample labels, and determining the co-occurrence probability of each sample label with other sample labels according to the sample label matrix;

[0137] Determine each sample label as a node, and determine the co-occurrence probability of each sample label with other sample labels as an edge;

[0138] A second machine learning model is constructed based on the nodes and edges determined by the sample labels.

[0139] Specifically, a sample label matrix is ​​first constructed based on the sample labels of the sample images, and then the co-occurrence probability of each sample label with other sample labels is calculated according to the sample label matrix.

[0140] See also Figure 3 , Figure 3 A schematic diagram of a sample label matrix in a model training method provided according to an embodiment of the present specification is shown.

[0141] in, Figure 3 The label network co-occurrence probability matrix in is the sample label matrix.

[0142] Depend on Figure 3 It can be seen that the training data formed by the sample labels of the sample images include: pattern: solid color, applicable to: youth, style: single shoes; style: modern Chinese, design element: splicing, design element: log; ... silhouette: X-type, pattern: large flower, waist type: high waist, skirt length: long.

[0143] Then, an initial sample label matrix is ​​constructed based on the above training data. For example, the three sample labels in the training data are "pattern: solid color, applicable to youth, style: single shoes". Each row represents a sample label. For example, "solid color" is the first row, "youth" is the second row, and "single shoes" is the third row. If there are 10,000 labels in the training data, there will be 10,000 rows. Each label will be arranged in a fixed position and replaced by a unique number. The arrangement of the columns is the same. The initial sample label matrix is ​​constructed based on this rule.

[0144] After constructing the initial sample label matrix, the co-occurrence probability of each sample label in the initial sample label matrix with other sample labels is counted. For example, in the training data, "pattern: solid color, applicable to: youth, style: single shoes", these three sample labels appear together, so the number between them will be increased by 1, that is, the co-occurrence of these three labels is increased by 1. In this way, all sample labels in the initial sample label matrix are counted. For example, when the sample label "solid color" appears, what is the probability of the sample label "youth" appearing, and what is the probability of the sample label "single shoe" appearing. After all the sample labels in the initial sample label matrix are counted, the sample labels in each row or column are normalized after the statistics, and the co-occurrence probability of sample labels with a co-occurrence probability greater than or equal to 0.1 is retained, and the co-occurrence probability of sample labels with a co-occurrence probability less than 0.1 is set to 0.

[0145] by Figure 3 Taking the sample label of the first row and first column in the data set as "pure color", the sample label of the third column as "youth", and the sample label of the last column as "single shoe" as an example, it can be determined that the co-occurrence probability of the sample label "pure color" with itself is 1.0, the co-occurrence probability with the sample label "youth" is 0.2, and the co-occurrence probability with the sample label "single shoe" is 0.9.

[0146] The final sample label matrix is ​​constructed in the above manner, through which the co-occurrence probability between each sample label and other sample labels can be determined.

[0147] Finally, all the training data are counted based on the sample label matrix, and each sample label in the sample label matrix is ​​set as a node in the label network. The edges between the nodes represent the co-occurrence probability between the sample labels. The second machine learning model, namely the message passing network model, is implemented based on the nodes and edges.

[0148] In the embodiments of the present specification, the model training method introduces a label network to model the label relationship. Subsequently, the message passing network model (i.e., Message Passing) obtained by the modeling can be used to iteratively update the initial prediction results of the label. Combined with the basic distinction ability of the image label of the basic label model, the recognition accuracy of the label recognition algorithm can be significantly improved.

[0149] Step 208: Adjust the initial prediction probability of the sample label according to the co-occurrence probability between the sample labels in the second machine learning model to obtain the target prediction probability of the sample label.

[0150] The first machine learning model is suitable for labeling the sample image and outputting an initial label of the sample image and an initial predicted probability of the initial label.

[0151] The second machine learning model is suitable for adjusting the initial prediction probability of the initial label and outputting the adjusted target prediction probability of the initial label.

[0152] Specifically, after the second machine learning model is constructed, the initial prediction probability of the sample label output by the convolutional neural network model is iteratively updated using the co-occurrence probability between the sample labels in the second machine learning model. The specific updating process is as follows:

[0153] initialization:

[0154] for iter in 1: M do

[0155] Message passing from label l to label i

[0156] Local node information update

[0157] Probability Normalization

[0158] end for

[0159] Among them, Q(x) in the initialization is the normalized value of ψ(x) The normalized vector is the probability vector of each sample label. Each probability vector element has a value between 0 and 1, and the sum is 1. And for iter in1:M is to iteratively update Q(x). The update process is each step in the for loop. Specifically, W i，l It is the sample label matrix. W is obtained by statistics of training data. For example, if two sample labels i and l appear together (that is, they appear at the same time in one training data), then W i，l The corresponding statistical value is added by 1. After all sample labels in the sample label matrix are counted, W is normalized by row / column, that is, the conditional probability is increased, and the probability card threshold is staged, and the ones that are retained are those with edges (that is, if two sample labels often appear at the same time, they are on the right, and if they occasionally or never appear at the same time, there is no edge). The second machine learning model is constructed through the sample labels with edges.

[0160] In actual use, the above-mentioned iterative cycle formula is used to iteratively update the initial prediction probability of the sample label, and finally obtain the target prediction probability of the sample label after iterative update.

[0161] Step 210: The first machine learning model and the second machine learning model are obtained by training based on the target prediction probability and the true prediction probability of the sample label.

[0162] Specifically, after obtaining the target prediction probability of the sample label, the target prediction probability is compared with the true prediction probability of the sample label before being predicted by the first machine learning model and the second machine learning model, and the first machine learning model and the second machine learning model are trained based on the comparison results. For example, if the desired true prediction probability of the sample label before being predicted by the first machine learning model and the second machine learning model is higher, and the target prediction probability of the sample label after being predicted by the first machine learning model and the second machine learning model is lower, then the training parameters in the first machine learning model and the second machine learning model will be adjusted to make the first machine learning model and the second machine learning model obtained by training more accurate. The specific implementation method is as follows:

[0163] The training of obtaining the first machine learning model and the second machine learning model based on the target prediction probability and the true prediction probability of the sample label includes:

[0164] A loss function is calculated based on the target prediction probability and the true prediction probability of the sample label, and the first machine learning model and the second machine learning model are trained according to the loss function to obtain the first machine learning model and the second machine learning model.

[0165] In specific implementation, the loss function is calculated based on the target prediction probability and the true prediction probability of the sample label, and then the first machine learning model and the second machine learning model are back-propagated and trained according to the loss function to obtain the final first machine learning model and the second machine learning model.

[0166] The model training method of the embodiment of the present specification first trains an initial deep convolutional neural network model using sample images and sample labels, and constructs a message passing network model using sample labels of sample images; then, the initial prediction probability of the sample labels of the sample images obtained by the initial deep convolutional neural network model is input into the message passing network model for iterative updating, and the initial deep convolutional neural network model and the message passing network model are trained according to the target prediction probability and the true prediction probability after the iterative update to obtain the final deep convolutional neural network model and the message passing network model with higher recognition accuracy.

[0167] See also Figure 4 , Figure 4 A schematic diagram of a specific processing process of a model training method provided according to an embodiment of the present specification is shown.

[0168] First, obtain sample images and sample labels to train a convolutional neural network, and then input the sample image (x) into the convolutional neural network (ψ(x) i ), obtain the sample label of the sample image and the initial prediction probability corresponding to each sample label; at the same time, use the sample label of the sample image to generate training data, and generate a label network co-occurrence probability matrix ({W i，j}), and construct a message passing network in the form of a label network through the label network co-occurrence probability matrix.

[0169] Then, the initial prediction probability of the sample label output by the convolutional neural network is iteratively updated using the co-occurrence probability of the sample label in the message passing network to obtain the predicted probability of the corrected sample label (Q(x) i ), and use it as the final predicted probability of the sample label.

[0170] Finally, the final predicted probability is compared with the true predicted probability of the sample label before prediction, and the convolutional neural network and message passing network are trained through back-propagation.

[0171] In the embodiments of this specification, a label relationship network is used to model sample labels, and a message passing network model obtained by modeling is used to iteratively update the initial prediction probability of the sample labels. The convolutional neural network model and the message passing network model are trained through the iteratively updated final prediction probability to obtain a convolutional neural network model and a message passing network model that are more accurate for label recognition.

[0172] See also Figure 5 , Figure 5 A flow chart of a first tag determination method provided according to an embodiment of the present specification is shown, which specifically includes the following steps.

[0173] Step 502: Receive an image to be predicted.

[0174] Among them, the images to be predicted include but are not limited to network images downloaded through a smart terminal, images shot by a shooting terminal, or video images formed by segmentation according to a preset segmentation method, etc., wherein the preset segmentation method can be segmentation by frame or segmentation by duration; and the specific content of the image to be predicted may include but is not limited to objects, scenery or people, etc.

[0175] In specific implementation, before receiving the image to be predicted, or after receiving the image to be predicted, before predicting the label of the image to be predicted, it is necessary to first train and obtain the first machine learning model and the second machine learning model, so as to obtain the target label of the image to be predicted and the target prediction probability of the target label through the trained first machine learning model and the second machine learning model. The specific implementation method is as follows:

[0176] The training process of the first machine learning model and the second machine learning model is as follows:

[0177] Acquire a sample image training set, wherein the sample image training set includes sample images, sample labels corresponding to the sample images, and true prediction probabilities of the sample labels;

[0178] Inputting the sample image into a first machine learning model to obtain a sample label of the sample image and an initial prediction probability of the sample label;

[0179] Building a second machine learning model based on the sample labels and the co-occurrence probabilities between the sample labels;

[0180] Adjusting the initial prediction probability of the sample label according to the co-occurrence probability between the sample labels in the second machine learning model to obtain the target prediction probability of the sample label;

[0181] The first machine learning model and the second machine learning model are obtained by training based on the target prediction probability and the true prediction probability of the sample label.

[0182] Among them, the sample images include but are not limited to network images downloaded through a smart terminal, images taken by a shooting terminal, or video frame images formed after video segmentation, etc., wherein the preset segmentation method can be segmentation by frame or segmentation by time length; and the content of the specific sample images can include but is not limited to objects, scenery or people, etc.

[0183] In specific implementation, each sample image in the sample image training set corresponds to at least one sample label and the true prediction probability of each sample label. For example, sample image 1 corresponds to sample label a1, sample label a2, and sample label a3, where the true prediction probability of sample label a1 is 0.6, the true prediction probability of sample label a2 is 0.5, and the true prediction probability of sample label a3 is 0.7. It can be seen that each sample image can correspond to multiple sample labels, and each sample label corresponds to a true prediction probability.

[0184] Specifically, after obtaining the sample image training set, each sample image is input into the first machine learning model to obtain the sample label corresponding to each sample image and the initial prediction probability corresponding to each sample label. At the same time, a second machine learning model is constructed based on the sample labels of the sample images in the sample image training set and the co-occurrence probability between each sample label and other sample labels. The initial prediction probability of the sample label is adjusted according to the co-occurrence probability between the sample labels in the second machine learning model to obtain the target prediction probability of the sample label. Finally, the first machine learning model and the second machine learning model are obtained by training based on the target prediction probability and the true prediction probability of the sample label.

[0185] Optionally, constructing a second machine learning model based on the sample labels and the co-occurrence probabilities between the sample labels includes:

[0186] Constructing a sample label matrix based on the sample labels, and determining the co-occurrence probability of each sample label with other sample labels according to the sample label matrix;

[0187] Determine each sample label as a node, and determine the co-occurrence probability of each sample label with other sample labels as an edge;

[0188] A second machine learning model is constructed based on the nodes and edges determined by the sample labels.

[0189] Optionally, the training of obtaining the first machine learning model and the second machine learning model based on the target prediction probability and the true prediction probability of the sample label includes:

[0190] A loss function is calculated based on the target prediction probability and the true prediction probability of the sample label, and the first machine learning model and the second machine learning model are trained according to the loss function to obtain the first machine learning model and the second machine learning model.

[0191] Optionally, the first machine learning model includes an image tag recognition model, and the second machine learning model includes a message passing network model.

[0192] The first machine learning model inputs the sample image and outputs a sample label corresponding to the sample image and an initial prediction probability of the sample label.

[0193] The second machine learning model inputs the initial prediction probability of the sample label and outputs the target prediction probability of the sample label.

[0194] Among them, the specific training process of the image label recognition model and the message passing network model in the embodiments of this specification can refer to the detailed description of the above-mentioned model training method, which will not be repeated here.

[0195] In the embodiments of this specification, an image label recognition model and a message passing network model are obtained through the above-mentioned model training method, so that the image to be predicted can subsequently have a more accurate image label based on the image label recognition model and the message passing network model.

[0196] Step 504: Input the image to be predicted into the first machine learning model to obtain an initial label of the image to be predicted and an initial prediction probability of the initial label.

[0197] Specifically, taking the first machine learning model as an image label recognition model as an example, the image to be predicted is input into the first machine learning model to obtain the initial label of the image to be predicted and the initial prediction probability of the initial label. It can be understood that the image to be predicted is input into the image label recognition model to obtain the initial label of the image to be predicted and the initial prediction probability of each initial label. For example, the image to be predicted is an image containing clothes, and the initial label of the image to be predicted output by the image label recognition model is: pattern: solid color, applicable object: elderly, waist type: low waist, and the initial prediction probability of each initial label is: pattern: solid color has an initial prediction probability of 0.6, applicable object: elderly has an initial prediction probability of 0.8, and waist type: low waist has an initial prediction probability of 0.7.

[0198] Step 506: Input the initial prediction probability of the initial label into the second machine learning model to obtain the target prediction probability of the initial label.

[0199] Specifically, taking the second machine learning model as a message passing network model as an example, the initial prediction probability of the initial label is input into the second machine learning model to obtain the target prediction probability of the initial label. It can be understood that the initial prediction probability of the initial label is input into the message passing network model to obtain the target prediction probability of each initial label.

[0200] Still taking the initial labels of the image to be predicted as: pattern: solid color, applicable object: elderly, waist type: low waist as an example, in actual applications, we can know that the probability of co-occurrence of the labels elderly and low waist is not high, then when the initial prediction probability of pattern: solid color is 0.6, the initial prediction probability of applicable object: elderly is 0.8, the initial prediction probability of low waist is 0.7, which may be biased. Therefore, the pre-trained message passing network model is used to correct the initial prediction probabilities of all initial labels, and the initial prediction probability of the initial label is adjusted with the co-occurrence probability between each label and other labels in the message passing network model. Finally, the target prediction probability of pattern: solid color is 0.7, the target prediction probability of applicable object: elderly is 0.8, and the target prediction probability of low waist is 0.3.

[0201] Step 508: Determine a target label of the image to be predicted based on the target prediction probability of the initial label.

[0202] The first machine learning model is suitable for labeling the image to be predicted, and outputting an initial label of the image to be predicted and an initial prediction probability of the initial label.

[0203] The second machine learning model is suitable for adjusting the initial prediction probability of the initial label and outputting the adjusted target prediction probability of the initial label.

[0204] Specifically, when the target prediction probability of the initial label is obtained, the target label of the image to be predicted can be screened out based on the target prediction probability of the initial label.

[0205] In a specific implementation, determining the target label of the image to be predicted based on the target prediction probability of the initial label includes:

[0206] The target prediction probability of the initial label is determined, and the initial label whose target prediction probability is greater than or equal to a preset probability threshold is determined as the target label of the image to be predicted.

[0207] The preset probability threshold can be set according to actual needs, for example, to 0.5 or 0.6.

[0208] Continuing with the above example, if the preset probability threshold is set to 0.5, then when the target prediction probability of pattern: solid color is 0.7, the target prediction probability of applicable object: elderly is 0.8, and the target prediction probability of low waist is 0.3, the target label of the image to be predicted that is greater than or equal to the preset probability threshold 0.5 determined based on the preset probability threshold 0.5 is: pattern: solid color, applicable object: elderly.

[0209] In practical applications, it is not limited to obtaining the target label of the image to be predicted through a preset probability threshold, and the target label of the image to be predicted can also be determined based on a preset number of labels. For example, after determining the target prediction probability of each initial label, the initial labels are arranged in descending order from large to small based on the target prediction probability, and then the initial labels with the previous preset number of labels are extracted as the target labels of the image to be predicted. The specific selection of the target label can be set according to actual needs, and this application does not make any limitation on this.

[0210] In the embodiments of the present specification, a more appropriate initial label can be selected as the target label of the image to be predicted by setting a preset probability threshold, so as to improve the accuracy of the identified label of the image to be predicted and enhance the user experience.

[0211] In another embodiment of the present specification, after determining the initial label whose target prediction probability is greater than or equal to a preset probability threshold as the target label of the image to be predicted, the method further includes:

[0212] Image description information is generated for the image to be predicted based on the target label.

[0213] In a specific implementation, after the target label of the image to be predicted is determined, description information of the image to be predicted can be generated based on the target label.

[0214] Still taking the above example, if the target label of the image to be predicted is: solid color, old age, then image description information can be generated for the image to be predicted based on the target label: the pattern is solid color, and the applicable object is clothes for the elderly.

[0215] In the embodiments of the present specification, after obtaining the target label of the image to be predicted, relatively accurate image description information can be generated for the image to be predicted based on the accurately identified target label of the image to be predicted. In actual applications, the user no longer needs to edit the description information for the object to be predicted through the target label, which saves time and greatly improves the user experience.

[0216] In practical applications, the label determination method provided in one or more embodiments of this specification can be applied to any scenario of labeling or describing images. For example, in a C2C e-commerce platform, ordinary sellers have not received professional training similar to that of professional sellers on a B2C platform. When publishing a product, the product description information edited and filled in is limited, and the product description information is often filled in incorrectly or omitted, resulting in the product not being effectively recalled by the search engine. In this case, the label determination method provided in the embodiments of this specification is used. After inputting a product image, the label determination method provided in one or more embodiments of this specification is used to match the product image with a relatively accurate and large number of target labels for the user to choose to use, or more conveniently, editable image description information is automatically generated for the user based on the target label to enhance the user experience.

[0217] In practical applications, the label determination method can not only be used to determine the target label of the predicted image, but also can be used to obtain the target label of other multimodal data, where the multimodal data includes but is not limited to text, voice, image and / or video.

[0218] When the input of the label determination method is other multimodal data, the first machine learning model and the second machine learning model are first trained for the labels corresponding to the multimodal data. Subsequently, the labels corresponding to the multimodal data and the predicted probabilities of the labels can be obtained based on the trained first machine learning model and the second machine learning model.

[0219] In the specific implementation, after receiving the multimodal data to be predicted, the multimodal data can be parsed, and the initial label corresponding to each modal data and the initial prediction probability of the initial label can be obtained using the first machine learning model corresponding to each modal data. The initial prediction probability of the initial label is then input into the second machine learning model for correction to obtain the target prediction probability of the corrected initial label. Finally, the target label of the multimodal data to be predicted is determined based on the initial labels corresponding to all modal data and the target prediction probability of the initial label.

[0220] For example, if the received multimodal data to be predicted is a video, the video is parsed into images and sounds, and then the image is input into a pre-trained image label recognition model to obtain the initial label of the image and the initial prediction probability of the initial label; at the same time, the parsed sound is input into a pre-trained voiceprint label extraction model to obtain the initial voiceprint label of the sound and the initial prediction probability of the initial voiceprint label; then the initial prediction probability of the initial label of the image is input into a message passing network model pre-trained by image labels to obtain the target prediction probability of the initial label of the image; and the initial prediction probability of the initial voiceprint label of the sound is input into a message passing network model pre-trained by sound labels to obtain the target prediction probability of the initial voiceprint label of the sound; finally, the target label of the video is determined based on the target prediction probability of the initial label of the image and the target prediction probability of the initial voiceprint label of the sound, for example, after all labels are sorted according to the target prediction probability, the top 20 labels are taken as the target labels of the video. In practical applications, sound tags can be used to revise or supplement image tags. For example, if there is a brand name in the sound tag but not in the image tag, the image tag can be supplemented based on the sound tag. This method can enrich the multimodal data labels while ensuring the accuracy of the multimodal data labels through the revision of the sound tags.

[0221] In an embodiment of the present specification, the label determination method inputs the initial prediction result of the image to be predicted obtained by the first machine learning model into the second machine learning model for secondary correction. By updating the initial prediction result by the second machine learning model, the recognition accuracy of the image label corresponding to the image to be predicted can be greatly improved; by introducing Message Passing Networks to model the relationship between labels, making full use of the co-occurrence and mutually exclusive relationship between labels, and combining the recognition ability of the original label model, the coverage range and recognition accuracy can be greatly expanded.

[0222] See also Figure 6 , Figure 6 A flow chart of a second tag determination method provided according to an embodiment of the present specification is shown, comprising the following steps.

[0223] Step 602: Displaying an image input interface to the user based on the user's call request.

[0224] Specifically, when receiving a call request from a user, an image input interface is determined according to the call request, and the image input interface is displayed to the user, and the user can input the image to be predicted through the image input interface.

[0225] In actual applications, users include but are not limited to merchants or individual users.

[0226] Step 604: receiving the image to be predicted input by the user based on the image input interface.

[0227] Step 606: Input the image to be predicted into the first machine learning model to obtain an initial label of the image to be predicted and an initial prediction probability of the initial label.

[0228] Step 608: Input the initial prediction probability of the initial label into the second machine learning model to obtain the target prediction probability of the initial label.

[0229] Step 610: Determine the target label of the image to be predicted based on the target prediction probability of the initial label and return it to the user.

[0230] It should be noted that, for the part of the second tag determination method provided in the embodiments of this specification that corresponds to the embodiment of the first tag determination method mentioned above, reference can be made to the detailed description in the embodiment of the first tag determination method mentioned above, and will not be repeated here.

[0231] In an embodiment of the present specification, the label determination method inputs the initial prediction result of the image to be predicted obtained by the first machine learning model into the second machine learning model for secondary correction. By updating the initial prediction result by the second machine learning model, the recognition accuracy of the image label corresponding to the image to be predicted can be greatly improved; by introducing Message Passing Networks to model the relationship between labels, making full use of the co-occurrence and mutually exclusive relationship between labels, and combining the recognition ability of the original label model, the coverage range and recognition accuracy can be greatly expanded.

[0232] See also Figure 7 , Figure 7 A flowchart of a third tag determination method provided according to an embodiment of the present specification is shown, comprising the following steps.

[0233] Step 702: receiving a call request sent by a user, wherein the call request carries an image to be predicted.

[0234] Step 704: Input the image to be predicted into the first machine learning model to obtain an initial label of the image to be predicted and an initial prediction probability of the initial label.

[0235] Step 706: Input the initial prediction probability of the initial label into the second machine learning model to obtain the target prediction probability of the initial label.

[0236] Step 708: Determine the target label of the image to be predicted based on the target prediction probability of the initial label and return it to the user.

[0237] It should be noted that, for the part of the third tag determination method provided in the embodiments of this specification that corresponds to the embodiment of the first tag determination method mentioned above, reference can be made to the detailed description in the embodiment of the first tag determination method mentioned above, and will not be repeated here.

[0238] In actual applications, the label determination method is applied to the local server and an API interface is provided for the user. After receiving the API call request sent by the user, based on the image to be predicted carried in the user's call request, the target label of the image to be predicted is quickly and accurately obtained through the label determination method, and the target label is returned to the user to improve the user experience.

[0239] In an embodiment of the present specification, the label determination method inputs the initial prediction result of the image to be predicted obtained by the first machine learning model into the second machine learning model for secondary correction. By updating the initial prediction result by the second machine learning model, the recognition accuracy of the image label corresponding to the image to be predicted can be greatly improved; by introducing Message Passing Networks to model the relationship between labels, making full use of the co-occurrence and mutually exclusive relationship between labels, and combining the recognition ability of the original label model, the coverage range and recognition accuracy can be greatly expanded.

[0240] See also Figure 8 , Figure 8 A flowchart of a second model training method provided according to an embodiment of the present specification is shown, comprising the following steps.

[0241] Step 802: Displaying an image input interface to the user based on the user's call request.

[0242] Step 804: receiving a sample image training set input by the user based on the image input interface, wherein the sample image training set includes sample images, sample labels corresponding to the sample images, and true prediction probabilities of the sample labels.

[0243] Step 806: Input the sample image into the first machine learning model to obtain a sample label of the sample image and an initial prediction probability of the sample label.

[0244] Step 808: Construct a second machine learning model based on the sample labels and the co-occurrence probabilities between the sample labels.

[0245] Step 810: Adjust the initial prediction probability of the sample label according to the co-occurrence probability between the sample labels in the second machine learning model to obtain the target prediction probability of the sample label.

[0246] Step 812: The first machine learning model and the second machine learning model are trained based on the target prediction probability and the true prediction probability of the sample label and returned to the user.

[0247] It should be noted that the part of the second model training method provided in the embodiments of this specification that corresponds to the embodiment of the first model training method mentioned above can be found in the detailed description in the embodiment of the first model training method mentioned above, and will not be repeated here.

[0248] The model training method of the embodiment of the present specification first trains an initial deep convolutional neural network model using sample images and sample labels, and constructs a message passing network model using sample labels of sample images; then, the initial prediction probability of the sample labels of the sample images obtained by the initial deep convolutional neural network model is input into the message passing network model for iterative updating, and the initial deep convolutional neural network model and the message passing network model are trained according to the target prediction probability and the true prediction probability after the iterative update to obtain the final deep convolutional neural network model and the message passing network model with higher recognition accuracy.

[0249] See also Fig. 9 , Fig. 9 A flowchart of a third model training method provided according to an embodiment of the present specification is shown, comprising the following steps.

[0250] Step 902: receiving a call request sent by a user, wherein the call request carries a sample image training set, wherein the sample image training set includes sample images, sample labels corresponding to the sample images, and true prediction probabilities of the sample labels.

[0251] Step 904: Input the sample image into a first machine learning model to obtain a sample label of the sample image and an initial prediction probability of the sample label.

[0252] Step 906: Construct a second machine learning model based on the sample labels and the co-occurrence probabilities between the sample labels.

[0253] Step 908: Adjust the initial prediction probability of the sample label according to the co-occurrence probability between the sample labels in the second machine learning model to obtain the target prediction probability of the sample label.

[0254] Step 910: The first machine learning model and the second machine learning model are obtained by training based on the target prediction probability and the true prediction probability of the sample label and returned to the user.

[0255] It should be noted that, for the part of the third model training method provided in the embodiments of this specification that corresponds to the embodiment of the first model training method mentioned above, reference can be made to the detailed description in the embodiment of the first model training method mentioned above, and will not be repeated here.

[0256] The model training method of the embodiment of the present specification first trains an initial deep convolutional neural network model using sample images and sample labels, and constructs a message passing network model using sample labels of sample images; then, the initial prediction probability of the sample labels of the sample images obtained by the initial deep convolutional neural network model is input into the message passing network model for iterative updating, and the initial deep convolutional neural network model and the message passing network model are trained according to the target prediction probability and the true prediction probability after the iterative update to obtain the final deep convolutional neural network model and the message passing network model with higher recognition accuracy.

[0257] Corresponding to the above method embodiment, this specification also provides a tag determination device embodiment, Fig.10 FIG. 1 shows a schematic diagram of the structure of a first tag determination device provided by an embodiment of this specification. Fig.10 As shown, the device comprises:

[0258] A first receiving module 1002 is configured to receive an image to be predicted;

[0259] A first probability obtaining module 1004 is configured to input the image to be predicted into a first machine learning model to obtain an initial label of the image to be predicted and an initial prediction probability of the initial label;

[0260] A second probability acquisition module 1006 is configured to input the initial prediction probability of the initial label into a second machine learning model to obtain a target prediction probability of the initial label;

[0261] The first label determination module 1008 is configured to determine the target label of the image to be predicted based on the target prediction probability of the initial label,

[0262] The first machine learning model is suitable for labeling the image to be predicted, and outputting an initial label of the image to be predicted and an initial prediction probability of the initial label.

[0263] The second machine learning model is suitable for adjusting the initial prediction probability of the initial label and outputting the adjusted target prediction probability of the initial label.

[0264] Optionally, the first machine learning model and the second machine learning model are obtained by training a training module, wherein the training module is configured to:

[0265] Acquire a sample image training set, wherein the sample image training set includes sample images, sample labels corresponding to the sample images, and true prediction probabilities of the sample labels;

[0266] Inputting the sample image into a first machine learning model to obtain a sample label of the sample image and an initial prediction probability of the sample label;

[0267] Building a second machine learning model based on the sample labels and the co-occurrence probabilities between the sample labels;

[0268] Adjusting the initial prediction probability of the sample label according to the co-occurrence probability between the sample labels in the second machine learning model to obtain the target prediction probability of the sample label;

[0269] The first machine learning model and the second machine learning model are obtained by training based on the target prediction probability and the true prediction probability of the sample label.

[0270] Optionally, the training module is further configured to:

[0271] Constructing a sample label matrix based on the sample labels, and determining the co-occurrence probability of each sample label with other sample labels according to the sample label matrix;

[0272] Determine each sample label as a node, and determine the co-occurrence probability of each sample label with other sample labels as an edge;

[0273] A second machine learning model is constructed based on the nodes and edges determined by the sample labels.

[0274] Optionally, the training module is further configured to:

[0275] A loss function is calculated based on the target prediction probability and the true prediction probability of the sample label, and the first machine learning model and the second machine learning model are trained according to the loss function to obtain the first machine learning model and the second machine learning model.

[0276] Optionally, the first machine learning model includes an image tag recognition model, and the second machine learning model includes a message passing network model.

[0277] The first machine learning model inputs the sample image and outputs a sample label corresponding to the sample image and an initial prediction probability of the sample label.

[0278] The second machine learning model inputs the initial prediction probability of the sample label and outputs the target prediction probability of the sample label.

[0279] Optionally, the first tag determining module 1008 is further configured to:

[0280] The target prediction probability of the initial label is determined, and the initial label whose target prediction probability is greater than or equal to a preset probability threshold is determined as the target label of the image to be predicted.

[0281] Optionally, the device further includes:

[0282] The information generation module is configured to generate image description information for the image to be predicted based on the target label.

[0283] Optionally, the image to be predicted includes a video image formed after being segmented according to a preset segmentation method.

[0284] In an embodiment of the present specification, the label determination device inputs the initial prediction result of the image to be predicted obtained by the first machine learning model into the second machine learning model for secondary correction. By updating the initial prediction result by the second machine learning model, the recognition accuracy of the image label corresponding to the image to be predicted can be greatly improved; by introducing Message Passing Networks to model the relationship between labels, making full use of the co-occurrence and mutually exclusive relationships between labels, and combining the recognition ability of the original label model, the coverage range and recognition accuracy can be greatly expanded.

[0285] The above is a schematic scheme of the first tag determination device of this embodiment. It should be noted that the technical scheme of the tag determination device and the technical scheme of the first tag determination method mentioned above belong to the same concept, and the details not described in detail in the technical scheme of the tag determination device can be referred to the description of the technical scheme of the above tag determination method.

[0286] Corresponding to the above method embodiment, this specification also provides a tag determination device embodiment, Fig.11 FIG. 2 shows a schematic diagram of the structure of a second tag determination device provided by an embodiment of this specification. Fig.11As shown, the device comprises:

[0287] The first interface display module 1102 is configured to display an image input interface for the user based on a user's call request;

[0288] The second receiving module 1104 is configured to receive the image to be predicted input by the user based on the image input interface;

[0289] The third probability obtaining module 1106 is configured to input the image to be predicted into the first machine learning model to obtain an initial label of the image to be predicted and an initial prediction probability of the initial label;

[0290] The fourth probability acquisition module 1108 is configured to input the initial prediction probability of the initial label into the second machine learning model to obtain the target prediction probability of the initial label;

[0291] The second label determination module 1110 is configured to determine the target label of the image to be predicted based on the target prediction probability of the initial label and return it to the user.

[0292] In an embodiment of the present specification, the label determination device inputs the initial prediction result of the image to be predicted obtained by the first machine learning model into the second machine learning model for secondary correction. By updating the initial prediction result by the second machine learning model, the recognition accuracy of the image label corresponding to the image to be predicted can be greatly improved; by introducing Message Passing Networks to model the relationship between labels, making full use of the co-occurrence and mutually exclusive relationships between labels, and combining the recognition ability of the original label model, the coverage range and recognition accuracy can be greatly expanded.

[0293] The above is a schematic scheme of the second tag determination device of this embodiment. It should be noted that the technical scheme of the tag determination device and the technical scheme of the second tag determination method mentioned above belong to the same concept, and the details of the technical scheme of the tag determination device that are not described in detail can all be referred to the description of the technical scheme of the above-mentioned tag determination method.

[0294] Corresponding to the above method embodiment, this specification also provides a tag determination device embodiment, Fig.12 FIG. 2 shows a schematic diagram of the structure of a third tag determination device provided by an embodiment of this specification. Fig.12 As shown, the device comprises:

[0295] The third receiving module 1202 is configured to receive a call request sent by a user, wherein the call request carries an image to be predicted;

[0296] A fifth probability obtaining module 1204 is configured to input the image to be predicted into the first machine learning model to obtain an initial label of the image to be predicted and an initial prediction probability of the initial label;

[0297] The sixth probability acquisition module 1206 is configured to input the initial prediction probability of the initial label into the second machine learning model to obtain the target prediction probability of the initial label;

[0298] The third label determination module 1208 is configured to determine the target label of the image to be predicted based on the target prediction probability of the initial label and return it to the user.

[0299] In an embodiment of the present specification, the label determination device inputs the initial prediction result of the image to be predicted obtained by the first machine learning model into the second machine learning model for secondary correction. By updating the initial prediction result by the second machine learning model, the recognition accuracy of the image label corresponding to the image to be predicted can be greatly improved; by introducing Message Passing Networks to model the relationship between labels, making full use of the co-occurrence and mutually exclusive relationships between labels, and combining the recognition ability of the original label model, the coverage range and recognition accuracy can be greatly expanded.

[0300] The above is a schematic scheme of the third tag determination device of this embodiment. It should be noted that the technical scheme of the tag determination device and the technical scheme of the third tag determination method mentioned above belong to the same concept, and the details of the technical scheme of the tag determination device that are not described in detail can be found in the description of the technical scheme of the above-mentioned tag determination method.

[0301] Corresponding to the above method embodiment, this specification also provides a model training device embodiment, Fig.13 FIG. 1 shows a schematic diagram of the structure of a first model training device provided by an embodiment of this specification. Fig.13 As shown, the device comprises:

[0302] A first sample acquisition module 1302 is configured to acquire a sample image training set, wherein the sample image training set includes sample images, sample labels corresponding to the sample images, and true prediction probabilities of the sample labels;

[0303] A seventh probability acquisition module 1304 is configured to input the sample image into the first machine learning model to obtain a sample label of the sample image and an initial prediction probability of the sample label;

[0304] A first model building module 1306 is configured to build a second machine learning model based on the sample labels and the co-occurrence probability between the sample labels;

[0305] An eighth probability obtaining module 1308 is configured to adjust the initial prediction probability of the sample label according to the co-occurrence probability between the sample labels in the second machine learning model to obtain a target prediction probability of the sample label;

[0306] The first model training module 1310 is configured to obtain the first machine learning model and the second machine learning model through training based on the target prediction probability and the true prediction probability of the sample label.

[0307] Optionally, the first model building module 1306 is further configured to:

[0308] Constructing a sample label matrix based on the sample labels, and determining the co-occurrence probability of each sample label with other sample labels according to the sample label matrix;

[0309] Determine each sample label as a node, and determine the co-occurrence probability of each sample label with other sample labels as an edge;

[0310] A second machine learning model is constructed based on the nodes and edges determined by the sample labels.

[0311] Optionally, the first model training module 1310 is further configured to:

[0312] A loss function is calculated based on the target prediction probability and the true prediction probability of the sample label, and the first machine learning model and the second machine learning model are trained according to the loss function to obtain the first machine learning model and the second machine learning model.

[0313] Optionally, the first machine learning model includes an image tag recognition model, and the second machine learning model includes a message passing network model.

[0314] The first machine learning model inputs the sample image and outputs a sample label corresponding to the sample image and an initial prediction probability of the sample label.

[0315] The second machine learning model inputs the initial prediction probability of the sample label and outputs the target prediction probability of the sample label.

[0316] The model training device of the embodiment of the present specification first trains an initial deep convolutional neural network model using sample images and sample labels, and constructs a message passing network model using sample labels of sample images; then, the initial prediction probability of the sample labels of the sample images obtained by the initial deep convolutional neural network model is input into the message passing network model for iterative updating, and the initial deep convolutional neural network model and the message passing network model are trained according to the target prediction probability and the true prediction probability after the iterative update to obtain the final deep convolutional neural network model and the message passing network model with higher recognition accuracy.

[0317] The above is a schematic scheme of the first model training device of this embodiment. It should be noted that the technical scheme of the model training device and the technical scheme of the first model training method mentioned above belong to the same concept, and the details not described in detail in the technical scheme of the model training device can be referred to the description of the technical scheme of the above model training method.

[0318] Corresponding to the above method embodiment, this specification also provides a model training device embodiment, Fig.14 FIG. 2 shows a schematic diagram of the structure of a second model training device provided by an embodiment of the present specification. Fig.14 As shown, the device comprises:

[0319] The second interface display module 1402 is configured to display an image input interface for the user based on a call request from the user;

[0320] The fourth receiving module 1404 is configured to receive a sample image training set input by the user based on the image input interface, wherein the sample image training set includes sample images, sample labels corresponding to the sample images, and true prediction probabilities of the sample labels;

[0321] A ninth probability acquisition module 1406 is configured to input the sample image into the first machine learning model to obtain a sample label of the sample image and an initial prediction probability of the sample label;

[0322] A second model building module 1408 is configured to build a second machine learning model based on the sample labels and the co-occurrence probability between the sample labels;

[0323] a tenth probability obtaining module 1410, configured to adjust the initial prediction probability of the sample label according to the co-occurrence probability between the sample labels in the second machine learning model to obtain a target prediction probability of the sample label;

[0324] The second model training module 1412 is configured to obtain the first machine learning model and the second machine learning model based on the target prediction probability and the true prediction probability of the sample label and return them to the user.

[0325] The model training device of the embodiment of the present specification first trains an initial deep convolutional neural network model using sample images and sample labels, and constructs a message passing network model using sample labels of sample images; then, the initial prediction probability of the sample labels of the sample images obtained by the initial deep convolutional neural network model is input into the message passing network model for iterative updating, and the initial deep convolutional neural network model and the message passing network model are trained according to the target prediction probability and the true prediction probability after the iterative update to obtain the final deep convolutional neural network model and the message passing network model with higher recognition accuracy.

[0326] The above is a schematic scheme of the second model training device of this embodiment. It should be noted that the technical scheme of the model training device and the technical scheme of the second model training method mentioned above belong to the same concept, and the details not described in detail in the technical scheme of the model training device can be referred to the description of the technical scheme of the above model training method.

[0327] Corresponding to the above method embodiment, this specification also provides a model training device embodiment, Fig.15 FIG. 2 shows a schematic diagram of the structure of a third model training device provided by an embodiment of this specification. Fig.15 As shown, the device comprises:

[0328] The fifth receiving module 1502 is configured to receive a call request sent by a user, wherein the call request carries a sample image training set, wherein the sample image training set includes a sample image, a sample label corresponding to the sample image, and a true prediction probability of the sample label;

[0329] An eleventh probability acquisition module 1504 is configured to input the sample image into a first machine learning model to obtain a sample label of the sample image and an initial prediction probability of the sample label;

[0330] A third model building module 1506 is configured to build a second machine learning model based on the sample labels and the co-occurrence probability between the sample labels;

[0331] A twelfth probability obtaining module 1508 is configured to adjust the initial prediction probability of the sample label according to the co-occurrence probability between the sample labels in the second machine learning model to obtain the target prediction probability of the sample label;

[0332] The third model training module 1510 is configured to obtain the first machine learning model and the second machine learning model based on the target prediction probability and the true prediction probability of the sample label and return them to the user.

[0333] The model training device of the embodiment of the present specification first trains an initial deep convolutional neural network model using sample images and sample labels, and constructs a message passing network model using sample labels of sample images; then, the initial prediction probability of the sample labels of the sample images obtained by the initial deep convolutional neural network model is input into the message passing network model for iterative updating, and the initial deep convolutional neural network model and the message passing network model are trained according to the target prediction probability and the true prediction probability after the iterative update to obtain the final deep convolutional neural network model and the message passing network model with higher recognition accuracy.

[0334] The above is a schematic scheme of the third model training device of this embodiment. It should be noted that the technical scheme of the model training device and the technical scheme of the third model training method mentioned above belong to the same concept, and the details not described in detail in the technical scheme of the model training device can be referred to the description of the technical scheme of the model training method mentioned above.

[0335] See also Fig.16 , Fig.16 A flowchart of a commodity identification method provided by an embodiment of this specification is shown, which specifically includes the following steps.

[0336] Step 1602: receiving a product image, and determining a target label of the product in the product image based on the label recognition method.

[0337] Among them, the above-mentioned tag identification method is the first tag identification method, the second tag identification method or the third tag identification method in the embodiments of this specification.

[0338] The product image is an image containing products of any category, such as food products, clothing products, etc.

[0339] Specifically, after receiving product images of multiple products, the target label of the product in each product image is identified using any of the above-mentioned label recognition methods. For example, a product image containing a skirt can be identified by using any of the above-mentioned label recognition methods to obtain the target label of the product image: skirt, red, long, etc.

[0340] Step 1604: Receive a user's search request for a product, wherein the search request carries description information of the product.

[0341] The description information of a product is text information describing a product. For example, if the product is a skirt, the description information of the skirt may be a red skirt, a long skirt, etc.

[0342] Specifically, a search request for a certain product to be searched is received from a user, and the search request carries description information of the product, and a certain product can be searched in a targeted manner through the description information. For example, a search request for a skirt is received from a user, and the search request carries description information of the skirt: red skirt.

[0343] Step 1606: Match the description information of the product with the target tag to determine the target tag that matches the description information of the product.

[0344] Specifically, the description information of the product is matched with the target tag, that is, keywords or key words in the description information of the product are extracted, and these keywords or key words are matched with the target tag to determine the target tag that matches the description information of the product.

[0345] Using the above example, if the description of the product is a red skirt, then the keywords extracted from the description of the product are red and skirt. Then red and skirt are matched with the target tags to match all target tags corresponding to red and skirt.

[0346] Step 1608: Determine the product corresponding to the target tag matching the description information of the product as the target product, and send the target product to the user.

[0347] Specifically, after the target tag corresponding to the description information of the product is matched, the corresponding product is determined based on the target tag, and the products are sent to the user as target products.

[0348] Still using the above example, all products corresponding to the target labels matching red and skirt can be recommended to the user as target products.

[0349] In the embodiments of this specification, the commodity identification method is applied in the C2C e-commerce platform. In the C2C e-commerce platform, ordinary sellers have not received professional training similar to that of professional sellers on the B2C platform. When publishing commodities, the commodity information filled in by the editor is limited, and the commodity information is often filled in incorrectly or omitted, resulting in the commodity not being effectively recalled by the search engine when it is searched by buyers. Then, in specific implementation, the C2C e-commerce platform can use the label identification method provided in the embodiments of this specification to first automatically add detailed and accurate commodity information, i.e., the target label of the commodity, to all commodities in the C2C e-commerce platform, simplifying the process of commodity release (i.e., the seller can directly generate commodity description information based on the target label without manual editing), and at the same time, when performing commodity search tasks, it can also promote the search and recall of commodities based on rich and accurate structured commodity target labels, so that the commodity search and recall effect is obvious in the C2C e-commerce scenario.

[0350] See also Fig.17 , Fig.17 A schematic diagram of the structure of a commodity identification device provided by an embodiment of this specification is shown, and the device includes:

[0351] The product labeling module 1702 is configured to receive a product image and determine a target label of the product in the product image based on the label recognition method;

[0352] The search request receiving module 1704 is configured to receive a user's search request for a product, wherein the search request carries description information of the product;

[0353] The tag matching module 1706 is configured to match the description information of the product with the target tag to determine the target tag that matches the description information of the product;

[0354] The target product determination module 1708 is configured to determine the product corresponding to the target tag matching the description information of the product as the target product, and send the target product to the user.

[0355] In the embodiments of this specification, the commodity identification device is applied in the C2C e-commerce platform. In the C2C e-commerce platform, ordinary sellers have not received professional training similar to that of professional sellers on the B2C platform. When publishing commodities, the commodity information filled in by the editor is limited, and the commodity information is often filled in incorrectly or omitted, resulting in the commodity not being effectively recalled by the search engine when it is searched by buyers. Then, in specific implementation, the C2C e-commerce platform can use the label identification method provided in the embodiments of this specification to first automatically add detailed and accurate commodity information, i.e., the target label of the commodity, to all commodities in the C2C e-commerce platform, simplifying the process of commodity release (i.e., the seller can directly generate commodity description information based on the target label without manual editing), and at the same time, when performing commodity search tasks, it can also promote the search and recall of commodities based on rich and accurate structured commodity target labels, making the commodity search and recall effect obvious in the C2C e-commerce scenario.

[0356] Fig.18 The block diagram of a computing device 1800 according to one embodiment of the present specification is shown. The components of the computing device 1800 include but are not limited to a memory 1810 and a processor 1820. The processor 1820 is connected to the memory 1810 via a bus 1830, and the database 1850 is used to store data.

[0357] The computing device 1800 also includes an access device 1840 that enables the computing device 1800 to communicate via one or more networks 1860. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 1840 may include one or more of any type of network interface (e.g., a network interface card (NIC)) wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a World Wide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, a near field communication (NFC) interface, and the like.

[0358] In one embodiment of the present specification, the above components of the computing device 1800 and Fig.18 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Fig.18 The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0359] Computing device 1800 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or PC. Computing device 1800 may also be a mobile or stationary server.

[0360] Among them, the processor 1820 is used to execute computer-executable instructions, and when the instructions are executed by the processor 1820, the steps of the label determination method or the steps of the model training method are implemented.

[0361] The above is a schematic scheme of a computing device of this embodiment. It should be noted that the technical scheme of the computing device and the technical scheme of the above-mentioned label determination method or model training method belong to the same concept, and the details not described in detail in the technical scheme of the computing device can be referred to the description of the technical scheme of the above-mentioned label determination method or model training method.

[0362] An embodiment of the present specification also provides a computer-readable storage medium storing computer instructions, which, when executed by a processor, implement the steps of the label determination method or the steps of the model training method.

[0363] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the above-mentioned label determination method or model training method belong to the same concept, and the details not described in detail in the technical scheme of the storage medium can be referred to the description of the technical scheme of the above-mentioned label determination method or model training method.

[0364] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0365] The computer instructions include computer program codes, which may be in source code form, object code form, executable files or some intermediate forms, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0366] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of this specification are not limited by the order of the actions described, because according to the embodiments of this specification, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of this specification.

[0367] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0368] The preferred embodiments of this specification disclosed above are only used to help explain this specification. The optional embodiments do not describe all the details in detail, nor do they limit the invention to only the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of the embodiments of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of this specification, so that technicians in the relevant technical field can well understand and use this specification. This specification is only limited by the claims and their full scope and equivalents.

Claims

1. A label determination method, comprising: Receiving an image to be predicted; Inputting the image to be predicted into a first machine learning model to obtain an initial label of the image to be predicted and an initial prediction probability of the initial label; Inputting the initial prediction probability of the initial label into a second machine learning model to obtain a target prediction probability of the initial label, wherein the second machine learning model is constructed according to the sample labels of the sample images and the co-occurrence probability between each sample label and other sample labels; Determine the target label of the image to be predicted based on the target prediction probability of the initial label, The first machine learning model is suitable for labeling the image to be predicted, and outputting an initial label of the image to be predicted and an initial prediction probability of the initial label. The second machine learning model is adapted to adjust the initial prediction probability of the initial label and output the adjusted target prediction probability of the initial label, The first machine learning model and the second machine learning model are trained according to the target prediction probability of the sample labels. The target prediction probability of the sample labels is obtained by adjusting the initial prediction probability of the sample labels according to the co-occurrence probability between the sample labels in the second machine learning model. The initial prediction probability of the sample labels is obtained based on the first machine learning model.

2. According to the label determination method of claim 1, the training process of the first machine learning model and the second machine learning model is as follows: Get a sample image training set, where The sample image training set includes sample images, sample labels corresponding to the sample images, and true prediction probabilities of the sample labels; Inputting the sample image into a first machine learning model to obtain a sample label of the sample image and an initial prediction probability of the sample label; Building a second machine learning model based on the sample labels and the co-occurrence probabilities between the sample labels; Adjusting the initial prediction probability of the sample label according to the co-occurrence probability between the sample labels in the second machine learning model to obtain the target prediction probability of the sample label; The first machine learning model and the second machine learning model are obtained by training based on the target prediction probability and the true prediction probability of the sample label.

3. According to the label determination method of claim 2, constructing a second machine learning model based on the sample labels and the co-occurrence probability between the sample labels comprises: Constructing a sample label matrix based on the sample labels, and determining the co-occurrence probability of each sample label with other sample labels according to the sample label matrix; Determine each sample label as a node, and determine the co-occurrence probability of each sample label with other sample labels as an edge; A second machine learning model is constructed based on the nodes and edges determined by the sample labels.

4. According to the label determination method of claim 2, the training of the first machine learning model and the second machine learning model based on the target prediction probability and the true prediction probability of the sample label comprises: A loss function is calculated based on the target prediction probability and the true prediction probability of the sample label, and the first machine learning model and the second machine learning model are trained according to the loss function to obtain the first machine learning model and the second machine learning model.

5. The tag determination method according to claim 2, wherein the first machine learning model comprises an image tag recognition model, and the second machine learning model comprises a message passing network model. in, The first machine learning model inputs the sample image, outputs a sample label corresponding to the sample image and an initial prediction probability of the sample label, The second machine learning model inputs the initial prediction probability of the sample label and outputs the target prediction probability of the sample label.

6. The label determination method according to claim 1, wherein determining the target label of the image to be predicted based on the target prediction probability of the initial label comprises: The target prediction probability of the initial label is determined, and the initial label whose target prediction probability is greater than or equal to a preset probability threshold is determined as the target label of the image to be predicted.

7. The label determination method according to claim 6, after determining the initial label whose target prediction probability is greater than or equal to a preset probability threshold as the target label of the image to be predicted, further comprising: Image description information is generated for the image to be predicted based on the target label.

8. According to the label determination method according to any one of claims 1 to 7, the image to be predicted includes a video image formed after being segmented according to a preset segmentation method.

9. A model training method, comprising: Acquire a sample image training set, wherein the sample image training set includes sample images, sample labels corresponding to the sample images, and true prediction probabilities of the sample labels; Inputting the sample image into a first machine learning model to obtain a sample label of the sample image and an initial prediction probability of the sample label; Building a second machine learning model based on the sample labels and the co-occurrence probabilities between the sample labels; Adjusting the initial prediction probability of the sample label according to the co-occurrence probability between the sample labels in the second machine learning model to obtain the target prediction probability of the sample label; The first machine learning model and the second machine learning model are obtained by training based on the target prediction probability and the true prediction probability of the sample label.

10. According to the model training method of claim 9, constructing a second machine learning model based on the sample labels and the co-occurrence probabilities between the sample labels comprises: Constructing a sample label matrix based on the sample labels, and determining the co-occurrence probability of each sample label with other sample labels according to the sample label matrix; Determine each sample label as a node, and determine the co-occurrence probability of each sample label with other sample labels as an edge; A second machine learning model is constructed based on the nodes and edges determined by the sample labels.

11. According to the model training method of claim 9, the training based on the target prediction probability and the true prediction probability of the sample label to obtain the first machine learning model and the second machine learning model comprises: A loss function is calculated based on the target prediction probability and the true prediction probability of the sample label, and the first machine learning model and the second machine learning model are trained according to the loss function to obtain the first machine learning model and the second machine learning model.

12. The model training method according to any one of claims 9 to 11, wherein the first machine learning model comprises an image tag recognition model, and the second machine learning model comprises a message passing network model. in, The first machine learning model inputs the sample image, outputs a sample label corresponding to the sample image and an initial prediction probability of the sample label, The second machine learning model inputs the initial prediction probability of the sample label and outputs the target prediction probability of the sample label.

13. A label determination method, comprising: Displaying an image input interface to the user based on a call request from the user; receiving an image to be predicted input by the user based on the image input interface; Inputting the image to be predicted into a first machine learning model to obtain an initial label of the image to be predicted and an initial prediction probability of the initial label; Inputting the initial prediction probability of the initial label into a second machine learning model to obtain a target prediction probability of the initial label, wherein the second machine learning model is constructed according to the sample labels of the sample images and the co-occurrence probability between each sample label and other sample labels; Determine the target label of the image to be predicted based on the target prediction probability of the initial label and return it to the user, The first machine learning model is suitable for labeling the image to be predicted, and outputting an initial label of the image to be predicted and an initial prediction probability of the initial label. The second machine learning model is adapted to adjust the initial prediction probability of the initial label and output the adjusted target prediction probability of the initial label, The first machine learning model and the second machine learning model are trained according to the target prediction probability of the sample labels. The target prediction probability of the sample labels is obtained by adjusting the initial prediction probability of the sample labels according to the co-occurrence probability between the sample labels in the second machine learning model. The initial prediction probability of the sample labels is obtained based on the first machine learning model.

14. A label determination method, comprising: Receiving a call request sent by a user, wherein the call request carries an image to be predicted; Inputting the image to be predicted into a first machine learning model to obtain an initial label of the image to be predicted and an initial prediction probability of the initial label; Inputting the initial prediction probability of the initial label into a second machine learning model to obtain a target prediction probability of the initial label, wherein the second machine learning model is constructed according to the sample labels of the sample images and the co-occurrence probability between each sample label and other sample labels; Determine the target label of the image to be predicted based on the target prediction probability of the initial label and return it to the user, The first machine learning model is suitable for labeling the image to be predicted, and outputting an initial label of the image to be predicted and an initial prediction probability of the initial label. The second machine learning model is adapted to adjust the initial prediction probability of the initial label and output the adjusted target prediction probability of the initial label, The first machine learning model and the second machine learning model are trained according to the target prediction probability of the sample labels. The target prediction probability of the sample labels is obtained by adjusting the initial prediction probability of the sample labels according to the co-occurrence probability between the sample labels in the second machine learning model. The initial prediction probability of the sample labels is obtained based on the first machine learning model.

15. A model training method, comprising: Displaying an image input interface to the user based on a call request from the user; Receiving a sample image training set input by the user based on the image input interface, wherein the sample image training set includes sample images, sample labels corresponding to the sample images, and true prediction probabilities of the sample labels; Inputting the sample image into a first machine learning model to obtain a sample label of the sample image and an initial prediction probability of the sample label; Building a second machine learning model based on the sample labels and the co-occurrence probabilities between the sample labels; Adjusting the initial prediction probability of the sample label according to the co-occurrence probability between the sample labels in the second machine learning model to obtain the target prediction probability of the sample label; The first machine learning model and the second machine learning model are trained based on the target prediction probability and the true prediction probability of the sample label and returned to the user.

16. A model training method, comprising: Receive a call request sent by a user, wherein the call request carries a sample image training set, wherein the sample image training set includes a sample image, a sample label corresponding to the sample image, and a true prediction probability of the sample label; Inputting the sample image into a first machine learning model to obtain a sample label of the sample image and an initial prediction probability of the sample label; Building a second machine learning model based on the sample labels and the co-occurrence probabilities between the sample labels; Adjusting the initial prediction probability of the sample label according to the co-occurrence probability between the sample labels in the second machine learning model to obtain the target prediction probability of the sample label; The first machine learning model and the second machine learning model are trained based on the target prediction probability and the true prediction probability of the sample label and returned to the user.

17. A commodity identification method, comprising: Receiving a product image, and determining a target label of the product in the product image based on the label determination method according to any one of claims 1 to 8; Receiving a user's search request for a product, wherein the search request carries description information of the product; Matching the description information of the product with the target tag to determine the target tag that matches the description information of the product; The commodity corresponding to the target tag matching the description information of the commodity is determined as a target commodity, and the target commodity is sent to the user.

18. A label determination device, comprising: A first receiving module is configured to receive an image to be predicted; A first probability acquisition module is configured to input the image to be predicted into a first machine learning model to obtain an initial label of the image to be predicted and an initial prediction probability of the initial label; A second probability acquisition module is configured to input the initial prediction probability of the initial label into a second machine learning model to obtain a target prediction probability of the initial label, wherein the second machine learning model is constructed according to the sample labels of the sample images and the co-occurrence probability between each sample label and other sample labels; A first label determination module is configured to determine a target label of the image to be predicted based on a target prediction probability of the initial label, The first machine learning model is suitable for labeling the image to be predicted, and outputting an initial label of the image to be predicted and an initial prediction probability of the initial label. The second machine learning model is adapted to adjust the initial prediction probability of the initial label and output the adjusted target prediction probability of the initial label, The first machine learning model and the second machine learning model are trained according to the target prediction probability of the sample labels. The target prediction probability of the sample labels is obtained by adjusting the initial prediction probability of the sample labels according to the co-occurrence probability between the sample labels in the second machine learning model. The initial prediction probability of the sample labels is obtained based on the first machine learning model.

19. A label determination device, comprising: A first interface display module, configured to display an image input interface for the user based on a call request from the user; A second receiving module is configured to receive the image to be predicted input by the user based on the image input interface; A third probability acquisition module is configured to input the image to be predicted into the first machine learning model to obtain an initial label of the image to be predicted and an initial prediction probability of the initial label; a fourth probability acquisition module, configured to input the initial prediction probability of the initial label into a second machine learning model to obtain a target prediction probability of the initial label, wherein the second machine learning model is constructed according to the sample labels of the sample images and the co-occurrence probability between each sample label and other sample labels; A second label determination module is configured to determine a target label of the image to be predicted based on the target prediction probability of the initial label and return it to the user, The first machine learning model is suitable for labeling the image to be predicted, and outputting an initial label of the image to be predicted and an initial prediction probability of the initial label. The second machine learning model is adapted to adjust the initial prediction probability of the initial label and output the adjusted target prediction probability of the initial label, The first machine learning model and the second machine learning model are trained according to the target prediction probability of the sample labels. The target prediction probability of the sample labels is obtained by adjusting the initial prediction probability of the sample labels according to the co-occurrence probability between the sample labels in the second machine learning model. The initial prediction probability of the sample labels is obtained based on the first machine learning model.

20. A label determination device, comprising: A third receiving module is configured to receive a call request sent by a user, wherein the call request carries an image to be predicted; a fifth probability acquisition module, configured to input the image to be predicted into the first machine learning model to obtain an initial label of the image to be predicted and an initial prediction probability of the initial label; a sixth probability acquisition module, configured to input the initial prediction probability of the initial label into a second machine learning model to obtain a target prediction probability of the initial label, wherein the second machine learning model is constructed according to the sample labels of the sample images and the co-occurrence probability between each sample label and other sample labels; A third label determination module is configured to determine a target label of the image to be predicted based on the target prediction probability of the initial label and return it to the user, The first machine learning model is suitable for labeling the image to be predicted, and outputting an initial label of the image to be predicted and an initial prediction probability of the initial label. The second machine learning model is adapted to adjust the initial prediction probability of the initial label and output the adjusted target prediction probability of the initial label, The first machine learning model and the second machine learning model are trained according to the target prediction probability of the sample labels. The target prediction probability of the sample labels is obtained by adjusting the initial prediction probability of the sample labels according to the co-occurrence probability between the sample labels in the second machine learning model. The initial prediction probability of the sample labels is obtained based on the first machine learning model.

21. A model training device, comprising: A first sample acquisition module is configured to acquire a sample image training set, wherein the sample image training set includes sample images, sample labels corresponding to the sample images, and true prediction probabilities of the sample labels; a seventh probability acquisition module, configured to input the sample image into the first machine learning model to obtain a sample label of the sample image and an initial prediction probability of the sample label; A first model building module is configured to build a second machine learning model based on the sample labels and the co-occurrence probability between the sample labels; an eighth probability obtaining module, configured to adjust the initial prediction probability of the sample label according to the co-occurrence probability between the sample labels in the second machine learning model to obtain a target prediction probability of the sample label; The first model training module is configured to obtain the first machine learning model and the second machine learning model through training based on the target prediction probability and the true prediction probability of the sample label.

22. A model training device, comprising: A second interface display module is configured to display an image input interface to the user based on a call request from the user; a fourth receiving module, configured to receive a sample image training set input by the user based on the image input interface, wherein the sample image training set includes sample images, sample labels corresponding to the sample images, and true prediction probabilities of the sample labels; a ninth probability acquisition module, configured to input the sample image into the first machine learning model to obtain a sample label of the sample image and an initial predicted probability of the sample label; A second model building module is configured to build a second machine learning model based on the sample labels and the co-occurrence probability between the sample labels; a tenth probability obtaining module, configured to adjust the initial prediction probability of the sample label according to the co-occurrence probability between the sample labels in the second machine learning model to obtain a target prediction probability of the sample label; The second model training module is configured to obtain the first machine learning model and the second machine learning model based on the target prediction probability and the true prediction probability of the sample label and return them to the user.

23. A model training device, comprising: A fifth receiving module is configured to receive a call request sent by a user, wherein the call request carries a sample image training set, wherein the sample image training set includes a sample image, a sample label corresponding to the sample image, and a true prediction probability of the sample label; an eleventh probability acquisition module, configured to input the sample image into a first machine learning model to obtain a sample label of the sample image and an initial predicted probability of the sample label; A third model building module is configured to build a second machine learning model based on the sample labels and the co-occurrence probability between the sample labels; a twelfth probability acquisition module, configured to adjust the initial prediction probability of the sample label according to the co-occurrence probability between the sample labels in the second machine learning model to obtain a target prediction probability of the sample label; The third model training module is configured to obtain the first machine learning model and the second machine learning model based on the target prediction probability and the true prediction probability of the sample label and return them to the user.

24. A commodity identification device, comprising: A product labeling module, configured to receive a product image and determine a target label of the product in the product image based on the label determination method according to any one of claims 1 to 8; A search request receiving module, configured to receive a user's search request for a product, wherein the search request carries description information of the product; a tag matching module, configured to match the description information of the commodity with the target tag, and determine a target tag that matches the description information of the commodity; The target product determination module is configured to determine the product corresponding to the target tag matching the description information of the product as the target product, and send the target product to the user.

25. A computing device comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the instructions are executed by the processor, the steps of the label determination method described in any one of claims 1-8, 13, and 14 are implemented, or the steps of the model training method described in any one of claims 9-12, 15, and 16 are implemented, or the steps of the commodity identification method described in claim 17 are implemented.

26. A computer-readable storage medium storing computer instructions, which, when executed by a processor, implement the steps of the label determination method described in any one of claims 1-8, 13, and 14, or the steps of the model training method described in any one of claims 9-12, 15, and 16, or the steps of the product identification method described in claim 17.

Images