A method, apparatus, electronic device and storage medium for determining image similarity

Abstract

The embodiment of the present invention provides a method, device, electronic device and readable storage medium for determining image similarity, the method comprising: obtaining an image to be detected and multiple candidate images; inputting the image to be detected and the candidate images into a pre-trained similarity distribution model, outputting a predicted distribution, the predicted distribution is used to reflect the distribution probability of the similarity between the image to be detected and the candidate images; the similarity distribution model is trained based on multiple training samples, the training samples include a first sample image, multiple second sample images, and the similarity value between the first sample image and each second sample image; mapping the predicted distribution based on a mapping function, determining multiple target similarities, the mapping function is fitted based on the predicted distribution and similarity values ​​of the training samples. According to the embodiment of the present invention, the accuracy of similarity determination can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method, device, electronic device and storage medium for determining image similarity. Background Art

[0002] With the continuous development of information technology and artificial intelligence, image recognition is now widely used, such as identifying images of the same product in online shopping. Among them, the current recognition method uses sample consistency as the goal of model learning, which requires obtaining a large number of consistent sample images for training during the training process.

[0003] However, in practice it is usually difficult to collect a sufficient number of consistent sample images, that is, the sample images used for model training are insufficient, resulting in low accuracy of the trained model in predicting image similarity. Summary of the invention

[0004] The purpose of the embodiments of the present invention is to provide a method, device, electronic device and readable storage medium for determining image similarity, which can improve the accuracy of predicting image similarity. The specific technical solution is as follows:

[0005] In a first aspect of the present invention, a method for determining image similarity is provided, comprising:

[0006] Obtain an image to be detected and multiple candidate images;

[0007] The image to be detected and the candidate image are input into a pre-trained similarity distribution model, and a predicted distribution is output, where the predicted distribution is used to reflect the distribution probability of the similarity between the image to be detected and the candidate image; the similarity distribution model is trained based on a plurality of training samples, where the training samples include a first sample image, a plurality of second sample images, and a similarity value between the first sample image and each second sample image;

[0008] The predicted distribution is mapped based on a mapping function to determine multiple target similarities. The mapping function is fitted according to the predicted distribution and similarity values ​​of the training samples.

[0009] In a second aspect of the present invention, there is also provided an image similarity determination device, comprising:

[0010] An acquisition module, used for acquiring an image to be detected and multiple candidate images;

[0011] An input module is used to input the image to be detected and the candidate image into a pre-trained similarity distribution model, and output a predicted distribution, where the predicted distribution is used to reflect the distribution probability of the similarity between the image to be detected and the candidate image; the similarity distribution model is trained based on a plurality of training samples, where the training samples include a first sample image, a plurality of second sample images, and a similarity value between the first sample image and each second sample image;

[0012] The determination module is used to map the predicted distribution based on a mapping function to determine multiple target similarities, and the mapping function is fitted according to the predicted distribution and similarity values ​​of the training samples.

[0013] In another aspect of the present invention, a computer-readable storage medium is provided, wherein instructions are stored in the computer-readable storage medium, and when the computer-readable storage medium is run on a computer, the computer executes any of the above-mentioned data forwarding methods.

[0014] In another aspect of the present invention, a computer program product comprising instructions is provided. When the computer program product is run on a computer, the computer is enabled to execute any of the above-mentioned methods for determining image similarity.

[0015] The image similarity determination method provided by the embodiment of the present invention obtains an image to be detected and multiple candidate images that meet preset similarity conditions, and inputs each group of images to be detected and candidate images into a pre-trained similarity distribution model in turn, and outputs a predicted distribution for reflecting the distribution probability of the similarity between the image to be detected and the candidate images, wherein, since the similarity distribution model is obtained by training based on multiple training samples, the training samples include a first sample image, multiple second sample images, and the similarity value between the first sample image and each second sample image, the similarity distribution model obtained by training in this way can accurately predict the similarity distribution between the image to be detected and each candidate image, that is, the output predicted distribution. Finally, based on the mapping function obtained by fitting the predicted distribution and similarity value of the sample image, the predicted distribution is mapped, and the predicted distribution can be converted into the target similarity corresponding to each candidate image, thereby quickly and accurately determining the similarity between different candidate images and the image to be detected. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art are briefly introduced below.

[0017] Figure 1 is a flow chart of a method for determining image similarity provided by an embodiment of the present invention;

[0018] Figure 2 is a flow chart of a training method for a similarity distribution model provided by an embodiment of the present invention;

[0019] Figure 3 It is a schematic diagram of a training process and an application process of a similarity distribution model provided by an embodiment of the present invention;

[0020] Figure 4 is a schematic diagram of the structure of an image similarity determination device provided by an embodiment of the present invention;

[0021] Figure 5 It is a schematic diagram of the structure of a training device for a similarity distribution model provided by an embodiment of the present invention;

[0022] Figure 6 It is a schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0023] The technical solutions in the embodiments of the present invention will be described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0024] The image similarity determination method provided by the embodiment of the present invention can be applied to at least the following application scenarios, which are described below.

[0025] At present, in video scenes, it is usually difficult to accurately locate the exact style of clothing worn by the characters in the video. Since it is difficult to collect images of the same product, a large number of clothing images appearing in video scenes cannot be used for model training, that is, the samples for model training are insufficient, which leads to the problem that the trained model has poor performance in predicting similar images of the image to be detected.

[0026] Based on the above application scenarios, the image similarity determination method provided by the embodiment of the present invention is described in detail below.

[0027] The following is an explanation of the method for determining image similarity provided by an embodiment of the present invention.

[0028] Figure 1 The present invention provides a flowchart of a method for determining image similarity.

[0029] like Figure 1 As shown, the image similarity determination method may include steps 110 to 130, and the method is applied to an image similarity determination device, as shown below:

[0030] Step 110: Acquire an image to be detected and multiple candidate images.

[0031] Step 120, input the image to be detected and the candidate image into a pre-trained similarity distribution model, and output a predicted distribution, which is used to reflect the distribution probability of the similarity between the image to be detected and the candidate image; the similarity distribution model is trained based on multiple training samples, and the training samples include a first sample image, multiple second sample images, and a similarity value between the first sample image and each second sample image.

[0032] Step 130 , mapping the predicted distribution based on a mapping function to determine multiple target similarities, where the mapping function is fitted according to the predicted distribution and similarity values ​​of the training samples.

[0033] In the disclosed embodiment, by obtaining an image to be detected and multiple candidate images that meet preset similarity conditions, multiple candidate images can be retrieved from a preset image library according to the image features of the image to be detected, and each group of images to be detected and candidate images are sequentially input into a pre-trained similarity distribution model, and a predicted distribution for reflecting the distribution probability of the similarity between the image to be detected and the candidate image is output. Wherein, since the similarity distribution model is obtained by training based on multiple training samples, the training samples include a first sample image, multiple second sample images, and the similarity value between the first sample image and each second sample image, in the training process of the similarity distribution model, a neural network of a preset dimension is trained according to the sample prediction distribution determined by the first sample image and the second sample image, and the loss function determined by the sample target distribution determined by the multiple similarity values. When the loss function meets the preset training conditions, the difference between the sample prediction distribution and the sample target distribution is continuously reduced, and a similarity distribution model is obtained. Thus, the trained similarity distribution model can quickly and accurately output a group of similarity distributions between the image to be detected and the candidate images. The similarity distribution model trained in this way can accurately predict the similarity distribution between the image to be detected and each candidate image, that is, the output predicted distribution. Finally, the predicted distribution is mapped based on the mapping function obtained by fitting the predicted distribution and similarity value of the sample image, which can convert the predicted distribution into the target similarity corresponding to each candidate image, thereby quickly and accurately determining the similarity between different candidate images and the image to be detected.

[0034] The specific implementation methods of the above steps are introduced below.

[0035] This involves step 110 .

[0036] Obtain an image to be detected and multiple candidate images;

[0037] An image to be detected and multiple candidate images are obtained, and the image to be detected and the candidate images meet a preset similarity condition. Specifically, the image to be detected can be obtained, image features of the image to be detected can be extracted, and multiple candidate images can be retrieved from a preset image library according to the image features of the image to be detected. The preset similarity condition can be that the similarity is greater than a preset threshold.

[0038] This involves step 120 .

[0039] The image to be detected and the candidate images are input into a pre-trained similarity distribution model, and a predicted distribution is output, where the predicted distribution is used to reflect the distribution probability of the similarity between the image to be detected and the candidate images; the similarity distribution model is trained based on multiple training samples, where the training samples include a first sample image, multiple second sample images, and a similarity value between the first sample image and each second sample image.

[0040] The image to be detected and the candidate image are input into a pre-trained similarity distribution model, and a predicted distribution is output. The predicted distribution is used to reflect the distribution probability of the similarity between the image to be detected and the candidate image.

[0041] The similarity distribution model is trained based on multiple training samples, and the training samples include a first sample image, multiple second sample images, and a similarity value between the first sample image and each second sample image. For example, if the first sample image is a, the second sample images may be b1, b2, b3, ... The similarity values ​​are the similarity between a and b1, the similarity between b2, and the similarity between b3.

[0042] This involves step 130 .

[0043] The predicted distribution is mapped based on a mapping function to determine multiple target similarities. The mapping function is fitted according to the predicted distribution and similarity values ​​of the training samples.

[0044] For example, the image to be detected is A, and the candidate images are: B1, B2, B3... A and B1, A and B2, A and B3 are respectively input into the pre-trained similarity distribution model, the predicted distribution of each group of images is output, the predicted distribution is mapped based on the mapping function, the target similarity of each group of images is determined, and multiple candidate images can be sorted according to the target similarity.

[0045] Therefore, the predicted distributions corresponding to the multiple images to be detected determined above can be mapped based on the mapping function, multiple target similarities can be determined quickly and accurately, and multiple candidate images can be ranked according to the target similarities.

[0046] In a possible embodiment, before step 130, the following steps may also be included:

[0047] Establish the fitting function;

[0048] According to the data pairs of multiple groups of similarity values ​​and sample prediction distribution, the parameters of the fitting function are solved;

[0049] Substitute the parameters into the fitting function to obtain the mapping function.

[0050] Function fitting is performed on the similarity value and the sample prediction distribution to convert the sample prediction distribution into the corresponding similarity value, to map the prediction distribution based on the mapping function, and to output the target similarity between the two images (the image to be detected and the candidate image).

[0051] Specifically, linear regression is used to map the sample prediction distribution q to the similarity value y, y=w*q+b. In this way, the parameters (w, b) of the mapping function are fitted through multiple sets of data pairs of similarity values ​​and sample prediction distributions, and the parameters (w, b) of the fitting function are solved. The solved parameters are substituted into the fitting function to obtain the mapping function.

[0052] Therefore, by establishing a fitting function and multiple sets of data pairs of similarity values ​​and sample prediction distributions, a mapping function is obtained, which can facilitate the rapid and accurate conversion of the output prediction distribution into the corresponding target similarity.

[0053] In a possible embodiment, the image to be detected and the candidate image are input into a pre-trained similarity distribution model, and the target similarity of each candidate image is output. The similarity distribution model is obtained by training based on multiple training samples, and the training samples include a first sample image, multiple second sample images, and the similarity value between the first sample image and each second sample image. The similarity distribution model may include a distribution prediction structure and a scoring prediction structure, wherein the distribution prediction structure is obtained by training based on multiple training samples, and the training samples include a first sample image, multiple second sample images, and the similarity value between the first sample image and each second sample image. During the training process of the similarity distribution model, a neural network of a preset dimension is trained according to a loss function determined by a sample prediction distribution determined by the first sample image and the second sample image, and a sample target distribution determined by multiple similarity values. When the loss function meets the preset training conditions, the difference between the sample prediction distribution and the sample target distribution is continuously reduced, and a distribution prediction structure is obtained. Thus, the trained similarity distribution model can quickly and accurately output a set of similarity distributions between the image to be detected and the candidate image. In the scoring prediction structure, a function is fitted for the similarity value and the sample prediction distribution to convert the sample prediction distribution into a corresponding similarity value, and to map the prediction distribution based on the mapping function to output the target similarity between the two images (the image to be detected and the candidate image). Thus, in the similarity distribution model, the target similarity between the image to be detected and each candidate image can be output quickly and accurately.

[0054] In summary, in an embodiment of the present invention, by obtaining an image to be detected and multiple candidate images that meet preset similarity conditions, each group of images to be detected and candidate images is sequentially input into a pre-trained similarity distribution model, and a predicted distribution is output to reflect the distribution probability of the similarity between the image to be detected and the candidate image, wherein, since the similarity distribution model is obtained by training based on multiple training samples, the training samples include a first sample image, multiple second sample images, and the similarity value between the first sample image and each second sample image, the similarity distribution model obtained by training in this way can accurately predict the similarity distribution between the image to be detected and each candidate image, that is, the output predicted distribution. Finally, based on the mapping function obtained by fitting the predicted distribution and similarity value of the sample image, the predicted distribution is mapped, and the predicted distribution can be converted into the target similarity corresponding to each candidate image respectively. The target similarity is used to sort multiple candidate images, and the similarity between different candidate images and the image to be detected can be quickly and accurately determined.

[0055] Next, the training process of the similarity distribution model provided by the embodiment of the present invention is described.

[0056] Figure 2 is a flow chart of a training method for a similarity distribution model provided by an embodiment of the present invention;

[0057] like Figure 2 As shown, the training method of the similarity distribution model may include steps 210 to 240, which are specifically as follows:

[0058] Step 210: Acquire multiple training samples.

[0059] Step 220, determining the sample target distribution according to the multiple similarity values.

[0060] Step 230, input the first sample image and the second sample image into a neural network of preset dimensions, and output a sample prediction distribution; the sample prediction distribution is used to reflect the distribution probability of the similarity between the first sample image and the second sample image.

[0061] Step 240, training a neural network of a preset dimension according to a loss function determined by the sample prediction distribution and the sample target distribution, and obtaining a similarity distribution model when the loss function meets the preset training conditions.

[0062] The specific implementation methods of the above steps are introduced below.

[0063] This involves step 210 .

[0064] A plurality of training samples are obtained, each of which includes a first sample image, a plurality of second sample images, and a similarity value between the first sample image and each of the second sample images.

[0065] In a possible embodiment, step 210 includes:

[0066] Acquire a first sample image and a plurality of second sample images;

[0067] Score the similarity between the first sample image and the second sample image from a preset dimension to obtain a similarity value for each preset dimension;

[0068] According to the similarity value of each preset dimension, a similarity value between the first sample image and each second sample image is determined.

[0069] A first sample image is obtained, and a second sample image that satisfies a preset similarity condition with the first sample image is retrieved according to the first sample image. Then, the similarity between the first sample image and the second sample image is scored to obtain a similarity score between them. The first sample image includes a first object, and the second sample image includes a second object.

[0070] The first sample image and the second sample image are scored for similarity from a preset dimension D to obtain a similarity value for each preset dimension, where D is a positive integer. The preset dimension can be understood as a dimension of an image feature.

[0071] If the first object and the second object are clothing, the corresponding preset dimensions may include: clothing type, collar type, color, material, pattern and details, a total of 6 dimensions. The score range of the similarity value of each preset dimension may be [0, c_d], where a similarity value of 0 indicates complete inconsistency, and c_d indicates complete consistency. The similarity values ​​of each preset dimension are summed to determine the similarity value between the first sample image and each second sample image, that is, the similarity value y=sum(c_d, d=1, 2, ..., 6).

[0072] Therefore, by scoring the similarity of the first sample image and the second sample image from the preset dimension, the similarity value of each preset dimension can be obtained, and the detailed similarity value of each preset dimension can be obtained. Then, based on the similarity value of each preset dimension, the similarity value between the first sample image and each second sample image can be determined. The similarity values ​​of different preset dimensions can be integrated, and the characteristics of the images in different preset dimensions can be fully considered. The similarity value between the first sample image and each second sample image obtained is more accurate.

[0073] The step of acquiring a first sample image and a plurality of second sample images comprises:

[0074] Acquire a sample video, where the sample video includes a plurality of first sample images;

[0075] Capturing a plurality of first sample images from a sample video;

[0076] From a preset image library, a second sample image corresponding to each first sample image is obtained respectively, and the first sample image and the second sample image meet a preset similarity condition.

[0077] A plurality of first sample images are intercepted from a sample video, wherein the first sample image may be a clothing area image, and a second sample image that satisfies a preset similarity condition with the first sample image is retrieved based on the first sample image. Here, in a video scene, a plurality of first sample images are intercepted from a sample video, and a second sample image corresponding to each first sample image is obtained from a preset image library, that is, each first sample image corresponds to a plurality of second sample images. Here, the second sample image and the first sample image can meet the preset similarity condition, for example, the similarity between the second sample image and the first sample image is greater than a preset threshold. Here, the problem that a large number of images of the same product appearing in a video scene cannot be applied to model training because it is difficult to collect images of the same product in a video frame in a video scene is solved.

[0078] This involves step 220 .

[0079] Based on multiple similarity values, the sample target distribution is determined.

[0080] The multiple manually annotated similarity values ​​are mapped to the similarity distribution as the sample target distribution for model learning. Among them, one first sample image corresponds to one target distribution, and N first sample images correspond to N sample target distributions, so the model can be trained through N sample target distributions.

[0081] In a possible embodiment, the second sample image includes: a third sample image and a fourth sample image, a first similarity value between the first sample image and the third sample image is greater than a first threshold, and a second similarity value between the first sample image and the fourth sample image is less than a second threshold, and step 220 includes:

[0082] Determine the sample target distribution based on multiple similarity values, including:

[0083] A distribution mapping process is performed on the first similarity value and the second similarity value to obtain a sample target distribution.

[0084] The second sample image includes: a third sample image having a first similarity value with the first sample image greater than a first threshold, and a fourth sample image having a second similarity value with the first sample image less than a second threshold, wherein the first threshold is greater than the second threshold.

[0085] In a specific embodiment, the acquisition of the first sample image and multiple second sample images involved above may include: querying from a preset image library K / 2 third sample images that are most similar to the first sample image (a first similarity value with the first sample image is greater than a first threshold) and K / 2 fourth sample images that are least similar (a second similarity value with the first sample image is less than a second threshold), where K is the number of second sample images and K is a positive integer.

[0086] Therefore, K / 2 third sample images whose first similarity value with the first sample image is greater than the first threshold can be collected from the preset image library as positive training samples, and K / 2 fourth sample images whose second similarity value with the first sample image is less than the second threshold can be collected as negative training samples, which can help improve the recognition accuracy of the trained model.

[0087] In a possible embodiment, step 220 includes:

[0088] Discretizing the multiple similarity values ​​to obtain multiple discrete intervals, each discrete interval including multiple similarity values;

[0089] Determine the probability value of each similarity value relative to each discrete interval;

[0090] Determines a sample target distribution based on multiple probability values.

[0091] Specifically, the similarity score intervals may be discretized by equal intervals or probability equalization to obtain M discretized intervals, where M is a positive integer. The specific discretization method is not specifically limited, and is subject to the implementation of the solution.

[0092] The equal interval method is to divide the range of similarity values ​​into M discrete intervals, and make the interval length of each interval the same. For example, the discrete intervals may include: [0,1], (1,2], ...

[0093] Among them, the probability distribution method is to divide the value range of the similarity value into M intervals, so that the number of similarity values ​​in each interval is basically the same; each interval corresponds to a range of similarity scores, that is, the discrete intervals divided where the similarity values ​​are concentrated are also relatively dense. For example, the discrete intervals can include: [0,5], (5,7], .... Although the lengths of the discrete intervals [0,5] and (5,7] are different, the number of similarity values ​​included in each discrete interval is basically the same.

[0094] The step of determining the probability value of each similarity value relative to each discrete interval may specifically be: determining the probability value of each similarity value relative to the midpoint c of each discrete interval mThe midpoint of a discrete interval is the median of the discrete interval. It can be determined by the following formula:

[0095]

[0096] Among them, qm is the probability value of each similarity value relative to each discrete interval;

[0097] y is the similarity value;

[0098] exp is the exponential function;

[0099] β is the weighting coefficient, which can be set to 32;

[0100] c m is the midpoint of the mth discrete interval and is the representative value of the mth discrete interval;

[0101] The ||XY|| operation is the square root of the sum of the squares of the components after the vector difference is taken (when X and Y are scalars, the absolute value is taken for calculation);

[0102] c i Represents the midpoint of each discrete interval. Here, i and m are distinguished to indicate the c i Perform a sum operation on M intervals.

[0103] Finally, we obtain q=(q 1 ,q 2 , ..., q m ) can be used as the sample target distribution corresponding to the training sample.

[0104] Here, yc m It can reflect the relative relationship between the similarity value between the first sample image and each second sample image and the representative value of each discrete interval, yc i It can reflect the relative relationship between the similarity value between the first sample image and each second sample image with respect to any value of each discrete interval. The discrete interval represents the value range of multiple similarity values. Therefore, the sample prediction distribution obtained by the above method can be used to reflect the relative relationship between a certain similarity value and multiple similarity values.

[0105] That is, the sample prediction distribution can be used to reflect all possible distributions of the similarity between the first sample image and the second sample image, that is, the possibility that the similarity is equal to a specified value. Therefore, the distribution probability can be used to characterize the probability that the similarity between the first sample image and the second sample image is equal to a certain value.

[0106] This involves step 230 .

[0107] The first sample image and the second sample image are input into a neural network of preset dimensions, and a sample prediction distribution is output; the sample prediction distribution is used to reflect the distribution probability of the similarity between the first sample image and the second sample image. Among them, for a training sample, the target distribution is the distribution calculated based on the manual annotation result, that is, the similarity value (a D-dimensional vector); the prediction distribution is the prediction distribution obtained by inference using a neural network of preset dimensions, which is also a D-dimensional vector.

[0108] Among them, the neural network of preset dimension refers to a neural network with an output dimension of preset dimension. Specifically, a fully connected layer with an output dimension of D can be used to output the sample prediction distribution corresponding to the first sample image and the second sample image.

[0109] For example, the above is to score the similarity of the first sample image and the second sample image from the preset dimension D to obtain the similarity value of each preset dimension. The corresponding preset dimensions may include: clothing type, collar type, color, material, pattern and details, a total of 6 dimensions. Then, the neural network also outputs the sample prediction distribution from the 6 dimensions of clothing type, collar type, color, material, pattern and details. Therefore, the prediction distribution obtained by the neural network reasoning of the preset dimensions is a D-dimensional vector.

[0110] This involves step 240 .

[0111] A neural network of preset dimensions is trained according to a loss function determined by sample prediction distribution and sample target distribution, and a similarity distribution model is obtained when the loss function meets preset training conditions.

[0112] Specifically, KL (Kullback-Leibler divergence) divergence can be used as the loss function, the loss function is determined according to the sample prediction distribution and the sample target distribution, and the model parameters are learned by gradient descent. Specifically, the model parameters can be learned in an end-to-end manner to minimize the difference between the prediction distribution and the target distribution of the model output.

[0113] Among them, KL divergence is also called KL distance, which is a measure of the distance between two random distributions. Metric, also known as distance function, is a mathematical concept. It is a special function that meets specific conditions in the metric space, generally represented by d. Metric function is a common term in neural network learning, and can also be considered as a loss function.

[0114] In summary, the training method of the similarity distribution model provided in the embodiment of the present invention obtains multiple training samples including a first sample image, multiple second sample images, and the similarity value between the first sample image and each second sample image, and determines the sample target distribution according to the multiple similarity values. The first sample image and the second sample image are input into a neural network of preset dimensions, and a sample prediction distribution reflecting the distribution probability of the similarity between the first sample image and the second sample image is output; the neural network of preset dimensions is trained according to the loss function determined by the sample prediction distribution and the sample target distribution, which can continuously reduce the difference between the sample prediction distribution and the sample target distribution, and obtain a similarity distribution model when the loss function meets the preset training conditions. Thus, the trained similarity distribution model can quickly and accurately output the similarity distribution between a set of images to be detected and candidate images.

[0115] The following is an overall description of the similarity distribution model structure provided by the embodiment of the present invention.

[0116] Figure 3 is a schematic diagram of a training process and an application process of a similarity distribution model provided by an embodiment of the present invention, such as Figure 3 As shown, it is divided into a training process 310 and an application process 320.

[0117] In the training process 310, a plurality of training samples including a first sample image, a plurality of second sample images, and a similarity value between the first sample image and each second sample image are obtained, and a sample target distribution 314 is determined according to the plurality of similarity values. Each group of sample images 311 (the first sample image and the second sample image) is respectively input into a neural network 312 of a preset dimension, and a sample prediction distribution 313 is outputted to reflect the distribution probability of the similarity between the first sample image and the second sample image.

[0118] By training a neural network of preset dimensions according to the loss function 315 determined by the sample prediction distribution 313 and the sample target distribution 314, the difference between the sample prediction distribution and the sample target distribution can be continuously reduced, and when the loss function 315 meets the preset training conditions, a similarity distribution model 322 is obtained. Thus, the trained similarity distribution model can quickly and accurately output a set of similarity distributions between the image to be detected and the candidate image.

[0119] In the application process 320, by obtaining an image to be detected and multiple candidate images that meet preset similarity conditions, each group of images to be detected and candidate images 321 is input into the pre-trained similarity distribution model 322 in turn, and a predicted distribution 323 for reflecting the distribution probability of the similarity between the image to be detected and the candidate images is output, wherein, since the similarity distribution model is trained based on multiple training samples, the training samples include a first sample image, multiple second sample images, and the similarity value between the first sample image and each second sample image, the similarity distribution model trained in this way can accurately predict the similarity distribution between the image to be detected and each candidate image, that is, the output predicted distribution 323.

[0120] Finally, the predicted distribution 323 is mapped based on the mapping function obtained by fitting the predicted distribution and similarity value of the sample image. The predicted distribution 323 can be converted into the target similarity 324 corresponding to each candidate image. The target similarity 324 can be used to sort multiple candidate images, so as to quickly and accurately determine the similarity between different candidate images and the image to be detected.

[0121] Based on the above Figure 1 The image similarity determination method shown in the figure, the embodiment of the present invention also provides an image similarity determination device, such as Figure 4 As shown, the image similarity determination device 400 may include:

[0122] The acquisition module 410 is used to acquire an image to be detected and multiple candidate images.

[0123] The input module 420 is used to input the image to be detected and the candidate image into a pre-trained similarity distribution model, and output a predicted distribution, which is used to reflect the distribution probability of the similarity between the image to be detected and the candidate image; the similarity distribution model is trained based on multiple training samples, and the training samples include a first sample image, multiple second sample images, and a similarity value between the first sample image and each second sample image.

[0124] The determination module 430 maps the predicted distribution based on a mapping function to determine multiple target similarities, where the mapping function is fitted according to the predicted distribution and similarity values ​​of the training samples.

[0125] In a possible embodiment, the image similarity determination device 400 may further include: a fitting module, specifically configured to:

[0126] Establish the fitting function;

[0127] According to the data pairs of multiple groups of similarity values ​​and sample prediction distribution, the parameters of the fitting function are solved;

[0128] Substitute the parameters into the fitting function to obtain the mapping function.

[0129] In summary, in an embodiment of the present invention, by obtaining an image to be detected and multiple candidate images that meet preset similarity conditions, each group of images to be detected and candidate images is sequentially input into a pre-trained similarity distribution model, and a predicted distribution is output to reflect the distribution probability of the similarity between the image to be detected and the candidate image, wherein, since the similarity distribution model is obtained by training based on multiple training samples, the training samples include a first sample image, multiple second sample images, and the similarity value between the first sample image and each second sample image, the similarity distribution model obtained by training in this way can accurately predict the similarity distribution between the image to be detected and each candidate image, that is, the output predicted distribution. Finally, based on the mapping function obtained by fitting the predicted distribution and similarity value of the sample image, the predicted distribution is mapped, and the predicted distribution can be converted into the target similarity corresponding to each candidate image respectively. The target similarity is used to sort multiple candidate images, and the similarity between different candidate images and the image to be detected can be quickly and accurately determined.

[0130] Based on the above Figure 2 The training method of the similarity distribution model shown in FIG. 1 is a method for training a similarity distribution model. The embodiment of the present invention further provides a training device 500 for a similarity distribution model. Figure 5 As shown, the training device 500 of the similarity distribution model may include:

[0131] The acquisition module 510 acquires a plurality of training samples.

[0132] The determination module 520 determines the sample target distribution according to the multiple similarity values.

[0133] The input module 530 inputs the first sample image and the second sample image into a neural network of a preset dimension, and outputs a sample prediction distribution; the sample prediction distribution is used to reflect the distribution probability of the similarity between the first sample image and the second sample image.

[0134] The training module 540 is used to train a neural network of a preset dimension according to a loss function determined by the sample prediction distribution and the sample target distribution, and obtain a similarity distribution model when the loss function meets the preset training conditions.

[0135] In a possible embodiment, the second sample image includes: a third sample image and a fourth sample image, a first similarity value between the first sample image and the third sample image is greater than a first threshold, and a second similarity value between the first sample image and the fourth sample image is less than a second threshold;

[0136] The determination module 520 is specifically configured to:

[0137] A distribution mapping process is performed on the first similarity value and the second similarity value to obtain a sample target distribution.

[0138] In a possible embodiment, the acquisition module 510 is specifically configured to:

[0139] Acquire a first sample image and a plurality of second sample images;

[0140] Score the similarity between the first sample image and the second sample image from a preset dimension to obtain a similarity value for each preset dimension;

[0141] According to the similarity value of each preset dimension, a similarity value between the first sample image and each second sample image is determined.

[0142] In a possible embodiment, the determination module 520 is specifically configured to:

[0143] Discretizing the multiple similarity values ​​to obtain multiple discrete intervals, each discrete interval including multiple similarity values;

[0144] Determine the probability value of each similarity value relative to each discrete interval;

[0145] Determines a sample target distribution based on multiple probability values.

[0146] In a possible embodiment, the acquisition module 510 is specifically configured to:

[0147] Acquire a sample video, where the sample video includes a plurality of first sample images;

[0148] Capturing a plurality of first sample images from a sample video;

[0149] From a preset image library, a second sample image corresponding to each first sample image is obtained respectively, and the first sample image and the second sample image meet a preset similarity condition.

[0150] In summary, the training method of the similarity distribution model provided in the embodiment of the present invention obtains multiple training samples including a first sample image, multiple second sample images, and the similarity value between the first sample image and each second sample image, and determines the sample target distribution according to the multiple similarity values. The first sample image and the second sample image are input into a neural network of preset dimensions, and a sample prediction distribution reflecting the distribution probability of the similarity between the first sample image and the second sample image is output; the neural network of preset dimensions is trained according to the loss function determined by the sample prediction distribution and the sample target distribution, which can continuously reduce the difference between the sample prediction distribution and the sample target distribution, and obtain a similarity distribution model when the loss function meets the preset training conditions. Thus, the trained similarity distribution model can quickly and accurately output the similarity distribution between a set of images to be detected and candidate images.

[0151] The embodiment of the present invention further provides an electronic device, such as Figure 6 As shown, it includes a processor 601, a communication interface 602, a memory 603 and a communication bus 604, wherein the processor 601, the communication interface 602, and the memory 603 communicate with each other through the communication bus 604.

[0152] Memory 603, used for storing computer programs;

[0153] The processor 601 is used to execute the program stored in the memory 603 to implement the following steps:

[0154] Acquire an image to be detected and multiple candidate images; input the image to be detected and the candidate images into a pre-trained similarity distribution model, and output a predicted distribution, which is used to reflect the distribution probability of the similarity between the image to be detected and the candidate images; the similarity distribution model is trained based on multiple training samples, and the training samples include a first sample image, multiple second sample images, and a similarity value between the first sample image and each second sample image; map the predicted distribution based on a mapping function to determine multiple target similarities, and the mapping function is fitted according to the predicted distribution and similarity values ​​of the training samples.

[0155] The communication bus mentioned in the above terminal can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0156] The communication interface is used for communication between the above terminal and other devices.

[0157] The memory may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0158] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0159] In another embodiment of the present invention, a computer-readable storage medium is provided. The computer-readable storage medium stores instructions. When the instructions are executed on a computer, the computer executes the image similarity determination method described in any one of the above embodiments.

[0160] In another embodiment of the present invention, a computer program product including instructions is provided. When the computer program product is run on a computer, the computer executes the method for determining image similarity described in any one of the above embodiments.

[0161] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk Solid State Disk (SSD)), etc.

[0162] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0163] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0164] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention.

Claims

1. A method for determining image similarity, characterized in that: The method comprises: Acquire an image to be detected and multiple candidate images, and combine the image to be detected with each of the multiple candidate images to obtain multiple target image combinations, each of which includes the image to be detected and any of the candidate images; Inputting a plurality of target image combinations of the image to be detected and the candidate images into a pre-trained similarity distribution model, and outputting a plurality of prediction distributions about the plurality of target image combinations, wherein the prediction distributions are used to reflect the distribution probability of the similarity between the image to be detected and the candidate images; the similarity distribution model is trained based on a plurality of training samples, wherein the training samples include a first sample image, a plurality of second sample images, and a similarity value between the first sample image and each of the second sample images; The plurality of predicted distributions are mapped based on a mapping function to determine a plurality of target similarities, wherein the mapping function is obtained by fitting the predicted distributions of the training samples and the similarity values.

2. The method according to claim 1, before inputting the image to be detected and the candidate image into a pre-trained similarity distribution model and outputting a predicted distribution, the method further comprises: Acquire a plurality of the training samples; Determining a plurality of sample target distributions according to a plurality of the similarity values; Inputting the first sample image and any of the second sample images in a plurality of the second sample images into a neural network of a preset dimension, respectively, and outputting a plurality of sample prediction distributions; the sample prediction distributions are used to reflect the distribution probability of the similarity between the first sample image and the second sample image; The neural network of the preset dimension is trained according to the loss function determined by the sample prediction distribution and the sample target distribution, and the similarity distribution model is obtained when the loss function meets the preset training conditions.

3. The method according to claim 2, wherein the second sample image comprises: a third sample image and a fourth sample image, wherein a first similarity value between the first sample image and the third sample image is greater than a first threshold, and a second similarity value between the first sample image and the fourth sample image is less than a second threshold; Determining multiple sample target distributions according to the multiple similarity values ​​includes: Distribution mapping processing is performed on the plurality of the first similarity values ​​and the plurality of the second similarity values ​​to obtain the plurality of the sample target distributions.

4. The method according to claim 2, wherein obtaining a plurality of the training samples comprises: Acquire a first sample image and a plurality of second sample images; Scoring the similarity between the first sample image and the second sample image from the preset dimension to obtain a similarity value for each preset dimension; According to the similarity value of each of the preset dimensions, a similarity value between the first sample image and each of the second sample images is determined.

5. The method according to claim 2 or 3, wherein determining a plurality of sample target distributions according to a plurality of similarity values ​​comprises: Discretizing the plurality of similarity values ​​to obtain a plurality of discrete intervals, each of the discrete intervals including a plurality of similarity values; Determining a probability value of each of the similarity values ​​relative to each of the discrete intervals; The sample target distribution is determined according to the plurality of probability values.

6. The method according to claim 2, before mapping the predicted distribution based on a mapping function to determine a plurality of target similarities, the method further comprises: Establish the fitting function; Solving and obtaining the parameters of the fitting function according to a plurality of data pairs of the similarity values ​​and the sample prediction distribution; Substituting the parameters into the fitting function, the mapping function is obtained.

7. The method according to claim 4, wherein acquiring the first sample image and the plurality of second sample images comprises: Acquire a sample video, where the sample video includes a plurality of first sample images; intercepting the plurality of first sample images from the sample video; A second sample image corresponding to each of the first sample images is obtained from a preset image library, and the first sample image and the second sample image meet a preset similarity condition.

8. An image similarity determination device, characterized in that: The device comprises: An acquisition module, used for acquiring an image to be detected and a plurality of candidate images, and combining the image to be detected with each of the plurality of candidate images to obtain a plurality of target image combinations, each of which includes the image to be detected and any of the candidate images; An input module, used to input a plurality of target image combinations of the image to be detected and the candidate images into a pre-trained similarity distribution model, and output a plurality of predicted distributions about the plurality of target image combinations, wherein the predicted distributions are used to reflect the distribution probability of the similarity between the image to be detected and the candidate images; the similarity distribution model is trained based on a plurality of training samples, wherein the training samples include a first sample image, a plurality of second sample images, and a similarity value between the first sample image and each of the second sample images; A determination module is used to map the multiple predicted distributions based on a mapping function to determine multiple target similarities, wherein the mapping function is obtained by fitting the predicted distributions of the training samples and the similarity values.

9. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, for implementing the method steps described in any one of claims 1 to 7 when executing a program stored in a memory.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images