Image similarity determination method and device, equipment, medium and product

Abstract

The invention discloses an image similarity determination method and device, equipment, a medium and a product, and the method comprises the steps: obtaining m first theme colors corresponding to a first image and n second theme colors corresponding to a second image, taking the minimum matching distance as a target, carrying out the matching of the m first theme colors and the n second theme colors, and obtaining k color pairs; each color pair comprises a first theme color and a second theme color, different color pairs comprise different first theme colors, and different color pairs comprise different second theme colors. Wherein the matching distance is used for representing the sum of the color distances corresponding to the k color pairs. And finally, determining the image similarity between the first image and the second image according to the shortest matching distance. Visibly, a mode of matching based on theme colors is provided, the similarity between the images is evaluated from the visual level, the similarity between the images can be reflected more visually, manual calibration of image pairs, sample labels and the like is not needed any more, and cost is saved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a method, device, equipment, medium and product for determining image similarity. Background Art

[0002] An image usually refers to a digital visual media composed of pixels, which can be generated by digital cameras, scanners, computers, etc. It is the basis of human vision.

[0003] The similarity between images can reflect the degree of similarity or difference between images, and can be used for image classification, image screening, image synthesis, etc. Therefore, in practical applications, how to determine the similarity between images is of great significance. In related technologies, image similarity assessment models are trained based on artificial intelligence technology to improve efficiency. Specifically, by manually screening and marking image pairs (which can be recorded as pairs) as training samples, a large number of training samples enable the model to fully learn the ability to assess image similarity.

[0004] However, this method requires manual labeling of image pairs and sample labels, which is time-consuming and labor-intensive. On the other hand, the model maps images to abstract feature representations such as vectors, and then calculates the similarity between two vectors as image similarity. The similarity between vectors is the similarity between abstract feature representations, so it is difficult to intuitively reflect the similarity between images. Summary of the invention

[0005] In order to solve the above technical problems, the present application provides a method, device, equipment, medium and product for determining image similarity, and provides a matching method based on theme color, which evaluates the similarity between images from a visual level, can more intuitively reflect the similarity between images, and no longer requires manual calibration of image pairs, sample labels, etc., which is conducive to cost savings.

[0006] The embodiments of the present application disclose the following technical solutions:

[0007] On the one hand, an embodiment of the present application provides a method for determining image similarity, the method comprising:

[0008] Obtain m first theme colors corresponding to the first image, and obtain n second theme colors corresponding to the second image;

[0009] With the goal of minimizing the matching distance, m first theme colors are matched with n second theme colors to obtain k color pairs, each of which includes one first theme color and one second theme color, different color pairs include different first theme colors, and different color pairs include different second theme colors, k takes the minimum value of m and n, the matching distance is used to represent the sum of the color distances corresponding to the k color pairs, and the color distance of the color pair is used to represent the color distance between the first theme color in the color pair and the second theme color in the color pair;

[0010] The image similarity between the first image and the second image is determined according to the shortest matching distance.

[0011] On the other hand, an embodiment of the present application provides a device for determining image similarity, the device comprising an acquisition unit, a matching unit, and a determination unit:

[0012] The acquisition unit is used to acquire m first theme colors corresponding to the first image, and acquire n second theme colors corresponding to the second image;

[0013] The matching unit is used to match m first theme colors with n second theme colors with the goal of minimizing the matching distance, so as to obtain k color pairs, each of which includes one first theme color and one second theme color, different color pairs include different first theme colors, and different color pairs include different second theme colors, k takes the minimum value of m and n, the matching distance is used to represent the sum of the color distances corresponding to the k color pairs, and the color distance of the color pair is used to represent the color distance between the first theme color in the color pair and the second theme color in the color pair;

[0014] The determining unit is used to determine the image similarity between the first image and the second image according to the shortest matching distance.

[0015] In another aspect, an embodiment of the present application provides a computer device, the computer device comprising a processor and a memory:

[0016] The memory is used to store program code and transmit the program code to the processor;

[0017] The processor is used to execute the method for determining image similarity described in the above aspects according to the instructions in the program code.

[0018] The computer device may include a terminal device or a server, and the aforementioned image similarity determination device may be configured in the computer device.

[0019] On the other hand, an embodiment of the present application further provides a storage medium, wherein the storage medium is used to store a computer program, and the computer program is used to execute the method for determining image similarity provided in the above embodiment.

[0020] In addition, an embodiment of the present application further provides a computer program product including instructions, which, when executed on a computer, enables the computer to execute the method for determining image similarity provided in the above embodiment.

[0021] It can be seen from the above technical solution that for the first image and the second image, first, m first theme colors corresponding to the first image are obtained, and n second theme colors corresponding to the second image are obtained. Based on this, the theme colors are used to represent the visual features of the image, and the similarity between the images is subsequently evaluated from the image visual level. Specifically, with the minimum matching distance as the goal, the m first theme colors are matched with the n second theme colors to obtain k color pairs. Each color pair includes a first theme color and a second theme color. Different color pairs include different first theme colors, and different color pairs include different second theme colors. K takes the smaller value of m and n. Among them, the matching distance is used to represent the sum of the color distances corresponding to the k color pairs, and the color distance of the color pair is used to represent the color distance between the first theme color and the second theme color in the color pair. When the matching distance is the smallest, it can be considered as the optimal match between the theme colors of the two images. At this time, the matching distance (that is, the minimum value of the matching distance) can reflect the similarity of the two images at the visual level. Therefore, the image similarity between the first image and the second image can be determined based on the shortest matching distance. Based on this, a method for matching based on theme colors is provided to evaluate the similarity between images from a visual level. Compared with the method of using models in related technologies, image similarity determined from a visual level can more intuitively reflect the similarity between images, and no longer requires manual calibration of image pairs, sample labels, etc., which is conducive to cost savings. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0023] Figure 1 A flowchart of a method for determining image similarity provided in an embodiment of the present application;

[0024] Figure 2 A schematic diagram of the architecture of a process for determining image similarity provided in an embodiment of the present application;

[0025] Figure 3 A structural diagram of a device for determining image similarity provided in an embodiment of the present application. DETAILED DESCRIPTION

[0026] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0027] The method for determining image similarity provided in the embodiment of the present application can be implemented by a computer device, which can be a terminal device or a server, wherein the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. Terminal devices include but are not limited to mobile phones, computers, intelligent voice interaction devices, smart home appliances, vehicle-mounted terminals, etc. The terminal device and the server can be directly or indirectly connected via wired or wireless communication, and this application does not impose any restrictions on this.

[0028] The specific examples are as follows:

[0029] Figure 1 A flowchart of a method for determining image similarity provided in an embodiment of the present application is provided, and a server is used as the aforementioned computer device for illustration. The method includes S101-S103:

[0030] S101: Obtain m first theme colors corresponding to a first image, and obtain n second theme colors corresponding to a second image.

[0031] Wherein, m and n are both positive integers, the m first theme colors may refer to the main colors in the first image, and the n second theme colors may refer to the main colors in the second image. The theme colors, as the main colors in the image, can represent the visual features of the image. That is, in the present application, the theme colors in the image represent the visual features of the image. Correspondingly, a matching method based on theme colors is provided to evaluate the similarity between images from a visual level. For details, please refer to the subsequent embodiment description.

[0032] It should be noted that the present application does not make any limitation on how to obtain m first theme colors and how to obtain n second theme colors. For ease of understanding, the present application embodiment provides the following method as an example:

[0033] In a possible implementation, the theme color that can represent the image can be selected based on the color ratio of the color in the image. In a specific implementation, the first color whose color ratio is greater than or equal to the ratio threshold can be selected from the multiple first colors included in the first image as the first theme color, and m first theme colors can be obtained. Correspondingly, the second color whose color ratio is greater than or equal to the ratio threshold can be selected from the multiple second colors included in the second image as the second theme color, and n second theme colors can be obtained.

[0034] The first color may refer to a color included in the first image, the second color may refer to a color included in the second image, and the color proportion of the first color may refer to the color proportion of the first color in the first image, and the color proportion of the second color may refer to the color proportion of the second color in the second image. The proportion threshold may refer to a preset color proportion limit, and if the color proportion is greater than or equal to the proportion threshold, it is considered that the color has a large proportion in the image and can be used as the theme color.

[0035] Exemplarily, the color ratio can be determined by counting the ratio of the number of pixels, and the value of the color ratio can be 0-100%, and the color ratio can also be recorded as ratio. The ratio threshold can be flexibly set according to the number of colors contained in the image. For example, if the image contains a large number of colors, a lower ratio threshold is set, and if the image contains a small number of colors, a higher ratio threshold is set. For example, the ratio threshold can be recorded as th1, and th1=10% can be set.

[0036] Based on this, by counting the color ratios of the colors contained in the image and filtering out colors whose color ratios exceed a threshold, the theme color that can represent the visual characteristics of the image can be quickly and effectively filtered out.

[0037] In practical applications, colors can be described based on the HSV color space and are described by three components: H (Hue), S (Saturation), and V (Value). Hue indicates the type or kind of color, saturation indicates the purity or vividness of the color, and value indicates the brightness of the color.

[0038] For example, the m first theme colors corresponding to the first image (for example, image A) may be recorded as theme_th1_A, and the n second theme colors corresponding to the second image (for example, image B) may be recorded as theme_th1_B. Specifically:

[0039] theme_th1_A=[[[h1_A, s1_A, v1_A], ratio1_A], [[h2_A, s2_A, v2_A], ratio2_A], ..., [[hm_A, sm_A, vm_A], ratio_A]], m in total.

[0040] theme_th1_B=[[[h1_B,s1_B,v1_B],ratio1_B],[[h2_B,s2_B,v2_B],ratio2_B],…,[[hn_B,sn_B,vn_B],ration_B]],n in total.

[0041] S102: With the goal of minimizing the matching distance, the m first theme colors are matched with the n second theme colors to obtain k color pairs.

[0042] Among them, each color pair includes a first theme color and a second theme color, different color pairs include different first theme colors, and different color pairs include different second theme colors, k takes the minimum value of m and n, the matching distance is used to represent the sum of the color distances corresponding to the k color pairs respectively, and the color distance of the color pair is used to represent the color distance between the first theme color in the color pair and the second theme color in the color pair.

[0043] In practical applications, the color distance between two colors is used to represent the similarity or difference between the two colors, and the theme color of an image can represent the overall visual features of the image. Therefore, the similarity or difference between the theme colors of two images can reflect the similarity or difference between the two images in terms of visual features. Therefore, the similarity or difference between the theme colors of images can be used to represent the similarity or difference between the images by matching theme colors. In the matching process, different matching combinations have different matching distances. When the matching distance is the smallest, it can be considered that the theme colors of the two images have achieved the best match, indicating that the two most similar theme colors in the two images form a color pair. Therefore, the matching distance at this time can more accurately reflect the similarity between the two images at the visual level. As an example, image A includes theme colors a1 (red), a2 (green), and a3 (black), and image B includes theme colors b1 (pink), b2 (light green), b3 (black), and b4 (white). The best match between the theme colors of the two images is that a1 matches b1, a2 matches b2, and a3 matches b3, but not a3 matches b4. Therefore, with the minimum matching distance as the goal, the matching distances corresponding to the aforementioned k color pairs are the shortest matching distances.

[0044] It should be noted that this application does not impose any limitation on how to perform the matching. For ease of understanding, this application embodiment provides the following methods as examples:

[0045] Since the size of the color proportion makes the degree of contribution of color to the visual characteristics of the image different, in order to more accurately determine the similarity of two images at the visual level, in a possible implementation method, the color proportion of the theme color can be combined for matching, such as controlling the color proportions of two theme colors that can be matched into a color pair to be similar.

[0046] Correspondingly, for the aforementioned k color pairs, the color ratio parameter of the color pair is less than the ratio threshold. The color ratio parameter is the same as the difference between the color ratio of the first theme color in the color pair and the color ratio of the second theme color in the color pair. That is, the color ratio parameter of the color pair is less than the ratio threshold, indicating that the color ratios of the two colors in the color pair in the two images are relatively similar, that is, the difference between the color ratios of the two is not large.

[0047] Based on this, it is possible to avoid matching the first theme color with a larger color proportion with the second theme color with a smaller color proportion into a color pair, or avoid matching the first theme color with a smaller color proportion with the second theme color with a larger color proportion into a color pair. This is conducive to matching the first theme color and the second theme color that have similar contributions to the visual features of the image into a color pair, which is conducive to ensuring the accuracy of subsequently determining the image similarity between the two images based on the color distance of the color pair.

[0048] For example, the aforementioned ratio threshold can be set to 3. Assuming that a first theme color [[h1, s1, v1], ratio1] in image A matches a second theme color [[h2, s2, v2], ratio2] in image B, the difference between the color proportions of the two is determined. If ratio1 / ratio2 is greater than 3, or ratio2 / ratio1 is greater than 3, the matching result is deleted, that is, the two theme colors cannot be matched into a color pair. In other words, the difference in color proportions of two theme colors that can be matched into a color pair needs to be less than three times, so as to prevent incorrect matching in extreme cases.

[0049] Since the purpose of matching is to determine the shortest matching distance, in order to improve efficiency, in another possible implementation, the color distance can be calculated first, the color distance matrix can be determined, and then based on the color distance matrix, a matching process with the goal of minimizing the matching distance can be implemented.

[0050] In specific implementation, the color distance matrix can be determined based on the m first theme colors and the n second theme colors. Among them, the matrix size of the color distance matrix is ​​m*n, i∈m, j∈n, that is, i takes an integer greater than or equal to 1 and less than or equal to m, j takes an integer greater than or equal to 1 and less than or equal to n, and the matrix element in the i-th row and j-th column is used to represent the color distance between the i-th first theme color and the j-th second theme color. Then, based on the color distance matrix, the shortest matching path can be determined with the minimum matching distance as the goal, and the shortest matching path is used to represent k color pairs. In practical applications, unreasonable matching paths are deleted during the matching process (such as the aforementioned color ratio parameter exceeds the ratio threshold, it is deleted), and finally the path with the smallest matching distance is found in the remaining paths to obtain the shortest matching path.

[0051] For example, the shortest matching path can be recorded as path, specifically:

[0052] path=[[[[h1_A,s1_A,v1_A],ratio1_A],[[h2_B,s2_B,v2_B],ratio2_B],dist1],…]. Among them, [[[[h1_A,s1_A,v1_A],ratio1_A],[[h2_B,s2_B,v2_B],ratio2_B],dist1] is used to represent a color pair, and dist1 is used to represent the color distance corresponding to this color pair, that is, the color distance between the first theme color in this color pair and the second theme color in this color pair. In this example, dist1 is the color distance between the first theme color [h1_A,s1_A,v1_A] and the second theme color [h2_B,s2_B,v2_B].

[0053] It can be understood that path is used to represent the specific conditions of k color pairs (such as what the two colors in each color pair are, what the proportion of each color is, and what the color distance is). Accordingly, the specific conditions of path depend on the specific conditions of k color pairs. For example, for image A and image B, corresponding k=3, then path can include the specific conditions of 3 color pairs. For another example, for image C and image D, corresponding k=1, then path can only include the specific conditions of 1 color pair.

[0054] After the matching is completed, the shortest matching distance can be determined based on the k color pairs identified by the shortest matching path. In specific implementation, the shortest matching distance can be determined by the following formula:

[0055]

[0056] In the above formula, dist sumis used to represent the shortest matching distance, k is used to represent the number of color pairs, and dist k Used to represent the color distance corresponding to the k-th color pair.

[0057] It can be understood that the above formula refers to the matching distance corresponding to the shortest matching path obtained by the final matching (i.e., the shortest matching distance). However, in the matching process, the matching distance corresponding to each matching path is also the sum of the color distances corresponding to the k color pairs under the matching path, but it is not the shortest match. In this regard, the matching distance corresponding to each matching path can be determined by referring to the above formula, and will not be repeated here.

[0058] In the above embodiment, it should be noted that the present application does not impose any limitation on how to determine the color distance between two colors. For ease of understanding, the present application provides the following example description:

[0059] Considering that the proportion of colors in the image as a whole will also affect the visual perception of color feedback, the degree of visual difference between color a in image A and color b in image B is different from the degree of difference between color a and color b when evaluated separately. Therefore, in a possible implementation, the proportion of colors in the image can be comprehensively considered to evaluate the color distance between the two colors, so as to improve the accuracy of the color distance of the color pair, thereby facilitating the accuracy of evaluating the similarity between the two images.

[0060] In practical applications, the color distance can be determined by the following formula:

[0061] dist=(1+|ratio1-ratio2|)*dist0

[0062] In the above formula, dist is used to represent the color distance between the first theme color and the second theme color, dist0 is used to represent the initial distance between the first theme color and the second theme color, and the initial distance can be determined based on the HSV values ​​of the two colors. ratio1 is used to represent the color ratio of the first theme color in the first image, and ratio2 is used to represent the color ratio of the second theme color in the second image.

[0063] S103: Determine the image similarity between the first image and the second image according to the shortest matching distance.

[0064] Finally, the image similarity between the first image and the second image can be determined based on the shortest matching distance. Based on this, a method of matching based on theme color is provided to evaluate the similarity between images from a visual level. Compared with the method of using models in related technologies, the image similarity determined from a visual level can more intuitively reflect the similarity between images, and no longer requires manual calibration of image pairs, sample labels, etc., which is conducive to cost saving.

[0065] It should be noted that the present application does not impose any limitation on how to determine the image similarity based on the shortest matching distance. For ease of understanding, the present application provides the following method as an example:

[0066] In a possible implementation, the image similarity can be determined directly based on the shortest matching distance, based on which the similarity between the theme colors is used to represent the similarity between the images. In practical applications, the image similarity is negatively correlated with the shortest matching distance. The smaller the shortest matching distance, the smaller the difference between the theme colors of the two images, and the closer the color visual perception of the two images is. Therefore, it can be determined that the image similarity between the two images is greater.

[0067] In practical applications, after seeing an image, the human eye first perceives those colors that account for a relatively large proportion, that is, the theme color of the image. Therefore, the image similarity between two images can be determined based on the theme color. Of course, in addition to the theme color, the remaining colors in the image can reflect more details of the image, so they can also reflect the visual characteristics of the image, that is, the remaining colors in the two images will also affect the image similarity between the two images. Therefore, in order to improve the accuracy of image similarity, in another possible implementation method, on the basis of considering the theme color, the remaining colors other than the theme color can also be comprehensively considered. In specific implementation, the first remaining colors corresponding to the first image can be obtained first, and the second remaining colors corresponding to the second image can be obtained.

[0068] Among them, the color proportion of the first theme color in the first image is greater than or equal to the proportion threshold, and the color proportion of the first remaining color in the first image is less than the proportion threshold, that is, the first remaining color and the first theme color together constitute the aforementioned multiple first colors. The color proportion of the second theme color in the second image is greater than or equal to the proportion threshold, and the color proportion of the second remaining color in the second image is less than the proportion threshold, that is, the second remaining color and the second theme color together constitute the aforementioned multiple second colors.

[0069] Since the color proportion of the remaining colors will affect the accuracy of the shortest matching distance determined based on the theme color in representing the similarity between the two images, after the remaining colors are determined, the compensation distance can be determined according to the color proportion of the first remaining color and the color proportion of the second remaining color. Correspondingly, in the specific implementation of the aforementioned S103, the image similarity between the first image and the second image can be determined according to the shortest matching distance and the compensation distance.

[0070] Based on this, the similarity between two images is evaluated based on the theme color, and the contribution of other colors to the visual features of the image is comprehensively considered. In this way, the image similarity is determined by combining the shortest matching distance and the compensation distance, which is conducive to improving the accuracy of image similarity.

[0071] Thus, for two images, even if the two images are very similar in terms of theme color, the other colors of the two images are quite different, making the two images not so similar at the visual level. Therefore, the above method can determine a more accurate image similarity.

[0072] For example, the theme color of image A is red (color proportion is 30%), and the color proportion of the other colors is very small, so they will not be classified as the theme color, but the total color proportion of these other colors reaches 70%, so they are still important to the visual features presented by image A. The theme color of image B is also red (color proportion is 30%), and the color proportion of the other colors is also very small, so they will not be classified as the theme color, but the total color proportion of these other colors reaches 70%, which is still important to the visual features presented by image B.

[0073] For the above-mentioned image A and image B, from the perspective of the similarity between the theme colors alone, the similarity between the two images is 100%, but in fact, the total proportion of the remaining colors is very high, so that the red (theme color) accounting for 30% is mixed in, and ultimately it is impossible to visually perceive that these are two images with a similarity close to 100% through the red. At this time, after adopting the above method, because the colors other than the theme colors are taken into account, it can assist in evaluating the similarity between the two images and obtain a more accurate image similarity, thereby reflecting the visually perceptible similarity between the two images.

[0074] It should be noted that the present application does not make any limitation on how to determine the image similarity based on the shortest matching distance and the compensation distance, and how to determine the compensation distance. For ease of understanding, the present application will provide the following examples one by one.

[0075] First, regarding how to determine image similarity based on the shortest matching distance and the compensation distance, the present application embodiment provides the following example description:

[0076] In a possible implementation, the average distance corresponding to the k color pairs may be determined based on the shortest matching distance and k. The shortest matching distance is the sum of the color distances of the k color pairs, and the average distance may refer to the mean of the distances when evaluating the similarity between two images from a single color pair. Then, the image similarity between the first image and the second image may be determined based on the average distance and the compensation distance.

[0077] In practical applications, different images may contain different amounts of theme colors. In this case, there may be theme colors that are not matched to form a color pair, and the theme colors are colors that have a significant impact on the image vision.

[0078] Therefore, in a possible implementation, if the aforementioned m is not equal to n, then the theme colors that do not fall into the k color pairs can be screened from the m first theme colors and the n second theme colors as unmatched theme colors, and the unmatched theme colors can refer to the theme colors that are left alone and cannot be matched to form a color pair. Then, the compensation distance can be determined according to the color proportion of the first remaining colors, the color proportion of the second remaining colors, and the color proportion of the unmatched theme colors.

[0079] Based on this, the contribution of the theme color that does not match the color pair and the other colors other than the theme color to the visual features of the image is comprehensively considered. The determined compensation distance can comprehensively reflect the visual difference between the two images from these two aspects, which is conducive to improving the accuracy of image similarity.

[0080] In specific implementation, the sum of the color proportion of the first remaining colors, the color proportion of the second remaining colors and the color proportion of the unmatched theme color can be determined as the compensation color proportion, and then the compensation distance can be determined according to the preset compensation factor and the compensation color proportion.

[0081] Based on this, by setting the compensation factor, the compensation distance is adjusted to be as close as possible in magnitude to the distance result determined based on the theme color to avoid extreme situations.

[0082] For ease of understanding, based on the example description of the above embodiment, the embodiment of the present application also provides the following formula example. Specifically, the image similarity can be determined by the following formula:

[0083] sim=1-(dist avg +w*ratio extra )

[0084] In the above formula, sim is used to represent image similarity, dist avg is used to represent the aforementioned average distance, w is used to represent the aforementioned compensation factor (usually it can be set to w = 1.1), ratioextra Used to indicate the aforementioned compensation color ratio.

[0085] The average distance can be determined by the following formula:

[0086]

[0087] In the above formula, dist avg is used to represent the average distance, k is used to represent the number of color pairs, and dist sum Used to indicate the shortest matching distance.

[0088] Exemplarily, assuming that m is less than n, it means that each first theme color in the first image (i.e., image A) will be matched with the second theme color in the second image (i.e., image B), so there will be one second theme color among the n second theme colors that does not match the first theme color, which can be used as the aforementioned unmatched theme color. Next, the remaining colors are screened based on the proportion threshold, and the color proportions of all colors that do not participate in the match in image A and image B are calculated to obtain the compensation color proportion. Finally, the compensation item is determined based on the compensation color proportion to more comprehensively evaluate the color similarity between images, which is conducive to improving the accuracy of image similarity.

[0089] It can be seen from the above technical solution that for the first image and the second image, first, m first theme colors corresponding to the first image are obtained, and n second theme colors corresponding to the second image are obtained. Based on this, the theme colors are used to represent the visual features of the image, and the similarity between the images is subsequently evaluated from the image visual level. Specifically, with the minimum matching distance as the goal, the m first theme colors are matched with the n second theme colors to obtain k color pairs. Each color pair includes a first theme color and a second theme color. Different color pairs include different first theme colors, and different color pairs include different second theme colors. K takes the smaller value of m and n. Among them, the matching distance is used to represent the sum of the color distances corresponding to the k color pairs, and the color distance of the color pair is used to represent the color distance between the first theme color and the second theme color in the color pair. When the matching distance is the smallest, it can be considered as the optimal match between the theme colors of the two images. At this time, the matching distance (that is, the minimum value of the matching distance) can reflect the similarity of the two images at the visual level. Therefore, the image similarity between the first image and the second image can be determined based on the shortest matching distance. Based on this, a method for matching based on theme colors is provided to evaluate the similarity between images from a visual level. Compared with the method of using models in related technologies, image similarity determined from a visual level can more intuitively reflect the similarity between images, and no longer requires manual calibration of image pairs, sample labels, etc., which is conducive to cost savings.

[0090] Through the above embodiments, the method for determining image similarity provided by the present application is described in detail. In summary, please refer to Figure 2 The schematic diagram of the architecture of a process for determining image similarity is shown, which may specifically include the following parts:

[0091] The first part is theme color extraction. For image A, theme color extraction is performed to obtain the corresponding m first theme colors, and for image B, theme color extraction is performed to obtain the corresponding n second theme colors.

[0092] The second part is the calculation of the color distance matrix. That is, according to m first theme colors and n second theme colors, the color distance between a first theme color and a second theme color is calculated to obtain a color distance matrix. In the color distance matrix, the matrix element in the i-th row and j-th column is used to represent the color distance between the i-th first theme color and the j-th second theme color.

[0093] The third part is the calculation of the shortest matching distance. Based on the color distance matrix obtained in the second part, the matching process is implemented with the goal of minimizing the matching distance to determine the shortest matching path.

[0094] The fourth part is the compensation distance calculation. That is, the contribution of those colors that are not involved in the matching to the image vision is comprehensively considered to improve the accuracy of image similarity.

[0095] Part V: Image Similarity Calculation Based on the shortest matching distance determined in Part III and the compensation distance determined in Part IV, the similarity between the two images is evaluated to obtain the image similarity.

[0096] For better understanding, the embodiment of the present application takes the poster design application scenario as an example, and performs image similarity evaluation on the framed images captured from the film and television video and the poster images in the poster library, thereby selecting poster images with similar visual features as references, adjusting and designing the framed images, and obtaining corresponding posters. For example, using similar poster images as references, determine the color of the logo material that needs to be added to the framed images. Based on this, it is helpful to improve the efficiency of designing corresponding posters for film and television dramas based on framed images.

[0097] It can be understood that it basically corresponds to the method embodiment, so the relevant parts can refer to the partial description of the method embodiment.

[0098] Through the above embodiments, the method for determining image similarity provided by the present application is described in detail. For better understanding, the embodiments of the present application also provide the following example description:

[0099] The image similarity between two images can reflect the degree of similarity between the two images. Therefore, in a possible implementation, the visual design of one image can be used as a reference for the visual design of another image based on the image similarity between the two images. For ease of understanding, take the scenario of designing a poster image for video (such as film and television drama) promotion as an example:

[0100] The aforementioned first image may be a video frame image from a video, the second image may be a poster template image stored in a poster library, and the number of the second images is multiple. The video frame image may be an image captured from a video, such as an image of a certain frame including a protagonist of a film or TV series, and the poster template image stored in the poster library may refer to a previously designed poster image.

[0101] Accordingly, according to the image similarities between the first image and the plurality of second images, the second image with an image similarity greater than a similarity threshold can be determined as a poster reference image. The image similarity greater than the similarity threshold indicates that the two images are very similar at the visual level, for example, the color combinations in the two images are very similar. Therefore, such a second image can be used as a poster reference image.

[0102] Next, if the poster reference image includes a reference theme identifier, the color of the video theme identifier of the video can be adjusted according to the color of the reference theme identifier until the color distance between the adjusted video theme identifier and the reference theme identifier is greater than the distance threshold. The reference theme identifier can be the theme identifier of the video corresponding to the poster reference image (such as the logo of the title A of the film and television series A), and the video theme identifier can refer to the theme identifier of the aforementioned video for which the poster image is to be generated (such as the logo of the title B of the film and television series B).

[0103] Among them, the color distance between the adjusted video theme identifier and the reference theme identifier is greater than the distance threshold, indicating that the two theme identifiers are visually similar. Since the poster reference image is a designed poster image in the poster library, the reference theme identifier and other contents in the poster reference image except the reference theme representation are visually matched. Therefore, the adjusted video theme identifier can be added to the first image to obtain the first image carrying the adjusted video theme identifier, and the first image carrying the adjusted video theme identifier is determined as the poster image of the video.

[0104] It can be seen that by evaluating the image similarity between images, a poster image similar to the video frame image of the current video is selected from the poster template images stored in the poster library, and used as a reference poster image. Then, the color of the theme logo of the current video can be adjusted according to the theme logo in the reference poster image, so that the adjusted video theme logo and the video frame image can have a better visual matching effect. Based on this, poster images can be generated for videos more quickly, and the generated poster images can be used to promote videos, etc.

[0105] For example, the first image (for example, the video frame image of TV series B) is blue, the logo of TV series B is black, the poster reference image (for example, the poster image of TV series X) is also blue, and the logo in the poster reference image is silver-gray. Based on this, the logo of TV series B is adjusted to silver-gray, and then the adjusted logo is added to the first image, so as to quickly generate a poster image with better visual effect for TV series B.

[0106] In practical applications, the image similarity between two images can be a value between 0 and 1, which simply and intuitively reflects the similarity between the two images. Accordingly, 0.8 can be set as the aforementioned similarity threshold. Similarly, the color distance between two colors can be a value between 0 and 1, which simply and intuitively reflects the similarity between the two colors. Accordingly, 0.7 can be set as the aforementioned distance threshold.

[0107] Figure 3 This is a structural diagram of a device for determining image similarity provided in an embodiment of the present application, wherein the device includes an acquisition unit 301, a matching unit 302, and a determination unit 303:

[0108] The acquisition unit 301 is used to acquire m first theme colors corresponding to the first image, and acquire n second theme colors corresponding to the second image;

[0109] The matching unit 302 is used to match m first theme colors with n second theme colors with the goal of minimizing the matching distance, to obtain k color pairs, each of the color pairs including one first theme color and one second theme color, different color pairs including different first theme colors, and different color pairs including different second theme colors, k is the minimum value of m and n, the matching distance is used to represent the sum of the color distances corresponding to the k color pairs, and the color distance of the color pair is used to represent the color distance between the first theme color in the color pair and the second theme color in the color pair;

[0110] The determining unit 303 is configured to determine the image similarity between the first image and the second image according to the shortest matching distance.

[0111] In a possible implementation manner, the acquiring unit is further configured to:

[0112] Acquire a first remaining color corresponding to the first image, and acquire a second remaining color corresponding to the second image, wherein a color proportion of the first theme color in the first image is greater than or equal to a proportion threshold, a color proportion of the first remaining color in the first image is less than the proportion threshold, a color proportion of the second theme color in the second image is greater than or equal to the proportion threshold, and a color proportion of the second remaining color in the second image is less than the proportion threshold;

[0113] The determining unit is further configured to:

[0114] Determine a compensation distance according to the color proportion of the first remaining colors and the color proportion of the second remaining colors;

[0115] The image similarity between the first image and the second image is determined according to the shortest matching distance and the compensation distance.

[0116] In a possible implementation manner, if the m is not equal to the n, the acquiring unit is further configured to:

[0117] From the m first theme colors and the n second theme colors, select a theme color that does not fall into the k color pairs as an unmatched theme color;

[0118] The determining unit is further configured to determine the compensation distance according to a color proportion of the first remaining colors, a color proportion of the second remaining colors, and a color proportion of the unmatched theme color.

[0119] In a possible implementation manner, the determining unit is further configured to:

[0120] Determine the sum of the color proportion of the first remaining colors, the color proportion of the second remaining colors, and the color proportion of the unmatched theme color as the compensation color proportion;

[0121] The compensation distance is determined according to a preset compensation factor and the compensation color ratio.

[0122] In a possible implementation manner, the determining unit is further configured to:

[0123] Determine the average distances corresponding to k color pairs according to the shortest matching distance and k;

[0124] The image similarity between the first image and the second image is determined according to the average distance and the compensated distance.

[0125] In a possible implementation manner, the matching unit is further configured to:

[0126] Determine a color distance matrix according to m first theme colors and n second theme colors, wherein the matrix size of the color distance matrix is ​​m*n, i∈m, j∈n, and the matrix element in the i-th row and j-th column is used to represent the color distance between the i-th first theme color and the j-th second theme color;

[0127] According to the color distance matrix, with the goal of minimizing the matching distance, a shortest matching path is determined, and the shortest matching path is used to represent the k color pairs.

[0128] In a possible implementation manner, the acquiring unit is further configured to:

[0129] From the plurality of first colors included in the first image, select a first color whose color proportion is greater than or equal to a proportion threshold as the first theme color, and obtain m first theme colors;

[0130] From the plurality of second colors included in the second image, a second color whose color proportion is greater than or equal to the proportion threshold is selected as the second theme color, to obtain n second theme colors.

[0131] In a possible implementation, a color ratio parameter of the color pair is less than a ratio threshold, and the color ratio parameter is used to indicate a difference between a color ratio of the first theme color in the color pair and a color ratio of the second theme color in the color pair.

[0132] In a possible implementation, if the first image is a video frame image from a video, the second image is a poster template image stored in a poster library, and there are multiple second images, the determining unit is further configured to:

[0133] According to the image similarities between the first image and the plurality of second images respectively, determining the second image whose image similarity is greater than a similarity threshold as a poster reference image;

[0134] If the poster reference image includes a reference theme identifier, adjusting the color of the video theme identifier of the video according to the color of the reference theme identifier until the color distance between the adjusted video theme identifier and the reference theme identifier is greater than a distance threshold;

[0135] Adding the adjusted video theme identifier to the first image to obtain the first image carrying the adjusted video theme identifier;

[0136] The first image carrying the adjusted video theme identifier is determined as the poster image of the video.

[0137] It can be seen from the above technical solution that for the first image and the second image, first, m first theme colors corresponding to the first image are obtained, and n second theme colors corresponding to the second image are obtained. Based on this, the theme colors are used to represent the visual features of the image, and the similarity between the images is subsequently evaluated from the image visual level. Specifically, with the minimum matching distance as the goal, the m first theme colors are matched with the n second theme colors to obtain k color pairs. Each color pair includes a first theme color and a second theme color. Different color pairs include different first theme colors, and different color pairs include different second theme colors. K takes the smaller value of m and n. Among them, the matching distance is used to represent the sum of the color distances corresponding to the k color pairs, and the color distance of the color pair is used to represent the color distance between the first theme color and the second theme color in the color pair. When the matching distance is the smallest, it can be considered as the optimal match between the theme colors of the two images. At this time, the matching distance (that is, the minimum value of the matching distance) can reflect the similarity of the two images at the visual level. Therefore, the image similarity between the first image and the second image can be determined based on the shortest matching distance. Based on this, a method for matching based on theme colors is provided to evaluate the similarity between images from a visual level. Compared with the method of using models in related technologies, image similarity determined from a visual level can more intuitively reflect the similarity between images, and no longer requires manual calibration of image pairs, sample labels, etc., which is conducive to cost savings.

[0138] In another aspect, an embodiment of the present application provides a computer device, the computer device comprising a processor and a memory:

[0139] The memory is used to store program code and transmit the program code to the processor;

[0140] The processor is used to execute the method for determining image similarity provided in the above embodiment according to the instructions in the program code.

[0141] The computer device may include a terminal device or a server, and the aforementioned image similarity determination device may be configured in the computer device.

[0142] On the other hand, an embodiment of the present application further provides a storage medium, wherein the storage medium is used to store a computer program, and the computer program is used to execute the method for determining image similarity provided in the above embodiment.

[0143] In addition, an embodiment of the present application further provides a computer program product including instructions, which, when executed on a computer, enables the computer to execute the method for determining image similarity provided in the above embodiment.

[0144] A person of ordinary skill in the art can understand that: all or part of the steps of implementing the above-mentioned method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above-mentioned method embodiment; and the aforementioned storage medium can be at least one of the following media: read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, etc., various media that can store program codes.

[0145] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The device embodiment described above is only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative work.

[0146] It should be noted that, in this article, relational terms such as "first" and "second", etc. (if any) are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0147] The above is a detailed introduction to a method, device, equipment, medium and product for determining image similarity provided by the embodiment of the present application. The principle and implementation method of the present application are described in detail using specific examples herein. The description of the above embodiments is only used to help understand the method of the present application. At the same time, for those skilled in the art, according to the method of the present application, there will be changes in the specific implementation method and application scope.

[0148] In summary, the content of this specification should not be understood as limiting the present application, and any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in this application should be included in the protection scope of this application. Moreover, based on the implementations provided in the above aspects, this application can also be further combined to provide more implementations.

Claims

1. A method for determining image similarity, characterized in that: The method comprises: Obtain m first theme colors corresponding to the first image, and obtain n second theme colors corresponding to the second image; With the goal of minimizing the matching distance, m first theme colors are matched with n second theme colors to obtain k color pairs, each of which includes one first theme color and one second theme color, different color pairs include different first theme colors, and different color pairs include different second theme colors, k takes the minimum value of m and n, the matching distance is used to represent the sum of the color distances corresponding to the k color pairs, and the color distance of the color pair is used to represent the color distance between the first theme color in the color pair and the second theme color in the color pair; The image similarity between the first image and the second image is determined according to the shortest matching distance.

2. The method according to claim 1, characterized in that The method further comprises: Acquire a first remaining color corresponding to the first image, and acquire a second remaining color corresponding to the second image, wherein a color proportion of the first theme color in the first image is greater than or equal to a proportion threshold, a color proportion of the first remaining color in the first image is less than the proportion threshold, a color proportion of the second theme color in the second image is greater than or equal to the proportion threshold, and a color proportion of the second remaining color in the second image is less than the proportion threshold; Determine a compensation distance according to the color proportion of the first remaining colors and the color proportion of the second remaining colors; The determining, according to the shortest matching distance, the image similarity between the first image and the second image includes: The image similarity between the first image and the second image is determined according to the shortest matching distance and the compensation distance.

3. The method according to claim 2, characterized in that If the m is not equal to the n, the method further includes: From the m first theme colors and the n second theme colors, select a theme color that does not fall into the k color pairs as an unmatched theme color; The determining the compensation distance according to the color proportion of the first remaining colors and the color proportion of the second remaining colors includes: The compensation distance is determined according to the color proportion of the first remaining colors, the color proportion of the second remaining colors, and the color proportion of the unmatched theme color.

4. The method according to claim 3, characterized in that The determining the compensation distance according to the color proportion of the first remaining colors, the color proportion of the second remaining colors, and the color proportion of the unmatched theme color includes: Determine the sum of the color proportion of the first remaining colors, the color proportion of the second remaining colors, and the color proportion of the unmatched theme color as the compensation color proportion; The compensation distance is determined according to a preset compensation factor and the compensation color ratio.

5. The method according to claim 2, characterized in that: The method of determining the image similarity between the first image and the second image according to the shortest matching distance and the compensation distance comprises: Determine the average distances corresponding to k color pairs according to the shortest matching distance and k; The image similarity between the first image and the second image is determined according to the average distance and the compensation distance.

6. The method according to claim 1, characterized in that With the goal of minimizing the matching distance, m first theme colors are matched with n second theme colors to obtain k color pairs, including: Determine a color distance matrix according to m first theme colors and n second theme colors, wherein the matrix size of the color distance matrix is ​​m*n, i∈m, j∈n, and the matrix element in the i-th row and j-th column is used to represent the color distance between the i-th first theme color and the j-th second theme color; According to the color distance matrix, with the goal of minimizing the matching distance, a shortest matching path is determined, and the shortest matching path is used to represent the k color pairs.

7. The method according to claim 1, characterized in that The step of obtaining m first theme colors corresponding to the first image includes: From the plurality of first colors included in the first image, select a first color whose color proportion is greater than or equal to a proportion threshold as the first theme color, and obtain m first theme colors; The obtaining of n second theme colors corresponding to the second image includes: From the plurality of second colors included in the second image, a second color whose color proportion is greater than or equal to the proportion threshold is selected as the second theme color, to obtain n second theme colors.

8. The method according to any one of claims 1 to 7, characterized in that: A color ratio parameter of the color pair is less than a ratio threshold, and the color ratio parameter is used to indicate a ratio difference between a color ratio of a first theme color in the color pair and a color ratio of a second theme color in the color pair.

9. The method according to any one of claims 1 to 7, characterized in that: If the first image is a video frame image from a video, the second image is a poster template image stored in a poster library, and there are multiple second images, the method further includes: According to the image similarities between the first image and the plurality of second images respectively, determining the second image whose image similarity is greater than a similarity threshold as a poster reference image; If the poster reference image includes a reference theme identifier, adjusting the color of the video theme identifier of the video according to the color of the reference theme identifier until the color distance between the adjusted video theme identifier and the reference theme identifier is greater than a distance threshold; Adding the adjusted video theme identifier to the first image to obtain the first image carrying the adjusted video theme identifier; The first image carrying the adjusted video theme identifier is determined as the poster image of the video.

10. A device for determining image similarity, characterized in that: The device comprises an acquisition unit, a matching unit and a determination unit: The acquisition unit is used to acquire m first theme colors corresponding to the first image, and acquire n second theme colors corresponding to the second image; The matching unit is used to match m first theme colors with n second theme colors with the goal of minimizing the matching distance, so as to obtain k color pairs, each of which includes one first theme color and one second theme color, different color pairs include different first theme colors, and different color pairs include different second theme colors, k takes the minimum value of m and n, the matching distance is used to represent the sum of the color distances corresponding to the k color pairs, and the color distance of the color pair is used to represent the color distance between the first theme color in the color pair and the second theme color in the color pair; The determining unit is used to determine the image similarity between the first image and the second image according to the shortest matching distance.

11. A computer device, characterized in that: The computer device comprises a processor and a memory: The memory is used to store program code and transmit the program code to the processor; The processor is configured to execute the method according to any one of claims 1 to 9 according to the instructions in the program code.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store a computer program, and the computer program is used to execute the method according to any one of claims 1 to 9.

13. A computer program product comprising instructions, which, when executed on a computer, causes the computer to execute the method according to any one of claims 1 to 9.

Images