Image processing method and device

Abstract

The invention discloses an image processing method and device, and relates to the technical field of image processing. A specific embodiment of the method comprises the steps of extracting multiple types of pixel features from two to-be-compared images, calculating feature similarity of each pixel feature and a weight coefficient corresponding to the pixel feature to determine the similarity of the to-be-compared images, and determining the similarity of the to-be-compared images based on multiple preset similarity ranges. According to the embodiment of the invention, the similarity and the weight coefficient are respectively calculated by utilizing multiple types of pixel features, the precision degree and the accuracy of calculating the image similarity are improved, and the comparison result is directly determined through the preset similarity range. The problem that the accuracy and efficiency of image processing are low due to the fact that manual judgment is needed in an existing method is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method and device for processing an image. Background Art

[0002] With the development of image processing technology, the demand for contrast images has also been applied to various image processing application scenarios, such as using contrast images to retrieve online products that users want to search for.

[0003] At present, the method of comparing images usually uses a certain image processing model to directly calculate the image similarity of two images. The existing methods have the problem of low accuracy due to the single method of determining image similarity; and in the fuzzy case where the calculated image similarity does not clearly indicate the same or different, manual comparison of images is required. However, manual processing has the problem of low accuracy of compared images due to subjective factors, which reduces the efficiency of image processing. Summary of the invention

[0004] In view of this, an embodiment of the present invention provides a method and device for processing an image, which can respectively extract multiple types of pixel features from two images to be compared, and for each of the pixel features, calculate its feature similarity and the weight coefficient corresponding to the pixel feature to determine the similarity of the images to be compared, and determine the comparison result based on multiple preset similarity ranges. The embodiment of the present invention uses multiple types of pixel features to respectively calculate the similarity and the weight coefficient, which improves the precision and accuracy of calculating the image similarity, and directly determines the comparison result through the preset similarity range, overcoming the problem of low accuracy and efficiency in image processing caused by the need for manual judgment in the existing method.

[0005] To achieve the above-mentioned purpose, according to one aspect of an embodiment of the present invention, a method for processing an image is provided, characterized in that it includes: determining two images to be compared; for each image to be compared, extracting multiple types of pixel features from the image to be compared; for each of the pixel features, performing the following operations: calculating the feature similarity of the two images to be compared for the pixel features and the weight coefficient corresponding to the pixel features; determining the comparison results of the images to be compared according to the feature similarity of each of the pixel features, the weight coefficient corresponding to the feature similarity and a plurality of preset similarity ranges.

[0006] Optionally, the calculating the feature similarity of the two images to be compared with respect to the pixel feature includes: using a first similarity model and a second similarity model to respectively calculate a first pixel feature similarity and a second pixel feature similarity of the two images to be compared with respect to the pixel feature.

[0007] Optionally, the calculation of the weight coefficient corresponding to the pixel feature includes: constructing a similarity matrix using the first pixel feature similarity and the second pixel feature similarity of each of the pixel features, and calculating the proportion of the pixel feature based on the elements of each row or column included in the similarity matrix; and calculating the weight coefficient of the pixel feature using the proportion.

[0008] Optionally, determining the comparison result of the images to be compared based on the feature similarity of each pixel feature, the weight coefficient corresponding to the feature similarity and multiple preset similarity ranges includes: calculating the weighted similarity of the two images to be compared for each pixel feature in combination with the feature similarity and the weight coefficient corresponding to the feature similarity; selecting the maximum similarity from the multiple weighted similarities; searching for a target similarity range containing the maximum similarity from multiple preset similarity ranges, and determining the image similarity category indicated by the target similarity range as the comparison result of the images to be compared.

[0009] Optionally, the method for processing images further includes: obtaining multiple sample combinations, each of the sample combinations including two image samples; for each of the sample combinations, calculating the image similarity of the two image samples included therein; dividing into multiple preset similarity ranges according to the multiple image similarities of the multiple sample combinations, and determining the image similarity category corresponding to each preset similarity range.

[0010] Optionally, the calculating the image similarity of the two image samples included in the sample combination includes: calculating the first feature similarity of the two image samples using the first similarity model, and calculating the second feature similarity of the two image samples using the second similarity model; the dividing the multiple image similarities calculated according to the multiple groups of image samples into multiple preset similarity ranges includes: calculating the first similarity model according to the multiple first feature similarities obtained by the first similarity model and the corresponding multiple first preset similarity ranges calculated by the second similarity model.

[0011] Optionally, the calculation of the weighted similarity of the two images to be compared for each pixel feature by combining the feature similarity and the weight coefficient corresponding to the feature similarity includes: for each pixel feature, performing the following operations: obtaining a first pixel feature similarity and a second pixel feature similarity corresponding to the pixel feature; determining a first target similarity range to which the first pixel feature similarity belongs from a plurality of first preset similarity ranges, and determining a second target similarity range to which the second pixel feature similarity belongs from a plurality of second preset similarity ranges; constructing an evaluation matrix using the first pixel feature similarity and the corresponding first target similarity range, the second pixel feature similarity and the corresponding second target similarity range of each pixel feature; and determining the weighted similarity of the pixel features in combination with the weight coefficient and the evaluation matrix.

[0012] To achieve the above object, according to a second aspect of an embodiment of the present invention, there is provided an apparatus for processing an image, characterized in that it comprises: a feature extraction module, a calculation module and a comparison module; wherein:

[0013] The feature extraction module is used to determine two images to be compared; for each image to be compared, extract multiple types of pixel features from the image to be compared;

[0014] The calculation module is used to perform the following operations for each of the pixel features: calculating the feature similarity of the two images to be compared for the pixel feature and the weight coefficient corresponding to the pixel feature;

[0015] The comparison module is used to determine the comparison result of the to-be-compared images according to the feature similarity of each of the pixel features, the weight coefficient corresponding to the feature similarity and a plurality of preset similarity ranges.

[0016] Optionally, the image processing device is used to calculate the feature similarity of the two images to be compared for the pixel feature, including: using a first similarity model and a second similarity model to respectively calculate a first pixel feature similarity and a second pixel feature similarity of the two images to be compared for the pixel feature.

[0017] Optionally, the device for processing images is used to calculate the weight coefficient corresponding to the pixel feature, including: constructing a similarity matrix using the first pixel feature similarity and the second pixel feature similarity of each of the pixel features, and calculating the proportion of the pixel feature based on the elements of each row or column included in the similarity matrix; and calculating the weight coefficient of the pixel feature using the proportion.

[0018] Optionally, the image processing device is used to determine the comparison result of the images to be compared based on the feature similarity of each pixel feature, the weight coefficient corresponding to the feature similarity and multiple preset similarity ranges, including: calculating the weighted similarity of the two images to be compared for each pixel feature in combination with the feature similarity and the weight coefficient corresponding to the feature similarity; selecting the maximum similarity from the multiple weighted similarities; searching for a target similarity range containing the maximum similarity from multiple preset similarity ranges, and determining the image similarity category indicated by the target similarity range as the comparison result of the images to be compared.

[0019] Optionally, the image processing device is further used to obtain multiple sample combinations, each sample combination includes two image samples; for each sample combination, calculate the image similarity of the two image samples included therein; divide into multiple preset similarity ranges according to the multiple image similarities of the multiple sample combinations, and determine the image similarity category corresponding to each preset similarity range.

[0020] Optionally, the image processing device is used to calculate the image similarity of the two image samples included in the sample combination, including: using the first similarity model to calculate the first feature similarity of the two image samples, and using the second similarity model to calculate the second feature similarity of the two image samples; the multiple image similarities calculated based on the multiple groups of image samples are divided into multiple preset similarity ranges, including: using the first similarity model to calculate the first similarity model for the multiple first feature similarities corresponding to the multiple first preset similarity ranges; using the second similarity model to calculate the second similarity model for the multiple second feature similarities corresponding to the multiple second preset similarity ranges.

[0021] Optionally, the device for processing images is used to calculate the weighted similarity of the two images to be compared for each pixel feature in combination with the feature similarity and the weight coefficient corresponding to the feature similarity, including: for each pixel feature, performing the following operations: obtaining a first pixel feature similarity and a second pixel feature similarity corresponding to the pixel feature; determining a first target similarity range to which the first pixel feature similarity belongs from a plurality of first preset similarity ranges, and determining a second target similarity range to which the second pixel feature similarity belongs from a plurality of second preset similarity ranges; constructing an evaluation matrix using the first pixel feature similarity and the corresponding first target similarity range, the second pixel feature similarity and the corresponding second target similarity range of each pixel feature; and determining the weighted similarity of the pixel features in combination with the weight coefficient and the evaluation matrix.

[0022] To achieve the above-mentioned purpose, according to the third aspect of an embodiment of the present invention, there is provided an electronic device for processing images, characterized in that it comprises: one or more processors; a storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement any of the methods described in the above-mentioned methods for processing images.

[0023] To achieve the above-mentioned purpose, according to a fourth aspect of an embodiment of the present invention, a computer-readable medium is provided, on which a computer program is stored, characterized in that when the program is executed by a processor, any method as described in the above-mentioned method for processing an image is implemented.

[0024] An embodiment of the above invention has the following advantages or beneficial effects: it is capable of extracting multiple types of pixel features from the two images to be compared, respectively; for each of the pixel features, calculating its feature similarity and the weight coefficient corresponding to the pixel feature to determine the similarity of the images to be compared, and determining the comparison result based on multiple preset similarity ranges. The embodiment of the present invention uses multiple types of pixel features to calculate the similarity and weight coefficient respectively, improves the precision and accuracy of calculating the image similarity, and directly determines the comparison result through the preset similarity range, overcoming the problem of low accuracy and efficiency of image processing caused by the need for manual judgment in the existing method.

[0025] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with the specific implementation manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The accompanying drawings are used to better understand the present invention and do not constitute an improper limitation of the present invention.

[0027] Figure 1 is a flowchart of a method for processing an image provided by an embodiment of the present invention;

[0028] Figure 2A is a schematic image to be compared provided by an embodiment of the present invention;

[0029] Figure 2B It is a schematic diagram of distribution of multiple pixel features extracted from a schematic image provided by an embodiment of the present invention;

[0030] Figure 3A is another schematic diagram of an image processing flow provided by an embodiment of the present invention;

[0031] Figure 3B is a coordinate schematic diagram including multiple preset similarity ranges provided by an embodiment of the present invention;

[0032] Figure 4 is a schematic structural diagram of a device for processing an image provided by an embodiment of the present invention;

[0033] Figure 5 is an exemplary system architecture diagram to which embodiments of the present invention may be applied;

[0034] Figure 6 It is a schematic diagram of the structure of a computer system of a terminal device or a server suitable for implementing an embodiment of the present invention. DETAILED DESCRIPTION

[0035] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0036] It should be noted that the collection, collection, updating, analysis, processing, use, transmission, storage and other aspects of user personal information involved in the technical solution of the present invention are in compliance with the provisions of relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. Necessary measures are taken for user personal information to prevent illegal access to user personal information data and maintain the security of user personal information, network security and national security.

[0037] In the technical solution of the present invention, the collection, use, storage, sharing and transfer of user personal information involved are in compliance with the provisions of relevant laws and regulations, and it is necessary to inform the user and obtain the user's consent or authorization. When applicable, the user's personal information is de-identified and / or anonymized and / or encrypted.

[0038] After collecting the image information, we will use technical means to de-identify the image data.

[0039] In many application scenarios (for example, searching images by image, removing duplicates from similar images, verifying whether the information in two images is consistent, etc.), it is necessary to calculate image similarity to compare images.

[0040] Existing methods for calculating image similarity usually use a certain method (for example, a certain method from a collection of algorithms such as structural similarity algorithm, hash algorithm, image processing neural network model, etc.) to perform calculations, resulting in inaccurate comparison results. Regarding the method of using image processing neural network model to calculate image similarity, from the perspective of resource cost, the training of the neural network model requires a large amount of data sets, and the data sets need to be manually labeled, which has high requirements on hardware and results in high resource consumption.

[0041] When the images to be compared are in the critical state of being the same or different, it is impossible to determine whether the images are the same or different through image similarity. Further manual judgment is required, and for the same two images, some people may judge them to be the same while others may judge them to be different, resulting in low accuracy and efficiency in judging image similarity due to subjective factors.

[0042] In view of this, if Figure 1 As shown, an embodiment of the present invention provides a method for processing an image, which may include the following steps:

[0043] Step S101: determining two images to be compared; and extracting multiple types of pixel features from each image to be compared.

[0044] The embodiments of the present invention improve the accuracy of calculating image similarity by extracting multiple types of pixel features and comprehensively calculating image similarity based on the multiple types of pixel features.

[0045] Specifically, according to different application scenarios, two images to be compared are determined. For example, the two images to be compared can be directly received; or one image can be received, and based on the image, an image to be compared with it is determined from a picture data set, or two images are selected from multiple picture data as images to be compared, etc.

[0046] Furthermore, multiple types of pixel features are extracted from the images to be compared respectively; in an embodiment of the present invention, the multiple types of pixel features are R channel pixel features (represented by R), G channel pixel features (represented by G), B channel pixel features (represented by B), and grayscale pixel features of the image, wherein the grayscale pixel features can be obtained by calculating the statistical values ​​(e.g., mean, weighted average, etc.) of the pixel features of the three RGB channels.

[0047] Figure 2A is a schematic image to be compared between the two images to be compared, Figure 2B The schematic diagram of the distribution of four pixel features (RGB channels and grayscale) extracted from the schematic image is shown in FIG. Figure 2B As shown, there are large differences in the distribution of the four pixel features (RGB and grayscale) in the schematic image. It can be seen that the present invention improves the fineness of the compared image by comparing two images based on different pixel features, thereby improving the accuracy of the compared image.

[0048] Step S102: for each of the pixel features, perform the following operations: calculate the feature similarity of the two images to be compared for the pixel feature and the weight coefficient corresponding to the pixel feature.

[0049] Specifically, for example, if the images to be compared are image A and image B, then according to the description of step S101, four pixel features (RGB and grayscale) of image A and four pixel features (RGB and grayscale) of image B are extracted respectively; further, calculation is performed based on the R channel pixel features of image A and the R channel pixel features of image B, and so on, the similarity is calculated for each pixel feature RGB and grayscale respectively.

[0050] Further preferably, in an embodiment of the present invention, two similarity models are used to respectively calculate the similarity of two images on different pixel features. For example, the two similarity models are an SSIM (Structural Similarity, SSIM for short) model and a histogram model. The two similarity models can calculate image similarity from different angles to improve the accuracy of calculating image similarity. For example, the SSIM model is used to calculate the similarity of high-frequency information of images, and the histogram model is used to calculate the similarity of low-frequency information of images. High-frequency information generally refers to the part where the image intensity (such as brightness, grayscale, etc.) changes dramatically, indicating information such as image edge or texture, and low-frequency information generally refers to the part where the image intensity (such as brightness, grayscale, etc.) changes gently (such as a large color block area). Therefore, the embodiment of the present invention calculates the corresponding similarities for the high-frequency information and the low-frequency information in the image through different similarity models, thereby further improving the fineness of the contrast image and thus improving the accuracy of the contrast image. That is, the calculation of the feature similarity of the two images to be compared for the pixel feature includes: using a first similarity model and a second similarity model to respectively calculate a first pixel feature similarity and a second pixel feature similarity of the two images to be compared for the pixel feature. In an embodiment of the present invention, two other similarity models other than the SSIM model and the histogram model may also be selected to calculate the similarity, and the present invention takes the SSIM model and the histogram model as examples for explanation.

[0051] Step S103: determining the comparison result of the to-be-compared images according to the feature similarity of each of the pixel features, the weight coefficient corresponding to the feature similarity, and a plurality of preset similarity ranges.

[0052] Specifically, in an embodiment of the present invention, two similarity models and four pixel features can be combined to calculate the similarity of two images. Furthermore, the present invention determines the corresponding weight coefficients for different pixel features. That is, the calculation of the weight coefficient corresponding to the pixel feature includes: constructing a similarity matrix using the first pixel feature similarity and the second pixel feature similarity of each pixel feature, and calculating the weight of the pixel feature according to the elements of each row or column included in the similarity matrix; and calculating the weight coefficient of the pixel feature using the weight.

[0053] The following describes a specific method for calculating the weight coefficient of pixel features using weight:

[0054] 1) Constructing a similarity matrix using the first pixel feature similarity and the second pixel feature similarity of each pixel feature, for example, X is a similarity matrix:

[0055]

[0056] Among them, x 11 represents the similarity of the first pixel feature of the R channel pixel features of the two images to be compared (obtained using the SSIM model), x 12 Represents the second pixel feature similarity of the pixel features of the two images to be compared in the R channel (obtained using the histogram model), and so on, to obtain other matrix elements in X.

[0057] 2) Calculate the weight p of the pixel feature according to the elements of each row or column in the similarity matrix ij :As shown in formula (1):

[0058]

[0059] Among them, x′ ij Represents the similarity of pixel features, x ij Represents the elements of each row or column. The denominator in formula (1) is the value obtained by summing the elements. ij Can be based on x ij The calculation result, for example, for the R channel, can be obtained based on the statistical values ​​(eg, maximum value, minimum value, mean value, etc.) of the first pixel feature similarity and the second pixel feature similarity.

[0060] 3) Based on specific gravity p ij , calculate the entropy value of pixel features, as shown in formula (2):

[0061]

[0062] Where k is related to n, let k = 1 / ln(n), where ln is the natural logarithm, and e j ≥0; that is, the entropy value e is calculated based on the specific gravity j .

[0063] 4) Based on the entropy value calculated in 3), calculate the difference coefficient of the pixel feature. The larger the difference coefficient, the higher the weight of the pixel feature in the calculation of the similarity of the image. The method for calculating the difference coefficient is shown in formula (3):

[0064] g j =1-e j (3)

[0065] Among them, g j This is the coefficient of variation.

[0066] 5) Calculate the weight coefficient of pixel features, as shown in formula (4):

[0067]

[0068] Among them, Wr represents the weight coefficient of the R channel pixel feature. Similarly, the weight coefficients corresponding to each pixel feature are calculated: Wg represents the weight coefficient of the G channel pixel feature, Wb represents the weight coefficient of the B channel pixel feature, and Wz represents the weight coefficient of the grayscale pixel feature; and W r +W g +W b +W z =1.

[0069] Furthermore, the comparison result of the images to be compared is determined according to the feature similarity of each of the pixel features, the weight coefficient corresponding to the feature similarity, and a plurality of preset similarity ranges.

[0070] In an embodiment of the present invention, multiple preset similarity ranges can be determined for the application scenario, for example, three preset similarity ranges are determined. Table 1 shows an example of three preset similarity ranges. As shown in Table 1, the three preset similarity ranges are indicated as the same, different, and relatively the same, and the boundary values ​​corresponding to the preset similarity ranges are shown in Table 1.

[0071] Table 1

[0072]

[0073] Furthermore, according to the feature similarity of each of the pixel features and the weight coefficient corresponding to the feature similarity, a corresponding similarity value is obtained, and it is further determined to which preset similarity range the similarity value belongs, thereby directly determining the comparison result (same, different or relatively same).

[0074] It can be seen that the embodiments of the present invention determine the comparison results of the images to be compared based on the feature similarity of each of the pixel features, the weight coefficient corresponding to the feature similarity and a plurality of preset similarity ranges, thereby overcoming the need for further manual comparison processing of the images, the problem of low accuracy of the comparison images due to subjective factors, and reducing the efficiency of image processing.

[0075] Furthermore, in an embodiment of the present invention, a method for determining multiple preset similarity ranges is: obtaining multiple sample combinations, each of the sample combinations including two image samples; for each of the sample combinations, calculating the image similarity of the two image samples included therein; dividing multiple preset similarity ranges according to the multiple image similarities of the multiple sample combinations, and determining the image similarity category corresponding to each preset similarity range.

[0076] Specifically, the multiple sample combinations can be image samples of a larger data volume, and by calculating the image similarity of the larger data volume, statistics and comparison are performed, so as to analyze multiple preset similarity ranges (same, different, and relatively similar, etc.). When performing statistics, the similarity of each group of image samples obtained can be used as a data point, and clustering can be performed using a clustering model (such as a k-means model) to divide different preset similarity ranges through different clustering clusters, and determine the boundary values ​​of different preset similarity ranges.

[0077] like Figure 3A As shown, an embodiment of the present invention provides a method for processing an image, which may include the following steps:

[0078] Step S301: determining two images to be compared; and extracting multiple types of pixel features from each image to be compared.

[0079] Step S302: for each of the pixel features, perform the following operations: calculate the feature similarity of the two images to be compared for the pixel feature and the weight coefficient corresponding to the pixel feature.

[0080] Specifically, the description of step S301 to step S302 is consistent with the description of step S101 to step S102 and will not be repeated here.

[0081] Step S303: Calculate the weighted similarity of the two images to be compared for each pixel feature by combining the feature similarity and the weight coefficient corresponding to the feature similarity. Specifically, in an embodiment of the present invention, after calculating the weight coefficient, the weighted similarity of the pixel feature is further calculated.

[0082] The method for calculating the weighted similarity of each pixel feature is:

[0083] Preferably, the corresponding preset similarity ranges are determined for the first similarity model and the second similarity model respectively. It can be understood that, since the first similarity model and the second similarity model have different focuses on images, the boundary values ​​representing the same or different preset similarity ranges can be different. The method for determining the corresponding preset similarity ranges for the first similarity model and the second similarity model respectively can be to calculate the similarity using the first similarity model and the second similarity model for a plurality of sample images, and to determine the corresponding preset similarity ranges by statistical and clustering methods respectively; that is, calculating the image similarity of the two image samples included in the sample combination, including: calculating the first feature similarity of the two image samples using the first similarity model, and calculating the second feature similarity of the two image samples using the second similarity model; dividing the multiple preset similarity ranges according to the multiple image similarities calculated from the multiple groups of image samples, including: calculating the multiple first preset similarity ranges corresponding to the first similarity model according to the multiple first feature similarities obtained from the first similarity model; calculating the multiple second preset similarity ranges corresponding to the second similarity model according to the multiple second feature similarities obtained from the second similarity model.

[0084] The following takes the first similarity model or the second similarity model as an example to illustrate the corresponding multiple preset similarity ranges, such as Figure 3B In the coordinate system shown, the horizontal axis represents the value of the similarity index, k1, k2, k3, k4 are the boundary values ​​of each preset similarity range, and the vertical axis represents whether the calculated similarity belongs to any preset similarity range, that is, the range of [0-1].

[0085] Furthermore, the calculation of the weighted similarity of the two images to be compared for each of the pixel features by combining the feature similarity and the weight coefficient corresponding to the feature similarity includes: for each of the pixel features, performing the following operations: obtaining a first pixel feature similarity and a second pixel feature similarity corresponding to the pixel feature; determining a first target similarity range to which the first pixel feature similarity belongs from a plurality of first preset similarity ranges, and determining a second target similarity range to which the second pixel feature similarity belongs from a plurality of second preset similarity ranges; constructing an evaluation matrix using the first pixel feature similarity and the corresponding first target similarity range, the second pixel feature similarity and the corresponding second target similarity range of each of the pixel features; and determining the weighted similarity of the pixel features in combination with the weight coefficient and the evaluation matrix.

[0086] The following describes a method for determining which preset similarity range the similarity belongs to, such as the following calculation formulas (5)-(7) for f1, f2, and f3, where f1 represents a preset similarity range for different images, f2 represents a preset similarity range for relatively similar images, and f3 represents a preset similarity range for the same images; Figure 3B As shown, k1, k2, k3, k4 are statistical boundary values ​​of each preset similarity range, a and b represent boundary values ​​calculated based on k1, k2, k3, k4; x represents the calculated first pixel feature similarity or the second pixel feature similarity. Further, through the following calculation formulas of f1, f2, f3, the target similarity range (first target similarity range or second target similarity range) to which the calculated pixel feature similarity belongs can be determined.

[0087]

[0088]

[0089]

[0090] Furthermore, an evaluation matrix is ​​constructed using the first pixel feature similarity and the corresponding first target similarity range, the second pixel feature similarity and the corresponding second target similarity range of each of the pixel features; for example, the evaluation matrix is ​​represented by R, as shown below:

[0091]

[0092] Among them, r 11 represents the first pixel feature similarity calculated using the first similarity model (SSIM model) when the target similarity range is the same, r 21 The second pixel feature similarity calculated by the second similarity model (histogram model) when the target similarity range is the same, and so on, the evaluation matrix R is constructed according to the first pixel feature similarity or the second pixel feature similarity corresponding to each similarity range; wherein r is determined 11 The method is to obtain it through the calculation formula (5) of f1. Similarly, the elements contained in other evaluation matrices R are calculated through formulas (5) to (7).

[0093] Furthermore, the weighted similarity of the pixel features is determined by combining the weight coefficient and the evaluation matrix. The weighted similarity is as follows:

[0094]

[0095] Among them, S represents any one of the four weighted similarities, Wr represents the weight coefficient of the R channel pixel feature, and similarly, the weight coefficients corresponding to each pixel feature are calculated: Wg represents the weight coefficient of the G channel pixel feature, Wb represents the weight coefficient of the B channel pixel feature, Wz represents the weight coefficient of the grayscale pixel feature, and R represents the evaluation matrix. It can be seen that the weighted similarities of the four pixel features can be obtained respectively through the method of this step.

[0096] Step S304: selecting a maximum similarity from the multiple weighted similarities; searching for a target similarity range including the maximum similarity from multiple preset similarity ranges, and determining the image similarity category indicated by the target similarity range as the comparison result of the to-be-compared images.

[0097] Specifically, the maximum similarity is selected from the four weighted similarities corresponding to the four calculated pixel features, and then a target similarity range containing the maximum similarity is searched from multiple preset similarity ranges, wherein according to the similarity model corresponding to the maximum similarity, the corresponding preset similarity range can be used to determine the target similarity range, and the target similarity range is, for example, any one of the image similarity categories of the same, different, and relatively the same, so as to determine the comparison result for the two images to be compared.

[0098] like Figure 4 As shown, the embodiment of the present invention provides an apparatus 400 for processing an image, comprising: a feature extraction module 401, a calculation module 402 and a comparison module 403; wherein,

[0099] The feature extraction module 401 is used to determine two images to be compared; for each image to be compared, extract multiple types of pixel features from the image to be compared;

[0100] The calculation module 402 is used to perform the following operations for each of the pixel features: calculating the feature similarity of the two images to be compared for the pixel feature and the weight coefficient corresponding to the pixel feature;

[0101] The comparison module 403 is used to determine the comparison result of the to-be-compared images according to the feature similarity of each of the pixel features, the weight coefficient corresponding to the feature similarity, and a plurality of preset similarity ranges.

[0102] An embodiment of the present invention also provides an electronic device for processing images, comprising: one or more processors; a storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the method provided in any of the above embodiments.

[0103] An embodiment of the present invention further provides a computer-readable medium having a computer program stored thereon, and when the program is executed by a processor, the method for processing an image provided by any of the above embodiments is implemented.

[0104] Figure 5 An exemplary system architecture 500 is shown to which the method for processing an image or the apparatus for processing an image according to the embodiments of the present invention can be applied.

[0105] like Figure 5 As shown, system architecture 500 may include terminal devices 501, 502, 503, a network 504 and a server 505. Network 504 is used to provide a medium for communication links between terminal devices 501, 502, 503 and server 505. Network 504 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0106] Users can use terminal devices 501, 502, 503 to interact with server 505 through network 504 to receive or send messages, etc. Various client applications can be installed on terminal devices 501, 502, 503, such as e-commerce client applications, web browser applications, search applications, instant messaging tools, and email clients.

[0107] The terminal devices 501 , 502 , and 503 may be various electronic devices having a display screen and supporting various client applications, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0108] The server 505 may be a server providing various services, such as a background management server providing support for client applications used by users using the terminal devices 501, 502, 503. The background management server may process the received request for image comparison and feed back the comparison result to the terminal device.

[0109] It should be noted that the method for processing an image provided in the embodiment of the present invention is generally executed by the server 505 , and accordingly, the device for processing an image is generally disposed in the server 505 .

[0110] It should be understood that Figure 5 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0111] Reference below Figure 6 , which shows a schematic diagram of the structure of a computer system 600 of a terminal device suitable for implementing an embodiment of the present invention. Figure 6 The terminal device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0112] like Figure 6 As shown, the computer system 600 includes a central processing unit (CPU) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage part 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the system 600 are also stored. The CPU 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0113] The following components are connected to the I / O interface 605: an input section 606 including a keyboard, a mouse, etc.; an output section 607 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN card, a modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the I / O interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as needed, so that a computer program read therefrom is installed into the storage section 608 as needed.

[0114] In particular, according to the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 609, and / or installed from the removable medium 611. When the computer program is executed by the central processing unit (CPU) 601, the above-mentioned functions defined in the system of the present invention are executed.

[0115] It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0116] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0117] The modules and / or units involved in the embodiments of the present invention may be implemented in software or hardware. The modules and / or units described may also be arranged in a processor, for example, they may be described as: a processor includes a feature extraction module, a calculation module and a comparison module. The names of these modules do not constitute a limitation on the modules themselves in some cases, for example, the feature extraction module may also be described as a "module for extracting multiple types of pixel features from the image to be compared".

[0118] As another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiment; or it may exist independently and not be assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by a device, the device includes: determining two images to be compared; for each image to be compared, extracting multiple types of pixel features from the image to be compared; for each of the pixel features, performing the following operations: calculating the feature similarity of the two images to be compared for the pixel feature and the weight coefficient corresponding to the pixel feature; determining the comparison result of the images to be compared according to the feature similarity of each pixel feature, the weight coefficient corresponding to the feature similarity and multiple preset similarity ranges.

[0119] The embodiments of the present invention can respectively extract multiple types of pixel features from the two images to be compared, and for each of the pixel features, calculate its feature similarity and the weight coefficient corresponding to the pixel feature to determine the similarity of the images to be compared, and determine the comparison result based on multiple preset similarity ranges. The embodiments of the present invention use multiple types of pixel features to respectively calculate the similarity and the weight coefficient, thereby improving the precision and accuracy of calculating the image similarity, and directly determining the comparison result through the preset similarity range, thereby overcoming the problem of low accuracy and efficiency in processing images caused by the need for manual judgment in the existing methods.

[0120] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for processing an image, characterized in that: include: Determine two images to be compared; For each image to be compared, extracting multiple types of pixel features from the image to be compared; For each of the pixel features, calculating the feature similarity of the two images to be compared for the pixel feature and the weight coefficient corresponding to the pixel feature; The comparison result of the images to be compared is determined according to the feature similarity of each of the pixel features, the weight coefficient corresponding to the feature similarity, and a plurality of preset similarity ranges.

2. The method according to claim 1, characterized in that The calculating the feature similarity of the two images to be compared with respect to the pixel features comprises: The first similarity model and the second similarity model are used to respectively calculate the first pixel feature similarity and the second pixel feature similarity of the two images to be compared with respect to the pixel feature.

3. The method according to claim 2, characterized in that The calculating of the weight coefficient corresponding to the pixel feature includes: Constructing a similarity matrix using the first pixel feature similarity and the second pixel feature similarity of each pixel feature, and calculating the weight of the pixel feature according to the elements of each row or column included in the similarity matrix; The weight coefficient of the pixel feature is calculated using the proportion of the pixel feature.

4. The method according to claim 1, characterized in that: Determining the comparison result of the to-be-compared images according to the feature similarity of each of the pixel features, the weight coefficient corresponding to the feature similarity, and a plurality of preset similarity ranges, includes: Calculating the weighted similarity of the two images to be compared for each of the pixel features according to the feature similarity and the weight coefficient corresponding to the feature similarity; Selecting the maximum similarity from the multiple weighted similarities; A target similarity range including the maximum similarity is searched from a plurality of preset similarity ranges, and the image similarity category indicated by the target similarity range is determined as a comparison result of the to-be-compared images.

5. The method according to claim 1 or 4, characterized in that: Further including: Acquire a plurality of sample combinations, each of which includes two image samples; For each of the sample combinations, calculating the image similarity of the two image samples included therein; According to the image similarities of the plurality of sample combinations, a plurality of preset similarity ranges are divided, and the image similarity category corresponding to each preset similarity range is determined.

6. The method according to claim 5, characterized in that The calculating the image similarity of the two image samples included in the sample combination comprises: Calculating the first feature similarity of the two image samples by using the first similarity model, and calculating the second feature similarity of the two image samples by using the second similarity model; The method of dividing a plurality of preset similarity ranges according to the plurality of image similarities calculated based on the plurality of groups of image samples comprises: Calculating a plurality of first preset similarity ranges corresponding to the first similarity model according to the plurality of first feature similarities obtained by the first similarity model; A plurality of second feature similarities obtained according to the second similarity model are used to calculate a plurality of second preset similarity ranges corresponding to the second similarity model.

7. The method according to claim 4, characterized in that The step of calculating the weighted similarity of the two images to be compared for each pixel feature by combining the feature similarity and the weight coefficient corresponding to the feature similarity includes: For each of the pixel features, perform the following operations: Obtaining a first pixel feature similarity and a second pixel feature similarity corresponding to the pixel feature; Determine a first target similarity range to which the first pixel feature similarity belongs from a plurality of first preset similarity ranges, and determine a second target similarity range to which the second pixel feature similarity belongs from a plurality of second preset similarity ranges; Constructing an evaluation matrix using the first pixel feature similarity and the corresponding first target similarity range, the second pixel feature similarity and the corresponding second target similarity range of each of the pixel features; The weighted similarity of the pixel features is determined by combining the weight coefficients and the evaluation matrix.

8. A device for processing an image, characterized in that: include: Extract feature module, calculation module and comparison module; among them, The feature extraction module is used to determine two images to be compared; for each image to be compared, extract multiple types of pixel features from the image to be compared; The calculation module is used to perform the following operations for each of the pixel features: calculating the feature similarity of the two images to be compared for the pixel feature and the weight coefficient corresponding to the pixel feature; The comparison module is used to determine the comparison result of the to-be-compared images according to the feature similarity of each of the pixel features, the weight coefficient corresponding to the feature similarity and a plurality of preset similarity ranges.

9. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

10. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images