Image comparison method and device

Abstract

The invention discloses an image comparison method and device, and relates to the technical field of computers. A specific embodiment of the method comprises the following steps: in response to a received image comparison request, obtaining a reference webpage graph and an experimental webpage graph; respectively extracting a reference picture and an experimental picture from the reference webpage graph and the experimental webpage graph; and comparing the similarity between the reference picture and the experiment picture, performing composition element comparison on the reference picture and the experiment picture under the condition that the similarity comparison is not passed, and taking a composition element comparison result as a response of the image comparison request. According to the embodiment, efficient, fine-grained and accurate comparison of the images is realized, the comparison of the webpage images is finely grained to the pictures contained in the webpage images and the composition elements in the pictures, the situation that the image comparison is not passed only due to the difference of the display screen size and the resolution is avoided, the comparison accuracy is improved, and the user experience is improved. And compared with manual comparison, the comparison efficiency is also improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to an image comparison method and device. Background Art

[0002] Software compatibility testing aims to verify the compatibility and stability of software or systems across different environments, platforms, devices, or configurations. For example, it can compare experimental web images with benchmark web images displayed at different resolutions and on different mobile phone models to ensure they are identical. Existing image comparison methods primarily rely on manual methods or use relevant image similarity algorithms to directly compare experimental and benchmark web images.

[0003] In the process of implementing the present invention, the inventors found that the prior art has the following problems:

[0004] Existing image comparison methods suffer from high labor costs and low efficiency, or the image comparison results are unreliable. Due to the influence of screen size and resolution, different devices may display different content when displaying the same webpage. For example, a baseline webpage image displays a complete product list, but due to differences in mobile phone resolution and screen size, other phones may only display a portion of the product list. It is unreasonable to fail the image comparison simply because of differences in screen size and resolution. Summary of the Invention

[0005] In view of this, an embodiment of the present invention provides a method and device for image comparison, which realizes efficient, fine-grained and accurate image comparison, and fine-grained comparison of web page images to the pictures contained therein and the constituent elements of the pictures. This not only avoids the situation where image comparison fails simply due to differences in display screen size and resolution, but also improves the accuracy of comparison and also improves the comparison efficiency compared with manual comparison.

[0006] To achieve the above object, according to one aspect of an embodiment of the present invention, a method for image comparison is provided, comprising:

[0007] In response to receiving the image comparison request, obtaining a reference web page image and an experimental web page image;

[0008] Extracting a benchmark image and an experimental image from the benchmark web page image and the experimental web page image respectively;

[0009] The similarity between the reference image and the experimental image is compared, and if the similarity comparison fails, the reference image and the experimental image are compared in terms of component elements, and a result of the component element comparison is used as a response to the image comparison request.

[0010] Optionally, before extracting the benchmark picture and the experimental picture from the benchmark web page image and the experimental web page image respectively, the method also includes: using a histogram equalization algorithm to perform image enhancement on the benchmark web page image and the experimental web page image respectively, and then performing grayscale processing on the image-enhanced benchmark web page image and experimental web page image.

[0011] Optionally, a benchmark image and an experimental image are extracted from the benchmark web image and the experimental web image, respectively, including: identifying the image distribution positions in the benchmark web image and the experimental web image; performing regional segmentation on the benchmark web image and the experimental web image according to the identified image distribution positions of the benchmark web image and the identified image distribution positions of the experimental web image, respectively, to obtain a benchmark segmentation map and an experimental segmentation map; clarifying the comparison area in the benchmark segmentation map and the comparison area in the experimental segmentation map; and using the images in the comparison areas in the benchmark segmentation map and the experimental segmentation map as benchmark images and experimental images, respectively.

[0012] Optionally, the distribution positions of the images in the benchmark web page image and the experimental web page image are identified, including: converting the benchmark web page image and the experimental web page image into one-dimensional vector images respectively to obtain a benchmark one-dimensional vector image and an experimental one-dimensional vector image; calculating the position start point and position end point of the image in the benchmark web page image according to the sizes of the benchmark one-dimensional vector image and the benchmark web page image; calculating the position start point and position end point of the image in the experimental web page image according to the sizes of the experimental one-dimensional vector image and the experimental web page image.

[0013] Optionally, before comparing the similarity between the reference image and the experimental image, the method further includes: adjusting the sizes of the reference image and the experimental image for consistency, so as to ensure that the size of the adjusted reference image is the same as the size of the adjusted experimental image.

[0014] Optionally, comparing the similarity between the reference image and the experimental image includes: comparing the overall similarity between the reference image and the experimental image using a structural similarity algorithm based on pixel values ​​of the reference image and the experimental image.

[0015] Optionally, before comparing the component elements of the reference image and the experimental image, the method further includes: performing image binarization processing on the reference image and the experimental image.

[0016] Optionally, the comparison of component elements of the benchmark image and the experimental image includes: identifying the contour positioning of the respective component elements from the benchmark image and the experimental image respectively; obtaining the benchmark component elements of the benchmark image and the experimental component elements of the experimental image based on the contour positioning of the component elements of the benchmark image and the contour positioning of the component elements of the experimental image, and determining the correspondence between the benchmark component elements and the experimental component elements; and performing a similarity comparison between the benchmark component elements and the corresponding experimental component elements.

[0017] Optionally, the experimental picture has a corresponding distribution position in the experimental web page diagram; in the case where the component element comparison fails, the method further includes: in response to receiving a comparison result visualization request, visually displaying the contour positioning corresponding to the experimental component element that fails the comparison in the experimental picture; according to the corresponding distribution position of the experimental picture in the experimental web page diagram, restoring the experimental picture with contour positioning to the corresponding position of the experimental web page diagram.

[0018] According to a second aspect of an embodiment of the present invention, there is provided an image comparison apparatus, comprising:

[0019] a web page image acquisition module, configured to acquire a reference web page image and an experimental web page image in response to receiving an image comparison request;

[0020] An image extraction module is used to extract a reference image and an experimental image from the reference web page image and the experimental web page image respectively;

[0021] The comparison module is used to compare the similarity between the reference image and the experimental image, and if the similarity comparison fails, compare the component elements of the reference image and the experimental image, and use the result of the component element comparison as a response to the image comparison request.

[0022] According to a third aspect of an embodiment of the present invention, there is provided an electronic device for image comparison, comprising:

[0023] one or more processors;

[0024] a storage device for storing one or more programs,

[0025] When the one or more programs are executed by the one or more processors, the one or more processors implement the method provided in the first aspect of the embodiment of the present invention.

[0026] According to a fourth aspect of an embodiment of the present invention, a computer-readable medium is provided, on which a computer program is stored. When the program is executed by a processor, the method provided by the first aspect of the embodiment of the present invention is implemented.

[0027] One embodiment of the invention has the following advantages or beneficial effects: obtaining a reference web page image and an experimental web page image in response to receiving an image comparison request; extracting a reference image and an experimental image from the reference web page image and the experimental web page image, respectively; comparing the similarity between the reference image and the experimental image, and if the similarity comparison fails, performing a component element comparison on the reference image and the experimental image, and using the result of the component element comparison as a response to the image comparison request. This invention implements an efficient, fine-grained, and accurate image comparison method, which fine-grains the comparison of web page images to the images contained therein and the component elements within the images. This not only avoids the situation where image comparison fails solely due to differences in display screen size and resolution, thereby improving the accuracy of the comparison, but also improves the comparison efficiency compared to manual comparison. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The accompanying drawings are provided for a better understanding of the present invention and are not intended to limit the present invention.

[0029] Figure 1 is a schematic diagram of the main process of the image comparison method according to an embodiment of the present invention;

[0030] Figure 2 is a schematic diagram of a benchmark web page graph and an experimental web page graph according to an embodiment of the present invention;

[0031] Figure 3 is a schematic diagram of a reference web page graph and a corresponding reference one-dimensional vector graph according to an embodiment of the present invention;

[0032] Figure 4 is a schematic diagram comparing the constituent elements of a reference image and an experimental image according to an embodiment of the present invention;

[0033] Figure 5 is a schematic diagram of an experimental web page diagram with comparative results according to an embodiment of the present invention;

[0034] Figure 6 This is a flow chart of the overall solution of the image comparison method according to an embodiment of the present invention;

[0035] Figure 7 is a schematic diagram of main modules of an image comparison device according to an embodiment of the present invention;

[0036] Figure 8 is an exemplary system architecture diagram in which embodiments of the present invention may be applied;

[0037] Figure 9 It is a schematic diagram of the structure of a computer system of a terminal device or a server suitable for implementing an embodiment of the present invention. DETAILED DESCRIPTION

[0038] It should be noted that the acquisition, storage and application of user personal information involved in the technical solution of this disclosure are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0039] The following description of exemplary embodiments of the present invention is made in conjunction with the accompanying drawings, in which various details of the embodiments of the present invention are included to facilitate understanding. These details should be considered as merely exemplary. Therefore, it should be appreciated by those skilled in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0040] Existing image comparison methods suffer from high labor costs and low efficiency, or the reliability of image comparison results is low. Due to the influence of screen size and resolution, different devices may display different content when displaying the same webpage. For example, a baseline webpage image displays a complete product list, but due to differences in mobile phone resolution and screen size, other phones may only display a portion of the product list. This type of image comparison failure caused solely by differences in screen size and resolution is unreasonable and cannot effectively meet practical applications.

[0041] To address the aforementioned problems in the prior art, the present invention proposes an image comparison method, wherein a benchmark image and an experimental image extracted from a benchmark webpage image and an experimental webpage image are compared for image similarity. If the similarity comparison fails, the benchmark image and the experimental image are compared at a more fine-grained level based on their constituent elements, thereby obtaining a result of the constituent element comparison. This method achieves an efficient, fine-grained, and accurate image comparison method, which fine-grains the comparison of webpage images down to the images contained therein and the constituent elements within them. This method not only avoids image comparison failures due solely to differences in display screen size and resolution, thereby improving comparison accuracy, but also increases comparison efficiency compared to manual comparison.

[0042] In the introduction to the embodiments of the present invention, the terms and their meanings are as follows:

[0043] SSIM: is a method used to measure the similarity between two images;

[0044] RGB: It is a color model that represents the combination of three basic colors: red, green, and blue.

[0045] Figure 1 FIG. 1 is a schematic diagram of the main process of the image comparison method according to an embodiment of the present invention. Figure 1 As shown, the image comparison method according to the embodiment of the present invention includes the following steps S101 to S103.

[0046] Step S101: In response to receiving an image comparison request, obtaining a reference web page image and an experimental web page image.

[0047] Specifically, for the web page image compatibility test of different resolutions and different mobile phone models in the software compatibility test, it is necessary to use different mobile phones to conduct comparative tests on the web page images of the software. The embodiment of the present invention utilizes a cloud test platform based on a cloud real machine, which can quickly and easily obtain the web page images displayed by all mobile phone models that need to undergo web page image compatibility testing, solving the time-consuming problem of manually opening the mobile phone and obtaining images. When the system receives an image comparison request for web page image compatibility testing, it will obtain the web page image displayed in the model to be tested from the cloud test platform as the experimental web page image based on the model to be tested and the web page address to be tested in the image comparison request, and at the same time obtain the benchmark web page image in the image comparison request. The benchmark web page image is a standard and correct image for image comparison. The experimental web page image and the benchmark web page image are the displays of the same web page image on the mobile phone to be tested and the benchmark mobile phone. All experimental web pages and benchmark web page images are compared.

[0048] Step S102: extracting a reference image and an experimental image from the reference web page image and the experimental web page image respectively.

[0049] Specifically, considering that webpage images often contain multiple images, for example, an e-commerce platform's product listing page consists of multiple images of products, each of which includes a text description and a photo of the product. Specifically, each image can be pre-labeled in the HTML documents of the benchmark and experimental webpages. Based on the labels, information related to the benchmark and experimental images can be extracted from the HTML documents. The benchmark and experimental images can then be obtained through parsing and rendering.

[0050] Figure 2 It is a schematic diagram of the benchmark web page diagram and the experimental web page diagram of an embodiment of the present invention. It can be seen that the benchmark web page diagram on the left displays a list of transaction items, specifically showing three transaction items, while the experimental web page diagram on the right also displays the list of transaction items, but due to the difference in the displayed mobile phone models, only two complete transaction items can be displayed.

[0051] According to one embodiment of the present invention, before extracting the benchmark picture and the experimental picture from the benchmark web page image and the experimental web page image respectively, the method also includes: using a histogram equalization algorithm to perform image enhancement on the benchmark web page image and the experimental web page image respectively, and then performing grayscale processing on the image-enhanced benchmark web page image and experimental web page image.

[0052] Specifically, in order to more accurately extract the benchmark image from the benchmark web page image and the experimental image from the experimental web page image, before extracting the benchmark image and the experimental image from the benchmark web page image and the experimental web page image respectively, a histogram equalization algorithm is used to enhance the contrast of the benchmark web page image and the contrast of the experimental web page image, so that the dark areas in the web page image become brighter and the bright areas become darker. For example, optionally, the equalizeHist(img) method in the open source computer vision library can be used to perform image enhancement, but it is not limited to this. In addition, since the benchmark web page image and the experimental web page image both provide RGB images, in order to subsequently identify the distribution positions of the images in the benchmark web page image and the experimental web page image, it is also necessary to convert the enhanced benchmark web page image and the experimental web page into grayscale images respectively.

[0053] According to another embodiment of the present invention, a benchmark image and an experimental image are extracted from the benchmark web image and the experimental web image, respectively, including: identifying the image distribution positions in the benchmark web image and the experimental web image; performing regional segmentation on the benchmark web image and the experimental web image according to the identified image distribution positions in the benchmark web image and the identified image distribution positions in the experimental web image, respectively, to obtain a benchmark segmentation map and an experimental segmentation map; clarifying a comparison area in the benchmark segmentation map and a comparison area in the experimental segmentation map; and using the images in the comparison areas in the benchmark segmentation map and the experimental segmentation map as benchmark images and experimental images, respectively.

[0054] Specifically, in addition to the above-mentioned method of parsing the HTML image tags of the benchmark web page image and the experimental web page image to obtain the distribution positions of the benchmark image and the experimental image, under normal circumstances, before the benchmark web page image is generated, the developers will agree on the image distribution rules of the benchmark web page image in advance. According to the distribution rules, the image distribution positions in the benchmark web page image can be identified; and the experimental web page image is the display image of the benchmark web page image on the mobile phone to be tested, and then according to the resolution and screen size of the model to be tested, the corresponding distribution rules are adaptively converted to obtain the image distribution characteristics corresponding to the experimental web page image of the model to be tested, and then the image distribution positions in the experimental web page image are identified.

[0055] Furthermore, based on the identified image distribution positions of the benchmark web page image and the image distribution positions of the experimental web page image, the benchmark web page image and the experimental web page image are respectively segmented to obtain a benchmark segmentation map and an experimental segmentation map with segmentation lines. Considering that the head navigation bar area in the experimental web page image and the benchmark web page image displays the application icons currently being used by the mobile phone, this part does not need to participate in the image comparison, so the comparison area that needs to be compared is determined from the benchmark segmentation map and the experimental segmentation map. Finally, the various benchmark images contained in the comparison area of ​​the benchmark segmentation map are extracted in a pre-agreed order, and the various experimental images contained in the comparison area of ​​the experimental segmentation map are extracted in the same order, and a correspondence between the benchmark images and the experimental images is established in the order of extraction, so that the experimental images and the corresponding benchmark images can be compared according to the correspondence.

[0056] According to another embodiment of the present invention, the distribution positions of the images in the benchmark web page image and the experimental web page image are identified, including: converting the benchmark web page image and the experimental web page image into one-dimensional vector images respectively to obtain a benchmark one-dimensional vector image and an experimental one-dimensional vector image; calculating the position start point and position end point of the image in the benchmark web page image according to the sizes of the benchmark one-dimensional vector image and the benchmark web page image; calculating the position start point and position end point of the image in the experimental web page image according to the sizes of the experimental one-dimensional vector image and the experimental web page image.

[0057] Specifically, the embodiment of the present invention adopts a more efficient and more applicable image data analysis method to identify the distribution position of images in the benchmark web page image and the experimental web page image. The benchmark web page image and the experimental web page image or the benchmark web page image and the experimental web page image after the above-mentioned grayscale processing are converted into a one-dimensional vector image. Specifically, a one-dimensional conversion method in a specified programming language library can be used to perform one-dimensional conversion to obtain a benchmark one-dimensional vector image and an experimental one-dimensional vector image. The one-dimensional conversion of the benchmark web page image and the experimental web page image is to find the distribution pattern of each area in the benchmark web page image and the experimental web page image, but is not limited to this.

[0058] Furthermore, based on the obtained benchmark one-dimensional vector graph, the distribution pattern of each area in the benchmark web page graph can be obtained. According to the starting point and ending point of each trough in the benchmark one-dimensional vector graph, the starting point and ending point are used, and then combined with the total number of pixels of the benchmark web page graph, the image size (horizontal width and vertical height), and the horizontally extracted width are fixed features, the position starting point and position end point of each image contained in the benchmark web page graph are calculated. Similarly, according to the starting point and ending point of each trough in the experimental one-dimensional vector graph, the starting point and ending point are used, and then combined with the total number of pixels of the experimental web page graph, the image size (horizontal width and vertical height), and the horizontally extracted width are fixed features, the position starting point and position end point of each image contained in the experimental web page graph are calculated.

[0059] For example, Figure 3 It is a schematic diagram of the reference web page diagram of an embodiment of the present invention, and the corresponding reference one-dimensional vector diagram. The upper figure is the reference web page diagram, and the lower figure is the reference one-dimensional vector diagram. The rectangular boxes in the lower reference one-dimensional vector diagram correspond to the pictures of the various rectangular areas in the upper reference web page diagram. Assuming that the number of pixels of the reference web page diagram is X, the width is W, and the height is H, and the width of each picture is the width of the screen by default, then the key to calculating the starting point and the end point of each picture lies in the starting point height and the end point height of each area in the reference web page diagram. The starting point height and the end point height of the first transaction product smart watch picture area are marked with circles in the figure. The specific calculation formulas for the starting point height and the end point height of each picture area are:

[0060]

[0061]

[0062] Where ImgH1 is the starting height of the image area, ImgH2 is the ending height of the image area, Xbegin represents the pixel value corresponding to the starting point of the trough in the area, and correspondingly, Xfinal represents the pixel value corresponding to the ending point of the trough in the area.

[0063] Step S103 : comparing the similarity between the reference image and the experimental image, and if the similarity comparison fails, performing a component element comparison on the reference image and the experimental image, and using the result of the component element comparison as a response to the image comparison request.

[0064] Specifically, based on the above-mentioned extracted benchmark images and experimental images, it can be understood that, under normal circumstances, there are multiple experimental web pages, and one experimental web page has multiple experimental images, which corresponds to multiple groups of experimental images. Accordingly, each group of experimental images is subjected to a similarity comparison with the corresponding benchmark images extracted from the benchmark web page. Whether the similarity comparison is passed is determined based on the set similarity threshold. The embodiment of the present invention takes into account individual factors such as the depth of the image color difference and sets a similarity threshold of 98%. If the similarity threshold is not reached, it is considered that the similarity comparison fails. If the comparison between the experimental image and the benchmark image fails, considering that the experimental image and the benchmark image also include components such as photos of transaction items and various texts, in order to improve the comparison accuracy and facilitate the acquisition of more accurate comparison results, a more fine-grained comparison of the component elements of the experimental image and the benchmark image is performed, and the result of the component element comparison is fed back to the requester of the comparison request.

[0065] According to one embodiment of the present invention, before comparing the similarity between the reference image and the experimental image, the method further includes: adjusting the size of the reference image and the experimental image for consistency, so as to ensure that the size of the adjusted reference image is the same as the size of the adjusted experimental image.

[0066] Specifically, in order to ensure the accuracy of image comparison, before comparing the similarity between the reference image and the experimental image, the reference image and the experimental image are resized for consistency, and the minimum value of the two images is taken as the target size for adjustment. Specifically, the image scaling method in the open source computer vision library can be used, and the minimum width and minimum height of the two images are taken as the target size for adjustment. For example, optionally, the image scaling method is: imageA = cv.resize(imageA, dsize = (min(imageA.shape[1], imageB.shape[1]), min(imageA.shape[0], imageB.shape[0])), interpolation = cv.INTER_AREA), but it is not limited to this. For the example image scaling method, in the open source computer vision library, cv.INTER_AREA is a method of image scaling and interpolation, which is used to calculate pixel values ​​in the image resampling operation. When using the resize() function to scale an image, if you need to reduce the image size (i.e., shrink the image), cv.INTER_AREA is a commonly used interpolation method, where min(imageA.shape[1], imageB.shape[1]) refers to the minimum width of imageA and imageB; min(imageA.shape[0], imageB.shape[0]) refers to the minimum height of imageA and imageB.

[0067] According to another embodiment of the present invention, comparing the similarity between the reference image and the experimental image includes: comparing the overall similarity between the reference image and the experimental image using a structural similarity algorithm based on pixel values ​​of the reference image and the experimental image.

[0068] Specifically, based on the pixel values ​​of the benchmark image and the experimental image, the pixel mean and pixel variance of the benchmark image are calculated. Based on the pixel mean and pixel variance, as well as the pixel value, the benchmark image is set as x and the experimental image is set as y. The structural similarity algorithm is used. Optionally, the SSIM algorithm is used to calculate the overall similarity between the benchmark image and the experimental image:

[0069]

[0070] Among them, μ xand μ y Respectively represent the mean value of the pixels in the image x and the mean value of the pixels in the image y, σ x and σ y Indicates the variance of the image x pixels and the variance of the image y pixels, x i and y i represents each pixel value of image x and image y, and C1 and C2 are constants. Of course, other structural similarity algorithms can also be used, and the embodiment of the present invention does not make specific limitations.

[0071] According to yet another embodiment of the present invention, before comparing the component elements of the reference image and the experimental image, the method further includes: performing image binarization processing on the reference image and the experimental image.

[0072] Specifically, before comparing the components of the baseline image and the experimental image, the baseline image and the experimental image are binarized using a threshold operation method in an open source computer vision library, such as, but not limited to, cv.threshold(diff, 0, 125, cv.THRESH_BINARY_INV | cv.THRESH_OTSU). A threshold of 125 is specified, and if the pixel is greater than the threshold, the pixel is set to 125, otherwise it is set to 0. The binarized baseline image and experimental image can facilitate subsequent comparison of their components.

[0073] According to another embodiment of the present invention, the comparison of constituent elements of the benchmark image and the experimental image includes: identifying the contour positioning of the respective constituent elements from the benchmark image and the experimental image respectively; obtaining the benchmark constituent elements of the benchmark image and the experimental constituent elements of the experimental image based on the contour positioning of the constituent elements of the benchmark image and the contour positioning of the constituent elements of the experimental image, and determining the correspondence between the benchmark constituent elements and the experimental constituent elements; and performing a similarity comparison between the benchmark constituent elements and the corresponding experimental constituent elements.

[0074] Specifically, the benchmark and experimental images are analyzed and identified for their constituent elements, obtaining contour location information for each element. Based on this contour location information, the constituent elements are extracted from the benchmark and experimental images. The corresponding relationship between the benchmark and experimental elements is determined based on the element identifiers within each element. Finally, an existing similarity comparison algorithm is used to compare the similarity between the benchmark and experimental elements. If the similarity value does not meet the preset requirements, it is determined that the experimental and benchmark elements differ, and this difference needs to be recorded and reported.

[0075] According to another embodiment of the present invention, the experimental picture has a corresponding distribution position in the experimental web page diagram; in the case where the component element comparison fails, the method further includes: in response to receiving a comparison result visualization request, visually displaying the contour positioning corresponding to the experimental component element that fails the comparison in the experimental picture; based on the corresponding distribution position of the experimental picture in the experimental web page diagram, restoring the experimental picture with the contour positioning to the corresponding position of the experimental web page diagram.

[0076] Specifically, when the experimental picture is extracted from the experimental web page diagram, the distribution position information of the experimental picture in the experimental web page diagram will be recorded. In order to provide a clearer comparison result to the comparison requester, when a comparison result visualization request is received, the contour positioning corresponding to the experimental component elements that fail the comparison is visualized in the experimental picture according to the comparison results of the above-mentioned component elements. For example, the contour positioning of the component elements that fail the comparison is displayed in the experimental picture with a rectangular frame using the rectangular frame drawing method in the open source computer vision library. In addition, considering that the experimental picture may have been adjusted for size consistency before the comparison, the original size is determined according to the distribution position information of the experimental picture in the experimental web page diagram, and the size of the experimental picture with contour positioning is restored. According to the distribution position information of the experimental picture, the restored experimental picture with contour positioning is used to replace the original experimental picture in the experimental web page diagram to obtain the experimental web page diagram with the comparison results.

[0077] Figure 4 This is a schematic diagram comparing the components of a baseline image and an experimental image according to an embodiment of the present invention. The first image is the baseline image, the second image is the experimental image, and the third image is the annotated experimental image with the outline (rectangular frame) displayed after comparison. The difference between the experimental image and the baseline image is the lack of the purchase data text box. Therefore, the purchase data text box is added to the experimental image to produce the annotated experimental image.

[0078] Figure 5 This is a schematic diagram of an experimental web page with comparative results according to an embodiment of the present invention. Figure 4 After comparison, it is shown that the experimental image marked with a rectangular frame has been restored to the corresponding position of the experimental web page image.

[0079] Figure 6It is a flow chart of the overall solution of the image comparison method of an embodiment of the present invention. The image comparison mainly includes five parts. The first is to obtain a benchmark web page image and multiple experimental web page images to be tested from the cloud test platform when an image comparison request is received. Next, before extracting the benchmark image and experimental image for comparison from the benchmark web page image and the experimental web page image, the benchmark web page image and the experimental web page image can be subjected to image enhancement, grayscale processing, and other image preprocessing in advance to facilitate subsequent image extraction. The benchmark web page image and the experimental web page image after grayscale processing are converted into one dimension to obtain a one-dimensional vector image. According to the one-dimensional vector image, the image distribution positions in the benchmark web page image and the experimental web page image are identified. According to the identified image distribution positions of the benchmark web page image and the image distribution positions of the experimental web page image, the benchmark web page image and the experimental web page image are respectively segmented to obtain each segmented area of ​​the benchmark web page image and each segmented area of ​​the experimental web page image. Next, a benchmark image to be compared is selected from each segmented region of the benchmark web image, and a corresponding experimental image is selected from each segmented region of the experimental web image. An open-source computer vision library is used to compare the similarity between the benchmark and experimental images to determine whether the experimental and benchmark images are identical. If the similarity comparison fails, a more fine-grained similarity comparison of the components of the experimental and benchmark images is performed to obtain the comparison results for each experimental image. Finally, based on the comparison result visualization request received, Experimental Images 1 and 2, along with the comparison results, are restored to the experimental web image, resulting in the final visual comparison results of the benchmark and experimental web images.

[0080] The embodiment of the present invention is based on a cloud-based testing platform, which conveniently obtains various experimental web page images to be compared; and by segmenting the regions of the benchmark web page image and the experimental web page image, determines the comparison regions that need to be compared, and effectively filters out the extra noise in the web page image that does not need to be compared, thereby improving the comparison efficiency and ensuring the accuracy of the comparison; finally, by extracting the benchmark image and the experimental image to be compared from the web page image, performing a similarity comparison of the image granularity, and performing a similarity comparison of the granularity of the constituent elements of the benchmark image and the experimental image that fail the comparison, fine-grained image comparison is achieved. The image comparison using the embodiment of the present invention can effectively avoid the situation where the image comparison fails only due to differences in display screen size and resolution, improves the accuracy of the comparison, and also improves the comparison efficiency compared with manual comparison.

[0081] Figure 7 FIG. 1 is a schematic diagram of the main modules of the image comparison device according to an embodiment of the present invention. Figure 7 As shown, the image comparison device 700 mainly includes a web page image acquisition module 701, an image extraction module 702 and a comparison module 703.

[0082] The web page image acquisition module 701 is configured to acquire a reference web page image and an experimental web page image in response to receiving an image comparison request;

[0083] An image extraction module 702 is configured to extract a reference image and an experimental image from the reference web page image and the experimental web page image, respectively;

[0084] The comparison module 703 is used to compare the similarity between the reference image and the experimental image, and if the similarity comparison fails, perform a component element comparison on the reference image and the experimental image, and use the result of the component element comparison as a response to the image comparison request.

[0085] According to one embodiment of the present invention, the image comparison device 700 also includes an image preprocessing module (not shown in the figure), which is used to: before extracting the benchmark picture and the experimental picture from the benchmark web page image and the experimental web page image respectively, use a histogram equalization algorithm to perform image enhancement on the benchmark web page image and the experimental web page image respectively, and then perform grayscale processing on the enhanced benchmark web page image and experimental web page image.

[0086] According to another embodiment of the present invention, the image extraction module 702 is further used to: identify the image distribution positions in the benchmark web page image and the experimental web page image; perform regional segmentation on the benchmark web page image and the experimental web page image according to the identified image distribution positions of the benchmark web page image and the identified image distribution positions of the experimental web page image, to obtain a benchmark segmentation map and an experimental segmentation map; clarify the comparison area in the benchmark segmentation map and the comparison area in the experimental segmentation map; and use the images in the comparison areas in the benchmark segmentation map and the experimental segmentation map as benchmark images and experimental images, respectively.

[0087] According to another embodiment of the present invention, the image extraction module 702 is further used to: convert the benchmark web page image and the experimental web page image into one-dimensional vector images respectively to obtain a benchmark one-dimensional vector image and an experimental one-dimensional vector image; calculate the position start point and position end point of the image in the benchmark web page image according to the sizes of the benchmark one-dimensional vector image and the benchmark web page image; calculate the position start point and position end point of the image in the experimental web page image according to the sizes of the experimental one-dimensional vector image and the experimental web page image.

[0088] According to another embodiment of the present invention, the image comparison device 700 further includes a size adjustment module (not shown in the figure), which is used to: before comparing the similarity between the benchmark image and the experimental image, adjust the size of the benchmark image and the experimental image for consistency, so as to determine that the size of the adjusted benchmark image is the same as the size of the adjusted experimental image.

[0089] According to another embodiment of the present invention, the comparison module 703 is further configured to compare the overall similarity between the reference image and the experimental image using a structural similarity algorithm based on pixel values ​​of the reference image and the experimental image.

[0090] According to another embodiment of the present invention, the image comparison device 700 also includes an image processing module (not shown in the figure), which is used to: perform image binarization processing on the reference image and the experimental image before comparing the component elements of the reference image and the experimental image.

[0091] According to another embodiment of the present invention, the comparison module 703 is further used to: identify the contour positioning of each component element from the benchmark image and the experimental image respectively; obtain the benchmark component elements of the benchmark image and the experimental component elements of the experimental image based on the contour positioning of the component elements of the benchmark image and the contour positioning of the component elements of the experimental image, and determine the correspondence between the benchmark component elements and the experimental component elements; and compare the benchmark component elements and the corresponding experimental component elements for similarity.

[0092] According to another embodiment of the present invention, the experimental picture has a corresponding distribution position in the experimental web page diagram; in the case where the component element comparison fails, the image comparison device 700 also includes a result visualization processing module (not shown in the figure), which is used to: in response to receiving a comparison result visualization request, visualize the contour positioning corresponding to the experimental component element that fails the comparison in the experimental picture; according to the corresponding distribution position of the experimental picture in the experimental web page diagram, restore the experimental picture with contour positioning to the corresponding position of the experimental web page diagram.

[0093] Figure 8 is an exemplary system architecture diagram to which embodiments of the present invention can be applied.

[0094] like Figure 8 As shown, system architecture 800 may include terminal devices 801, 802, 803, a network 804, and a server 805. Network 804 is used to provide a medium for communication links between terminal devices 801, 802, 803 and server 805. Network 804 may include various connection types, such as wired or wireless communication links or fiber optic cables.

[0095] Users can use terminal devices 801, 802, 803 to interact with server 805 via network 804 to receive or send messages, etc. Terminal devices 801, 802, 803 can be installed with various communication client applications, such as image comparison applications, etc. (only as an example).

[0096] The terminal devices 801 , 802 , and 803 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers.

[0097] The server 805 may be a server that provides various services, such as a background management server (for example only) that supports image comparison performed by users using the terminal devices 801, 802, and 803. In response to receiving an image comparison request, the background management server may obtain a reference web page image and an experimental web page image; extract a reference image and an experimental image from the reference web page image and the experimental web page image, respectively; compare the similarity between the reference image and the experimental image, and if the similarity comparison fails, perform a component element comparison on the reference image and the experimental image, process the result of the component element comparison as a response to the image comparison request, and feed back the processing result (for example, the comparison result, etc. - for example only) to the terminal device.

[0098] It should be noted that the image comparison method provided in the embodiment of the present invention is generally executed by the server 805 , and accordingly, the image comparison device is generally provided in the server 805 .

[0099] It should be understood that Figure 8 The number of terminal devices, networks and servers in the embodiment is merely illustrative. Any number of terminal devices, networks and servers may be provided as required.

[0100] Reference below Figure 9 , which shows a schematic structural diagram of a computer system of a terminal device or server suitable for implementing an embodiment of the present invention. Figure 9 The terminal device or server shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0101] like Figure 9 As shown, the computer system 900 includes a central processing unit (CPU) 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage unit 908 into a random access memory (RAM) 903. Various programs and data required for the operation of the system 900 are also stored in the RAM 903. The CPU 901, ROM 902, and RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0102] The following components are connected to the I / O interface 905: an input section 906 including a keyboard, a mouse, and the like; an output section 907 including devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), and a speaker; a storage section 908 including a hard disk and the like; and a communication section 909 including a network interface card such as a LAN card or a modem. The communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the I / O interface 905 as needed. A removable medium 911, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is installed in the drive 910 as needed, so that computer programs read therefrom can be installed into the storage section 908 as needed.

[0103] In particular, according to the embodiments disclosed in the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 909, and / or installed from a removable medium 911. When the computer program is executed by the central processing unit (CPU) 901, the above-mentioned functions defined in the system of the present invention are performed.

[0104] It should be noted that the computer-readable medium described in the present invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media can include, but are not limited to, an electrical connection having one or more conductors, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. This propagated data signal can take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wireline, optical cable, RF, or any suitable combination thereof.

[0105] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of the boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0106] The units involved in the embodiments of the present invention may be implemented in software or hardware. The units described may also be provided in a processor. For example, the processor may include: a web page image acquisition module, an image extraction module, and a comparison module.

[0107] Among them, the names of these modules do not constitute a limitation of the modules themselves in some cases. For example, the web page image acquisition module can also be described as "a module for obtaining a baseline web page image and an experimental web page image in response to receiving an image comparison request."

[0108] On the other hand, the present invention also provides a computer-readable medium, which may be included in the device described in the embodiment; or may exist independently and not be assembled into the device. The computer-readable medium carries one or more programs, and when the one or more programs are executed by a device, the device includes: in response to receiving an image comparison request, obtaining a reference web page image and an experimental web page image; extracting a reference image and an experimental image from the reference web page image and the experimental web page image, respectively; comparing the similarity between the reference image and the experimental image, and if the similarity comparison fails, performing a component element comparison on the reference image and the experimental image, and using the result of the component element comparison as a response to the image comparison request.

[0109] The technical solution according to the embodiment of the present invention has the following advantages or beneficial effects: by responding to receiving an image comparison request, a baseline web page image and an experimental web page image are obtained; a baseline picture and an experimental picture are extracted from the baseline web page image and the experimental web page image respectively; the similarity between the baseline picture and the experimental picture is compared, and when the similarity comparison fails, the baseline picture and the experimental picture are compared in terms of constituent elements, and the result of the constituent element comparison is used as a technical solution in response to the image comparison request, thereby realizing an efficient, fine-grained and accurate image comparison method, which fine-grains the comparison of web page images to the pictures contained therein and the constituent elements in the pictures, not only avoiding the situation where the image comparison fails only due to differences in display screen size and resolution, but also improving the accuracy of the comparison, and also improving the comparison efficiency compared with manual comparison.

[0110] The specific embodiments described herein do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A method for image comparison, characterized in that: include: In response to receiving the image comparison request, obtaining a reference web page image and an experimental web page image; Extracting a benchmark image and an experimental image from the benchmark web page image and the experimental web page image respectively; The similarity between the reference image and the experimental image is compared, and if the similarity comparison fails, the reference image and the experimental image are compared in terms of component elements, and a result of the component element comparison is used as a response to the image comparison request.

2. The method according to claim 1, characterized in that Before extracting the benchmark image and the experimental image from the benchmark web page image and the experimental web page image respectively, the method further includes: The image enhancement of the reference web page image and the experimental web page image is performed respectively using a histogram equalization algorithm, and then the enhanced reference web page image and the experimental web page image are subjected to grayscale processing.

3. The method according to claim 1, characterized in that Extracting a benchmark image and an experimental image from the benchmark web page image and the experimental web page image respectively includes: Identifying distribution positions of images in the benchmark web page graph and the experimental web page graph; According to the identified distribution positions of the images in the benchmark webpage graph and the identified distribution positions of the images in the experimental webpage graph, the benchmark webpage graph and the experimental webpage graph are respectively segmented to obtain a benchmark segmentation graph and an experimental segmentation graph; Determine the comparison area in the benchmark segmentation map and the comparison area in the experimental segmentation map; The images in the comparison areas of the benchmark segmentation map and the experimental segmentation map are respectively used as the benchmark image and the experimental image.

4. The method according to claim 3, characterized in that Identifying the distribution positions of the images in the benchmark web page graph and the experimental web page graph includes: Converting the reference web page graph and the experimental web page graph into one-dimensional vector graphs respectively to obtain a reference one-dimensional vector graph and an experimental one-dimensional vector graph; Calculating the starting point and ending point of the image in the reference web page image according to the sizes of the reference one-dimensional vector image and the reference web page image; According to the sizes of the experimental one-dimensional vector graph and the experimental web page graph, the position start point and the position end point of the image in the experimental web page graph are calculated.

5. The method according to claim 1, wherein Before comparing the similarity between the reference image and the experimental image, the method further includes: The reference image and the experimental image are adjusted for size consistency to ensure that the size of the adjusted reference image is the same as the size of the adjusted experimental image.

6. The method according to claim 1, characterized in that Comparing the similarity between the reference image and the experimental image includes: According to the pixel values ​​of the reference image and the experimental image, a structural similarity algorithm is used to compare the overall similarity between the reference image and the experimental image.

7. The method according to claim 1, characterized in that Before comparing the reference image and the experimental image with respect to the composition elements, the method further includes: Perform image binarization processing on the benchmark image and the experimental image.

8. The method according to claim 1, characterized in that Comparing the composition elements of the reference image and the experimental image includes: Identifying the contour locations of the respective components from the reference image and the experimental image respectively; Obtaining, based on the contour positioning of the constituent elements of the reference image and the contour positioning of the constituent elements of the experimental image, a reference constituent element of the reference image and an experimental constituent element of the experimental image, and determining a correspondence between the reference constituent element and the experimental constituent element; The benchmark component elements and the corresponding experimental component elements are compared for similarity.

9. The method according to claim 8, characterized in that The experimental pictures have corresponding distribution positions in the experimental web page diagram; If the comparison of the constituent elements fails, the method further includes: In response to receiving a comparison result visualization request, visually displaying the contours corresponding to the experimental component elements that failed the comparison in the experimental picture; According to the corresponding distribution positions of the experimental pictures in the experimental web page diagram, the experimental pictures with outline positioning are restored to the corresponding positions in the experimental web page diagram.

10. An image comparison device, characterized in that: include: a web page image acquisition module, configured to acquire a reference web page image and an experimental web page image in response to receiving an image comparison request; An image extraction module is used to extract a reference image and an experimental image from the reference web page image and the experimental web page image respectively; The comparison module is used to compare the similarity between the reference image and the experimental image, and if the similarity comparison fails, compare the component elements of the reference image and the experimental image, and use the result of the component element comparison as a response to the image comparison request.

11. A mobile electronic device terminal, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 9.

12. A computer-readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Images