Methods, apparatus, equipment, media, and products for detecting page layout differences

Abstract

This application relates to a method, apparatus, device, medium, and product for detecting layout differences. The method includes: detecting pairs of difference images corresponding to at least one image difference region between two interface images; performing sliding matching on the first difference image in the same pair of difference images in the second interface image to which the second difference image belongs, and determining the similarity between the graphic element corresponding to each sliding match in the second interface image and the first difference image; when the similarity of each graphic element in the second interface image is lower than a preset similarity threshold, determining the image difference region corresponding to the same pair of difference images as the corresponding layout difference region between the two interface images; and marking the layout difference region in the image content of the two interface images. This application can not only determine the layout difference region between two interface images, but also specifically locate the specific page element causing the layout change between the two interface images. It has low computational complexity and can improve the efficiency of interface inspection during page development.

Description

Technical Field

[0001] This application relates to image detection technology, and more particularly to a method, apparatus, device, medium, and product for detecting layout differences. Background Technology

[0002] Layout design is an almost unavoidable part of website and application development, and its quality directly impacts the overall value of the website or application. For example, the interface images generated on the pages of various e-commerce platform websites or applications directly affect users' aesthetic perception and user experience. Therefore, introducing interface testing techniques to ensure the quality of layout design in these scenarios is essential.

[0003] In practice, website and application layout designers are responsible for designing and finalizing the interface display effects of the corresponding pages. Before the pages are published, they often need to conduct interface testing to confirm whether editing the pages has resulted in changes to the corresponding interface images. One purpose of this testing is to identify discrepancies, especially those caused by adjustments to interface controls. These discrepancies are used to troubleshoot layout anomalies and correct the layout design in a timely manner. Reasons for layout anomalies include, but are not limited to, text and image formatting errors, string breaks in text, missing elements, and so on, all of which will be reflected in the corresponding interface images. Relying on manual inspection of the massive number of pages generated by each website, application, and even platform to identify layout anomalies is clearly inefficient. Therefore, it is best to technicalize layout anomaly detection.

[0004] On the path to technological advancement, the applicant recognized a significant obstacle: in page layout design, the spatial positions of various elements are interconnected and constantly changing. Traditional image comparison techniques typically only provide general information about large areas of change, while the details causing these large-scale shifts are the most direct indicators of the problem. For example, a slight resizing of a control on the page can cause subsequent elements to shift, and traditional techniques cannot pinpoint the exact control responsible for these changes. Furthermore, attempting to implement this using deep learning models is clearly not cost-effective and lacks efficiency advantages.

[0005] Therefore, given the inherent limitations of traditional interface testing technology, the process of deepening interface testing technology not only needs to consider the potential need to determine the root cause of anomalies when investigating layout abnormalities, but also needs to take into account various factors, including cost. Summary of the Invention

[0006] The purpose of this application is to provide a method for detecting page layout differences and the corresponding apparatus, device, non-volatile readable storage medium, and computer program product.

[0007] According to one aspect of this application, a method for detecting page layout differences is provided, comprising the following steps:

[0008] Detect each pair of difference images corresponding to at least one image difference region between two interface images;

[0009] The first difference map belonging to the first interface map in the same pair of difference maps is subjected to sliding matching in the second interface map to which the second difference map belongs, and the similarity between the corresponding graphic element in the second interface map and the first difference map is determined.

[0010] When the similarity of each graphic element in the second interface image is lower than the preset similarity threshold, the image difference region corresponding to the same pair of difference images is determined as the corresponding layout difference region between the two interface images.

[0011] The areas of difference in layout are marked in the image content of the two interface diagrams.

[0012] In an optional embodiment, detecting pairs of difference maps corresponding to at least one image difference region between two interface maps includes:

[0013] Obtain two interface images corresponding to different design versions of the same page;

[0014] Identify the areas of image difference between the two interface images;

[0015] For each of the image differences, a corresponding rectangular box is extended to indicate the corresponding image difference region;

[0016] Based on the corresponding screenshots of each rectangle from the two interface images, we obtain the pairs of difference images corresponding to each rectangle.

[0017] In an optional embodiment, determining the locations of image differences between the two interface images includes:

[0018] Calculate the structural similarity between the two interface diagrams to obtain a structural similarity diagram;

[0019] A morphological erosion operation is performed on the structural similarity map to obtain an erosion correction map;

[0020] Binarization is performed on the erosion correction map based on a preset edge threshold to obtain a binarized image;

[0021] In the binarized image, at least one closed edge contour is found to determine the image difference locations accordingly.

[0022] In an optional embodiment, taking screenshots from the two interface images according to each rectangle includes:

[0023] Calculate the image area occupied by each rectangle, delete rectangles whose image area is lower than a preset area threshold, and retain valid rectangles to indicate each valid image difference region.

[0024] Based on the corresponding screenshots of each valid rectangle from the two interface images, we obtain the pairs of difference images corresponding to each rectangle.

[0025] In an optional embodiment, a sliding match is performed between the first difference map belonging to the first interface map in the same pair of difference maps and the second interface map to which the second difference map belongs, to determine the similarity between the graphic elements corresponding to each sliding match in the second interface map and the first difference map, including:

[0026] In the same pair of difference maps, the image with the larger texture deviation is designated as the first difference map, and the one with the smaller deviation is designated as the second difference map.

[0027] In the second interface map to which the second difference map belongs, an image matching region with a larger preset area is expanded based on the image difference region occupied by the second difference map in the second interface map.

[0028] The first difference map is used to perform sliding matching within the region image corresponding to the image matching region, and the similarity between the primitives corresponding to each sliding match in the region image and the first difference map is determined.

[0029] In an optional embodiment, before determining that the image with the larger texture deviation in the same pair of difference maps is the first difference map and the smaller one is the second difference map, the process includes:

[0030] Calculate the variance of the two difference plots in the same pair of difference plots respectively;

[0031] When the maximum variance between the two difference graphs is less than a preset variance threshold, the pair of difference graphs will no longer be processed, and the next pair of difference graphs will be called for iterative processing.

[0032] When the maximum variance in the two difference maps exceeds the variance threshold, the difference map corresponding to the maximum variance is set as the first difference map, and the other difference map is set as the second difference map.

[0033] In an optional embodiment, a sliding match is performed on the first difference map within the region image corresponding to the image matching region to determine the similarity between the primitives corresponding to each sliding match in the region image and the first difference map, including:

[0034] Binarization is performed on the first difference map based on a preset mask threshold to obtain a reference mask;

[0035] Based on the image size setting of the first difference map, the sliding window of equal size, and the preset step size, sliding matching is initiated within the image area of ​​the region.

[0036] Each time the sliding window slides, its corresponding primitive is determined, and the similarity between the primitive and the first difference map is calculated using the reference mask as the weight.

[0037] According to another aspect of this application, a layout difference detection device is provided, comprising:

[0038] The difference detection module is configured to detect pairs of difference images corresponding to at least one image difference region between two interface images.

[0039] The sliding matching module is configured to perform sliding matching on the first difference map belonging to the first interface map in the same pair of difference maps, and on the second interface map to which the second difference map belongs, to determine the similarity between the corresponding graphic element in the second interface map and the first difference map for each sliding matching;

[0040] The difference detection module is configured to determine the image difference region corresponding to the same pair of difference images as the corresponding layout difference region between the two interface images when the similarity of each graphic element in the second interface image is lower than a preset similarity threshold.

[0041] The difference annotation module is configured to mark the differences in the layout of the two interface images.

[0042] According to another aspect of this application, a layout difference detection device is provided, including a central processing unit and a memory, wherein the central processing unit is used to invoke and run a computer program stored in the memory to perform the steps of the layout difference detection method described in this application.

[0043] According to another aspect of this application, a non-volatile readable storage medium is provided, which stores a computer program implemented according to the layout difference detection method in the form of computer-readable instructions, wherein the computer program, when invoked by a computer, executes the steps included in the method.

[0044] According to another aspect of this application, a computer program product is provided, including a computer program / instructions that, when executed by a processor, implement the steps of the method described in any embodiment of this application.

[0045] Compared with existing technologies, this application has several technological advantages, including but not limited to:

[0046] First, this application identifies the image difference regions from two interface images. It then obtains pairs of difference images corresponding to each image difference region from the two interface images. For the same pair of difference images, using the first difference image as a template, it performs a sliding match with the second interface image to which the second difference image belongs. This determines the similarity of the corresponding graphic elements in the second interface image for each sliding match. When the similarity of each graphic element determined relative to the first difference image is lower than a preset similarity threshold, it can be determined that the image difference region indicated by the pair of difference images is the layout difference region between the two interface images. This allows for the marking of the corresponding layout difference regions in the two interface images, achieving efficient screening of image content differences between the two interface images. This can replace manual screening and improve page development efficiency.

[0047] Secondly, since this application performs a wider range of image matching based on the first difference map in the second interface map where the second difference map is located, and uses the condition that the similarity of each corresponding graphic element is lower than the preset similarity threshold as the condition for identifying the layout difference area, it can effectively ignore the false differences caused by the displacement of page elements between the two interface maps, and determine the real layout difference to the image position that caused the displacement of other elements. This makes it convenient for users to accurately locate the corresponding page element that caused the layout change by marking the layout difference area, and realize the root cause analysis and location of layout anomalies, which can further improve the efficiency of layout change investigation.

[0048] Furthermore, this application can achieve interface difference detection based on sliding matching, without the need to introduce deep learning models or implement corresponding training, which can save labeling and training costs and has extremely low computational load. When performing interface difference detection on websites and applications of large sites such as e-commerce platforms, it can yield significant economies of scale. Attached Figure Description

[0049] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0050] Figure 1 This is a flowchart illustrating one embodiment of the layout difference detection method of this application;

[0051] Figure 2 This is a schematic diagram illustrating the principle of sliding matching in the region image of the second interface image using the first difference map in this application;

[0052] Figure 3 , Figure 4 , Figure 5All of these are illustrations of identifying differences in the layout of the two corresponding interface diagrams based on this application.

[0053] Figure 6 This is a flowchart illustrating the process of determining the pairs of difference diagrams for two interface diagrams in an embodiment of this application.

[0054] Figure 7 This is a flowchart illustrating the process of determining the differences between two interface images in an embodiment of this application.

[0055] Figure 8 This is a schematic diagram of the process for determining multiple similarities for the same pair of difference maps in an embodiment of this application;

[0056] Figure 9 This is a schematic diagram illustrating the process of distinguishing different difference maps within the same pair of difference maps in an embodiment of this application;

[0057] Figure 10 This is a schematic diagram illustrating the process of implementing sliding matching to determine multiple similarities in an embodiment of this application;

[0058] Figure 11 This is a schematic diagram of the layout difference detection device of this application;

[0059] Figure 12 This is a schematic diagram of the structure of a layout difference detection device used in this application. Detailed Implementation

[0060] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0061] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this application means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or wireless coupling. The term “and / or” as used herein includes all or any units and all combinations of one or more associated listed items.

[0062] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0063] Those skilled in the art will understand that the terms "client," "terminal," and "terminal device" as used herein include both devices that receive wireless signals, devices that only possess wireless signal receiver capabilities without transmission capabilities, and devices with receiving and transmitting hardware, devices that have receiving and transmitting hardware capable of bidirectional communication over a bidirectional communication link. Such devices may include: cellular or other communication devices such as personal computers or tablets, having single-line displays, multi-line displays, or cellular or other communication devices without multi-line displays; PCS (Personal Communications Service) that can combine voice, data processing, fax, and / or data communication capabilities; PDA (Personal Digital Assistant) that may include a radio frequency receiver, pager, internet / intranet access, web browser, notepad, calendar, and / or GPS (Global Positioning System) receiver; and conventional laptops and / or handheld computers or other devices that have and / or include radio frequency receivers. As used herein, "client," "terminal," and "terminal device" can be portable, transportable, installed in a means of transportation (air, sea, and / or land), or suitable and / or configured to operate locally and / or in a distributed manner, operating in any other location on Earth and / or in space. "Client," "terminal," and "terminal device" as used herein can also be a communication terminal, an internet access terminal, or a music / video playback terminal, such as a PDA, a MID (Mobile Internet Device), and / or a mobile phone with music / video playback capabilities, or a smart TV, set-top box, etc.

[0064] The hardware referred to by the names "server," "client," and "service node" in this application is essentially an electronic device with the equivalent capabilities of a personal computer. It is a hardware device with the necessary components revealed by the von Neumann architecture, such as a central processing unit (including an arithmetic logic unit and a control unit), memory, input devices, and output devices. The computer program is stored in its memory, and the central processing unit loads the program stored in the secondary storage into the main memory to run it, execute the instructions in the program, and interact with the input and output devices to complete specific functions.

[0065] It should be noted that the concept of "server" used in this application can also be extended to the case of server clusters. Based on the network deployment principles understood by those skilled in the art, the servers should be logically divided. Physically, these servers can be independent of each other but accessible through interfaces, or they can be integrated into a single physical computer or a computer cluster. Those skilled in the art should understand this flexibility and should not use it to constrain the implementation of the network deployment method in this application.

[0066] One or more of the technical features of this application, unless explicitly specified herein, can be deployed on a server and accessed by a client remotely calling the online service interface provided by the server, or can be directly deployed and run on a client for access.

[0067] Unless otherwise specified, all data involved in this application may be stored remotely on a server or on a local terminal device, as long as it is suitable for use by the technical solution of this application.

[0068] Those skilled in the art will understand that although the various methods in this application are described based on the same concept and thus present commonality among them, they can be performed independently unless otherwise specified. Similarly, the various embodiments disclosed in this application are all based on the same inventive concept; therefore, concepts expressed in the same way, as well as concepts that are appropriately changed for convenience but are expressed differently, should be understood equivalently.

[0069] Unless otherwise expressly stated, the various embodiments disclosed in this application can be combined in a cross-cutting manner to flexibly construct new embodiments, as long as such combination does not depart from the inventive spirit of this application and can meet the needs of the prior art or solve a certain deficiency in the prior art. Those skilled in the art should be aware of such modifications.

[0070] Based on the principles revealed above, please refer to Figure 1 According to a layout difference detection method provided in this application, in one embodiment, it includes the following steps:

[0071] Step S1100: Detect each pair of difference images corresponding to at least one image difference region between two interface images;

[0072] This application performs layout difference detection based on every two interface images, two interfaces Figure 1 Generally, these images are generated based on different versions of the same page. For example, in one embodiment, before modifying the layout of a page, the developer saves a screenshot of the previous version as one of the interface images. After completing the editing of the current version of the page, another interface image is generated based on the current version. Thus, the two interface images, one before the other, can be used to detect layout differences and identify layout changes caused by page editing. In another embodiment, one interface image can be a screenshot generated based on the current version of the page, while the other interface image can be a static image generated by the interface designer.

[0073] It's easy to understand that since the two interface images are usually generated from the same page, they are generally of equal size. If their sizes differ, adjustments can be made beforehand to ensure they are the same. By using any feasible traditional algorithm to roughly compare the image differences between the two interface images, one or more regions of image difference can be identified. For example, various development frameworks generally provide tools for rough image difference comparison; these tools can be directly called. For instance, the C / C++ image processing library OpenCV and the image processing toolbox in Matlab both have corresponding methods that can be directly called. By calling these tools to perform a rough image difference comparison on the two interface images, one or more regions of image difference between them can be obtained.

[0074] Image difference regions can be represented using window coordinates, for example, using the coordinates of the top-left and bottom-right corners of the image difference region: [x0, y0, x1, y1], where [x0, y0] are the coordinates of the top-left corner and [x1, y1] are the coordinates of the bottom-right corner. Representing image difference regions using window coordinates can indicate the respective image differences in two interface images. These two image difference regions correspond in position and generally also in image content. Therefore, screenshots can be taken on each interface image corresponding to each image difference region to obtain the corresponding image content as a difference map. The two difference maps obtained for each image difference region corresponding to the two interface images can be constructed into a pair of difference maps, thus obtaining one or more pairs of difference maps corresponding to the number of image difference regions.

[0075] Step S1200: Perform sliding matching on the first difference map belonging to the first interface map in the same pair of difference maps, and on the second interface map to which the second difference map belongs, to determine the similarity between the graphic elements corresponding to each sliding match in the second interface map and the first difference map.

[0076] After determining one or more pairs of difference maps based on two interface maps, the difference details can be detected for each pair of difference maps. In this application, the detection of difference details in the same pair of difference maps can be implemented by sliding matching.

[0077] In one embodiment, one difference map in a pair of difference maps is first identified as the first difference map, and the other as the second difference map. Correspondingly, the interface map to which the first difference map belongs is designated as the first interface map, and the interface map to which the second difference map belongs is designated as the second interface map. The first difference map can be a difference map with richer image texture; therefore, when using the first difference map as a reference, it is easier to discover image differences by leveraging the rich image texture. The method for distinguishing the first and second difference maps within the same pair of difference maps can be either based on human visual recognition and user-defined settings, or it can be based on statistical techniques, such as variance comparison.

[0078] Once the first and second difference maps are determined, an image matching region can be defined within the second interface map to which the second difference map belongs. This image matching region is the area containing the image content of the second difference map; for example, it could be the entire second interface map. Then, the first difference map is slidably matched within this image matching region.

[0079] Slide matching method, such as Figure 2 As shown, in the image matching region, each primitive of the same size as the first difference map is determined. These primitives may have overlapping areas. The similarity between each primitive and the first difference map is calculated iteratively. In this way, each primitive can obtain a similarity corresponding to the first difference map.

[0080] Step S1300: When the similarity of each graphic element in the second interface image is lower than the preset similarity threshold, the image difference region corresponding to the same pair of difference images is determined as the corresponding layout difference region between the two interface images.

[0081] To determine whether there is content in the second interface image that is highly similar to the first difference image, the similarity of each element in the second interface image can be judged. For this purpose, a similarity threshold is preset to measure the threshold at which an element corresponds to a sufficient difference with the first difference image. Then, using this similarity threshold, it is checked whether the similarity of each element is lower than the similarity threshold. When the similarity of all elements is lower than the similarity threshold, it can be confirmed that the image difference area corresponding to the first difference image is the layout difference area corresponding to the first interface image and the second interface image.

[0082] In one embodiment, to improve decision-making efficiency, the maximum similarity among the similarities corresponding to each graphic element can be obtained. Then, the maximum similarity is compared with a similarity threshold. When the maximum similarity is lower than the similarity threshold, it can be confirmed that the image difference region corresponding to the first difference map belongs to the layout difference region. The similarity threshold is usually selected from a higher value within the similarity range to represent a high degree of similarity. For example, when the similarity dimension is within the range of [0,1], the similarity threshold can be set to any fixed value within the sub-range of [0.97,0.99], such as 0.98. Thus, if the similarity between each graphic element and the first difference map is all lower than 0.98, it indicates that no graphic element highly consistent with the first difference map can be found in the second interface map. In other words, although the graphic element corresponding to the second difference map in the second interface map will theoretically have a high similarity to the first difference map, sufficient differences still exist. Furthermore, in other words, no image content highly consistent with the first difference image was found in the second interface image. Therefore, the image content of the first difference image did not appear in any other location in the second interface image except for the corresponding image difference area. The difference between the second difference image and the first difference image is not caused by image displacement.

[0083] Therefore, it is not difficult to understand that if a graphic element with a similarity higher than the similarity threshold can be found within the image matching area of ​​the second interface image, it often means that this graphic element is the content corresponding to the first difference image. The difference between the first difference image and the second difference image is a "pseudo-difference", which may be due to the displacement effect caused by the adjustment of page element size. In this case, the first difference image and the second difference image do not essentially constitute the same pair of difference images, so the corresponding image difference area of ​​this pair of difference images is not determined as the layout difference area.

[0084] Step S1400: Mark the layout difference areas in the image content of the two interface images.

[0085] After determining the layout difference area for each pair of difference images using the above process, the two interface images can be labeled accordingly based on this layout difference area. As mentioned earlier, the layout difference area is essentially the identified image difference area, which can be represented by window coordinates. Therefore, based on the window coordinates of the layout difference area, two corresponding image contents in the first and second interface images can be determined. Labeling within the corresponding image content area using windows or masks effectively marks the corresponding layout difference area.

[0086] To facilitate reading, the two interface images can be combined side-by-side into a single image file.

[0087] In real-world scenarios, page layout issues such as misaligned elements, missing content, inconsistent colors, and text misalignment can all lead to layout discrepancies. The method described in this application can accurately identify these issues. Figure 3 , Figure 4 , Figure 5 Examples of each case are given below, with brief explanations:

[0088] Figure 3 This mainly shows the situation where page elements are misaligned. In the two interface images, the background image in the lower left corner of the area of ​​layout difference indicated by the larger rectangle in the left interface image (corresponding to the text "Two Line") is higher than the background image in the corresponding position in the right interface image, which is a misalignment that can be directly identified.

[0089] Figure 5 This mainly shows situations where page content is missing or colors are inconsistent. In the two interface images on the left and right, the three rectangles at the bottom indicating the price information serve as windows to indicate the differences between the various page layouts. By comparing the positions of the rectangles in the two interface images, it can be seen that there are differences in price data, font color, and the presence or absence of content between the two interface images. All of these are marked.

[0090] Figure 5 This mainly shows the situation where the text on the page is displayed in a single line. See the rectangle in the upper right corner of each of the two interface images. The difference between the two interface images in the layout difference area is mainly because the text in the right interface image is displayed in two lines, while the corresponding text in the left interface image is still displayed in a single line. This difference is indicated by the corresponding window.

[0091] comprehensive Figures 3 to 5It is not difficult to see that although the reasons for layout differences are different, the method of this application can accurately locate the area of ​​layout difference to the position that caused the layout change, thereby accurately indicating the page element that caused the layout change and quickly locating the root cause of the layout difference.

[0092] Therefore, based on the above embodiments, it can be understood that this application has multiple technical advantages, including but not limited to:

[0093] First, this application identifies the image difference regions from two interface images. It then obtains pairs of difference images corresponding to each image difference region from the two interface images. For the same pair of difference images, using the first difference image as a template, it performs a sliding match with the second interface image to which the second difference image belongs. This determines the similarity of the corresponding graphic elements in the second interface image for each sliding match. When the similarity of each graphic element determined relative to the first difference image is lower than a preset similarity threshold, it can be determined that the image difference region indicated by the pair of difference images is the layout difference region between the two interface images. This allows for the marking of the corresponding layout difference regions in the two interface images, achieving efficient screening of image content differences between the two interface images. This can replace manual screening and improve page development efficiency.

[0094] Secondly, since this application performs a wider range of image matching based on the first difference map in the second interface map where the second difference map is located, and uses the condition that the similarity of each corresponding graphic element is lower than the preset similarity threshold as the condition for identifying the layout difference area, it can effectively ignore the false differences caused by the displacement of page elements between the two interface maps, and determine the real layout difference to the image position that caused the displacement of other elements. This makes it convenient for users to accurately locate the corresponding page element that caused the layout change by marking the layout difference area, and realize the root cause analysis and location of layout anomalies, which can further improve the efficiency of layout change investigation.

[0095] Furthermore, this application can achieve interface difference detection based on sliding matching, without the need to introduce deep learning models or implement corresponding training, which can save labeling and training costs and has extremely low computational load. When performing interface difference detection on websites and applications of large sites such as e-commerce platforms, it can yield significant economies of scale.

[0096] Based on any embodiment of the method in this application, please refer to Figure 6 Detecting at least one image difference region between two interface images, including:

[0097] Step S1110: Obtain two interface images corresponding to different design versions of the same page;

[0098] The two interface images used to perform image difference detection are usually from the same page. There are several ways to obtain these two interface images, such as the method exported by the developer as shown in the previous example. In addition, there are other automated methods that can further improve efficiency, such as the various workarounds detailed below:

[0099] In one embodiment, the method of this application provides a corresponding tool plugin in the development framework of a website or application, and provides a corresponding interface image export control. After the developer completes the editing of a certain page, he can touch the interface image export control to trigger the page interface image export instruction. The tool plugin responds to the instruction and performs the function of exporting the interface image of the page, which can be saved locally and loaded into memory for layout difference detection with another pre-stored interface image.

[0100] In another embodiment, the process of obtaining two interface diagrams corresponding to different design versions of the same page includes:

[0101] Step S1111: In response to the page load event of the target page, cache the previous version of the target page;

[0102] When the method of this application is implemented as a tool plugin for a development framework, it can run in the background when the development framework starts, thus monitoring the operation process of pages in the website or application under development. In this case, when a user loads a target page, the tool plugin responds to the page load event triggered by the loading operation by first caching the unmodified version of the target page, that is, its original version in its unmodified state, which is usually the code file and resource data of the target page.

[0103] Step S1112: In response to the edit event of the target page, cache the modified version of the target page in the latest modified state;

[0104] When a user performs any editing operation on the target page, a corresponding editing event is triggered. The tool plugin responds to each editing event and further caches the modified version of the target page in its latest modified state. Similarly, this includes the target page's code files and its resource data.

[0105] Step S1113: In response to the page interface diagram export command of the target page, generate interface diagrams corresponding to the two states of the target page before and after modification based on the version before modification and the version after modification.

[0106] As can be seen, the tool plugin actually tracks the entire editing process of the target page, caching the code and resource data of the page before and after modification. Therefore, when the user triggers the command to export the page interface image, the tool plugin responds to the command, parses the code and resource data of the two versions before and after modification to generate the corresponding interface, and exports two interface images accordingly, thus directly obtaining two interface images of the same page before and after modification.

[0107] By using the above method, two interface images of the target page before and after modification are generated by listening to the editing process of the target page. This improves the level of automation and eliminates the need for users to manually provide interface images. By performing image difference detection on these two interface images, it is possible to quickly find out whether the page editing operation has caused differences in the images before and after the editing, making it convenient for users to quickly find problems and deal with them in a timely manner.

[0108] In another embodiment, which is a further modification of the previous embodiment, the process of obtaining two interface diagrams corresponding to different design versions of the same page includes:

[0109] Step S1111': In response to the page load event of the target page, generate the interface image corresponding to the previous version of the target page and save it to the cache area;

[0110] Step S1112': In response to the edit event of the target page, generate an interface image corresponding to the modified version of the target page in the latest modified state; save it to the cache area;

[0111] Step S1113': In response to the page interface diagram export command of the target page, call the two interface diagrams before and after modification for image difference detection.

[0112] As can be seen, in this embodiment, the tool plugin directly generates the corresponding interface image in the background in response to each event. When image difference detection needs to be performed at the end, the corresponding two interface images can be called. Through background processing, the efficiency of importing interface images for subsequent image difference detection can be further improved.

[0113] As can be seen from the above description of various implementation methods, the two interface diagrams can be manually imported by the user or automatically generated by the tool plugin. The manual import method helps to open up flexibility to the user, while the automatic generation method can further improve efficiency, each with its own advantages.

[0114] Step S1120: Determine the locations of image differences between the two interface images;

[0115] To easily identify differences between two interface images, various feasible interfaces provided by the development framework can be used, or the corresponding functions can be implemented manually. By comparing the two interface images at the pixel level, the differences can be found. These differences can be represented as a feature map, which provides corresponding indication information for each differing pixel, thus reflecting the image differences between the two interfaces. In another embodiment, the two interface images can be considered as two image frames, and the frame difference information between the two image frames can be calculated to obtain a feature map, which can also be used to represent the various image differences between the two interface images.

[0116] Each image difference region is typically a set of pixels in the feature map whose indicator information values ​​are similar and whose locations are connected. Therefore, in one embodiment, the image difference regions can be determined based on the indicator information of each pixel in the feature map representing the image difference region, according to the similarity of the indicator information values ​​and the connectivity of the pixels.

[0117] Step S1130: Expand the corresponding rectangular box based on each of the image difference regions to indicate the corresponding image difference areas;

[0118] There may be multiple image differences between two interface images. These differences are described based on pixels and are not regular shapes. To facilitate obtaining regular-shaped difference images later and to include more image content to improve the accuracy of difference recognition, the area of ​​the image region occupied by the image difference can be expanded to determine the corresponding rectangle. This rectangle should contain the range of all pixels of the corresponding image difference. Thus, the image range corresponding to this rectangle is the corresponding image difference region.

[0119] In one embodiment, the boundaries can be set based on the pixels at the outermost edges of the four sides of the screen within the coverage area of ​​each image difference region, forming four sides to constitute a corresponding rectangle. This rectangle is then represented as window coordinates to obtain the corresponding image difference region.

[0120] Step S1140: Take screenshots of each rectangle from the two interface images to obtain the corresponding pairs of difference images for each rectangle.

[0121] After identifying the bounding boxes corresponding to each image difference region, for each image difference region, using its bounding box as positional information, screenshots are taken at the corresponding positions in the two interface images, resulting in two screenshots. These screenshots serve as a pair of difference images corresponding to that image difference region. Following this method, multiple pairs of difference images can be obtained from the two interface images for multiple image difference regions. As can be understood from the above embodiment, regardless of how the two interface images are obtained, a rough identification of image differences is necessary. This involves first identifying the image differences between the two interface images using a pixel-level image comparison method, then expanding the image difference regions based on these differences, and finally extracting the corresponding difference image pairs from the two interface images based on these regions. This decomposes a large task into smaller tasks, improving task processing efficiency while effectively reducing computational load and technical development difficulty.

[0122] Based on any embodiment of the method in this application, please refer to Figure 7 Determine the locations of image differences between the two interface images, including:

[0123] Step S1121: Calculate the structural similarity between the two interface diagrams to obtain a structural similarity diagram;

[0124] In this embodiment, the Structural Similarity (SSIM) algorithm can be directly applied to determine the similarity between two interface images, resulting in a corresponding structural similarity map, which can also be considered a feature map, representing the structural differences between the two interface images. When applying the structural similarity algorithm to determine the structural similarity map, the corresponding interface provided by tools such as OpenCV can be directly called. In short, the resulting structural similarity map also represents the degree of difference between pixels in the two interface images, and therefore essentially belongs to the feature map mentioned earlier in this application.

[0125] Step S1122: Perform morphological erosion on the structural similarity map to obtain an erosion correction map;

[0126] Based on the structural similarity map, pre-encapsulated interfaces can be called to perform morphological erosion on the map. Morphological erosion is a process that eliminates boundary points of connected components, causing the boundaries to shrink inward, which can correct the boundaries of areas of image difference. After morphological erosion, the boundary points of connected components in the structural similarity map are eliminated, and the boundaries of connected components are appropriately shrunk, resulting in an eroded and corrected map.

[0127] Step S1123: Perform a binarization operation on the erosion correction map based on a preset edge threshold to obtain a binarized image;

[0128] To make the edges of continuous parts of the image in the erosion correction image visible, a relatively high edge threshold can be applied to the erosion correction image to perform a binarization operation to obtain a binarized image.

[0129] The edge threshold value should be set to match the high value range of edge pixels in the erosion correction image. For example, when the value range of each pixel in the erosion correction image is [0, 255], the edge threshold can be set to 230, which accounts for a relatively high proportion of the maximum value. In this way, when performing binarization, pixels with values ​​higher than this edge threshold will be represented as 1 in the binarized image, and otherwise as 0. This will allow the edge features corresponding to the differences between the two interface images to be displayed in the binarized image.

[0130] Step S1124: Find at least one closed edge contour in the binarized image to determine the image difference locations accordingly.

[0131] Based on a binarized image, by applying the corresponding interface of edge finding function provided by OpenCV, one or more closed edge contours can be found, that is, the edge contours closed by each pixel with a value of 1. It is easy to understand that each closed edge contour actually indicates a part of the image difference, thereby determining one or more parts of the image difference between two interface images.

[0132] As can be understood from the above embodiments, the process of determining the differences between two interface images can be achieved with a low computational load using only ordinary algorithms, making it more efficient and accurate.

[0133] Based on any embodiment of the method of this application, corresponding screenshots are taken from the two interface images according to each rectangle, including:

[0134] Step S1131: Calculate the image area occupied by each rectangle, delete rectangles whose image area is lower than a preset area threshold, and retain valid rectangles to indicate each valid image difference region.

[0135] As mentioned earlier, when expanding the corresponding rectangles based on the image differences between two interface images to indicate the corresponding image difference areas, screenshots can be taken from both interface images to determine each pair of difference images based on the corresponding rectangles of each image difference area. In practical applications, if the image area of ​​the image difference area is too small, such processing is not very meaningful. Therefore, this embodiment presets an area threshold. First, the image area of ​​each rectangle, which is also the image difference area, is calculated. The image area of ​​each rectangle is compared with the area threshold. If it is lower than the area threshold, the rectangle is deleted, indicating that the layout difference recognition of the corresponding image difference area of ​​the deleted rectangle will no longer be performed. Otherwise, if it is not lower than the area threshold, the corresponding rectangle is retained as a valid rectangle to indicate that its corresponding image difference area is valid, and layout difference recognition will continue for the valid image difference areas.

[0136] The range of the area threshold can be determined according to the actual situation. For example, it can be any value between [0, 60], such as 50. That is to say, when the image area (the product of the length and width pixels) corresponding to a rectangle is less than 50, the rectangle needs to be deleted.

[0137] Step S1132: Take screenshots from the two interface images according to each valid rectangle to obtain the corresponding pairs of difference images for each rectangle.

[0138] Once the valid rectangles are determined, screenshots can be taken from the two interface images based on each valid rectangle. This results in a pair of difference images for each valid rectangle and multiple pairs of difference images for all valid rectangles.

[0139] As can be easily understood from the above embodiments, before taking a screenshot based on the corresponding rectangles of the image difference regions, the rectangles can be filtered to ignore some small rectangles, thereby reducing unnecessary interference caused by some insignificant difference information or calculation errors, and further improving the efficiency of layout difference recognition.

[0140] Based on any embodiment of the method in this application, please refer to Figure 8 The first difference map, belonging to the first interface map in the same pair of difference maps, is subjected to sliding matching in the second interface map to which the second difference map belongs. The similarity between the corresponding graphic elements in the second interface map for each sliding match and the first difference map is determined, including:

[0141] Step S1210: Determine the image with the larger texture deviation in the same pair of difference images as the first difference image, and the one with the smaller deviation as the second difference image;

[0142] As mentioned earlier, for each pair of difference maps, the one with the larger image texture deviation can be selected as the first difference map, and the one with the smaller image texture deviation can be selected as the second difference map.

[0143] Step S1220: In the second interface map to which the second difference map belongs, expand the image matching area to a larger preset area based on the image difference area occupied by the second difference map in the second interface map.

[0144] Although the first difference image can be used as a template to determine whether there are any identical graphic elements in the second interface image within the scope of the first difference image, so as to determine whether the difference between the first difference image and the second difference image is only caused by the displacement of the image content of the first difference image in the second interface image, considering that the displacement range of page layout is generally not large in actual scenarios, it is not necessary to expand the image matching area to be searched to the entire frame of the second interface image before performing such a search. Instead, the size of the image matching area to be searched can be appropriately constrained.

[0145] Therefore, in one embodiment, the image matching region to be searched can be set to one to three times the corresponding image difference region of the second difference map, for example, preferably two times. Then, based on the window coordinates of the image difference region corresponding to the position of the second difference map in the second interface map, the boundaries of each side of the entire image difference region are expanded. Specifically, the window coordinates of the image matching region can be obtained based on the window coordinates (x0, y0, x1, y1). According to the new window coordinates, the corresponding region image can be determined in the second interface map, and subsequent sliding matching can be performed within the range of this region image.

[0146] Step S1230: Perform sliding matching on the first difference map within the region image corresponding to the image matching region, and determine the similarity between the primitives corresponding to each sliding match in the region image and the first difference map.

[0147] At this point, it is possible Figure 2 The principle example shown involves performing sliding matching on the first difference map in the obtained region image, and calculating the similarity between the corresponding primitives of each sliding match and the first difference map.

[0148] As can be understood from the above embodiments, appropriately setting the range of the image matching area corresponding to the sliding match can not only effectively meet the scenario requirements of accurate positioning of page interface image layout differences, but also reduce the amount of computation for accurate positioning of layout differences and improve the efficiency of layout difference positioning.

[0149] Based on any embodiment of the method in this application, please refer to Figure 9Before determining that the image with the larger texture deviation in the same pair of difference maps is the first difference map and the one with the smaller deviation is the second difference map, the following steps are taken:

[0150] Step S1211: Calculate the variance of the two difference plots in the same pair of difference plots respectively;

[0151] To quickly determine the image texture differences in the same pair of difference maps, statistical principles can be used to calculate the variance of each of the two difference maps in the same pair. The method of calculating the variance is known to those skilled in the art. The variance can measure the degree of difference between pixels in each difference map. When the variance of a difference map is larger, the degree of difference between its pixels is higher, and therefore its image texture is richer. Otherwise, the degree of difference between pixels is lower, and the image texture is correspondingly simpler.

[0152] Step S1212: When the maximum variance of the two difference maps is less than the preset variance threshold, the pair of difference maps will no longer be processed, and the next pair of difference maps will be called to continue iterative processing.

[0153] After determining the variances of both difference maps, their variances are comparable since they are at the same scale. A preset variance threshold can be used to test the variances of the two difference maps. When the maximum variance is less than the preset threshold, that is, when the variances of both difference maps are below the threshold, it indicates that each difference map is basically solid color or blank content. In this case, no further processing is needed, and the process can be repeated to return to step S1200 for iterative processing. Alternatively, if there are no more unidentified difference maps, the identification process for the two interface maps can be terminated.

[0154] The variance threshold set here should be able to effectively measure the solid color content and blank content that need to be excluded. Therefore, it is usually set to a minimum value below 20, such as 10. Those skilled in the art can flexibly adapt and implement it accordingly.

[0155] Step S1213: When the maximum variance in the two difference maps exceeds the variance threshold, the difference map corresponding to the maximum variance is set as the first difference map, and the other difference map is set as the second difference map.

[0156] Conversely, when the difference map with the largest variance exceeds a preset variance threshold, it indicates that at least one difference map does not belong to a solid color area or a blank area. Therefore, the difference map corresponding to the largest variance can be set as the first difference map, and the other difference map can be set as the second difference map. Obviously, the variance of the first difference map is larger than that of the second difference map, so the image texture of the first difference map is richer.

[0157] The above embodiments determine the richness of image texture between two difference maps in the same pair of difference maps based on variance, and use variance thresholding to filter out difference maps whose image content is solid color or blank. This can not only eliminate interference factors, but also reduce the amount of computation. More importantly, it can efficiently determine the first difference map with richer image texture in the same pair of difference maps. In this way, the rich texture can be used as a template to more effectively implement layout change detection.

[0158] Based on any embodiment of the method in this application, please refer to Figure 10 The first difference map is used for sliding matching within the region image corresponding to the image matching region, and the similarity between the primitives corresponding to each sliding match in the region image and the first difference map is determined, including:

[0159] Step S1231: Perform binarization on the first difference map based on a preset mask threshold to obtain a reference mask;

[0160] Before performing sliding matching, a reference mask can be generated based on the first difference map with richer image texture. The reference mask is used to indicate that only the pixels of interest in the reference mask are considered during the similarity calculation. This reduces the amount of computation and makes the similarity calculation results more accurate.

[0161] To generate a reference mask, the first difference image can be converted into a grayscale image. Then, a lower value is determined within the range of pixel values ​​in the grayscale image as the mask threshold. For each pixel in the grayscale image, its pixel value is compared with the mask threshold. When the pixel value is higher than the mask threshold, the corresponding pixel in the reference mask is represented as a value of 1, otherwise it is represented as a value of 0. In this way, colored pixels can be indicated.

[0162] Step S1232: Based on the image size setting of the first difference map and the sliding window of the same size and the preset step size, start sliding matching within the image range of the region;

[0163] To implement sliding matching, a sliding window and a preset step size for controlling the sliding window's movement need to be set. For this, the length and width of the image difference region corresponding to the first difference map can be calculated and used as the length and width of the sliding window. A preset step size can be flexibly set to the distance of multiple pixels, such as any distance between 1 and 10 pixels. After setting the sliding window and preset step size, sliding matching can be initiated on the region image already determined from the second interface image. Within the region image, the sliding window iterates continuously according to the preset step size. Each slide essentially applies the sliding window to determine a primitive at the corresponding position within the region image. By iterating continuously across the entire region image, multiple primitives are effectively traversed.

[0164] Step S1233: Determine the corresponding primitive each time the sliding window slides, and calculate the similarity between the primitive and the first difference map using the reference mask as the weight.

[0165] During the iterative sliding of the sliding window, each iteration determines the primitive corresponding to that sliding. Here, the reference mask can be associated as a weight to calculate the similarity between the primitive and the first difference map. In one embodiment, since the primitive, the first difference map, and the reference mask can all be represented by matrices, the calculation of the similarity between the primitive and the first difference map can be equivalent to the element-wise product, i.e., the Hadamard product. Finally, the average of the element-wise products of each pixel can be used as the similarity between the primitive and the first difference map.

[0166] As can be understood from the above embodiments, during the sliding matching process, the similarity of the corresponding graphic elements in each sliding motion can be quickly determined. The first difference map is slid-matched within a region of the image, and this region of the image itself already contains the second difference map corresponding to the first difference map. Therefore, by sliding matching, the displacement of the image content of the first difference map within the region of the image can be accurately identified. Combined with the subsequent judgment of similarity, the specific page element position that caused the layout change can be accurately identified. The calculation of the similarity of each graphic element by sliding matching itself can be implemented as an iterative mechanism, so its implementation difficulty and computational load are relatively low, it consumes less memory, and it runs more efficiently.

[0167] Please see Figure 11According to one aspect of this application, a layout difference detection device includes a difference detection module 1100, a sliding matching module 1200, a difference precision module 1300, and a difference annotation module 1400. The difference detection module 1100 is configured to detect pairs of difference images corresponding to at least one image difference region between two interface images. The sliding matching module 1200 is configured to perform sliding matching on a first difference image belonging to a first interface image within the same pair of difference images, and on a second interface image to which a second difference image belongs, to determine the similarity between the corresponding graphic element in the second interface image and the first difference image for each sliding match. The difference precision module 1300 is configured to determine that the image difference region corresponding to the same pair of difference images is the corresponding layout difference region between the two interface images when the similarity of each graphic element in the second interface image is lower than a preset similarity threshold. The difference annotation module 1400 is configured to mark the layout difference region in the image content of the two interface images.

[0168] Based on any embodiment of the device in this application, the difference detection module 1100 includes: a source image acquisition submodule, configured to acquire two interface images corresponding to different design versions of the same page; a difference recognition submodule, configured to determine the image difference parts between the two interface images; a part regularization submodule, configured to expand a corresponding rectangle based on each image difference part to indicate the corresponding image difference area; and a screenshot processing submodule, configured to take screenshots from the two interface images according to each rectangle to obtain each pair of difference images corresponding to each rectangle.

[0169] Based on any embodiment of the device in this application, the difference recognition submodule includes: a structure analysis unit configured to calculate the structural similarity between the two interface images to obtain a structure similarity image; an edge correction unit configured to perform a morphological erosion operation on the structure similarity image to obtain an erosion correction image; an edge enhancement unit configured to perform a binarization operation on the erosion correction image based on a preset edge threshold to obtain a binarized image; and a contour search unit configured to search for at least one closed edge contour in the binarized image to determine the image difference locations accordingly.

[0170] Based on any embodiment of the device in this application, the screenshot processing submodule includes: a region selection unit, configured to calculate the image area occupied by each rectangle, delete rectangles whose image area is lower than a preset area threshold, and retain valid rectangles to indicate each valid image difference region; and a region screenshot unit, configured to take screenshots from the two interface images according to each valid rectangle to obtain each pair of difference images corresponding to each rectangle.

[0171] Based on any embodiment of the device in this application, the sliding matching module 1200 includes: an image region sub-module, configured to determine the image texture deviation with the larger deviation in the same pair of difference images as the first difference image and the smaller deviation as the second difference image; a region expansion sub-module, configured to expand an image matching region with a larger preset area based on the image difference region occupied by the second difference image in the second interface image to which the second difference image belongs; and a region matching sub-module, configured to perform sliding matching of the first difference image within the region image corresponding to the image matching region, and determine the similarity between the primitives corresponding to each sliding match in the region image and the first difference image.

[0172] Based on any embodiment of the device in this application, the image region molecular module includes: a variance calculation unit, configured to calculate the variance of the two difference maps in the same pair of difference maps respectively; a normal processing unit, configured to stop further processing of the same pair of difference maps when the maximum variance in the two difference maps is less than a preset variance threshold, and instead call the next pair of difference maps for iterative processing; and an exception processing unit, configured to set the difference map corresponding to the maximum variance as the first difference map and the other difference map as the second difference map when the maximum variance in the two difference maps exceeds the variance threshold.

[0173] Based on any embodiment of the device in this application, the region matching submodule includes: a mask generation unit configured to perform a binarization operation on a first difference image based on a preset mask threshold to obtain a reference mask; a sliding execution unit configured to set a sliding window of the same size as the first difference image and a preset step size according to the image size of the first difference image, and initiate sliding matching within the region image range; and a similarity calculation unit configured to determine the corresponding primitive each time the sliding window slides, and calculate the similarity between the primitive and the first difference image using the reference mask as a weight.

[0174] Another embodiment of this application provides a layout difference detection device. For example... Figure 12 The diagram shows the internal structure of a layout difference detection device. This device includes a processor, a computer-readable storage medium, a memory, and a network interface connected via a system bus. The computer-readable, non-volatile storage medium stores an operating system, a database, and computer-readable instructions. The database stores information sequences, and when executed by the processor, these computer-readable instructions enable the processor to implement a layout difference detection method.

[0175] The processor of the layout difference detection device provides computing and control capabilities, supporting the operation of the entire device. The device's memory can store computer-readable instructions, which, when executed by the processor, cause the processor to perform the layout difference detection method of this application. The network interface of the device is used for communication with a terminal.

[0176] Those skilled in the art will understand that Figure 12 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the layout difference detection device to which the present application is applied. A specific layout difference detection device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0177] In this embodiment, the processor is used to execute... Figure 11 The specific functions of each module are described, and the memory stores the program code and various data required to execute the above modules or sub-modules. The network interface is used to realize data transmission between user terminals or servers. The non-volatile readable storage medium in this embodiment stores the program code and data required to execute all modules in the layout difference detection device of this application, and the server can call the server's program code and data to execute the functions of all modules.

[0178] This application also provides a non-volatile readable storage medium storing computer-readable instructions, which, when executed by one or more processors, cause the one or more processors to perform the steps of the layout difference detection method of any embodiment of this application.

[0179] This application also provides a computer program product, including a computer program / instructions that, when executed by one or more processors, implement the steps of the method described in any embodiment of this application.

[0180] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The aforementioned storage medium can be a computer-readable storage medium such as a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM).

[0181] In summary, this application can not only determine the layout difference area between two interface images, but also pinpoint the specific page element causing the layout change between the two interface images. It has low computational complexity and can improve the efficiency of interface troubleshooting during the page development process.

Claims

1. A method of detecting layout differences, characterized by, include: Detecting at least one image difference region between two interface images and corresponding pairs of difference images includes: determining each image difference location between the two interface images, determining a corresponding image difference region based on each image difference location, and obtaining each pair of difference images by taking corresponding screenshots from the two interface images according to each image difference region. The first difference map belonging to the first interface map in the same pair of difference maps is subjected to sliding matching in the second interface map to which the second difference map belongs, including: taking the image texture deviation with larger deviation in the same pair of difference maps as the first difference map and the image texture deviation with smaller deviation as the second difference map, expanding the image matching area based on the image difference area occupied by the second difference map in the second interface map to which the second difference map belongs, and performing sliding matching to determine the similarity between the graphic element corresponding to each sliding matching in the second interface map and the first difference map. When the similarity of each graphic element in the second interface image is lower than the preset similarity threshold, the image difference region corresponding to the same pair of difference images is determined as the corresponding layout difference region between the two interface images. The areas of difference in layout are marked in the image content of the two interface diagrams.

2. The layout difference detection method according to claim 1, characterized in that, Detecting at least one image difference region between two interface images, corresponding to each pair of difference images, including: Obtain two interface images corresponding to different design versions of the same page; Identify the areas of image difference between the two interface images; For each of the image differences, a corresponding rectangular box is extended to indicate the corresponding image difference region; Based on the corresponding screenshots of each rectangle from the two interface images, we obtain the pairs of difference images corresponding to each rectangle.

3. The layout difference detection method according to claim 2, characterized in that, Determining the locations of image differences between the two interface images includes: Calculate the structural similarity between the two interface diagrams to obtain a structural similarity diagram; A morphological erosion operation is performed on the structural similarity map to obtain an erosion correction map; Binarization is performed on the erosion correction map based on a preset edge threshold to obtain a binarized image; In the binarized image, at least one closed edge contour is found to determine the image difference locations accordingly.

4. The layout difference detection method according to claim 2, characterized by, Based on the corresponding screenshots taken from the two interface images according to each rectangle, including: Calculate the image area occupied by each rectangle, delete rectangles whose image area is lower than a preset area threshold, and retain valid rectangles to indicate each valid image difference region. Based on the corresponding screenshots of each valid rectangle from the two interface images, we obtain the pairs of difference images corresponding to each rectangle.

5. The layout difference detection method according to any one of claims 1 to 4, characterized by, The first difference map, belonging to the first interface map in the same pair of difference maps, is subjected to sliding matching in the second interface map to which the second difference map belongs. The similarity between the corresponding graphic elements in the second interface map for each sliding match and the first difference map is determined, including: In the same pair of difference maps, the image with the larger texture deviation is designated as the first difference map, and the one with the smaller deviation is designated as the second difference map. In the second interface map to which the second difference map belongs, an image matching region with a larger preset area is expanded based on the image difference region occupied by the second difference map in the second interface map. The first difference map is used to perform sliding matching within the region image corresponding to the image matching region, and the similarity between the primitives corresponding to each sliding match in the region image and the first difference map is determined.

6. The method for detecting layout differences according to claim 5, characterized in that, Before determining that the image with the larger texture deviation in the same pair of difference maps is the first difference map and the one with the smaller deviation is the second difference map, the following steps are taken: Calculate the variance of the two difference plots in the same pair of difference plots respectively; When the maximum variance between the two difference graphs is less than a preset variance threshold, the pair of difference graphs will no longer be processed, and the next pair of difference graphs will be called for iterative processing. When the maximum variance in the two difference maps exceeds the variance threshold, the difference map corresponding to the maximum variance is set as the first difference map, and the other difference map is set as the second difference map.

7. The method for detecting layout differences according to claim 5, characterized in that, The first difference map is used for sliding matching within the region image corresponding to the image matching region. The similarity between the primitives corresponding to each sliding match in the region image and the first difference map is determined, including: Binarization is performed on the first difference map based on a preset mask threshold to obtain a reference mask; Based on the image size setting of the first difference map, the sliding window of equal size, and the preset step size, sliding matching is initiated within the image area of ​​the region. Each time the sliding window slides, its corresponding primitive is determined, and the similarity between the primitive and the first difference map is calculated using the reference mask as the weight.

8. A page layout difference detection device, characterized in that, include: The difference detection module is configured to detect at least one image difference region between two interface images and corresponding pairs of difference images, including: determining the image difference locations between the two interface images, determining the corresponding image difference region based on each image difference location, and obtaining each pair of difference images by taking corresponding screenshots from the two interface images according to each image difference region. The sliding matching module is configured to perform sliding matching on the first difference map belonging to the first interface map in the same pair of difference maps, and on the second interface map to which the second difference map belongs. This includes: taking the image texture deviation with the larger deviation in the same pair of difference maps as the first difference map and the smaller deviation as the second difference map; expanding the image matching area based on the image difference area occupied by the second difference map in the second interface map to perform sliding matching; and determining the similarity between the graphic elements corresponding to each sliding matching in the second interface map and the first difference map. The difference detection module is configured to determine the image difference region corresponding to the same pair of difference images as the corresponding layout difference region between the two interface images when the similarity of each graphic element in the second interface image is lower than a preset similarity threshold. The difference annotation module is configured to mark the differences in the layout of the two interface images.

9. A layout difference detection device, comprising a central processing unit and a memory, characterized in that, The central processing unit is used to invoke and run a computer program stored in the memory to perform the steps of the method as described in any one of claims 1 to 7.

10. A non-volatile readable storage medium, characterized in that, It stores a computer program in the form of computer-readable instructions, which, when invoked by a computer, performs the steps of the method as described in any one of claims 1 to 7.

Images