A webpage rendering integrity detection method and device based on webpage snapshot and benchmark image comparison, equipment and medium

Abstract

This application discloses a method, apparatus, device, and medium for detecting the rendering integrity of a webpage based on the comparison of a webpage snapshot and a reference image, relating to the field of webpage testing and quality monitoring. The method includes: acquiring a visually rendered image, webpage metadata, and a webpage identifier corresponding to the current webpage sent by a client; acquiring a corresponding reference image from a preset reference image library based on the webpage identifier; aligning the spatial positions of regions corresponding to the same webpage element in the visually rendered image and the reference image; performing difference detection on the aligned image regions using a preset difference detection method to obtain target difference regions; determining the target webpage element corresponding to the target difference region based on the webpage metadata; generating a detection report including the target webpage element identifier, anomaly type, and difference information; and evaluating the rendering integrity of the current webpage in the current client environment based on the detection report. This application achieves precise element-level localization of rendering anomalies.

Description

Technical Field

[0001] This application relates to the field of web page testing and quality monitoring, and in particular to a method, apparatus, device and medium for detecting the integrity of web page rendering based on the comparison of web page snapshots and benchmark images. Background Technology

[0002] With the rapid development of internet technology and the significant increase in the complexity of web applications, modern web pages have evolved from simple information display pages into comprehensive applications containing rich interactivity, dynamic content, and complex visual designs. At the same time, user access environments have become highly fragmented, encompassing different brands and versions of operating systems, various browser engines, and a wide range of screen sizes and resolutions from mobile devices to desktop computers. Against this backdrop, ensuring that web pages render correctly and consistently across various client environments has become a key challenge for guaranteeing user experience and business reliability.

[0003] Currently, automated detection and monitoring of webpage rendering quality primarily rely on assertion checks of the Document Object Model (DOM) structure, validation of Cascading Style Sheets (CSS) style properties, and monitoring of network resource loading status. However, these methods struggle to comprehensively reflect visual differences in the rendered results and cannot directly detect issues such as structural misalignment, missing elements, or visual occlusion caused by client-side rendering. Furthermore, some existing solutions attempt to employ image comparison techniques. While these involve visual results, the image comparison typically occurs in a controlled, idealized server environment, failing to reflect the actual rendering on real user terminals. Moreover, such solutions often stop at pixel-level difference output, generating large-area difference maps but lacking the ability to accurately map pixel differences back to specific page elements. This makes it difficult for developers to quickly pinpoint the root cause of problems, resulting in low debugging efficiency.

[0004] In summary, existing technologies cannot comprehensively and accurately detect webpage rendering anomalies in real, ever-changing client environments, and cannot precisely locate the specific page elements causing visual differences when webpage rendering anomalies are detected. Summary of the Invention

[0005] In view of this, the purpose of this application is to provide a method, apparatus, device, and medium for detecting webpage rendering integrity based on the comparison of webpage snapshots and reference images. By capturing a real rendering snapshot on the client and comparing it with a reference image on the server, it is possible to accurately detect and locate rendering anomalies of webpages in real-world usage environments. The specific solution is as follows: Firstly, this application provides a method for detecting the integrity of webpage rendering based on comparing webpage snapshots with reference images, including: The system obtains the visual rendering image, webpage metadata, and webpage identifier corresponding to the current webpage sent by the client. The visual rendering image is a static image captured by the client that represents the final visual image of the webpage presented on the user's device at the target time. The webpage metadata is structured information collected synchronously by the client to describe the spatial attributes, structural attributes, and client environment information of the webpage elements. The reference image corresponding to the current webpage is obtained from a preset reference image library based on the webpage identifier; the reference image library is used to store reference images of the same webpage that meet preset standards for correct rendering results in different client environments; The regions corresponding to the same web page element in the visually rendered image and the reference image are spatially aligned, and a preset difference detection method is used to perform difference detection in the aligned image regions to obtain the target difference region. Based on the webpage metadata, the target webpage element corresponding to the target difference region is determined, and a detection report including the target webpage element identifier, anomaly type, and difference information is generated to evaluate the rendering integrity of the current webpage in the current client environment based on the detection report.

[0006] Optionally, before aligning the spatial positions of regions corresponding to the same webpage element in the visually rendered image and the reference image, the method further includes: Preprocessing operations are performed on the visually rendered image and the reference image to align the spatial positions of regions corresponding to the same web page element in the preprocessed visually rendered image and the reference image; the preprocessing operations include color normalization, scaling, viewport cropping, and noise reduction; The color normalization is used to adjust the color space, brightness, or contrast of the visually rendered image and the reference image to reduce image color deviation; the scaling is used to adjust the visually rendered image and the reference image to the same size; the window cropping is used to remove irrelevant background outside the web page rendering area in the visually rendered image and the reference image; and the denoising is used to remove noise interference in the visually rendered image and the reference image.

[0007] Optionally, aligning the spatial positions of regions corresponding to the same webpage element in the visually rendered image and the reference image includes: Feature point detection is performed on the visually rendered image and the reference image respectively to extract feature points that can characterize the local salient structure of the image; Feature point matching is performed between the feature points corresponding to the visually rendered image and the feature points corresponding to the reference image to obtain successfully matched feature point pairs. Based on the feature point pairs, target transformation parameters are determined, and corresponding spatial transformations are performed on the visually rendered image or the reference image according to the target transformation parameters to complete image alignment.

[0008] Optionally, aligning the spatial positions of regions corresponding to the same webpage element in the visually rendered image and the reference image includes: Based on the webpage element boundary information in the webpage metadata, the left and right boundaries of the main content area corresponding to the visual rendering image and the left and right boundaries of the main content area corresponding to the reference image are determined; the main content area is an effective content area that carries key functions and provides a focus object for rendering integrity detection. The center line of the main content area corresponding to the visually rendered image in the horizontal direction and the center line of the main content area corresponding to the reference image in the horizontal direction are determined by using the corresponding left and right boundaries. Based on the centerline and the preset extension radius, construct rectangular cropping windows corresponding to the visual rendering image and the reference image respectively, and use the corresponding rectangular cropping windows to crop the visual rendering image and the reference image respectively to obtain the first sub-image and the second sub-image; The first sub-image and the second sub-image are used as a spatially aligned image pair for difference detection.

[0009] Optionally, the step of performing difference detection in the aligned image region using a preset difference detection method includes: A pixel- or region-based difference detection method is used to perform difference detection on the first sub-image and the second sub-image to obtain a first difference detection result. And / or, based on the webpage element boundary information in the webpage metadata, determine the bounding box information of the same webpage element in the first sub-image and the second sub-image, compare the bounding box information corresponding to the first sub-image and the bounding box information corresponding to the second sub-image, and obtain the second difference detection result; And / or, perform optical character recognition on the predefined text regions in the first sub-image and the second sub-image respectively to obtain the corresponding text content, compare the text content of the first sub-image and the text content of the second sub-image to obtain the third difference detection result.

[0010] Optionally, the webpage rendering integrity detection method based on comparing webpage snapshots with reference images further includes: The obtained difference detection results are input into the pre-trained classifier or compared with the corresponding preset threshold. The confidence level of the difference detection results is determined based on the classifier output or the threshold comparison result. Based on the difference detection results with a confidence level greater than the preset confidence threshold, the target difference region that meets the preset error rendering result standard is determined.

[0011] Optionally, the detection report is a visual detection report, which also includes a difference heatmap and a severity score; the difference heatmap is used to mark the location and degree of difference of the target difference area with different color intensities, and the severity score is determined based on the size of the target difference area, the functional priority of the corresponding web page element and the anomaly type, and is used to determine the priority of anomaly repair.

[0012] Secondly, this application provides a webpage rendering integrity detection device based on comparing webpage snapshots with reference images, comprising: The data acquisition module is used to acquire the visual rendering image, webpage metadata, and webpage identifier corresponding to the current webpage sent by the client. The visual rendering image is a static image captured by the client that represents the final visual image of the webpage presented on the user's device at the target time. The webpage metadata is structured information collected synchronously by the client to describe the spatial attributes, structural attributes, and client environment information of the webpage elements. The image acquisition module is used to acquire a reference image corresponding to the current webpage from a preset reference image library based on the webpage identifier; the reference image library is used to store reference images of the same webpage that meet preset correct rendering result standards under different client environments. The detection module is used to spatially align regions corresponding to the same web page element in the visually rendered image and the reference image, and to perform difference detection in the aligned image region using a preset difference detection method to obtain the target difference region. The report generation module is used to determine the target webpage element corresponding to the target difference region based on the webpage metadata, and generate a detection report including the target webpage element identifier, anomaly type and difference information, so as to evaluate the rendering integrity of the current webpage in the current client environment based on the detection report.

[0013] Thirdly, this application provides an electronic device, comprising: Memory, used to store computer programs; A processor is used to execute the computer program to implement the aforementioned webpage rendering integrity detection method based on comparison of webpage snapshots and reference images.

[0014] Fourthly, this application provides a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the aforementioned method for detecting the integrity of web page rendering based on the comparison of web page snapshots and reference images.

[0015] In this application, the following steps are taken: First, a visually rendered image, webpage metadata, and webpage identifier corresponding to the current webpage sent by the client are obtained. The visually rendered image is a static image captured by the client, representing the final visual display of the webpage on the user's device at a target time. The webpage metadata is structured information synchronously collected by the client, describing the spatial attributes, structural attributes, and client environment information of webpage elements. Second, a reference image corresponding to the current webpage is obtained from a preset reference image library based on the webpage identifier. The reference image library stores reference images of the same webpage in different client environments that meet preset standards for correct rendering results. Third, the regions corresponding to the same webpage element in the visually rendered image and the reference image are spatially aligned. A preset difference detection method is used to perform difference detection on the aligned image regions to obtain target difference regions. Based on the webpage metadata, the target webpage element corresponding to the target difference region is determined, and a detection report including the target webpage element identifier, anomaly type, and difference information is generated to evaluate the rendering integrity of the current webpage in the current client environment based on the detection report. As can be seen from the above, on the one hand, this application does not rely on server simulation environments or underlying data inference, but directly obtains the visually rendered image captured by the client. This image is the final visual image of the webpage presented on the user's real device, completely restoring the actual rendering effect at the target moment, thus fundamentally solving the blind spots of traditional methods at the visual detection level. On the other hand, the webpage metadata collected synchronously by the client contains the spatial and structural attributes of webpage elements. This structured information establishes the correspondence between image pixel regions and webpage elements, providing a core basis for subsequent localization. Secondly, the difference detection does not stop at the overall pixel-level comparison, but first aligns the corresponding regions of the same webpage element, and then detects differences in a targeted manner. The target difference region obtained is a local region bound to a specific webpage element, rather than a large area of ​​meaningless difference map, ensuring that the detection results focus on real rendering anomalies, and achieving comprehensive and accurate anomaly detection. Finally, based on the element spatial and structural attributes stored in the webpage metadata, the target difference area is directly mapped to the corresponding target webpage element, generating a detection report containing element identifiers, anomaly types, and difference information. This allows developers to quickly pinpoint the specific elements causing the visual differences and the essence of the problem without manual investigation, completely solving the problem of "knowing there is an anomaly but not knowing why" in traditional solutions, and greatly improving debugging efficiency. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0017] Figure 1 This is a flowchart of a webpage rendering integrity detection method based on comparing webpage snapshots with a reference image, as disclosed in this application. Figure 2 This is a schematic diagram of the structure of a webpage rendering integrity detection device based on the comparison of webpage snapshots and reference images disclosed in this application; Figure 3 This is a schematic diagram of the structure of an electronic device disclosed in this application. Detailed Implementation

[0018] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0019] Existing technologies cannot comprehensively and accurately detect webpage rendering anomalies in real, ever-changing client environments, nor can they precisely locate the specific page elements causing visual differences when rendering anomalies are detected. Therefore, this application provides a webpage rendering integrity detection method based on comparing webpage snapshots with benchmark images. By capturing a real rendering snapshot on the client and comparing it with a benchmark image on the server, it can accurately detect and locate rendering anomalies in real-world usage environments, ensuring the rendering integrity and accuracy of webpages in various client environments.

[0020] See Figure 1 As shown in the figure, this application discloses a method for detecting the integrity of webpage rendering based on the comparison of webpage snapshots and reference images, including: Step S11: Obtain the visual rendering image, webpage metadata, and webpage identifier corresponding to the current webpage sent by the client; the visual rendering image is a static image captured by the client that represents the final visual image of the webpage presented on the user's device at the target time; the webpage metadata is structured information collected synchronously by the client to describe the spatial attributes, structural attributes, and client environment information of the webpage elements.

[0021] In this embodiment, when the page finishes loading or reaches a preset timed trigger condition, the client automatically captures a visual rendering snapshot of the current page, i.e., a static image. The content of this snapshot accurately reflects the final rendering result within the current viewport and supports taking screenshots of the entire page or a visible area. For example, a visual rendering snapshot can be obtained by calling native browser APIs (Application Programming Interfaces), such as canvas.drawWindow, html2canvas, or the screenshot API, or by using WebView on a mobile device. Simultaneously, the client synchronously collects structured metadata related to the page, including but not limited to the page URL (Uniform Resource Locator), timestamp, browser type and version, operating system and device information, viewport size, resource loading list, DOM (Document Object Model) tree snapshot, and the location information of key elements, such as the element's DOM path, CSS computed styles, and their bounding box coordinates within the viewport.

[0022] The captured visual rendering snapshots and corresponding metadata can then be uploaded to the server via secure channels such as HTTPS (Hypertext Transfer Protocol Secure). The upload request should include a page identifier that uniquely identifies the current page. This page identifier can be a combination of the page URL and status identifier, a session identifier, or a template identifier, ensuring that the server can accurately associate the corresponding baseline image and related information.

[0023] Step S12: Obtain the reference image corresponding to the current webpage from the preset reference image library according to the webpage identifier; the reference image library is used to store reference images of the same webpage in different client environments that meet the preset correct rendering result standard.

[0024] Based on the received page identifier, the server retrieves and obtains the corresponding benchmark image representing a normal rendering result from a pre-built benchmark image library. This benchmark image library can be maintained in multiple versions based on dimensions such as browser type, screen resolution, and user agent to ensure matching accuracy. Simultaneously, the server can also obtain element-level annotation information associated with the benchmark image.

[0025] Step S13: Spatial alignment of regions corresponding to the same web page element in the visually rendered image and the reference image; difference detection is performed on the aligned image regions using a preset difference detection method to obtain the target difference region.

[0026] In this embodiment, before aligning the spatial positions of regions corresponding to the same webpage element in the visually rendered image and the reference image, the method may further include: performing preprocessing operations on the visually rendered image and the reference image to align the spatial positions of regions corresponding to the same webpage element in the preprocessed visually rendered image and the reference image; the preprocessing operations include, but are not limited to, color normalization, scaling, window cropping, and denoising; wherein, color normalization is used to adjust the color space, brightness, or contrast of the visually rendered image and the reference image to reduce image color deviation, scaling is used to adjust the visually rendered image and the reference image to the same size, window cropping is used to remove irrelevant background outside the webpage rendering area in the visually rendered image and the reference image, and denoising is used to remove noise interference in the visually rendered image and the reference image.

[0027] Then, spatial alignment can be performed on the regions corresponding to the same web page element in the visually rendered image and the reference image.

[0028] In one specific implementation, feature point detection is first performed on both the visually rendered image and the reference image to extract feature points that characterize the local salient structures of the images. Then, feature point matching is performed between the feature points in the visually rendered image and the reference image to obtain successfully matched feature point pairs. Finally, target transformation parameters are determined based on the feature point pairs, and corresponding spatial transformations are performed on the visually rendered image or the reference image according to these parameters to complete image alignment. For example, methods such as ORB (Oriented Fast and Rotated BRIEF), SIFT (Scale Invariant Feature Transform), and SURF (Speeded Up Robust Features) can be used to achieve image alignment.

[0029] In another specific implementation, firstly, based on the boundary information of web page elements in the web page metadata, the left and right boundaries of the main content area corresponding to the visually rendered image, and the left and right boundaries of the main content area corresponding to the reference image, can be determined; whereby the main content area is the effective content area that carries key functions and provides the focus object for rendering integrity detection. Then, the corresponding left and right boundaries can be used to determine the horizontal centerline of the main content area corresponding to the visually rendered image, and the horizontal centerline of the main content area corresponding to the reference image. Next, rectangular cropping windows corresponding to the visually rendered image and the reference image can be constructed according to the centerline and a preset extension radius, and the visually rendered image and the reference image can be cropped using the corresponding rectangular cropping windows to obtain a first sub-image and a second sub-image. Finally, the first sub-image and the second sub-image can be used as an image pair with completed spatial alignment for difference detection. For example, a vertical cropping strategy based on the center of the main visual container can be used to achieve accurate reference alignment of the main content area of ​​the page across device environments. Assume the visually rendered image uploaded by the client is a two-dimensional image. Where H is the height, W is the width, and C is the number of color channels. The server obtains the corresponding reference image as follows: .

[0030] Suppose the main content area consists of a set of visible, detectable elements, denoted as: ; Each element A known bounding box in the client coordinate system That is, the coordinates of the top left corner of the element. The width of the element ,high All such information is uploaded to the server along with metadata.

[0031] Further extract the overall horizontal boundary of the main content area, defined as follows: Left Content Boundary: ; Right Content Boundary: ; Therefore, the total width of the main content area can be obtained as follows: ; Calculate the x-coordinate of the center point of the main content area: ; On image I, draw a straight line along the Y-axis through the center point, which is the vertical reference alignment line, defined as: ; Then, perform symmetrical horizontal trimming around this vertical alignment line. Set the trimming radius to... (Typical value) Then the left and right boundaries of the final cropping window are: ; Similarly, on the reference image G, according to its corresponding element set Extract the same logical structure and locate its ideal center point. Then perform a proportional cropping operation to obtain aligned sub-images. and This facilitates subsequent pixel-level difference analysis or feature matching.

[0032] Furthermore, a pre-defined difference detection method can be used to perform difference detection on the aligned image region. Specifically, this can include: performing difference detection on the first and second sub-images using pixel-based or region-based difference detection methods to obtain a first difference detection result. For example, pixel-based or region-based difference analysis can be performed using methods such as pixel difference, absolute difference, and structural similarity index measure (SSIM), and morphological operations can be combined to integrate discrete difference points to obtain a complete difference region; and / or, based on the boundary information of web page elements in the web page metadata, the bounding box information corresponding to the same web page element in the first and second sub-images can be determined, and the bounding box information corresponding to the first and second sub-images can be compared to obtain a second difference detection result. For example, the boundary of an element uploaded by the client can be compared with the boundary of the corresponding element in the reference image to determine whether the element exists, is occluded, has a positional offset exceeding a threshold, or has an abnormal size or style; and / or, optical character recognition can be performed on predefined text regions in the first and second sub-images to obtain the corresponding text content, and the text content of the first and second sub-images can be compared to obtain a third difference detection result. For example, optical character recognition (OCR) is performed on key text areas of the page to extract text content, and then a consistency comparison is performed with the baseline text to check for any missing or misaligned text.

[0033] The part on difference detection using pixel- or region-based difference detection methods can specifically include: First, calculate the client snapshot image. Compared with server baseline image The absolute difference is used to obtain the initial difference map: ; Then, the initial difference map is segmented based on a preset threshold to generate a binary difference mask, which initially identifies regions with pixel differences. Next, morphological closing operations are used to connect and merge discrete difference blocks in the mask, thereby identifying continuous and meaningful anomalous regions. Based on this, the quantified "overall difference rate" is calculated by statistically analyzing the ratio of the total number of true pixels in the difference mask to the total area of ​​the image. Furthermore, this method also supports the use of SSIM as an auxiliary evaluation metric. This metric, by comprehensively comparing brightness, contrast, and structural information, can more accurately assess the similarity between two images in a way that aligns with human visual perception. The SSIM value range is [0,1], with values ​​closer to 1 indicating higher similarity.

[0034] The difference detection results obtained above can then be input into a pre-trained classifier or compared with a corresponding preset threshold. The confidence level of the difference detection result is determined based on the classifier output or the threshold comparison result. Target difference regions that meet the preset error rendering result criteria are identified based on difference detection results with a confidence level greater than the preset confidence threshold. In this way, a comprehensive judgment can be made on whether the difference belongs to a significant rendering error based on the difference detection results obtained from multi-dimensional detection, ensuring the comprehensiveness and accuracy of difference detection.

[0035] Step S14: Based on the webpage metadata, determine the target webpage element corresponding to the target difference region, and generate a detection report including the target webpage element identifier, anomaly type, and difference information, so as to evaluate the rendering integrity of the current webpage in the current client environment according to the detection report.

[0036] The detection report in this embodiment is a visual detection report, which may also include a difference heatmap, severity score, and element-level anomaly list, such as element identifier, anomaly type, difference measure, and screenshot comparison with the baseline image. The difference heatmap is used to mark the location and degree of difference of the target difference area with different color intensities. The severity score is determined based on the size of the target difference area, the functional priority of the corresponding web page element, and the anomaly type, and is used to determine the priority of anomaly repair.

[0037] It should be noted that while generating the test report, the snapshot historical data of this test is stored simultaneously, and the above test report is accurately fed back to the developers or testers in the form of a visual report or real-time alarm, providing comprehensive support for them to quickly troubleshoot and fix web page rendering problems.

[0038] As shown above, this embodiment achieves accurate monitoring of webpage visual integrity in a real user environment by uploading the actual page rendering snapshot captured by the client and structured metadata to the server and intelligently comparing it with a pre-stored benchmark image. This method combines image alignment technology with element-level metadata mapping, effectively reducing the false alarm rate of traditional visual regression testing and precisely locating detected pixel-level differences to specific DOM elements, thus quickly pinpointing the root cause of rendering problems. Simultaneously, the system supports selecting benchmark images based on multiple dimensions such as browser and resolution, and allows for de-identification of sensitive information on the client side, balancing accuracy, practicality, and security. This significantly improves the efficiency of webpage quality monitoring and problem investigation, meeting the growing quality monitoring needs of web applications.

[0039] See Figure 2 As shown in the figure, this application also discloses a webpage rendering integrity detection device based on the comparison of webpage snapshots and reference images, including: The data acquisition module 11 is used to acquire the visual rendering image, webpage metadata and webpage identifier corresponding to the current webpage sent by the client; the visual rendering image is a static image captured by the client that represents the final visual image of the webpage presented on the user device at the target time; the webpage metadata is structured information collected synchronously by the client that describes the spatial attributes, structural attributes and client environment information of the webpage elements in the webpage. The image acquisition module 12 is used to acquire a reference image corresponding to the current webpage from a preset reference image library according to the webpage identifier; the reference image library is used to store reference images of the same webpage that meet the preset correct rendering result standard in different client environments; The detection module 13 is used to spatially align the regions corresponding to the same web page element in the visually rendered image and the reference image, and to perform difference detection in the aligned image region using a preset difference detection method to obtain the target difference region. The report generation module 14 is used to determine the target webpage element corresponding to the target difference area based on the webpage metadata, and generate a detection report including the target webpage element identifier, anomaly type and difference information, so as to evaluate the rendering integrity of the current webpage in the current client environment based on the detection report.

[0040] In some specific embodiments, the detection module 13 further includes: The position alignment unit is used to perform preprocessing operations on the visually rendered image and the reference image to align the spatial positions of regions corresponding to the same web page element in the preprocessed visually rendered image and the reference image; the preprocessing operations include color normalization, scaling, window cropping and noise reduction. The color normalization is used to adjust the color space, brightness, or contrast of the visually rendered image and the reference image to reduce image color deviation; the scaling is used to adjust the visually rendered image and the reference image to the same size; the window cropping is used to remove irrelevant background outside the web page rendering area in the visually rendered image and the reference image; and the denoising is used to remove noise interference in the visually rendered image and the reference image.

[0041] In some specific embodiments, the detection module 13 includes: The feature point extraction unit is used to perform feature point detection on the visually rendered image and the reference image respectively, so as to extract feature points that can characterize the local salient structure of the image; The feature point matching unit is used to perform feature point matching between the feature points corresponding to the visually rendered image and the feature points corresponding to the reference image, so as to obtain successfully matched feature point pairs. An image alignment unit is used to determine target transformation parameters based on the feature point pairs, and to perform corresponding spatial transformations on the visually rendered image or the reference image according to the target transformation parameters to complete image alignment.

[0042] In some specific embodiments, the detection module 13 includes: The boundary determination unit is used to determine the left and right boundaries of the main content area corresponding to the visual rendering image, and the left and right boundaries of the main content area corresponding to the reference image, based on the boundary information of web page elements in the web page metadata; the main content area is an effective content area that carries key functions and provides a focus object for rendering integrity detection; The centerline determination unit is used to determine the centerline of the main content area corresponding to the visual rendering image in the horizontal direction, and the centerline of the main content area corresponding to the reference image in the horizontal direction, using the corresponding left and right boundaries. The image cropping unit is used to construct rectangular cropping windows corresponding to the visual rendering image and the reference image respectively based on the center line and the preset extension radius, and to crop the visual rendering image and the reference image respectively using the corresponding rectangular cropping windows to obtain the first sub-image and the second sub-image. An image pair acquisition unit is used to treat the first sub-image and the second sub-image as an image pair with completed spatial alignment, and to perform difference detection based on the image pair.

[0043] In some specific embodiments, the detection module 13 includes: The first detection result acquisition unit is used to perform difference detection on the first sub-image and the second sub-image using a pixel- or region-based difference detection method to obtain a first difference detection result. The second detection result acquisition unit is used to determine the bounding box information of the same web page element in the first sub-image and the second sub-image based on the web page element boundary information in the web page metadata, and compare the bounding box information corresponding to the first sub-image and the bounding box information corresponding to the second sub-image to obtain the second difference detection result. The third detection result acquisition unit is used to perform optical character recognition on the predefined text regions in the first sub-image and the second sub-image respectively to obtain the corresponding text content, and compare the text content of the first sub-image and the text content of the second sub-image to obtain the third difference detection result.

[0044] In some specific embodiments, the webpage rendering integrity detection device based on comparing webpage snapshots with reference images further includes: The confidence determination unit is used to input the obtained difference detection result into the pre-trained classifier or compare it with the corresponding preset threshold, and determine the confidence level corresponding to the difference detection result based on the classifier output result or the threshold comparison result. The region determination unit is used to determine the target difference region that meets the preset error rendering result standard based on the difference detection results with a confidence level greater than a preset confidence threshold.

[0045] In some specific implementations, the detection report is a visual detection report, which also includes a difference heatmap and a severity score; the difference heatmap is used to mark the location and degree of difference of the target difference area with different color intensities, and the severity score is determined based on the size of the target difference area, the functional priority of the corresponding web page element and the anomaly type, and is used to determine the priority of anomaly repair.

[0046] Furthermore, embodiments of this application also disclose an electronic device, Figure 3 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application.

[0047] Figure 3This is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of this application. Specifically, the electronic device 20 may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the webpage rendering integrity detection method based on comparing webpage snapshots with reference images disclosed in any of the foregoing embodiments. Alternatively, the electronic device 20 in this embodiment may specifically be a computer.

[0048] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0049] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0050] The operating system 221 is used to manage and control the various hardware devices on the electronic device 20 and the computer program 222, which may be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of performing the webpage rendering integrity detection method based on webpage snapshot and benchmark image comparison executed by the electronic device 20 as disclosed in any of the foregoing embodiments, the computer program 222 may further include computer programs capable of performing other specific tasks.

[0051] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned disclosed method for detecting the integrity of webpage rendering based on the comparison of webpage snapshots and reference images. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0052] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0053] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0054] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0055] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0056] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for detecting the integrity of webpage rendering based on comparison between webpage snapshots and reference images, characterized in that, include: Obtain the visual rendering image, webpage metadata, and webpage identifier corresponding to the current webpage sent by the client; The visual rendering image is a static image captured by the client, representing the final visual image of the webpage presented on the user's device at the target time. The webpage metadata is structured information collected synchronously by the client to describe the spatial attributes, structural attributes, and client environment information of the webpage elements. Based on the webpage identifier, a reference image corresponding to the current webpage is obtained from a preset reference image library; The reference image library is used to store reference images of the same webpage that meet the preset standard for correct rendering results in different client environments. The regions corresponding to the same web page element in the visually rendered image and the reference image are spatially aligned, and a preset difference detection method is used to perform difference detection in the aligned image regions to obtain the target difference region. Based on the webpage metadata, the target webpage element corresponding to the target difference region is determined, and a detection report including the target webpage element identifier, anomaly type, and difference information is generated to evaluate the rendering integrity of the current webpage in the current client environment based on the detection report.

2. The webpage rendering integrity detection method based on comparison of webpage snapshots and reference images according to claim 1, characterized in that, Before aligning the spatial positions of regions corresponding to the same webpage element in the visually rendered image and the reference image, the method further includes: Preprocessing operations are performed on the visually rendered image and the reference image to align the spatial positions of regions corresponding to the same web page element in the preprocessed visually rendered image and the reference image; the preprocessing operations include color normalization, scaling, viewport cropping, and noise reduction; The color normalization is used to adjust the color space, brightness, or contrast of the visually rendered image and the reference image to reduce image color deviation; the scaling is used to adjust the visually rendered image and the reference image to the same size; the window cropping is used to remove irrelevant background outside the web page rendering area in the visually rendered image and the reference image; and the denoising is used to remove noise interference in the visually rendered image and the reference image.

3. The webpage rendering integrity detection method based on comparison of webpage snapshots and reference images according to claim 1, characterized in that, The step of aligning the spatial positions of regions corresponding to the same webpage element in the visually rendered image and the reference image includes: Feature point detection is performed on the visually rendered image and the reference image respectively to extract feature points that can characterize the local salient structure of the image; Feature point matching is performed between the feature points corresponding to the visually rendered image and the feature points corresponding to the reference image to obtain successfully matched feature point pairs. Based on the feature point pairs, target transformation parameters are determined, and corresponding spatial transformations are performed on the visually rendered image or the reference image according to the target transformation parameters to complete image alignment.

4. The webpage rendering integrity detection method based on comparison of webpage snapshots and reference images according to claim 1, characterized in that, The step of aligning the spatial positions of regions corresponding to the same webpage element in the visually rendered image and the reference image includes: Based on the webpage element boundary information in the webpage metadata, the left and right boundaries of the main content area corresponding to the visual rendering image and the left and right boundaries of the main content area corresponding to the reference image are determined; the main content area is an effective content area that carries key functions and provides a focus object for rendering integrity detection. The center line of the main content area corresponding to the visually rendered image in the horizontal direction and the center line of the main content area corresponding to the reference image in the horizontal direction are determined by using the corresponding left and right boundaries. Based on the centerline and the preset extension radius, construct rectangular cropping windows corresponding to the visual rendering image and the reference image respectively, and use the corresponding rectangular cropping windows to crop the visual rendering image and the reference image respectively to obtain the first sub-image and the second sub-image; The first sub-image and the second sub-image are used as a spatially aligned image pair for difference detection.

5. The webpage rendering integrity detection method based on comparison of webpage snapshots and reference images according to claim 4, characterized in that, The step of performing difference detection in the aligned image region using a preset difference detection method includes: A pixel- or region-based difference detection method is used to perform difference detection on the first sub-image and the second sub-image to obtain a first difference detection result. And / or, based on the webpage element boundary information in the webpage metadata, determine the bounding box information of the same webpage element in the first sub-image and the second sub-image, compare the bounding box information corresponding to the first sub-image and the bounding box information corresponding to the second sub-image, and obtain the second difference detection result; And / or, perform optical character recognition on the predefined text regions in the first sub-image and the second sub-image respectively to obtain the corresponding text content, compare the text content of the first sub-image and the text content of the second sub-image to obtain the third difference detection result.

6. The webpage rendering integrity detection method based on comparison of webpage snapshots and reference images according to claim 5, characterized in that, Also includes: The obtained difference detection results are input into the pre-trained classifier or compared with the corresponding preset threshold. The confidence level of the difference detection results is determined based on the classifier output or the threshold comparison result. Based on the difference detection results with a confidence level greater than the preset confidence threshold, the target difference region that meets the preset error rendering result standard is determined.

7. The webpage rendering integrity detection method based on comparison of webpage snapshots and reference images according to any one of claims 1 to 6, characterized in that, The detection report is a visual detection report, which also includes a difference heatmap and a severity score. The difference heatmap is used to mark the location and degree of difference of the target difference area with different color intensities. The severity score is determined based on the size of the target difference area, the functional priority of the corresponding web page element, and the anomaly type, and is used to determine the priority of anomaly repair.

8. A webpage rendering integrity detection device based on comparison of webpage snapshots and reference images, characterized in that, include: The data acquisition module is used to acquire the visual rendering image, webpage metadata, and webpage identifier corresponding to the current webpage sent by the client; The visual rendering image is a static image captured by the client, representing the final visual image of the webpage presented on the user's device at the target time. The webpage metadata is structured information collected synchronously by the client to describe the spatial attributes, structural attributes, and client environment information of the webpage elements. The image acquisition module is used to acquire a reference image corresponding to the current webpage from a preset reference image library based on the webpage identifier; The reference image library is used to store reference images of the same webpage that meet the preset standard for correct rendering results in different client environments. The detection module is used to spatially align regions corresponding to the same web page element in the visually rendered image and the reference image, and to perform difference detection in the aligned image region using a preset difference detection method to obtain the target difference region. The report generation module is used to determine the target webpage element corresponding to the target difference region based on the webpage metadata, and generate a detection report including the target webpage element identifier, anomaly type and difference information, so as to evaluate the rendering integrity of the current webpage in the current client environment based on the detection report.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the webpage rendering integrity detection method based on comparison of webpage snapshots and reference images as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, Used to store a computer program, which, when executed by a processor, implements the webpage rendering integrity detection method based on comparison of webpage snapshots and reference images as described in any one of claims 1 to 7.

Images