A method and apparatus for detecting interface fidelity

By combining semantic understanding and visual comparison technologies, similarity and difference are extracted from interface elements to generate quantitative restoration evaluation results, which solves the problem of low efficiency of manual detection in existing technologies and achieves efficient and accurate interface restoration detection.

CN123114928APending Publication Date: 2026-11-17BEIJING JINGDONG YUANSHENG TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202510592180.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-05-08
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing methods for detecting interface fidelity rely on manual identification and comparison, which cannot be quantified, resulting in low detection efficiency and poor objectivity and accuracy of results.

Method used

By combining semantic understanding and visual comparison technologies, interface elements are extracted from the interface to be detected and the target design interface. Through semantic similarity and visual similarity calculations, a restoration evaluation result based on a quantitative index system is generated.

Benefits of technology

It enables intelligent recognition and objective evaluation of changes in interface elements, improving detection efficiency and accuracy, and provides a visual difference map to assist in problem localization.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123114928A_ABST
    Figure CN123114928A_ABST
Patent Text Reader

Abstract

This invention discloses a method and apparatus for detecting interface fidelity, relating to the field of computer technology. One specific implementation of the method includes: extracting a first interface element and a second interface element from an interface to be detected and a target design interface, respectively; determining the similarity and difference between the first interface element and the second interface element; wherein the similarity includes semantic similarity and visual similarity; and generating a fidelity evaluation result of the interface to be detected based on a pre-configured quantitative index system, similarity, and difference. This embodiment can combine semantic understanding and visual comparison techniques to determine the similarity and difference between interface elements, intelligently identify changes in interface elements, and generate an objective fidelity evaluation result based on a pre-configured quantitative index system, thereby improving detection efficiency and the accuracy of the detection results.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a method and apparatus for detecting interface fidelity. Background Technology

[0002] In software development and design, interface fidelity refers to the degree of similarity between the actual product and the design draft. High interface fidelity ensures a consistent user experience and a positive brand image. However, existing interface fidelity testing methods primarily rely on manual identification and comparison. This approach is subjective, lacks quantification, and is relatively time-consuming, resulting in low testing efficiency and poor objectivity and accuracy of the results. Summary of the Invention

[0003] In view of this, embodiments of the present invention provide an interface fidelity detection method and apparatus, which can combine semantic understanding and visual comparison techniques to determine the similarity and difference between interface elements, intelligently identify changes in interface elements, and generate objective fidelity evaluation results based on a pre-configured quantitative index system, thereby improving detection efficiency and the accuracy of detection results.

[0004] To achieve the above objectives, according to one aspect of the present invention, an interface fidelity detection method is provided, comprising:

[0005] Extract the first interface elements and the second interface elements from the interface to be tested and the target design interface, respectively;

[0006] Determine the similarity and difference between the elements of the first interface and the elements of the second interface; whereby the similarity includes semantic similarity and visual similarity;

[0007] The restoration evaluation results of the interface to be detected are generated based on a pre-configured quantitative indicator system, similarity, and difference.

[0008] Optionally, determining the similarity between the first interface element and the second interface element includes:

[0009] Semantic recognition processing is performed on the first interface element and the second interface element respectively to obtain the attribute information corresponding to the first interface element and the second interface element; the attribute information includes semantic features and visual features.

[0010] The semantic similarity between the first interface element and the second interface element is determined based on semantic features, and the visual similarity between the first interface element and the second interface element is determined based on visual features.

[0011] Optionally, determining the degree of difference between the first interface element and the second interface element includes:

[0012] Obtain the first image region data corresponding to the first interface element and the second image region data corresponding to the second interface element;

[0013] The first image region data and the second image region data are decoded respectively to obtain the first pixel data and the second pixel data;

[0014] The first pixel data and the second pixel data are respectively converted into the first pixel set and the second pixel set in the luminance and chrominance color space;

[0015] The first pixel set and the second pixel set are compared pixel by pixel based on the perceptual color difference algorithm, and the degree of difference between the first interface element and the second interface element is determined based on the comparison results.

[0016] Optionally, a pixel-by-pixel comparison is performed between the first pixel set and the second pixel set based on a perceptual color difference algorithm, and the degree of difference between the first interface elements and the second interface elements is determined based on the comparison results, including:

[0017] Calculate the perceived color difference value between the first pixel in the first pixel set and the second pixel in the second pixel set. In response to the perceived color difference value exceeding a preset color difference threshold, mark the first pixel as a difference pixel.

[0018] Determine the quantity and distribution information of the differing pixels;

[0019] The degree of difference between the first interface element and the second interface element is determined based on the quantity and distribution information of the difference pixels.

[0020] Optionally, before determining the quantity and distribution information of the differing pixels, the method further includes:

[0021] Obtain the set of neighboring pixels of the differing pixel; the set of neighboring pixels includes multiple adjacent pixels of the differing pixel;

[0022] Determine the brightness difference between the differing pixel and each of its neighboring pixels, and determine the brightness distribution characteristics based on the brightness difference;

[0023] In response to the brightness distribution characteristics satisfying the preset anti-aliasing feature conditions, the difference pixels are changed to non-difference pixels.

[0024] Optionally, based on a pre-configured quantitative indicator system, similarity, and difference, a restoration evaluation result of the interface to be detected is generated, including:

[0025] Determine the corresponding evaluation weights of similarity and difference in the quantitative indicator system;

[0026] Based on similarity, difference, and evaluation weight, the result of the reproduction degree assessment of the interface to be detected relative to the target design interface is determined.

[0027] Optionally, after generating the restoration evaluation results of the interface to be detected based on the pre-configured quantitative indicator system, similarity, and difference, the process also includes:

[0028] An interface difference map with color annotations is generated based on the fidelity assessment results.

[0029] According to a second aspect of the present invention, an interface fidelity detection device is provided, comprising:

[0030] The extraction module is used to extract the first interface elements and the second interface elements from the interface to be detected and the target design interface, respectively.

[0031] The determination module is used to determine the similarity and difference between the first interface elements and the second interface elements; wherein, the similarity includes semantic similarity and visual similarity;

[0032] The generation module is used to generate the restoration evaluation results of the interface to be detected based on a pre-configured quantitative indicator system, similarity, and difference.

[0033] According to a third aspect of the present invention, an electronic device is provided, comprising:

[0034] One or more processors;

[0035] Memory, used to store one or more programs.

[0036] When one or more programs are executed by one or more processors, the one or more processors implement the methods of any of the above embodiments.

[0037] According to a fourth aspect of the present invention, a computer-readable medium is provided having a computer program stored thereon, which, when executed by a processor, implements the method of any of the above embodiments.

[0038] According to a fifth aspect of the present invention, a computer program product is provided, including a computer program that, when executed by a processor, implements the method of any of the above embodiments.

[0039] One embodiment of the above invention has the following advantages or beneficial effects: by extracting a first interface element and a second interface element from the interface to be detected and the target design interface respectively; determining the similarity and difference between the first interface element and the second interface element; wherein, the similarity includes semantic similarity and visual similarity; generating a restoration evaluation result of the interface to be detected based on a pre-configured quantitative index system, similarity, and difference; thereby, it is possible to combine semantic understanding and visual comparison technology to determine the similarity and difference between interface elements, intelligently identify changes in interface elements, and generate an objective restoration evaluation result based on a pre-configured quantitative index system, thereby improving detection efficiency and the accuracy of detection results.

[0040] The further effects of the aforementioned unconventional alternative methods will be explained below in conjunction with specific implementation methods. Attached Figure Description

[0041] The accompanying drawings are provided to better understand the invention and are not intended to unduly limit the scope of the invention. Wherein:

[0042] Figure 1 This is a schematic diagram of the main process of the interface fidelity detection method according to an embodiment of the present invention;

[0043] Figure 2 This is a schematic diagram of the main flow of an interface fidelity detection method according to a preferred embodiment of the present invention;

[0044] Figure 3 This is a schematic diagram of the main modules of the interface fidelity detection device according to an embodiment of the present invention;

[0045] Figure 4 This is an exemplary system architecture diagram in which embodiments of the present invention can be applied;

[0046] Figure 5 This is a schematic diagram of the structure of a computer system suitable for implementing terminal devices or servers of the present invention. Detailed Implementation

[0047] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of the present invention, including various details to aid understanding. These details should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the invention. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0048] It should be noted that the acquisition, storage, and application of personal information involved in the embodiments of the present invention comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0049] In software development and design, interface fidelity refers to the degree of similarity between the actual product and the design draft. High interface fidelity ensures a consistent user experience and a positive brand image. However, existing interface fidelity testing methods primarily rely on manual identification and comparison. This approach is subjective, lacks quantification, and is relatively time-consuming, resulting in low testing efficiency and poor objectivity and accuracy of the results.

[0050] In view of this, according to one aspect of the present invention, an interface fidelity detection method is provided.

[0051] Figure 1 This is a schematic diagram of the main flow of the interface fidelity detection method according to an embodiment of the present invention. Figure 1 As shown, the interface fidelity detection method according to an embodiment of the present invention includes the following steps S101 to S103.

[0052] Step S101: Extract the first interface element and the second interface element from the interface to be detected and the target design interface, respectively.

[0053] The interface to be tested refers to the user interface running in the actual system, such as the front-end interface view presented by a browser or application. The interface to be tested can be obtained through runtime screenshots or DOM (Document Object Model) rendering capture. The target design interface refers to the original design draft or prototype, such as a visual draft image output by a UI (User Interface) design tool, used to express the expected visual and interactive effects of the interface. The first interface element refers to the semantically meaningful functional areas extracted from the interface to be tested, such as buttons, input boxes, navigation bars, and card components, reflecting the element layout and style information in the actual running state of the interface. The second interface element is the interface component in the target design interface that corresponds to the first interface element in function and visually, reflecting the UI specifications and interaction intentions set in the design draft. Specifically, extracting the first and second interface elements can be achieved using a method based on DOM structure analysis and visual region mapping. This involves parsing the HTML (Hypertext Markup Language) structure of the interface to be detected, combining computed style and visual rendering position information, and mapping each visible node in the interface to be detected as a specific first interface element. Simultaneously, by combining preset layer annotations or auxiliary line data in the design draft, the design draft is structurally restored, and the corresponding layer regions are extracted as the second interface elements. Alternatively, an image processing and region detection method can be used. First, image segmentation and contour detection are performed on both the interface to be detected and the target design interface. Using low-level features such as color, boundaries, and texture, independent visual element regions in the image are identified. Then, template matching or layout alignment is used to establish visual correspondences, extracting the first and second interface elements.

[0054] Step S102: Determine the similarity and difference between the first interface element and the second interface element; wherein, the similarity includes semantic similarity and visual similarity.

[0055] Similarity refers to the degree of semantic and visual matching between a first interface element in the test interface and a second interface element in the target design interface. It measures the consistency of the two interface elements in terms of functional positioning and appearance. Similarity can include semantic similarity and visual similarity, etc.: Semantic similarity reflects the consistency of two interface elements in terms of functional purpose, control type, or behavior, such as buttons corresponding to buttons, input boxes corresponding to input boxes, etc.; visual similarity reflects the consistency of two interface elements in terms of visual details such as color, font, size, border, rounded corners, and shadows. Difference is a quantitative characterization of the inconsistent parts between two interface elements. It can be measured from the perspective of pixel-level perceived color difference, shape change, boundary offset, size inconsistency, etc., highlighting the degree of deviation between the current interface and the expected design.

[0056] Specifically, a semantic tag matching and attribute comparison approach can be used. This involves analyzing the semantic information of the first interface element in the target interface, such as DOM attributes, control types, and text content, and aligning it with the layer names, control annotations, and text content of the second interface element in the target design interface to calculate semantic similarity. Simultaneously, visual similarity is calculated by structurally extracting and comparing the style attributes of interface elements, such as color, font, border, and size, further comparing the degree of difference between attributes to form a preliminary judgment of the degree of difference. Alternatively, an image decoding and pixel comparison approach can be used. This involves decoding the image regions containing the first and second interface elements into pixel sets and uniformly converting them into a color space based on human visual perception characteristics (such as a space separating brightness and saturation). A perceptual color difference algorithm is then used to compare pixel by pixel to evaluate the consistency of image regions and calculate visual similarity. When significant differences are detected in dimensions such as color, contour, and texture, anti-aliasing detection mechanisms and edge-preserving algorithms are combined to quantify the degree of image difference. These methods can be used in combination according to actual application scenarios, with semantic and visual computation working together to improve the comprehensiveness and accuracy of interface fidelity assessment.

[0057] Step S103: Generate the restoration evaluation result of the interface to be detected based on the pre-configured quantitative index system, similarity and difference.

[0058] The pre-configured quantitative indicator system is a set of measurement standards set for user interface fidelity evaluation. This system defines various indicator parameters for interface similarity and difference, along with their corresponding evaluation dimensions and weight allocations. This quantitative indicator system can include multiple dimensions such as color consistency, layout alignment, element shape matching, text style consistency, and functional component semantic matching. Each dimension is configured with corresponding scoring rules and threshold standards to support multi-level quantitative evaluation of interface fidelity. The fidelity evaluation result is an overall matching score or evaluation level, reflecting the degree to which the tested interface replicates the target design interface in terms of appearance and functionality. This fidelity evaluation result can be output not only as a numerical score but also expressed through level labels (such as high, medium, and low) or visualizations (such as difference charts) to help designers or developers quickly identify interface deviation issues.

[0059] Specifically, a weighted aggregation calculation based on indicators can be used. The semantic similarity, visual similarity, and difference results calculated between the first and second interface elements are mapped to different evaluation dimensions in a pre-defined indicator system. By finding the scoring rules for each dimension, a local score for each dimension is calculated, and these local scores are weighted and aggregated according to pre-defined weighting coefficients to generate a comprehensive restoration score. This score can be used to measure the overall restoration level of the interface and provide detailed values ​​for each dimension to diagnose specific problems. Another approach is based on grading judgment and image annotation feedback. According to the grading rules in the pre-defined indicator system, similarity and difference are mapped to different scoring intervals or level labels. The restoration assessment level is generated by combining the scoring results, such as "complete restoration," "partial restoration," and "serious deviation." Simultaneously, a corresponding difference map is generated based on the pixel-level information in the difference score, and the deviation area is overlaid on the interface image to be tested using color annotation, enhancing interpretability and operability. Both approaches can be used in combination depending on the actual scenario. The former is suitable for structured scoring scenarios, while the latter is suitable for graphical diagnostic scenarios. Combining them allows for a unified quantitative and visual expression of the interface restoration assessment results.

[0060] Optionally, determining the similarity between the first interface element and the second interface element includes: performing semantic recognition processing on the first interface element and the second interface element respectively to obtain attribute information corresponding to the first interface element and the second interface element; the attribute information includes semantic features and visual features; determining the semantic similarity between the first interface element and the second interface element based on the semantic features, and determining the visual similarity between the first interface element and the second interface element based on the visual features.

[0061] In this embodiment, semantic recognition processing is performed on the first and second interface elements respectively to extract their respective attribute information. This attribute information includes semantic features and visual features. Semantic features include the element's function type, component name, and text meaning, while visual features include the element's color, size, position, font, and border style. Semantic similarity is obtained by comparing the semantic tags, keywords, or functional uses of the two interface elements based on semantic features. Simultaneously, visual similarity is calculated by comparing their appearance based on visual features, calculating indicators such as color matching, structural similarity, and style consistency. The semantic and visual similarity are combined to form an overall similarity judgment result. Alternatively, an end-to-end feature extraction and similarity calculation method based on a deep learning model can be used. This involves training a visual-semantic fusion model to vectorize interface elements and then calculating the overall similarity based on vector distance. This embodiment can effectively improve the accuracy of semantic and visual matching between interface elements, making the interface fidelity assessment more closely aligned with the actual design intent. This assists in the development and testing phases in promptly identifying fidelity deviations and improving the intelligence and accuracy of automated comparison and detection.

[0062] Optionally, determining the degree of difference between the first interface element and the second interface element includes: acquiring first image region data corresponding to the first interface element and second image region data corresponding to the second interface element; decoding the first image region data and the second image region data respectively to obtain first pixel data and second pixel data; converting the first pixel data and the second pixel data into a first pixel set and a second pixel set in a luminance-chrominance color space respectively; performing a pixel-by-pixel comparison of the first pixel set and the second pixel set based on a perceptual color difference algorithm, and determining the degree of difference between the first interface element and the second interface element based on the comparison result.

[0063] First, image region data of the first and second interface elements within their respective interfaces are acquired. This image region data can be the raw encoded data in the image buffer. Image decoding processing is then performed on these image region data to obtain the corresponding first and second pixel data. To improve the accuracy of the comparison and to better reflect human visual perception, the decoded first and second pixel data are converted into a first pixel set and a second pixel set in a luminance-chrominance color space. This color space can be a color model based on human visual characteristics, such as the YIQ color space. A pixel-by-pixel comparison of the two pixel sets is performed based on a perceptual color difference algorithm. This algorithm uses weighted calculations based on the human eye's sensitivity to different color differences, thus more realistically reflecting the visual differences between images. By comparing the perceptual color difference values ​​between each pair of corresponding pixels, the overall difference between the first and second interface elements is determined. Alternatively, the difference can also be determined using methods based on the Structural Similarity Index (SSIM) or image gradient comparison. By quantifying the differences in features such as image structure, texture, and edges, a higher-dimensional comparative evaluation can be achieved. This embodiment combines luminance and chromaticity color spaces with perceptual color difference algorithms, which better aligns with the judgment mechanism of the human visual system during the comparison process. This improves the accuracy and robustness of interface element comparison and provides a more reliable technical foundation for interface fidelity detection and automated testing.

[0064] Optionally, a pixel-by-pixel comparison is performed between the first pixel set and the second pixel set based on a perceptual color difference algorithm, and the degree of difference between the first interface element and the second interface element is determined based on the comparison result. This includes: calculating the perceptual color difference value between the first pixel in the first pixel set and the second pixel in the second pixel set; marking the first pixel as a difference pixel in response to the perceptual color difference value exceeding a preset color difference threshold; determining the quantity and distribution information of the difference pixels; and determining the degree of difference between the first interface element and the second interface element based on the quantity and distribution information of the difference pixels.

[0065] For each pair of pixels, the perceptual color difference value is calculated, reflecting the human eye's sensitivity to color differences. When the perceptual color difference value of a pair of pixels exceeds a preset color difference threshold, the corresponding first pixel is marked as a difference pixel, thus forming a set of difference pixels. The set of difference pixels is statistically processed to extract its quantity and distribution information. The quantity information characterizes the total number of pixels that differ between interface elements, while the distribution information reflects the spatial location and clustering degree of the difference regions in the image. For example, if difference pixels are concentrated around the edges of specific graphics or interactive controls, it may indicate a significant difference in visual style or interface layout. Based on the above quantity and distribution information, the degree of difference between the first and second interface elements is comprehensively determined. This degree of difference can be a quantitative indicator, such as the proportion of difference pixels or the density of difference regions, or a qualitative classification result, such as "highly consistent," "partially deviating," or "significantly different." In addition to the perceptual color difference algorithm, the calculation of the degree of difference can also be supplemented or enhanced by image gradient analysis, edge contour matching, or deep feature embedding vector comparison, thereby supporting the judgment of differences in more complex interface forms or cross-theme styles. This implementation method employs a color difference model based on the characteristics of human visual perception, combined with pixel-by-pixel comparison and spatial distribution analysis, to accurately identify real differences in images, thereby improving the accuracy of interface comparison and the engineering value of the evaluation results.

[0066] Optionally, before determining the quantity and distribution information of the differing pixels, the method further includes: obtaining a set of neighboring pixels of the differing pixels; the set of neighboring pixels includes multiple neighboring pixels of the differing pixels; determining the brightness difference between the differing pixels and each neighboring pixel, and determining brightness distribution features based on the brightness difference; and changing the differing pixels to non-discrepant pixels in response to the brightness distribution features satisfying a preset anti-aliasing feature condition.

[0067] In this embodiment, to improve the accuracy of difference recognition and avoid misjudging differences due to jagged edges in image rendering, anti-aliasing feature filtering is first applied to the initially identified difference pixels before determining the quantity and distribution information of the difference pixels. Specifically, a set of neighboring pixels around each difference pixel is obtained, which includes multiple spatially adjacent pixels. The brightness difference between the difference pixel and each neighboring pixel is calculated one by one to construct a brightness distribution map around the difference pixel. Based on this, it is further analyzed whether the brightness distribution exhibits a brightness gradient pattern that conforms to the preset jagged edge characteristics. For example, in common jagged edge effects, brightness changes usually show a smooth transition along the diagonal direction, with small and continuous brightness differences. If such a feature distribution is detected, it is determined that the difference pixel belongs to a non-real edge jagged edge change rather than a significant difference in interface content. Therefore, the labeling state of the difference pixel is changed to a non-difference pixel, and it is removed from subsequent difference statistics and distribution calculations. In addition to brightness distribution analysis, anti-aliasing can also be combined with color gradient vector directionality analysis and edge detection convolution kernel response to comprehensively determine whether there are false differences caused by rendering anti-aliasing, thereby further enhancing the robustness of recognition. This embodiment effectively eliminates false changes caused by pixel interpolation or edge anti-aliasing by introducing an anti-aliasing detection mechanism, ensuring the credibility of interface element difference assessment, reducing the false judgment rate, and providing a more accurate and clean basis for subsequent restoration assessment of difference pixels.

[0068] Optionally, the fidelity assessment result of the interface to be tested is generated based on a pre-configured quantitative indicator system, similarity, and difference, including: determining the corresponding evaluation weights of similarity and difference in the quantitative indicator system; and determining the fidelity assessment result of the interface to be tested relative to the target design interface based on similarity, difference, and evaluation weights.

[0069] The quantitative indicator system includes multiple dimensions of evaluation indicators and their corresponding weighting methods, such as semantic consistency, visual fidelity, and functional module mapping degree, to reflect various aspects of the interface reproduction effect. First, the evaluation dimensions corresponding to the currently identified similarities and differences are identified within the quantitative indicator system to determine the weight of each indicator in the overall evaluation system. For example, semantic similarity may have a higher weight in interactive applications, while visual differences are more important in presentation-type applications. Based on this weighting, similarity and differences are weighted and fused to obtain a comprehensive reproduction score. This reproduction score quantifies the degree of reproduction of the interface under test relative to the target design interface; a higher score indicates a better reproduction effect. In addition to weighted fusion, rule-based scoring mechanisms can also be used, such as setting multiple reproduction levels (high reproduction, basic reproduction, partial reproduction, no reproduction, etc.) and combining specific score ranges and weight offsets for judgment. Alternatively, machine learning models can be used to train historical interface comparison samples to automatically predict the reproduction level of the interface under test. This embodiment introduces a quantitative indicator system and an evaluation weighting mechanism, which can systematically integrate semantic and visual evaluation elements, making the evaluation of interface fidelity more objective, interpretable, and practical.

[0070] Optionally, after generating the restoration evaluation results of the interface to be detected based on the pre-configured quantitative indicator system, similarity, and difference, the method further includes: generating an interface difference map with color annotations based on the restoration evaluation results.

[0071] The similarity and difference mapping regions on the interface image are located to identify interface elements or pixel areas with significant differences. Based on preset color mapping rules, differences of varying degrees are highlighted with different colors; for example, red indicates severe differences, orange indicates moderate differences, and blue or green indicates slight differences or areas with no difference. During color annotation, the difference map can be layered by considering the numerical range of the difference, the spatial distribution of the difference pixels, and whether anti-aliasing detection has been passed, ensuring that the color representation is both accurate and visually recognizable. The generated interface difference map can serve as a visual supplement to the fidelity assessment results, helping users to more intuitively understand the deviation location and severity between the current interface and the design draft. In addition to color annotation based on static images, it can also be extended to support dynamic interactive difference feedback, such as using an interface structure tree to achieve interface module-level difference tracking, or using hotspot technology for difference highlighting, thereby improving the adaptability and usability of difference information across multiple types of terminal devices. This embodiment generates an interface difference map with color annotations, which not only enhances the interpretability and visualization of the fidelity assessment results, but also improves the positioning efficiency and decision-making basis for designers and testers in the interface consistency review process, thereby increasing the automation and intelligence of the interface development process.

[0072] Figure 2 This is a schematic diagram of the main flow of an interface fidelity detection method according to a preferred embodiment of the present invention. Figure 2 As shown, in this embodiment, the user first uploads the design draft image through a browser plugin. The plugin generates an overlay on the browser webpage to display the uploaded design draft and allows the user to adjust parameters such as the position, size, scaling ratio, and transparency of the design draft through an interactive interface to fit the visible area of ​​the actual page, ensuring basic spatial alignment between the design draft and the actual page, which facilitates subsequent pixel-level comparison.

[0073] After the user clicks the "Smart Detection" button, the plugin uses the html2canvas tool to take a screenshot of the current page and convert the screenshot image data into binary PNG data. html2Canvas is an open-source library that converts web pages or specific elements into images. The system uses the UPNG.js plugin to decode the uploaded design draft and the actual page screenshot separately, obtaining two PNG objects, which are then converted into RGBA format pixel data arrays. UPNG.js is a lightweight PNG image processing JavaScript library, primarily used for processing PNG images in browsers and Node.js environments. To achieve high-precision comparison, the plugin creates three Uint8Array arrays: two to store the pixel data of the design draft and the actual page, and one to store the data of the difference result image.

[0074] The system uses the pixelmatch library to compare the pixels of two images. Pixelmatch is a small, fast, open-source JavaScript library suitable for scenarios requiring image comparison. The library first verifies the validity, length, and size of the image pixel data. Using a perceptual color difference algorithm, based on the human eye's sensitivity to color, it calculates the difference in color space for each pixel in the image. If the perceptual color difference of a pixel exceeds a set threshold, it further determines whether the difference is due to anti-aliasing. To determine the anti-aliasing effect, it iterates through the eight neighboring pixels of the affected pixel and compares their brightness values. If two or more neighboring pixels have completely identical brightness, the difference is not classified as anti-aliasing but considered a genuine pixel difference, i.e., a valid difference. During processing, the plugin uses color to label different types of differences. Anti-aliasing differences are marked in yellow; valid differences are marked with bright colors such as red or orange; and similar or unchanged areas are drawn in gray to enhance the visualization of the differences. Finally, the plugin generates a color-coded difference image that clearly shows the pixel-level differences between the design draft and the actual page, and outputs the number of difference pixels, the difference image, and an analysis report for designers and developers to further review and correct.

[0075] This embodiment improves the efficiency and accuracy of interface fidelity detection through automated UI element parsing, pixel-level image comparison, and intelligent difference judgment. It also helps users locate problem areas in an intuitive image manner, effectively reducing manual comparison errors and accelerating the interface fidelity review process.

[0076] According to a second aspect of the present invention, an interface fidelity detection device is provided.

[0077] Figure 3 This is a schematic diagram of the main modules of the interface fidelity detection device according to an embodiment of the present invention. Figure 3 As shown, the interface fidelity detection device 300 includes:

[0078] The extraction module 301 is used to extract the first interface element and the second interface element from the interface to be detected and the target design interface, respectively.

[0079] The determining module 302 is used to determine the similarity and difference between the first interface elements and the second interface elements; wherein, the similarity includes semantic similarity and visual similarity;

[0080] The generation module 303 is used to generate the restoration evaluation results of the interface to be detected based on the pre-configured quantitative index system, similarity and difference.

[0081] Optionally, the determining module 302 is also used for:

[0082] Semantic recognition processing is performed on the first interface element and the second interface element respectively to obtain the attribute information corresponding to the first interface element and the second interface element; the attribute information includes semantic features and visual features.

[0083] The semantic similarity between the first interface element and the second interface element is determined based on semantic features, and the visual similarity between the first interface element and the second interface element is determined based on visual features.

[0084] Optionally, the determining module 302 is also used for:

[0085] Obtain the first image region data corresponding to the first interface element and the second image region data corresponding to the second interface element;

[0086] The first image region data and the second image region data are decoded respectively to obtain the first pixel data and the second pixel data;

[0087] The first pixel data and the second pixel data are respectively converted into the first pixel set and the second pixel set in the luminance and chrominance color space;

[0088] The first pixel set and the second pixel set are compared pixel by pixel based on the perceptual color difference algorithm, and the degree of difference between the first interface element and the second interface element is determined based on the comparison results.

[0089] Optionally, the determining module 302 is also used for:

[0090] Calculate the perceived color difference value between the first pixel in the first pixel set and the second pixel in the second pixel set. In response to the perceived color difference value exceeding a preset color difference threshold, mark the first pixel as a difference pixel.

[0091] Determine the quantity and distribution information of the differing pixels;

[0092] The degree of difference between the first interface element and the second interface element is determined based on the quantity and distribution information of the difference pixels.

[0093] Optionally, the determining module 302 is also used for:

[0094] Obtain the set of neighboring pixels of the differing pixel; the set of neighboring pixels includes multiple adjacent pixels of the differing pixel;

[0095] Determine the brightness difference between the differing pixel and each of its neighboring pixels, and determine the brightness distribution characteristics based on the brightness difference;

[0096] In response to the brightness distribution characteristics satisfying the preset anti-aliasing feature conditions, the difference pixels are changed to non-difference pixels.

[0097] Optionally, the generation module 303 is also used for:

[0098] Determine the corresponding evaluation weights of similarity and difference in the quantitative indicator system;

[0099] Based on similarity, difference, and evaluation weight, the result of the reproduction degree assessment of the interface to be detected relative to the target design interface is determined.

[0100] Optionally, the detection device 300 further includes a labeling module, which is used for:

[0101] An interface difference map with color annotations is generated based on the fidelity assessment results.

[0102] It should be noted that the specific implementation details of the interface fidelity detection device of the present invention have been described in detail in the interface fidelity detection method above, so the details will not be repeated here.

[0103] According to a third aspect of the present invention, an electronic device is provided, comprising: one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method provided in the first aspect of the present invention.

[0104] According to a fourth aspect of the present invention, a computer-readable medium is provided having a computer program stored thereon, which, when executed by a processor, implements the method provided in the first aspect of the present invention.

[0105] According to a fifth aspect of the present invention, a computer program product is provided, including a computer program that, when executed by a processor, implements the method provided in the first aspect of the present invention.

[0106] Figure 4 An exemplary system architecture 400 is shown that can be applied to the interface fidelity detection method or interface fidelity detection device of the present invention.

[0107] like Figure 4 As shown, system architecture 400 may include terminal devices 401, 402, and 403, a network 404, and a server 405. Network 404 serves as the medium for providing communication links between terminal devices 401, 402, and 403 and server 405. Network 404 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0108] Users can use terminal devices 401, 402, and 403 to interact with server 405 via network 404 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 401, 402, and 403, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social media platform software, etc. (for example only).

[0109] Terminal devices 401, 402, and 403 can be various electronic devices with displays that support web browsing, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0110] Server 405 can be a server that provides various services, such as a backend management server that supports shopping websites browsed by users using terminal devices 401, 402, and 403 (for example only). The backend management server can analyze and process data such as received interface fidelity detection requests, and feed back the processing results (such as fidelity evaluation results - for example only) to the terminal devices.

[0111] It should be noted that the interface fidelity detection method provided in this embodiment of the invention is generally run by server 405, and correspondingly, the interface fidelity detection device is generally set in server 405.

[0112] It should be understood that Figure 4 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0113] The following is for reference. Figure 5 It shows a schematic diagram of the structure of a computer system 500 suitable for implementing a terminal device of the present invention. Figure 5 The terminal device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0114] like Figure 5 As shown, the computer system 500 includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 502 or programs loaded from storage section 508 into random access memory (RAM) 503. The RAM 503 also stores various programs and data required for the operation of the system 500. The CPU 501, ROM 502, and RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0115] The following components are connected to I / O interface 505: an input section 506 including a keyboard, mouse, etc.; an output section 507 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN card, modem, etc. The communication section 509 performs communication processing via a network such as the Internet. Drive 510 is also connected to I / O interface 505 as needed. Removable media 511, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 510 as needed so that computer programs read from them can be installed into storage section 508 as needed.

[0116] In particular, according to the embodiments disclosed in this invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 509, and / or installed from removable medium 511. When the computer program is run by the central processing unit (CPU) 501, it performs the functions defined above in the system of this invention.

[0117] It should be noted that the computer-readable medium shown in this invention can be a computer-readable signal medium, a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit programs for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0118] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more operable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually operate substantially in parallel, and they may sometimes operate in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0119] The modules described in the embodiments of the present invention can be implemented in software or hardware. The described modules can also be housed in a processor; for example, a processor can be described as including an extraction module, a determination module, and a generation module. The names of these modules do not necessarily limit the module itself; for example, the extraction module can also be described as "a module for extracting a first interface element and a second interface element from the interface to be detected and the target design interface, respectively."

[0120] In another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiments; or it may exist independently and not assembled into the device. The computer-readable medium carries one or more programs, which, when executed by the device, cause the device to: extract a first interface element and a second interface element from an interface to be detected and a target design interface, respectively; determine the similarity and difference between the first interface element and the second interface element; wherein the similarity includes semantic similarity and visual similarity; and generate a restoration evaluation result of the interface to be detected based on a pre-configured quantitative index system, the similarity, and the difference.

[0121] The computer program product provided in this embodiment of the invention includes a computer program that, when executed by a processor, implements the interface fidelity detection method in this embodiment of the invention.

[0122] According to the technical solution of the present invention, the following advantages or beneficial effects are achieved: by extracting a first interface element and a second interface element from the interface to be detected and the target design interface respectively; determining the similarity and difference between the first interface element and the second interface element; wherein, the similarity includes semantic similarity and visual similarity; generating a restoration evaluation result of the interface to be detected based on a pre-configured quantitative index system, similarity, and difference; thereby, it is possible to combine semantic understanding and visual comparison technology to determine the similarity and difference between interface elements, intelligently identify changes in interface elements, and generate an objective restoration evaluation result based on a pre-configured quantitative index system, thereby improving detection efficiency and the accuracy of detection results.

[0123] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

[0124] It should be noted that the acquisition, storage, and application of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

Claims

1. An interface reduction degree detection method characterized by comprising: include: Extract the first interface elements and the second interface elements from the interface to be tested and the target design interface, respectively; Determine the similarity and difference between the first interface element and the second interface element; wherein, the similarity includes semantic similarity and visual similarity; The restoration evaluation result of the interface to be detected is generated based on the pre-configured quantitative index system, the similarity and difference.

2. The method of claim 1, wherein, Determining the similarity between the first interface element and the second interface element includes: Semantic recognition processing is performed on the first interface element and the second interface element respectively to obtain the attribute information corresponding to the first interface element and the second interface element; the attribute information includes semantic features and visual features. The semantic similarity between the first interface element and the second interface element is determined based on the semantic features, and the visual similarity between the first interface element and the second interface element is determined based on the visual features.

3. The method of claim 1, wherein, Determining the degree of difference between the first interface element and the second interface element includes: Obtain the first image region data corresponding to the first interface element and the second image region data corresponding to the second interface element; The first image region data and the second image region data are decoded respectively to obtain the first pixel data and the second pixel data; The first pixel data and the second pixel data are respectively converted into a first pixel set and a second pixel set in the luminance-chrominance color space; The first pixel set and the second pixel set are compared pixel by pixel based on the perceptual color difference algorithm, and the degree of difference between the first interface element and the second interface element is determined based on the comparison results.

4. The method of claim 3, wherein, A pixel-by-pixel comparison is performed between the first pixel set and the second pixel set based on a perceptual color difference algorithm. The degree of difference between the first interface element and the second interface element is determined based on the comparison results, including: Calculate the perceived color difference value between the first pixel in the first pixel set and the second pixel in the second pixel set; in response to the perceived color difference value exceeding a preset color difference threshold, mark the first pixel as a difference pixel. Determine the quantity and distribution information of the differing pixels; The degree of difference between the first interface element and the second interface element is determined based on the quantity and distribution information of the difference pixels.

5. The method of claim 4, wherein, Before determining the quantity and distribution information of the differing pixels, the method further includes: Obtain the neighborhood pixel set of the differing pixel; the neighborhood pixel set includes multiple adjacent pixels of the differing pixel; Determine the brightness difference between the differing pixel and each of the adjacent pixels, and determine the brightness distribution characteristics based on the brightness difference; In response to the brightness distribution feature satisfying the preset anti-aliasing feature condition, the difference pixel is changed to a non-difference pixel.

6. The method of claim 1, wherein, Based on a pre-configured quantitative indicator system, the similarity and difference are used to generate a restoration evaluation result for the interface to be detected, including: Determine the corresponding evaluation weights of the similarity and difference in the quantitative index system; Based on the similarity, difference, and evaluation weight, the fidelity assessment result of the interface to be detected relative to the target design interface is determined.

7. The method of claim 1, wherein, After generating the restoration evaluation result of the interface to be detected based on the pre-configured quantitative indicator system, the similarity and difference, the method further includes: An interface difference map with color annotations is generated based on the fidelity assessment results.

8. An interface reduction degree detection device characterized by comprising: include: The extraction module is used to extract the first interface elements and the second interface elements from the interface to be detected and the target design interface, respectively. A determining module is used to determine the similarity and difference between the first interface element and the second interface element; wherein the similarity includes semantic similarity and visual similarity; The generation module is used to generate the restoration evaluation result of the interface to be detected based on the pre-configured quantitative index system, the similarity and difference.

9. An electronic device, comprising: include: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-7.

10. A computer readable medium having stored thereon a computer program, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-7.

11. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-7.