Automatic walkthrough method, device and equipment for static UI page and medium

Abstract

The invention relates to the technical field of webpage design, and discloses an automatic walkthrough method and device for a static UI page, equipment and a medium, and the method comprises the following steps: carrying out image rendering on a page design file to obtain a page design image, and obtaining a real page image corresponding to the page design image; performing element target detection and element segmentation on the page design image to obtain a page design element image set; performing multi-type element feature extraction and element feature fusion on the page design element image set to obtain a design element feature set; extracting a real element feature set from the real page image, and performing element matching on the design element feature set according to the real element feature set to obtain an error element feature set; and obtaining a page code corresponding to the real page image, and performing element matching and error labeling on the page code according to the error element feature set to obtain a walkthrough detection result. According to the method and the device, the flexibility during static page walkthrough can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of webpage design, and in particular to an automatic checking method, device, equipment and medium for static UI pages. BACKGROUND

[0002] Page checking is a process of verifying the consistency between the design draft and the actual implementation effect of a page, aiming to find possible deviations, problems or errors in the development or deployment process of a front-end page. The process of page checking usually includes checking page layout, element style, function logic and interactive behavior one by one, and comparing them with the original design draft.

[0003] Existing page checking methods are mostly pixel-based page checking methods, such as comparing a real page screenshot with a design draft image pixel by pixel to find differences. In actual application, pixel-based page checking methods may misjudge minor differences due to factors such as resolution, anti-aliasing effect, font rendering, etc., and ignore user visual perception and page function logic, which may result in low flexibility in page checking. SUMMARY

[0004] The present application provides an automatic checking method, device and computer readable storage medium for static UI pages, which mainly aims to solve the problem of low flexibility in page checking.

[0005] To achieve the above-mentioned purpose, the present application provides an automatic checking method for static UI pages, which comprises: rendering an image of a page design file to obtain a page design image, and obtaining a real page image corresponding to the page design image; performing element target detection and element segmentation on the page design image to obtain a page design element atlas; performing multi-type element feature extraction and element feature fusion on the page design element atlas to obtain a design element feature set; extracting a real element feature set from the real page image, and performing element matching on the design element feature set according to the real element feature set to obtain an error element feature set; obtaining a page code corresponding to the real page image, performing element matching and error labeling on the page code according to the error element feature set to obtain a checking detection result.

[0006] Optionally, the rendering of the image of the page design file to obtain the page design image comprises: configuring the output format and output attributes of the page design file to obtain an output configuration file; grouping layers of the page design file to obtain a design layer group; initializing a rendering canvas according to the design layer group; configuring a margin and a background of the rendering canvas to obtain a configured rendering canvas; rendering the design layer group according to the configured rendering canvas and the output configuration file to obtain a page design image.

[0007] Optionally, the element target detection and element segmentation on the page design image to obtain a page design element atlas comprises: performing multi-scale image convolution on the page design image to obtain a design page image feature set; performing residual connection on the design page image feature set to obtain a design residual image feature set; performing multi-scale feature fusion on the design residual image feature set to obtain a fused design image feature; performing element target detection on the fused design image feature to obtain a page element bounding box set; performing image segmentation on the page design image according to the page element bounding box set to obtain a page design element atlas.

[0008] Optionally, the multi-type element feature extraction and element feature fusion on the page design element atlas to obtain a design element feature set comprises: performing text element screening on the page design element atlas to obtain a design text element atlas; screening a design text element atlas from the page design element atlas according to the element type set; performing text recognition on the design text element atlas to obtain a text content set; performing font recognition on the design text element atlas to obtain a text font set; generating an element text feature set according to the text content set and the text font set; extracting an element position feature set, an element color feature set, an element size feature set and an element texture feature set from the page design element atlas respectively; generating a design element feature set according to the element position feature set, the element color feature set, the element size feature set, the element text feature set and the element texture feature set.

[0009] Optionally, the real element feature set extraction from the real page image comprises: performing element target detection and element segmentation on the real page image to obtain a page real element atlas; The real element feature set is subjected to multi-type element feature extraction and element feature fusion to obtain a real element feature set.

[0010] Optionally, the element matching of the design element feature set according to the real element feature set to obtain an error element feature set comprises: Each real element feature in the real element feature set is selected as a target real element feature, and a design element feature corresponding to the target real element feature is filtered from the design element feature set as a target design element feature; The feature distance between each target real element feature and the corresponding target design element feature is calculated; It is judged whether each feature distance is greater than a preset distance threshold; All target design elements with a feature distance greater than the preset distance threshold are filtered out, and error type labeling is performed to obtain an error element feature set.

[0011] Optionally, the element matching of the page code according to the error element feature set and error labeling to obtain a review detection result comprises: The page code is subjected to structural analysis to obtain a page element structure; The error page element set is filtered from the page element structure according to the error element feature set; The error style code set corresponding to the error page element set is matched from the page code; The error type set is filtered from the error element feature set; The error style code set is subjected to error labeling according to the error type set to obtain a review detection result.

[0012] To solve the above problems, the application further provides an automatic review device for static UI pages, which comprises: An image rendering module is configured to perform image rendering on a page design file to obtain a page design image and acquire a real page image corresponding to the page design image; An element segmentation module is configured to perform element target detection and element segmentation on the page design image to obtain a page design element atlas; A feature extraction module is configured to perform multi-type element feature extraction and element feature fusion on the page design element atlas to obtain a design element feature set; An element matching module is configured to extract a real element feature set from the real page image and perform element matching of the design element feature set according to the real element feature set to obtain an error element feature set; The review annotation module is configured to obtain page codes corresponding to the real page image, perform element matching on the page codes according to the error element feature set, and perform error annotation to obtain a review detection result.

[0013] To solve the above problems, the application further provides an electronic device, which comprises: at least one processor; and a memory in communication with the at least one processor; wherein The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform the automatic review method of the static UI page.

[0014] To solve the above problems, the application further provides a computer readable storage medium, which stores at least one computer program executable by a processor in an electronic device to implement the automatic review method of the static UI page.

[0015] The embodiment of the application can obtain the designed page image and the real page image by image rendering on the page design file to obtain the page design image, thereby facilitating page review by machine vision, and can obtain the page design element atlas by element target detection and element segmentation on the page design image, thereby identifying the images of various elements in the page design image by target detection, segmenting the elements, improving the efficiency of element recognition, reducing the workload of manual visual comparison, improving the flexibility of page review, and improving the accuracy of subsequent page review by multi-scale extraction of features of various page elements through multi-type element feature extraction and element feature fusion.

[0016] The error element feature set is obtained by element matching on the design element feature set according to the real element feature set, and the automatic page element comparison can be realized by feature matching, thereby realizing automatic page review and improving the efficiency and flexibility of page review, and the review detection result is obtained by element matching on the page codes according to the error element feature set and error annotation, thereby automatically searching for the codes corresponding to the error elements and generating the final review result document, and the flexibility of page review is improved. Therefore, the automatic review method, device, equipment and medium of the static UI page can solve the problem of low flexibility in page review. BRIEF DESCRIPTION OF DRAWINGS

[0017] Figure 1A flowchart of the static UI page automatic checking method provided by an embodiment of the present application is shown in FIG. 1. Figure 2 A flowchart of the page design element atlas extraction method provided by an embodiment of the present application is shown in FIG. 4. Figure 3 A flowchart of the checking detection result generation method provided by an embodiment of the present application is shown in FIG. 5. Figure 4 A device architecture diagram of the static UI page automatic checking device provided by an embodiment of the present application is shown in FIG. 6. Figure 5 A structural diagram of an electronic device implementing the static UI page automatic checking method provided by an embodiment of the present application is shown in FIG. 7.

[0018] The object, functional characteristics and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0019] It should be understood that the specific embodiments described herein are merely intended to explain the present application and not to limit the present application.

[0020] An embodiment of the present application provides a static UI page automatic checking method. The execution subject of the static UI page automatic checking method includes but is not limited to at least one of a server, a terminal and other electronic devices that can be configured to execute the method provided by the present application. In other words, the static UI page automatic checking method can be executed by software or hardware installed in a terminal device or a server device. The server includes but is not limited to a single server, a server cluster, a cloud server or a cloud server cluster, etc. The server can be a stand-alone server, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content distribution networks (CDN), and big data and artificial intelligence platforms, etc. basic cloud computing services.

[0021] Referring to FIG. 1, Figure 1 A flowchart of the static UI page automatic checking method provided by an embodiment of the present application is shown in FIG. 1. In this embodiment, the static UI page automatic checking method includes: S1, image rendering is performed on a page design file to obtain a page design image and acquire a real page image corresponding to the page design image.

[0022] In detail, the page design file refers to a design draft file for a page, and formats of the page design file can be Sketch, Figma, Adobe XD, etc., the page design file is a vector format file or a proprietary format file specially used for design and prototyping, and contains design information, layout, style and interaction details of the page, the page design file has a specific format and cannot be directly used for code in a production environment, and the main function is to help designers and developers understand the appearance and structure of the page.

[0023] Specifically, the page design image is a page effect image presented by the page design file, and the real page image is a final page screenshot of a web page developed by a developer based on the page design file.

[0024] In the embodiment of the application, the page design file is rendered to obtain a page design image, comprising: The page design file is configured in output format and output attribute to obtain an output configuration file; The page design file is grouped to obtain a design layer group; The rendering canvas is initialized according to the design layer group; The rendering canvas is configured in margin and background to obtain a configured rendering canvas; The design layer group is rendered according to the configured rendering canvas and the output configuration file to obtain a page design image.

[0025] In detail, the output format configuration refers to the format of the output page design image, such as PNG and JPEG images, and the output attribute configuration refers to the resolution, pixel size, proportion and device adaptation type of the output page design image.

[0026] Specifically, the layer grouping refers to grouping all combined layers of the page design file, such as Group, Frame, etc., to a place, the initialized rendering canvas refers to selecting an area of the canvas according to the layer page size of the design layer group, and the margin configuration and background configuration refer to configuring the margin and background of the rendering canvas.

[0027] In the embodiment of the application, the page design file is rendered to obtain a page design image, which can obtain a designed page image and a real page image, so as to facilitate page checking by using a machine vision method.

[0028] S2, element target detection and element segmentation are performed on the page design image to obtain a page design element atlas.

[0029] Specifically, each page design element image in the page design element image set is a partial image corresponding to a page element in the page design image, for example, an image of a webpage title part or an image of a webpage form part, and the page element includes an image, text, a button, a form and the like.

[0030] In the embodiment of the present application, referring to Figure 2 As shown in the figure, the element target detection and element segmentation on the page design image to obtain the page design element image set comprises: S21, performing multi-scale image convolution on the page design image to obtain a design page image feature set; S22, performing residual connection on the design page image feature set to obtain a design residual image feature set; S23, performing multi-scale feature fusion on the design residual image feature set to obtain a fused design image feature; S24, performing element target detection on the fused design image feature to obtain a page element bounding box set; S25, performing image segmentation on the page design image according to the page element bounding box set to obtain a page design element image set.

[0031] Specifically, the multi-scale image convolution refers to layer-by-layer convolution of the page design image by using convolution layers of multiple different image scales, and the multi-scale image convolution can be performed by using CSPNet, Focus Layer, Conv BN LeakyReLU and Convolutional Block Attention Module (CBAM).

[0032] Specifically, the residual connection refers to residual connection of the features before convolution and the features after convolution of each design page image feature in the design page image feature set, so as to retain image feature details, and the residual connection can be performed by using a residual layer.

[0033] Specifically, the multi-scale feature fusion refers to fusion of each design residual image feature in the design residual image feature set into an image feature, and the multi-scale feature fusion can be performed by using a top-down and bottom-up feature pyramid network or a path aggregation network that transfers information from a bottom high-level feature to a top layer.

[0034] Specifically, the element target detection can be performed by using a Decoupled Head structure of a pre-trained YOLO model, and the image segmentation can be performed by using a mask segmentation method.

[0035] In the embodiment of the present application, the page design image is subjected to element target detection and element segmentation to obtain a page design element atlas, various elements in the page design image can be recognized by target detection, element segmentation is performed, the efficiency of element recognition is improved, the workload of manual visual comparison is reduced, and the flexibility of page checking is improved.

[0036] S3, multi-type element feature extraction and element feature fusion are performed on the page design element atlas to obtain a design element feature set.

[0037] In detail, the design element feature set is an element feature corresponding to each page design element image in the page design element atlas.

[0038] In the embodiment of the present application, the multi-type element feature extraction and element feature fusion performed on the page design element atlas to obtain a design element feature set comprises: Text element screening is performed on the page design element atlas to obtain a design text element atlas; The design text element atlas is screened from the page design element atlas according to the element type set; Text recognition is performed on the design text element atlas to obtain a text content set; Font recognition is performed on the design text element atlas to obtain a text font set; An element text feature set is generated according to the text content set and the text font set; An element position feature set, an element color feature set, an element size feature set and an element texture feature set are extracted from the page design element atlas respectively; A design element feature set is generated according to the element position feature set, the element color feature set, the element size feature set, the element text feature set and the element texture feature set.

[0039] Specifically, the element type recognition refers to determining the element type corresponding to each page design element image according to the page element bounding box set corresponding to the page design element atlas, and the element type can be a text type, an image type or a form type.

[0040] In detail, the design text element atlas refers to a group of page design element images of an element type of a text type, and text recognition can be performed by using a connection text proposal network (CRNN) of an optical character recognition (OCR) technology, a progressive scale expansion network (PSERN), and a differentiable binarization network (DBN).

[0041] Specifically, font recognition can be performed by using a VGG model, a ResNet model, and a Transformer-based model after training, and the text font in the text font set can be a font such as Song, Kai, or Bold.

[0042] In detail, the generation of the element text feature set according to the text content set and the text font set refers to vectorization of the text content set and the text font set to obtain a text content feature set and a text font feature set, and feature splicing and fusion of corresponding text content features and corresponding text font features according to a corresponding relationship to obtain the element text feature set.

[0043] Specifically, the element position feature set can be extracted by using a position coding method, the element color feature set can be extracted by using an HSV color space conversion method, the element size feature set can be generated according to a pixel resolution, the element texture feature set can be extracted by using a gabor filtering or Fourier transform method, and the design element feature set can be generated according to the element position feature set, the element color feature set, the element size feature set, the element text feature set, and the element texture feature set by using a feature splicing or full connection method.

[0044] In the embodiment of the application, by performing multi-type element feature extraction and element feature fusion, the features of each page element can be extracted in multiple scales, thereby improving the accuracy of subsequent page checking.

[0045] S4, a real element feature set is extracted from the real page image, and element matching is performed on the design element feature set according to the real element feature set to obtain an error element feature set.

[0046] In detail, the real element feature set is a set of element features of each page element in the real page image, and each error element feature in the error element feature set is a set of design element features in the design element feature set that do not match or have errors in the real element feature set.

[0047] Specifically, the extracting the real element feature set from the real page image comprises: performing element target detection and element segmentation on the real page image to obtain a real page element image set; performing multi-type element feature extraction and element feature fusion on the real page element image set to obtain the real element feature set.

[0048] In detail, the method of element target detection and element segmentation is consistent with the method in step S2, and the method of multi-type element feature extraction and element feature fusion is consistent with the method in step S3, which will not be repeated here.

[0049] In the embodiment of the application, the element matching of the design element feature set according to the real element feature set to obtain the error element feature set comprises: selecting a real element feature in the real element feature set as a target real element feature, and screening a design element feature corresponding to the target real element feature from the design element feature set as a target design element feature; calculating a feature distance between each target real element feature and the corresponding target design element feature; judging whether each feature distance is greater than a preset distance threshold; screening out all target design elements with a feature distance greater than the preset distance threshold, performing error type labeling to obtain the error element feature set.

[0050] In detail, the feature distance can be calculated by using a cosine similarity algorithm and an Euclidean distance algorithm, the distance threshold is a feature distance set by cosine, the error type labeling refers to error labeling according to a feature type exceeding a specific threshold in the feature distance, and the error elements include position error, color error, size error, text error and texture error, which correspond to the element position feature set, the element color feature set, the element size feature set, the element text feature set and the element texture feature set respectively.

[0051] In the embodiment of the present application, the error element feature set is obtained by element matching of the design element feature set according to the real element feature set, and automatic page element comparison is realized by the feature matching method, thereby realizing automatic page review and improving the efficiency and flexibility of page review.

[0052] S5, obtaining the page code corresponding to the real page image, performing element matching and error labeling on the page code according to the error element feature set, and obtaining a review detection result.

[0053] In detail, the page code is a web page code corresponding to a real page used for compiling the real page image.

[0054] In the embodiment of the present application, referring to Figure 3 As shown in the figure, the element matching and error labeling of the page code according to the error element feature set to obtain a review detection result comprises: S31, performing structure analysis on the page code to obtain a page element structure; S32, screening an error page element set from the page element structure according to the error element feature set; S33, matching an error style code set corresponding to the error page element set from the page code; S34, screening an error type set from the error element feature set; S35, performing error labeling on the error style code set according to the error type set to obtain a review detection result.

[0055] Specifically, the structure analysis can be performed by using a framework such as DevTools or DOM tree, and the structure analysis refers to analyzing the code structure corresponding to each element of the page code, such as div, button and other elements.

[0056] In detail, the error page element set can be screened according to the element position feature and element type corresponding to the error element feature set, and the error style code set corresponding to the error page element set can be matched from the page code by using element path positioning and CSS style positioning methods.

[0057] Specifically, the error labeling refers to labeling the error style code in the error style code set according to each corresponding error type in the error type set, and generating a corresponding result document.

[0058] In the embodiment of the present application, the page code is matched with the error elements according to the error element feature set, and the error elements are marked, so that the error element corresponding code can be automatically searched, and the final review result document can be generated, thereby improving the flexibility of page review.

[0059] In the embodiment of the present application, the page design file is rendered to obtain a page design image, so that the designed page image and the real page image can be obtained, thereby facilitating the page review by using the machine vision method. The page design element set is obtained by performing element target detection and element segmentation on the page design image, so that the images of various elements in the page design image can be recognized by using the target detection method, and the elements are segmented, thereby improving the element recognition efficiency, reducing the workload of manual visual comparison, improving the flexibility of page review, and improving the accuracy of subsequent page review by performing multi-type element feature extraction and element feature fusion.

[0060] The error element feature set is obtained by matching the design element feature set with the real element feature set, so that the page element comparison can be automatically realized by using the feature matching method, thereby realizing the automatic page review, improving the efficiency and flexibility of page review, and automatically searching the error element corresponding code by matching the page code with the error element feature set and marking the error elements, thereby generating the final review result document, and improving the flexibility of page review. Therefore, the automatic review method of the static UI page can solve the problem of low flexibility in page review.

[0061] As shown in Figure 4 FIG. 1 is a device architecture diagram of the automatic review device of the static UI page according to an embodiment of the present application.

[0062] The automatic review device 100 of the static UI page can be installed in an electronic device. According to the implemented function, the automatic review device 100 of the static UI page can include an image rendering module 101, an element segmentation module 102, a feature extraction module 103, an element matching module 104, and a review marking module 105. The modules of the present application can also be referred to as units, which refer to a series of computer program segments that can be executed by an electronic device processor and can complete a fixed function, and are stored in the memory of the electronic device.

[0063] In the embodiment, the functions of each module / unit are as follows: The image rendering module 101 is configured to perform image rendering on the page design file to obtain a page design image and acquire a real page image corresponding to the page design image. The element segmentation module 102 is configured to perform element target detection and element segmentation on the page design image to obtain a page design element atlas. The feature extraction module 103 is configured to perform multi-type element feature extraction and element feature fusion on the page design element atlas to obtain a design element feature set. The element matching module 104 is configured to extract a real element feature set from the real page image and perform element matching on the design element feature set according to the real element feature set to obtain an error element feature set. The walk-through annotation module 105 is configured to acquire page code corresponding to the real page image, perform element matching on the page code according to the error element feature set, and perform error annotation to obtain a walk-through detection result.

[0064] In detail, each module in the static UI page automatic walk-through device 100 in the embodiment of the present application adopts the same technical means as the static UI page automatic walk-through method in the above Figures 1 to 3 , and can produce the same technical effects, which will not be described here.

[0065] As shown in Figure 5 , it is a structural schematic diagram of an electronic device for implementing the static UI page automatic walk-through method according to an embodiment of the present application.

[0066] The electronic device 1 can include a processor 10, a memory 11, a communication bus 12, and a communication interface 13, and can further include a computer program stored in the memory 11 and executable on the processor 10, such as a static UI page automatic walk-through program.

[0067] In some embodiments, the processor 10 can be composed of an integrated circuit, for example, can be composed of a single packaged integrated circuit, or can be composed of multiple packaged integrated circuits with the same function or different functions, including one or more central processing units (CPU), microprocessors, digital processing chips, graphics processors, and combinations of various control chips, etc. The processor 10 is the control core of the electronic device, which connects all components of the electronic device through various interfaces and lines, executes or runs the programs or modules stored in the memory 11 (such as the static UI page automatic walk-through program), and calls the data stored in the memory 11 to perform various functions and process data of the electronic device.

[0068] The memory 11 includes at least one type of readable storage medium, such as a flash memory, a mobile hard disk, a multimedia card, a card-type memory (e.g., an SD or DX memory, etc.), a magnetic memory, a disk, an optical disk, etc. The memory 11 can be an internal storage unit of an electronic device in some embodiments, such as a mobile hard disk of the electronic device. The memory 11 can also be an external storage device of the electronic device in other embodiments, such as a plug-in mobile hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device. Further, the memory 11 can include both an internal storage unit and an external storage device of the electronic device. The memory 11 can be used to store application software and various data installed in the electronic device, such as the code of the automated walk-through program of the static UI page, and can also be used to temporarily store data that has been output or will be output.

[0069] The communication bus 12 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. The bus is configured to enable connection and communication between the memory 11 and at least one processor 10, etc.

[0070] The communication interface 13 is used for communication between the electronic device and other devices, and includes a network interface and a user interface. Optionally, the network interface can include a wired interface and / or a wireless interface (e.g., a WI-FI interface, a Bluetooth interface, etc.), and is typically used to establish a communication connection between the electronic device and other electronic devices. The user interface can be a display (Display), an input unit (such as a keyboard (Keyboard)), and optionally, the user interface can also be a standard wired interface, a wireless interface. Optionally, in some embodiments, the display can be an LED display, a liquid crystal display, a touch liquid crystal display, an organic light-emitting diode (OLED) touch, etc. The display can also be appropriately referred to as a display screen or a display unit, and is used to display information processed in the electronic device and to display a visualized user interface.

[0071] It should be understood that the electronic device shown in the figure is only for illustration, and the structure shown in the figure does not limit the electronic device. The electronic device can include fewer or more components than those shown in the figure, or combine some components, or arrange different components.

[0072] For example, although not shown, the electronic device can also include a power supply (such as a battery) for powering each component. Preferably, the power supply can be logically connected to the at least one processor 10 through a power management device, so that the power management device can realize functions such as charge management, discharge management, and power consumption management. The power supply can also include one or more direct current or alternating current power supplies, recharging devices, power failure detection circuits, power converters or inverters, power status indicators, and any other components. The electronic device can also include various sensors, Bluetooth modules, Wi-Fi modules, and the like, which are not described here.

[0073] It should be understood that the embodiments are only for illustration, and the scope of the patent application is not limited by the structure.

[0074] The length-based text analysis program stored in the memory 11 in the electronic device 1 is a combination of multiple instructions, which, when executed in the processor 10, can achieve: rendering an image of a page design file to obtain a page design image, and obtaining a real page image corresponding to the page design image; performing element target detection and element segmentation on the page design image to obtain a page design element atlas; performing multi-type element feature extraction and element feature fusion on the page design element atlas to obtain a design element feature set; extracting a real element feature set from the real page image, and performing element matching on the design element feature set according to the real element feature set to obtain an error element feature set; obtaining a page code corresponding to the real page image, performing element matching and error labeling on the page code according to the error element feature set to obtain a review detection result.

[0075] Specifically, the specific implementation method of the processor 10 for the above instructions can refer to the description of the related steps in the corresponding embodiments of the drawings, which is not described here.

[0076] Further, the modules / units integrated in the electronic device 1 are stored in a computer readable storage medium if realized in the form of software function units and sold or used as independent products. The computer readable storage medium can be volatile or non-volatile. For example, the computer readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (Read-Only Memory, ROM).

[0077] The application further provides a computer readable storage medium, which stores a computer program, and the computer program can realize the following when executed by a processor of an electronic device: render an image for a page design file to obtain a page design image, and acquire a real page image corresponding to the page design image; perform element target detection and element segmentation on the page design image to obtain a page design element atlas; perform multi-type element feature extraction and element feature fusion on the page design element atlas to obtain a design element feature set; extract a real element feature set from the real page image, and perform element matching on the design element feature set according to the real element feature set to obtain an error element feature set; acquire a page code corresponding to the real page image, perform element matching and error labeling on the page code according to the error element feature set to obtain a review detection result.

[0078] In several embodiments provided in the application, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and the division of the modules is merely a logical function division, and there can be another division manner in actual implementation.

[0079] The modules illustrated as separate components can or can not be physically separated, and the components illustrated as modules can or can not be physical units, i.e., can be located in one place, or can be distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0080] In addition, each functional module in each embodiment of the application can be integrated in one processing unit, or each unit can exist physically independently, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of hardware plus software function modules.

[0081] It will be obvious to a person skilled in the art that the application is not limited to the details of the above-described exemplary embodiments but can be implemented in other embodiments without departing from the scope of the application.

[0082] The embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the application being indicated by the appended claims rather than by the above description, all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. No single feature or combination of features should be considered essential unless expressly stated as such in the claims.

[0083] Embodiments of the present application can acquire and process related data based on artificial intelligence technology. Among them, artificial intelligence (Artificial Intelligence, AI for short) is to use digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. Theory, method, technology and application device.

[0084] In addition, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. The plurality of units or devices stated in the device embodiments can also be implemented by one unit or device through software or hardware. The words first, second, etc. are used to indicate names and not to indicate any particular order.

[0085] Finally, it should be pointed out that the above embodiments are only used to illustrate the technical solutions of the present application and are not limiting. Although the present application has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solutions of the present application can be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present application.

Claims

1. An automated walkthrough method for static UI pages, characterized in that: The method comprises: Performing image rendering on the page design file to obtain a page design image, and acquiring a real page image corresponding to the page design image; Performing element target detection and element segmentation on the page design image to obtain a page design element atlas; Performing multi-type element feature extraction and element feature fusion on the page design element atlas to obtain a design element feature set; Extracting a real element feature set from the real page image, and performing element matching on the design element feature set based on the real element feature set to obtain an error element feature set; A page code corresponding to the real page image is obtained, and element matching and error marking are performed on the page code according to the error element feature set to obtain a walkthrough detection result.

2. The method for automatically checking a static UI page according to claim 1, wherein: The image rendering of the page design file to obtain the page design image includes: Performing output format configuration and output attribute configuration on the page design file to obtain an output configuration file; Grouping the page design file by layers to obtain a design layer group; Initialize the rendering canvas according to the design layer group; Performing margin configuration and background configuration on the rendering canvas to obtain a configured rendering canvas; The design layer group is rendered according to the configured rendering canvas and the output configuration file to obtain a page design image.

3. The method for automatically checking a static UI page according to claim 1, wherein: The performing element target detection and element segmentation on the page design image to obtain a page design element atlas includes: Performing multi-scale image convolution on the page design image to obtain a design page graph feature set; Performing residual connection on the design page graph feature set to obtain a design residual graph feature set; Performing multi-scale feature fusion on the design residual graph feature set to obtain a fused design graph feature; Performing element target detection on the fused design features to obtain a page element detection frame set; The page design image is segmented according to the page element detection frame set to obtain a page design element atlas.

4. The method for automatically checking a static UI page according to claim 1, wherein: The extracting and fusing of multiple types of element features on the page design element atlas to obtain a design element feature set includes: Screening the page design element atlas for text elements to obtain a design text element atlas; Performing text recognition on the design text element atlas to obtain a text content set; Performing font recognition on the design text element atlas to obtain a text font set; generating an element text feature set according to the text content set and the text font set; extracting an element position feature set, an element color feature set, an element size feature set, and an element texture feature set from the page design element atlas respectively; A design element feature set is generated according to the element position feature set, the element color feature set, the element size feature set, the element text feature set, and the element texture feature set.

5. The method for automatically checking a static UI page according to claim 1, wherein: The extracting of the real element feature set from the real page image includes: Performing element target detection and element segmentation on the real page image to obtain a real element atlas of the page; Multi-type element feature extraction and element feature fusion are performed on the real element atlas of the page to obtain a real element feature set.

6. The method for automatically checking a static UI page according to claim 1, wherein: The performing element matching on the design element feature set according to the real element feature set to obtain the error element feature set includes: Selecting real element features in the real element feature set one by one as target real element features, and screening design element features corresponding to the target real element features from the design element feature set as target design element features; Calculating the characteristic distance between each target real element characteristic and the corresponding target design element characteristic; Determining whether each of the characteristic distances is greater than a preset distance threshold; All target design elements whose characteristic distance is greater than a preset distance threshold are screened out, and the error types are marked to obtain the error element feature set.

7. The method for automatically checking a static UI page according to claim 1, wherein: The performing element matching and error marking on the page code according to the error element feature set to obtain a walkthrough detection result includes: Performing structural analysis on the page code to obtain a page element structure; Filtering an error page element set from the page element structure according to the error element feature set; Matching the error style code set corresponding to the error page element set from the page code; Filtering an error type set from the error element feature set; Error marking is performed on the error pattern code set according to the error type set to obtain a walkthrough detection result.

8. An automatic walkthrough device for static UI pages, characterized in that: The device comprises: An image rendering module is used to render the page design file to obtain a page design image and acquire a real page image corresponding to the page design image; An element segmentation module is used to perform element target detection and element segmentation on the page design image to obtain a page design element atlas; A feature extraction module is used to extract and fuse multiple types of element features from the page design element atlas to obtain a design element feature set; An element matching module is used to extract a real element feature set from the real page image, and perform element matching on the design element feature set based on the real element feature set to obtain an error element feature set; The walkthrough and annotation module is used to obtain the page code corresponding to the real page image, perform element matching and error annotation on the page code according to the error element feature set, and obtain the walkthrough detection result.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the automatic walkthrough method for static UI pages according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for automatically checking a static UI page according to any one of claims 1 to 7 is implemented.

Images