Automatic detection and positioning method for application program user interface defects and electronic device

A user interface and application technology, applied in the computer field, can solve problems such as UI rendering errors, undetectable application user interface, and undisplayable pictures

Inactive Publication Date: 2020-10-20
INST OF SOFTWARE - CHINESE ACAD OF SCI
View PDF4 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Common application user interface defects are mainly divided into 5 categories: pictures cannot be displayed, components are blocked, text overlaps, the display is null, and the screen is blurred
Most of these defects are caused by compatibility issues. Although they will not cause the program to crash, UI rendering errors will greatly reduce the user experience
[0004] Most of the existing UI testing methods focus on code and UI coverage, and these application user interface defects will not cause problems such as program crashes, so it is difficult for t

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Automatic detection and positioning method for application program user interface defects and electronic device
  • Automatic detection and positioning method for application program user interface defects and electronic device
  • Automatic detection and positioning method for application program user interface defects and electronic device

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0035] The method will be further described through specific embodiments below in conjunction with the accompanying drawings.

[0036] The method process of the present invention is as figure 1 As shown, the specific steps are:

[0037] 1. Obtain the defect detection model

[0038] After preprocessing the training data, since a large amount of training data will improve the accuracy and universality of the model, the amount of existing labeled data is small, and it is necessary to expand the training data set through an automated method. A large number of bug-free application screenshots and their corresponding json files (related information about interface controls) were obtained from the existing public data sets. Input them together into the heuristic training data generator, and generate 5 kinds of applications containing bugs according to the characteristics of 5 types of application user interface defects (pictures cannot be displayed, components are blocked, text ove...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides an automatic detection and positioning method for defects of a user interface of an application program and an electronic device. The method comprises steps of zooming a to-be-detected user interface picture to a set size, and obtaining a zoomed picture; carrying out convolution on the zoom image for one or more times, and carrying out batch standardization on the mean valueand variance of each convolution result; pooling the batch standardization result, classifying the obtained pooled features, and judging whether the to-be-detected user interface picture contains defects and defect categories or not. The method is advantaged in that the feature information of defects of a user interface of an application program is obtained through automatic detection screenshotand modeling, the defects of the user interface are detected and positioned on the screenshot, detection accuracy is guaranteed, detection cost is reduced, the manual detection process is reduced, thespeed is high, adaptability is high, and the method can be suitable for various operating systems.

Description

technical field [0001] The invention belongs to the technical field of computers, and in particular relates to an automatic detection and positioning method and an electronic device for defects in an application program user interface. Background technique [0002] User Interface (UI) is the bridge for the interaction between the application and the user. A good UI design helps the user to use the application conveniently and efficiently, thereby improving the user experience of the application. UI design often requires proper user interaction, information architecture, and UI visual effects. At the same time, if there are UI bugs, it will greatly affect the user experience, resulting in the loss of users and economy. In the case of a relative shortage of professional testers in software companies, automated UI testing technology has been widely adopted by many software development companies. [0003] Automated testing tools mainly test various GUI components by simulating ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F11/36G06F9/451G06K9/62G06N3/04G06N3/08
CPCG06F11/3684G06F11/3688G06F9/451G06N3/08G06N3/047G06N3/045G06F18/24
Inventor 刘哲王俊杰黄悦凯胡军王青
Owner INST OF SOFTWARE - CHINESE ACAD OF SCI
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap