Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Software interface element detection method and system based on multi-granularity learning, computer equipment and storage medium

A software interface and detection method technology, applied in software testing/debugging, computing, error detection/correction, etc., can solve problems such as low efficiency, human factors, error-prone, etc., to reduce interference, ensure detection effectiveness, The effect of improving efficiency and accuracy

Active Publication Date: 2020-05-05
SHANGHAI YISAIQI SOFTWARE CO LTD
View PDF7 Cites 4 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

At present, it is mainly necessary to rely on testers to check whether the software interface is consistent with the software interface in the design drawing with the naked eye, which is inefficient and affected by human factors, and is prone to errors

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Software interface element detection method and system based on multi-granularity learning, computer equipment and storage medium
  • Software interface element detection method and system based on multi-granularity learning, computer equipment and storage medium
  • Software interface element detection method and system based on multi-granularity learning, computer equipment and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment example

[0079] Step (1), input the image of the software interface; input an image, the image is an RGB three-channel image, and the size is W*H.

[0080] Step (2), use the convolutional neural network to obtain the features of the image, and obtain an intermediate result of the same size; the intermediate result is an image of D*W*H.

[0081] Step (3), perform multi-granularity meshing operation on the features obtained in step (2); suppose the granularity size is (a 1 *b 1 ),(a 2 *b 2 ), (a 3 *b 3 )……(a n *b n ), then n groups of image grids of different sizes are obtained respectively. In this embodiment, assuming that the granularity is (4x4), (8x8), (16x16), three sets of grids can be obtained: the first set of 16 grids (4x4 equal parts of the original image), the second set of 64 grids, and the third group of 256 grids.

[0082] Step (4), use a multi-grain cascade background discriminator to calibrate the non-control background grid area of ​​the software interface; ass...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a software interface element detection and positioning method and system based on multi-granularity learning, computer equipment and a storage medium. The method comprises thesteps of: obtaining an operation interface image; inputting the image into a deep convolutional neural network to extract features to obtain the intermediate result of the same size; performing meshing division on the intermediate result; inputting the intermediate result into a multi-granularity cascade background discriminator, and carrying out background grid detection on the intermediate imageby a detector by taking a grid as a unit; after background grids are removed, partially inputting the remaining grids into a multi-granularity software interface element detection model for softwareinterface element detection, and outputting the type and the area size of a control to be detected. According to the multi-granularity cascaded background grid detection device provided by the invention, the background grid can be quickly and effectively found out, and the efficiency and accuracy of software interface element detection and positioning are improved.

Description

technical field [0001] The invention relates to a multi-granularity-based software interface element detection method, system, computer equipment and storage medium, and belongs to the field of automation software auxiliary technology. Background technique [0002] Automatic detection of software interface elements is the key technology of software intelligence. Software intelligence and some software auxiliary fields need to be able to automatically detect software interface elements. However, with the update iteration of software interface and the continuous enrichment of software interface element types, the complexity of automatic control detection is also increasing. [0003] In the process of software development and testing, it is also necessary to detect the software interface, that is, to detect whether the software interface is consistent with the software interface in the design drawing. At present, it is mainly necessary to rely on testers to check whether the s...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F11/36G06N3/04
CPCG06F11/3688G06N3/045Y02D10/00
Inventor 胡立军唐琦松林平吴鑫刘继明
Owner SHANGHAI YISAIQI SOFTWARE CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com