Automatic segmentation screenshot method and system for interface element

A technology of automatic segmentation and interface elements, applied in the electronic field, can solve the problems of high time cost, low efficiency, manual completion, etc., and achieve the effect of improving accuracy, reducing resource consumption, and reducing the generation of dirty data

Active Publication Date: 2020-01-24
深圳市比量科技传媒有限公司
View PDF7 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0008] The present invention provides a method and system for automatically segmenting screenshots of interface elements, which are used to solve such problems as screenshotting, segmenting and identifying pictures in the prior art. The process needs to be completed manually, the time cost is high, and the efficiency is low

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Automatic segmentation screenshot method and system for interface element
  • Automatic segmentation screenshot method and system for interface element
  • Automatic segmentation screenshot method and system for interface element

Examples

Experimental program
Comparison scheme
Effect test

example 1

[0068] First, if image 3 Shown is the system architecture corresponding to the method of the present invention. Based on the system architecture and specific implementation methods, the specific implementation principles of Example 1 are as follows:

[0069] Step 1: Open the AccessibilityService function in the Android phone

[0070] Step 2: Open Sina Weibo, and open any Weibo details, OCR image recognition system, call the "Start Screenshot Segmentation" method provided by the screenshot segmentation API, and pass in the control id that needs to be segmented (Sina Weibo The content id of the message is: com.sina.weibo:id / tweet_message).

[0071] Step 3: The screenshot segmentation API starts to automatically capture the screen, and at the same time uses the programming interface in the AccessibilityService to save the snapshot information of the content control on the microblog details interface.

[0072] Step 4: The screenshot segmentation API obtains the snapshot informa...

example 2

[0078] Step 1: Open the AccessibilityService function in the Android phone.

[0079] Step 2: The RPA system automatically opens Sina Weibo, and automatically opens the details of the Weibo that needs to check and accept the operation task. At the same time, it automatically calls the "Start screenshot and segment" method of the screenshot segment API, and passes in the comment user nickname id of Sina Weibo ( com.sina.weibo:id / tvItemCmtNickname), comment content id(com.sina.weibo:id / tvItemCmtContent).

[0080] Step 3: The screenshot segmentation API starts to automatically capture the screen, and at the same time uses the programming interface in the AccessibilityService to save the snapshot information of the content control on the microblog details interface.

[0081] Step 4: The screenshot segmentation API obtains the snapshot information according to the id of the above-mentioned incoming content control, and at the same time combines the screenshot image to crop the image...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses an automatic segmentation screenshot and system for interface elements, and the method comprises the steps: obtaining the recognition parameters of at least one control when anauxiliary service is started, executing the screenshot segmentation of a screen, obtaining a screenshot picture, and obtaining the information of all controls in the screen; according to the identification parameters of the controls, obtaining snapshot information of the corresponding controls; cutting an image of the control range according to the screenshot picture; and obtaining a segmented result set according to the snapshot information and the image, and sending the result set to a calling party. The content is identified by using the screenshot segmentation identification mode, so thatthe identification accuracy is qualitatively improved compared with the common identification accuracy, the generation of dirty data is reduced, the identification efficiency in the AI system is improved, and the resource consumption of the music AI service is reduced.

Description

technical field [0001] The present application relates to the field of electronic technology, and in particular to a method and system for automatically segmenting screenshots of interface elements. Background technique [0002] At present, with the popularization of artificial intelligence, the application of image recognition technology is becoming more and more extensive. We can extract text from images, recognize license plates, identify illegal content, and more. However, the accuracy of the current AI recognition technology is relatively dependent on the image source. For example, OCR text recognition cannot accurately extract a certain part of the text in the entire screenshot, but will recognize the text on the entire screen, resulting in unusable results. At the same time, the entire picture recognition will inevitably reduce the accuracy of AI recognition. [0003] At present, the AI ​​processing method for slicing and segmenting pictures has the following disadv...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & AuthorityApplications(China)
IPC IPC(8): G06F9/451G06F9/54G06T7/11
CPCG06F9/451G06F9/543G06T7/11G06T2207/20132
Inventor陈建维万里黄娜
Owner深圳市比量科技传媒有限公司