Unlock instant, AI-driven research and patent intelligence for your innovation.

Image recognition method, device, medium and electronic equipment

An image recognition and image technology, applied in the computer field, can solve problems such as inability to cross-platform and efficient use of cross-platform

Active Publication Date: 2020-07-07
BEIJING BYTEDANCE NETWORK TECH CO LTD
View PDF14 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] In view of the shortcomings of some image extraction technologies that cannot be efficiently used across platforms, considering that terminal devices (such as mobile phones) have their screen capture functions, relatively speaking, it is possible to avoid the occurrence of inability to cross-platform, so the input object of this program adopts is a screenshot of the terminal device

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Image recognition method, device, medium and electronic equipment
  • Image recognition method, device, medium and electronic equipment
  • Image recognition method, device, medium and electronic equipment

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0069] combined with figure 1 In this embodiment, a cross-platform image is input, and the recognition of the UI area in the image is realized through corresponding steps, specifically:

[0070] Step S101, acquiring an image to be recognized. The image in this embodiment can be selected as a cross-platform image, that is, an image that may have multiple UI areas. Considering that terminal devices (such as mobile phones, computers, etc.) generally have its screen capture function, the image can be a screen capture image. Use the screen capture function of the terminal device to perform a screen capture operation on a screen with multiple user interfaces to obtain a cross-platform screenshot image, which contains one or more UI areas, and the screen capture image is used as an image to be recognized, from which the UI area is extracted Information, the UI area information includes UI area location and category information.

[0071] Step S102, performing hierarchical segmentati...

Embodiment 2

[0131] See attached Image 6 , the difference between this embodiment and embodiment 1 is that the first potential UI area is determined when the straight line is extracted after the image is input, and the second potential UI area is generated after the image is synchronously input into the deep learning training module, and the above-mentioned first potential UI area is combined with The second potential UI area is compared and analyzed to obtain accurate UI information.

[0132] With reference to Embodiment 1, steps S601-S602 are the same as Embodiment 1 and will not be repeated here. Next, step S603 is to identify straight line features on the segmented image to determine the first potential user interface area. This embodiment also uses the method based on image morphology in OpenCV to perform image segmentation operations, and uses the line segment detector LSD to extract straight line features, and then determines potential UI regions.

[0133] Step S604, performing tr...

Embodiment 3

[0136] Such as Figure 7 As shown, according to a specific embodiment of the present invention, on the other hand, the present invention provides an image recognition device, including:

[0137] The obtaining unit 701 is configured to obtain an image to be recognized. The image can be a screenshot image, and the screenshot function of the terminal device can be used to perform a screenshot operation on a screen with multiple user interfaces to obtain a cross-platform screenshot image. The screenshot image includes one or more UI areas, and the screenshot image is used as The image to be recognized is extracted from the UI area information, and the UI area information includes UI area position and category information.

[0138] The segmentation unit 702 is configured to perform hierarchical segmentation on the image. There are some techniques for hierarchical image segmentation, such as frequency segmentation, gray threshold segmentation, histogram segmentation, feature space...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The present disclosure provides an image recognition method, device, medium and electronic device, the method includes: acquiring an image to be recognized; performing hierarchical segmentation on the image; performing line feature recognition on the segmented image to determine potential and training and identifying the potential user interface area, so as to determine accurate user interface information. Using the image extraction technology of the present disclosure to identify user interface information in cross-platform pictures, inputting a screenshot of a terminal device (such as a mobile phone), the position information of the user interface area in the image can be accurately identified, and deep learning training can be used to target to get the category information of the user interface area.

Description

technical field [0001] The present invention relates to the field of computer technology, in particular to an image recognition method, device, medium and electronic equipment, in particular to a user interface recognition method, device, medium and electronic equipment. Background technique [0002] With the development of mobile Internet technology, especially the development of mobile Internet, image processing and recognition are becoming more and more intelligent, and people are becoming more and more accustomed to performing various operations on the mobile Internet. It is a visual platform for user operation, which can make the software more personalized and tasteful, and also make the operation of the software comfortable, simple and free, fully reflecting the positioning and characteristics of the software. Interface design is not a simple art painting, it needs to locate the user, use environment, use method and design for the end user, it is a purely scientific ar...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06K9/00G06K9/34G06K9/62
CPCG06V20/35G06V10/26G06V10/267G06F18/214
Inventor 张钊杨萍邵珊珊
Owner BEIJING BYTEDANCE NETWORK TECH CO LTD