Unlock instant, AI-driven research and patent intelligence for your innovation.

Image recognition method and device, medium and electronic equipment

An image recognition and image technology, applied in the computer field, can solve problems such as inability to cross-platform and efficient use of cross-platform

Active Publication Date: 2019-07-26
BEIJING BYTEDANCE NETWORK TECH CO LTD
View PDF14 Cites 11 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] In view of the shortcomings of some image extraction technologies that cannot be efficiently used across platforms, considering that terminal devices (such as mobile phones) have their screen capture functions, relatively speaking, it is possible to avoid the occurrence of inability to cross-platform, so the input object of this program adopts is a screenshot of the terminal device

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Image recognition method and device, medium and electronic equipment
  • Image recognition method and device, medium and electronic equipment
  • Image recognition method and device, medium and electronic equipment

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0069] combined with figure 1 In this embodiment, a cross-platform image is input, and the recognition of the UI area in the image is realized through corresponding steps, specifically:

[0070] Step S101, acquiring an image to be recognized. The image in this embodiment can be selected as a cross-platform image, that is, an image that may have multiple UI areas. Considering that terminal devices (such as mobile phones, computers, etc.) generally have its screen capture function, the image can be a screen capture image. Use the screen capture function of the terminal device to perform a screen capture operation on a screen with multiple user interfaces to obtain a cross-platform screenshot image, which contains one or more UI areas, and the screen capture image is used as an image to be recognized, from which the UI area is extracted Information, the UI area information includes UI area location and category information.

[0071] Step S102, performing hierarchical segmentati...

Embodiment 2

[0131] See attached Figure 6 , the difference between this embodiment and embodiment 1 is that the first potential UI area is determined when the straight line is extracted after the image is input, and the second potential UI area is generated after the image is synchronously input into the deep learning training module, and the above-mentioned first potential UI area is combined with The second potential UI area is compared and analyzed to obtain accurate UI information.

[0132] With reference to Embodiment 1, steps S601-S602 are the same as Embodiment 1 and will not be repeated here. Next, step S603 is to identify straight line features on the segmented image to determine the first potential user interface area. This embodiment also uses the method based on image morphology in OpenCV to perform image segmentation operations, and uses the line segment detector LSD to extract straight line features, and then determines potential UI regions.

[0133] Step S604, performing t...

Embodiment 3

[0136] Such as Figure 7 As shown, according to a specific embodiment of the present invention, on the other hand, the present invention provides an image recognition device, including:

[0137] The obtaining unit 701 is configured to obtain an image to be recognized. The image can be a screenshot image, and the screenshot function of the terminal device can be used to perform a screenshot operation on a screen with multiple user interfaces to obtain a cross-platform screenshot image. The screenshot image includes one or more UI areas, and the screenshot image is used as The image to be recognized is extracted from the UI area information, and the UI area information includes UI area position and category information.

[0138] The segmentation unit 702 is configured to perform hierarchical segmentation on the image. There are some techniques for hierarchical image segmentation, such as frequency segmentation, gray threshold segmentation, histogram segmentation, feature space...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention provides an image recognition method and device, a medium and electronic equipment. The method comprises the steps of acquiring a to-be-recognized image; performing hierarchical segmentation on a image; performing linear feature recognition on a segmented image, and determining a potential user interface area; and performing training recognition on a potential user interface area, thereby determining accurate user interface information. According to the image extraction technology disclosed by the invention, the user interface information in a cross-platform image is identified.A screenshot image of the terminal equipment (such as a mobile phone) is input. The position information of a user interface region in the image can be accurately identified, and the category information of the user interface region can be obtained in a targeted manner through deep learning training.

Description

technical field [0001] The present invention relates to the field of computer technology, in particular to an image recognition method, device, medium and electronic equipment, in particular to a user interface recognition method, device, medium and electronic equipment. Background technique [0002] With the development of mobile Internet technology, especially the development of mobile Internet, image processing and recognition are becoming more and more intelligent, and people are becoming more and more accustomed to performing various operations on the mobile Internet. It is a visual platform for user operation, which can make the software more personalized and tasteful, and also make the operation of the software comfortable, simple and free, fully reflecting the positioning and characteristics of the software. Interface design is not a simple art painting, it needs to locate the user, use environment, use method and design for the end user, it is a purely scientific ar...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06K9/00G06K9/34G06K9/62
CPCG06V20/35G06V10/26G06V10/267G06F18/214
Inventor 张钊杨萍邵珊珊
Owner BEIJING BYTEDANCE NETWORK TECH CO LTD