Interface processing method, terminal and computer readable storage medium

Abstract

The present invention discloses an interface processing method, a terminal and a computer-readable storage medium, the method comprising: capturing an image of the entire or partial area of ​​an application interface; analyzing the features contained in the image; identifying the information of the interface according to the features; and processing the interface according to the information of the interface. In the technical solution of the present invention, a screenshot of the application interface is taken, and the features of the current interface are identified from the screenshot. The information of the interface can be analyzed through the interface features, that is, the type of interface of the current interface can be identified. After identification, the current interface can be processed in a targeted manner.

Description

Technical Field

[0001] The present invention relates to the field of mobile terminals, and in particular to an interface processing method, a terminal and a computer-readable storage medium. Background Art

[0002] For smart devices such as bracelets, when adapting different interfaces of third-party applications to perform different processing operations or adjusting the UI display content of different interfaces, it is necessary to determine which interface of the application is currently started. However, the current Android system can only distinguish different interfaces by the package name and class name to which the interface belongs.

[0003] At present, some third-party applications layout different Views in the same interface (same package name and class name), and control the display content of the current interface through the visibility and invisibility of the View. In this way, it is impossible to locate the specific content displayed on the current interface through the package name and class name of the interface (such as some instant messaging applications).

[0004] Therefore, a new technical solution is needed that can identify changes in the current interface when the application interface changes and make corresponding adjustments to the interface. Summary of the invention

[0005] The main purpose of the present invention is to provide an interface processing method, a terminal and a computer-readable storage medium, which are intended to identify changes in the current interface and make corresponding adjustments to the interface.

[0006] To achieve the above-mentioned purpose, the present invention provides an interface processing method, including: capturing an image of all or part of an application interface; analyzing features contained in the image; identifying information of the interface based on the features; and processing the interface based on the information of the interface.

[0007] To achieve the above-mentioned purpose, the present invention provides a terminal, which includes a processor, a memory, and a communication bus; the communication bus is used to realize the connection and communication between the processor and the memory; the processor is used to execute the program stored in the memory to implement the steps of the above-mentioned method.

[0008] To achieve the above objectives, the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of the aforementioned method.

[0009] According to the above technical solutions, it can be seen that the interface processing method, terminal and computer-readable storage medium of the present invention have at least the following advantages:

[0010] In the technical solution of the present invention, a screenshot of the application interface is taken, and the features of the current interface are identified from the screenshot. The information of the interface can be analyzed through the interface features, that is, the type of interface the current interface is, and after identification, targeted processing can be performed on the current interface. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 A schematic diagram of the hardware structure of a mobile terminal for implementing various embodiments of the present invention;

[0012] Figure 2 For Figure 1 A schematic diagram of a wireless communication system of a mobile terminal shown;

[0013] Figure 3 is a flow chart of an interface processing method according to an embodiment of the present invention;

[0014] Figure 4 is a flow chart of an interface processing method according to an embodiment of the present invention;

[0015] Figure 5 is a schematic diagram of an interface processing method according to an embodiment of the present invention;

[0016] Figure 6 is a schematic diagram of an interface processing method according to an embodiment of the present invention;

[0017] Figure 7 is a flow chart of an interface processing method according to an embodiment of the present invention;

[0018] Figure 8 is a block diagram of a terminal according to an embodiment of the present invention.

[0019] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0020] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0021] In the subsequent description, the suffixes such as "module", "component" or "unit" used to represent elements are only used to facilitate the description of the present invention, and have no special meanings. Therefore, "module", "component" or "unit" can be used in a mixed manner.

[0022] The terminal may be implemented in various forms. For example, the terminal described in the present invention may include mobile terminals such as mobile phones, tablet computers, laptop computers, PDAs, portable media players (PMPs), navigation devices, wearable devices, smart bracelets, pedometers, etc., as well as fixed terminals such as digital TVs and desktop computers.

[0023] The following description will be made by taking a mobile terminal as an example, and those skilled in the art will appreciate that, in addition to components specifically used for mobile purposes, the configuration according to the embodiments of the present invention can also be applied to fixed-type terminals.

[0024] See also Figure 1 , which is a schematic diagram of the hardware structure of a mobile terminal for implementing various embodiments of the present invention, the mobile terminal 100 may include: RF (Radio Frequency, radio frequency) unit 101, WiFi module 102, audio output unit 103, A / V (audio / video) input unit 104, sensor 105, display unit 106, user input unit 107, interface unit 108, memory 109, processor 110, and power supply 111 and other components. Those skilled in the art can understand that Figure 1 The structure of the mobile terminal shown in the figure does not constitute a limitation on the mobile terminal. The mobile terminal may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0025] Combine the following Figure 1 The following is a detailed introduction to the various components of the mobile terminal:

[0026] The radio frequency unit 101 can be used for receiving and sending signals during information transmission or communication. Specifically, after receiving the downlink information of the base station, it is sent to the processor 110 for processing; in addition, the uplink data is sent to the base station. Generally, the radio frequency unit 101 includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, etc. In addition, the radio frequency unit 101 can also communicate with the network and other devices through wireless communication. The above-mentioned wireless communication can use any communication standard or protocol, including but not limited to GSM (Global System of Mobile communication), GPRS (General Packet Radio Service), CDMA2000 (Code Division Multiple Access 2000), WCDMA (Wideband Code Division Multiple Access), TD-SCDMA (Time Division-Synchronous Code Division Multiple Access), FDD-LTE (Frequency Division Duplexing-Long Term Evolution) and TDD-LTE (Time Division Duplexing-Long Term Evolution), etc.

[0027] WiFi is a short-range wireless transmission technology. Mobile terminals can help users send and receive emails, browse web pages, and access streaming media through WiFi module 102. It provides users with wireless broadband Internet access. Figure 1 The WiFi module 102 is shown, but it is understandable that it is not an essential component of the mobile terminal and can be omitted as needed without changing the essence of the invention.

[0028] The audio output unit 103 can convert the audio data received by the RF unit 101 or the WiFi module 102 or stored in the memory 109 into an audio signal and output it as sound when the mobile terminal 100 is in a call signal reception mode, a talk mode, a recording mode, a voice recognition mode, a broadcast reception mode, etc. Moreover, the audio output unit 103 can also provide audio output related to the specific functions performed by the mobile terminal 100 (for example, a call signal reception sound, a message reception sound, etc.). The audio output unit 103 may include a speaker, a buzzer, etc.

[0029] The A / V input unit 104 is used to receive audio or video signals. The A / V input unit 104 may include a graphics processor (GPU) 1041 and a microphone 1042, and the graphics processor 1041 processes the image data of a static picture or video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The processed image frame can be displayed on the display unit 106. The image frame processed by the graphics processor 1041 can be stored in the memory 109 (or other storage medium) or sent via the radio frequency unit 101 or the WiFi module 102. The microphone 1042 can receive sound (audio data) via the microphone 1042 in a telephone call mode, a recording mode, a voice recognition mode, and other operating modes, and can process such sound into audio data. The processed audio (voice) data can be converted into a format output that can be sent to a mobile communication base station via the radio frequency unit 101 in the case of a telephone call mode. The microphone 1042 can implement various types of noise elimination (or suppression) algorithms to eliminate (or suppress) noise or interference generated in the process of receiving and sending audio signals.

[0030] The mobile terminal 100 also includes at least one sensor 105, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor includes an ambient light sensor and a proximity sensor, wherein the ambient light sensor can adjust the brightness of the display panel 1061 according to the brightness of the ambient light, and the proximity sensor can turn off the display panel 1061 and / or the backlight when the mobile terminal 100 is moved to the ear. As a type of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in all directions (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used for applications that identify the posture of the mobile phone (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; as for other sensors that can also be configured on the mobile phone, such as fingerprint sensors, pressure sensors, iris sensors, molecular sensors, gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., they will not be repeated here.

[0031] The display unit 106 is used to display information input by the user or information provided to the user. The display unit 106 may include a display panel 1061, which may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.

[0032] The user input unit 107 can be used to receive input digital or character information, and to generate key signal input related to the user settings and function control of the mobile terminal. Specifically, the user input unit 107 may include a touch panel 1071 and other input devices 1072. The touch panel 1071, also known as a touch screen, can collect the user's touch operation on or near it (such as the user's operation on the touch panel 1071 or near the touch panel 1071 using any suitable object or accessory such as a finger, stylus, etc.), and drive the corresponding connection device according to a pre-set program. The touch panel 1071 may include two parts: a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into the touch point coordinates, and then sends it to the processor 110, and can receive and execute the command sent by the processor 110. In addition, the touch panel 1071 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic waves. In addition to the touch panel 1071, the user input unit 107 may also include other input devices 1072. Specifically, the other input devices 1072 may include, but are not limited to, one or more of a physical keyboard, a function key (such as a volume control key, a switch key, etc.), a trackball, a mouse, a joystick, etc., which are not specifically limited here.

[0033] Furthermore, the touch panel 1071 may cover the display panel 1061. When the touch panel 1071 detects a touch operation on or near it, it transmits the information to the processor 110 to determine the type of the touch event. Then, the processor 110 provides a corresponding visual output on the display panel 1061 according to the type of the touch event. Figure 1 In the figure, the touch panel 1071 and the display panel 1061 are used as two independent components to implement the input and output functions of the mobile terminal. However, in some embodiments, the touch panel 1071 and the display panel 1061 can be integrated to implement the input and output functions of the mobile terminal, which is not limited here.

[0034] The interface unit 108 serves as an interface through which at least one external device can be connected to the mobile terminal 100. For example, the external device may include a wired or wireless headset port, an external power supply (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device with an identification module, an audio input / output (I / O) port, a video I / O port, a headphone port, etc. The interface unit 108 may be used to receive input (e.g., data information, power, etc.) from an external device and transmit the received input to one or more elements within the mobile terminal 100 or may be used to transmit data between the mobile terminal 100 and an external device.

[0035] The memory 109 can be used to store software programs and various data. The memory 109 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory 109 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0036] The processor 110 is the control center of the mobile terminal. It uses various interfaces and lines to connect various parts of the entire mobile terminal. It executes various functions of the mobile terminal and processes data by running or executing software programs and / or modules stored in the memory 109, and calling data stored in the memory 109, so as to monitor the mobile terminal as a whole. The processor 110 may include one or more processing units; preferably, the processor 110 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, etc., and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 110.

[0037] The mobile terminal 100 may also include a power supply 111 (such as a battery) for supplying power to various components. Preferably, the power supply 111 may be logically connected to the processor 110 via a power management system, thereby implementing functions such as charging, discharging, and power consumption management through the power management system.

[0038] although Figure 1 Not shown, the mobile terminal 100 may further include a Bluetooth module, etc., which will not be described in detail here.

[0039] To facilitate understanding of the embodiments of the present invention, the communication network system on which the mobile terminal of the present invention is based is described below.

[0040] See also Figure 2 , Figure 2 A communication network system architecture diagram is provided for an embodiment of the present invention. The communication network system is an LTE system of universal mobile communication technology. The LTE system includes a UE (User Equipment) 201, an E-UTRAN (Evolved UMTS Terrestrial Radio Access Network) 202, an EPC (Evolved Packet Core) 203 and an operator's IP service 204, which are sequentially connected for communication.

[0041] Specifically, UE201 may be the above-mentioned terminal 100, which will not be described in detail here.

[0042] E-UTRAN 202 includes eNodeB 2021 and other eNodeBs 2022 , etc. Among them, eNodeB 2021 can be connected to other eNodeBs 2022 through a backhaul (eg, an X2 interface), and eNodeB 2021 is connected to EPC 203 , and eNodeB 2021 can provide UE 201 with access to EPC 203 .

[0043] EPC203 may include MME (Mobility Management Entity) 2031, HSS (Home Subscriber Server) 2032, other MMEs 2033, SGW (Serving Gate Way) 2034, PGW (PDN Gate Way) 2035 and PCRF (Policy and Charging Rules Function) 2036, etc. Among them, MME2031 is a control node that processes signaling between UE201 and EPC203, and provides bearer and connection management. HSS2032 is used to provide some registers to manage functions such as home location register (not shown in the figure), and save some user-specific information such as service features and data rates. All user data can be sent through SGW2034, PGW2035 can provide IP address allocation and other functions for UE 201, PCRF2036 is the policy and charging control policy decision point for service data flow and IP bearer resources, which selects and provides available policy and charging control decisions for the policy and charging execution function unit (not shown in the figure).

[0044] The IP service 204 may include the Internet, an intranet, an IMS (IP Multimedia Subsystem) or other IP services.

[0045] Although the above introduction takes the LTE system as an example, those skilled in the art should know that the present invention is not only applicable to the LTE system, but also to other wireless communication systems, such as GSM, CDMA2000, WCDMA, TD-SCDMA and future new network systems, etc., which are not limited here.

[0046] Based on the above mobile terminal hardware structure and communication network system, various embodiments of the method of the present invention are proposed.

[0047] like Figure 3 As shown, an embodiment of the present invention provides an interface processing method, comprising:

[0048] Step S310: capturing an image of all or part of the application interface.

[0049] In this embodiment, there is no restriction on the specific type of application, and the entire image of its interface can be captured, or a portion of it can be captured, for example, the upper left corner, lower right corner or central part of the current interface can be captured.

[0050] Step S320, analyzing features contained in the image.

[0051] In this embodiment, there is no restriction on the feature type of the image, and any data that can reflect the interface information can be used as the image feature, such as text, graphics, virtual operation components, etc. contained in the image.

[0052] Step S330, identifying the information of the interface according to the features.

[0053] In this embodiment, the information identifying the interface is to clarify which interface the interface is.

[0054] Step S340: Process the interface according to the information of the interface.

[0055] In this embodiment, there is no restriction on the way of interface processing. Usually, the interface is processed to make it more adaptable to the current software and hardware environment. For example, the screen shape of smart devices such as bracelets is relatively special, and some application interfaces need to be adjusted to a shape suitable for use on the bracelet screen.

[0056] According to the technical solution of this embodiment, a screenshot of the application interface is taken, and the features of the current interface are identified from the screenshot. The information of the interface can be analyzed through the interface features, that is, the type of interface the current interface is, and after identification, targeted processing can be performed on the current interface.

[0057] like Figure 4 As shown, an embodiment of the present invention provides an interface processing method, comprising:

[0058] Step S410, when a change in the view of the interface is detected, an image is captured from the interface at a preset position.

[0059] In this embodiment, since the features on the interface (e.g., buttons or options) are usually located in a fixed position such as the upper left corner or the upper right corner of the interface, a screenshot can be taken and the features can be identified according to the fixed position. In this embodiment, the feature patterns of each interface of the instant messaging tool application are pre-learned and put into the local feature pattern library and the feature pattern position. For example, the feature pattern A on the homepage of the instant messaging tool, Figure 5 (The upper right corner box area is the characteristic position, and the button is the characteristic pattern); Instant messaging tool chat interface pattern B, such as Figure 6 (The square area in the upper right corner is the characteristic position, and the button in another form is the characteristic pattern.) During use, when the view of the current activity interface changes, it can be determined that the current interface has changed. At this time, according to the previously learned characteristic pattern position, the image S is intercepted at this position.

[0060] Step S420: identifying the content contained in the image and using the content as a feature of the image.

[0061] In this embodiment, the text or graphic content contained in the image can be analyzed based on the graphic recognition technology to serve as the image feature.

[0062] Step S430, identifying information of the interface according to the features.

[0063] In this embodiment, the captured feature image S is identified and matched with a pre-learned feature pattern library; if a feature pattern X is matched in the feature pattern, it can be determined that the current interface is the interface represented by the feature pattern X.

[0064] Step S440: adjusting the layout of the interface according to the information of the interface.

[0065] In this embodiment, different operation processes are performed according to the determined current interface, and the layout is adjusted mainly to adapt to the screen shape of smart devices such as bracelets.

[0066] According to the technical solution of this embodiment, a method based on image recognition technology is provided, which learns and acquires the feature points of different interfaces in advance by monitoring (for example, instant messaging tools have different icon shapes at the same position in the upper right corner of different interfaces), identifies and distinguishes different interfaces based on the acquired feature points, and performs corresponding interface processing. This solves the problem that an application with only one activity cannot locate the current interface through the package name and class name of the interface, and improves the accuracy and convenience of third-party application adaptation on the wristband device.

[0067] like Figure 7 As shown, an embodiment of the present invention provides an interface processing method, comprising:

[0068] Step S710, when an instruction of a change in the user input control interface is detected, an area in the image that meets a preset condition is identified and the image is captured.

[0069] In this embodiment, when the user inputs certain instructions, causing the current interface to change, the user's instruction input can be used as a triggering event for screenshots, so that screenshots are taken in time when the interface is updated and subsequent judgments are made. In this embodiment, there is no restriction on the preset conditions, for example, screenshots are taken for areas with significant color contrast, or for areas with specific text or patterns, which usually contain various features of the image.

[0070] Step S720: analyzing features contained in the image.

[0071] Step S730, identifying the information of the interface according to the features.

[0072] Step S740: adding operating components to the interface according to the information of the interface.

[0073] In this embodiment, the purpose of adding the operating component is to optimize the current interface function, and the user can operate the interface more conveniently through the operating component.

[0074] According to the technical solution of this embodiment, a technology based on image recognition will be provided, which can learn through listening and pre-acquire the feature points of different interfaces (such as instant messaging tools will have different icon shapes at the same position in the upper right corner of different interfaces), identify and distinguish different interfaces based on the acquired feature points, and perform corresponding interface processing.

[0075] like Figure 8 As shown, in one embodiment of the present invention, a terminal is provided, the terminal includes a processor 810, a memory 820, and a communication bus 830; the communication bus 830 is used to realize the connection and communication between the processor 810 and the memory 820; the processor 810 is used to execute the program stored in the memory 820 to implement the following steps:

[0076] Capture an image of all or part of the application's interface.

[0077] In this embodiment, there is no restriction on the specific type of application, and the entire image of its interface can be captured, or a portion of it can be captured, for example, the upper left corner, lower right corner or central part of the current interface can be captured.

[0078] Analyze the features contained in the image.

[0079] In this embodiment, there is no restriction on the feature type of the image, and any data that can reflect the interface information can be used as the image feature, such as text, graphics, virtual operation components, etc. contained in the image.

[0080] Information of the interface is identified based on features.

[0081] In this embodiment, the information identifying the interface is to clarify which interface the interface is.

[0082] Process the interface according to the information on the interface.

[0083] In this embodiment, there is no restriction on the way of interface processing. Usually, the interface is processed to make it more adaptable to the current software and hardware environment. For example, the screen shape of smart devices such as bracelets is relatively special, and some application interfaces need to be adjusted to a shape suitable for use on the bracelet screen.

[0084] According to the technical solution of this embodiment, a screenshot of the application interface is taken, and the features of the current interface are identified from the screenshot. The information of the interface can be analyzed through the interface features, that is, the type of interface the current interface is, and after identification, targeted processing can be performed on the current interface.

[0085] like Figure 8 As shown, in one embodiment of the present invention, a terminal is provided, the terminal includes a processor 810, a memory 820, and a communication bus 830; the communication bus 830 is used to realize the connection and communication between the processor 810 and the memory 820; the processor 810 is used to execute the program stored in the memory 820 to implement the following steps:

[0086] When a change in the view of the interface is detected, an image is captured from the interface at a preset position.

[0087] In this embodiment, since the features on the interface (e.g., buttons or options) are usually located in a fixed position such as the upper left corner or the upper right corner of the interface, a screenshot can be taken and the features can be identified according to the fixed position. In this embodiment, the feature patterns of each interface of the instant messaging tool application are pre-learned and put into the local feature pattern library and the feature pattern position. For example, the feature pattern A on the homepage of the instant messaging tool, Figure 5 (The upper right corner box area is the characteristic position, and the button is the characteristic pattern); Instant messaging tool chat interface pattern B, such as Figure 6 (The square area in the upper right corner is the characteristic position, and the button in another form is the characteristic pattern.) During use, when the view of the current activity interface changes, it can be determined that the current interface has changed. At this time, according to the previously learned characteristic pattern position, the image S is intercepted at this position.

[0088] Identify the content contained in the image and use the content as the feature of the image.

[0089] In this embodiment, the text or graphic content contained in the image can be analyzed based on the graphic recognition technology to serve as the image feature.

[0090] Information of the interface is identified based on features.

[0091] In this embodiment, the captured feature image S is identified and matched with a pre-learned feature pattern library; if a feature pattern X is matched in the feature pattern, it can be determined that the current interface is the interface represented by the feature pattern X.

[0092] Adjust the layout of the interface according to the information on the interface.

[0093] In this embodiment, different operation processes are performed according to the determined current interface, and the layout is adjusted mainly to adapt to the screen shape of smart devices such as bracelets.

[0094] According to the technical solution of this embodiment, a method based on image recognition technology is provided, which learns and acquires the feature points of different interfaces in advance by monitoring (for example, instant messaging tools have different icon shapes at the same position in the upper right corner of different interfaces), identifies and distinguishes different interfaces based on the acquired feature points, and performs corresponding interface processing. This solves the problem that an application with only one activity cannot locate the current interface through the package name and class name of the interface, and improves the accuracy and convenience of third-party application adaptation on the wristband device.

[0095] like Figure 8 As shown, in one embodiment of the present invention, a terminal is provided, the terminal includes a processor 810, a memory 820, and a communication bus 830; the communication bus 830 is used to realize the connection and communication between the processor 810 and the memory 820; the processor 810 is used to execute the program stored in the memory 820 to implement the following steps:

[0096] When an instruction of a change in the user input control interface is detected, an area in the image that meets the preset conditions is identified and the image is captured.

[0097] In this embodiment, when the user inputs certain instructions, causing the current interface to change, the user's instruction input can be used as a triggering event for screenshots, so that screenshots are taken in time when the interface is updated and subsequent judgments are made. In this embodiment, there is no restriction on the preset conditions, for example, screenshots are taken for areas with significant color contrast, or for areas with specific text or patterns, which usually contain various features of the image.

[0098] Analyze the features contained in the image.

[0099] Information of the interface is identified based on features.

[0100] Add operation components to the interface according to the information on the interface.

[0101] In this embodiment, the purpose of adding the operating component is to optimize the current interface function, and the user can operate the interface more conveniently through the operating component.

[0102] According to the technical solution of this embodiment, a technology based on image recognition will be provided, which can learn through listening and pre-acquire the feature points of different interfaces (such as instant messaging tools will have different icon shapes at the same position in the upper right corner of different interfaces), identify and distinguish different interfaces based on the acquired feature points, and perform corresponding interface processing.

[0103] In one embodiment of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the following steps:

[0104] Capture an image of all or part of the application's interface.

[0105] In this embodiment, there is no restriction on the specific type of application, and the entire image of its interface can be captured, or a portion of it can be captured, for example, the upper left corner, lower right corner or central part of the current interface can be captured.

[0106] Analyze the features contained in the image.

[0107] In this embodiment, there is no restriction on the feature type of the image, and any data that can reflect the interface information can be used as the image feature, such as text, graphics, virtual operation components, etc. contained in the image.

[0108] Information of the interface is identified based on features.

[0109] In this embodiment, the information identifying the interface is to clarify which interface the interface is.

[0110] Process the interface according to the information on the interface.

[0111] In this embodiment, there is no restriction on the way of interface processing. Usually, the interface is processed to make it more adaptable to the current software and hardware environment. For example, the screen shape of smart devices such as bracelets is relatively special, and some application interfaces need to be adjusted to a shape suitable for use on the bracelet screen.

[0112] According to the technical solution of this embodiment, a screenshot of the application interface is taken, and the features of the current interface are identified from the screenshot. The information of the interface can be analyzed through the interface features, that is, the type of interface the current interface is, and after identification, targeted processing can be performed on the current interface.

[0113] In one embodiment of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the following steps:

[0114] When a change in the view of the interface is detected, an image is captured from the interface at a preset position.

[0115] In this embodiment, since the features on the interface (e.g., buttons or options) are usually located in a fixed position such as the upper left corner or the upper right corner of the interface, a screenshot can be taken and the features can be identified according to the fixed position. In this embodiment, the feature patterns of each interface of the instant messaging tool application are pre-learned and put into the local feature pattern library and the feature pattern position. For example, the feature pattern A on the homepage of the instant messaging tool, Figure 5 (The upper right corner box area is the characteristic position, and the button is the characteristic pattern); Instant messaging tool chat interface pattern B, such as Figure 6 (The square area in the upper right corner is the characteristic position, and the button in another form is the characteristic pattern.) During use, when the view of the current activity interface changes, it can be determined that the current interface has changed. At this time, according to the previously learned characteristic pattern position, the image S is intercepted at this position.

[0116] Identify the content contained in the image and use the content as the feature of the image.

[0117] In this embodiment, the text or graphic content contained in the image can be analyzed based on the graphic recognition technology to serve as the image feature.

[0118] Information of the interface is identified based on features.

[0119] In this embodiment, the captured feature image S is identified and matched with a pre-learned feature pattern library; if a feature pattern X is matched in the feature pattern, it can be determined that the current interface is the interface represented by the feature pattern X.

[0120] Adjust the layout of the interface according to the information on the interface.

[0121] In this embodiment, different operation processes are performed according to the determined current interface, and the layout is adjusted mainly to adapt to the screen shape of smart devices such as bracelets.

[0122] According to the technical solution of this embodiment, a method based on image recognition technology is provided, which learns and acquires the feature points of different interfaces in advance by monitoring (for example, instant messaging tools have different icon shapes at the same position in the upper right corner of different interfaces), identifies and distinguishes different interfaces based on the acquired feature points, and performs corresponding interface processing. This solves the problem that an application with only one activity cannot locate the current interface through the package name and class name of the interface, and improves the accuracy and convenience of third-party application adaptation on the wristband device.

[0123] In one embodiment of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the following steps:

[0124] When an instruction of a change in the user input control interface is detected, an area in the image that meets the preset conditions is identified and the image is captured.

[0125] In this embodiment, when the user inputs certain instructions, causing the current interface to change, the user's instruction input can be used as a triggering event for screenshots, so that screenshots are taken in time when the interface is updated and subsequent judgments are made. In this embodiment, there is no restriction on the preset conditions, for example, screenshots are taken for areas with significant color contrast, or for areas with specific text or patterns, which usually contain various features of the image.

[0126] Analyze the features contained in the image.

[0127] Information of the interface is identified based on features.

[0128] Add operation components to the interface according to the information on the interface.

[0129] In this embodiment, the purpose of adding the operating component is to optimize the current interface function, and the user can operate the interface more conveniently through the operating component.

[0130] According to the technical solution of this embodiment, a technology based on image recognition will be provided, which can learn through listening and pre-acquire the feature points of different interfaces (such as instant messaging tools will have different icon shapes at the same position in the upper right corner of different interfaces), identify and distinguish different interfaces based on the acquired feature points, and perform corresponding interface processing.

[0131] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0132] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0133] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0134] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation modes, which are merely illustrative rather than restrictive. Under the guidance of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the present invention and the claims, all of which are within the protection of the present invention.

Claims

1. An interface processing method, characterized in that: include: Pre-learn the characteristic patterns of each interface and put them into the local characteristic pattern library; Capture images of all or part of the application interface; analyzing features contained in the image; Identify information of the interface according to the feature, specifically, match the feature with a local feature pattern library, and identify which interface of the application the current interface is according to the matching result; Processing the interface according to the information of the interface to make it more adaptable to the current software and hardware environment; The capturing of an image of all or part of an application interface includes: intercepting the image from the interface at a preset position, or identifying an area in the image that meets preset conditions and intercepting the image; The analyzing the features contained in the image includes: Identify content contained in the image and use the content as a feature of the image, wherein the content includes text, graphics or virtual operation components contained in the image; The processing of the interface according to the information of the interface includes: According to the information of the interface, the layout of the interface is adjusted to adapt to the screen shape of the smart device or operating components are added to the interface to facilitate the operation of the interface.

2. The method according to claim 1, characterized in that The capturing of an image of all or part of an application interface includes: When a change in the view of the interface is detected, the image is captured.

3. The method according to claim 1, characterized in that The capturing of an image of all or part of an application interface includes: When it is detected that the user inputs an instruction to control the interface to change, the image is captured.

4. A terminal, characterized in that: The terminal includes a processor, a memory, and a communication bus; the communication bus is used to realize connection and communication between the processor and the memory; the processor is used to execute the program stored in the memory to implement the steps of any one of the methods of claims 1 to 3.

5. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of any one of claims 1 to 3.

Images