Method for detecting page service and electronic equipment
By analyzing the visible text content in the user interface, identifying the services currently used by the user and recommending relevant information, the problem of impossible to accurately identify services in the prior art is solved, and the accuracy and efficiency of recommendations are improved.
Patent Information
- Application Number
- CN202310638149.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-31
- Publication Date
- 2024-12-06
AI Technical Summary
The prior art is difficult to accurately identify services supported by the user interface, resulting in the inability to accurately recommend relevant information.
By analyzing the visible text content in the interface, the service currently used by the user is determined, thereby inferring and recommending relevant information. Specific methods include obtaining text content in the interface, filtering nodes, using depth-first traversal and feature information matching, integrating feature information of multiple services, reducing resource consumption and improving efficiency.
It realizes accurate identification and recommendation of the current service users use, saves user search time and improves user experience.
Smart Images

Figure CN119088484A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of information technology, and in particular to a method and electronic device for detecting page services. Background Art
[0002] Electronic devices can identify services supported by a user interface and recommend relevant information to users based on the identified services. However, with the development of applications, a user interface can support multiple services. How to determine the services that the current user may use or the user is trying to use is an urgent problem to be solved in this field. Summary of the invention
[0003] The present application provides a method and electronic device for detecting page services, which can determine the services that the current user may use or attempt to use by determining the visible text in the current interface. In this way, the service currently used by the user can be inferred, and relevant information can be recommended more accurately for the service currently used by the user.
[0004] In a first aspect, the present application provides a method for detecting page services, the method comprising: an electronic device displays a first sub-interface of a first interface. The electronic device determines a service to be matched from all services supported by the electronic device, and the service to be matched includes services supported by all sub-interfaces contained in the first interface. The electronic device obtains text content of the first sub-interface. The electronic device determines the service provided by the first sub-interface from the services to be matched based on the text content of the first sub-interface.
[0005] After implementing the method provided in the present application, when the first interface supports multiple services, the services provided by the current first interface can be determined, wherein the text content of the first sub-interface includes the visible text content in the first interface.
[0006] In conjunction with the first aspect, in some implementations, before the electronic device displays the first sub-interface of the first interface, the method further includes: the electronic device obtains first characteristic information of each service among all services supported by the electronic device, the first characteristic information including the activity name of the user interface where the service is located. The to-be-matched service includes: among all services supported by the electronic device, the service whose activity name in the user interface is the activity name of the first interface.
[0007] In some implementations, the electronic device may obtain feature information of supported services stored in a cloud server after being powered on.
[0008] In this way, the electronic device can be pre-set with the characteristic information of the supported services. In this way, the electronic device can reduce the number of multiple services to be matched with the service currently used by the user in the subsequent process, saving resources and time, and being more efficient.
[0009] In combination with the first aspect, in some implementations, the electronic device obtains the text content of the first sub-interface, specifically including: the electronic device traverses the root node view of the first interface and obtains the text content of all sub-interfaces contained in the first interface. The electronic device determines the text content of the first sub-interface from the text content of all sub-interfaces contained in the first interface. The text content of the first sub-interface includes the visible text content in the first interface.
[0010] In combination with the first aspect, in some implementations, before the electronic device displays a sub-interface of the first interface, the method further includes: the electronic device obtains second characteristic information of each service among all services supported by the electronic device, the second characteristic information includes a first method, the first method is a filtering node method, and the first method indicates a filtering node and a sub-node of the filtering node. The electronic device traverses the root node view of the first interface, specifically including: the electronic device traverses the root node view of the first interface in a manner of filtering out the filtering node indicated by the first method and the sub-nodes of the filtering node.
[0011] In some implementations, the electronic device may use a depth-first search (DFS) method to traverse the root node view in the first interface.
[0012] In this way, it is not necessary to obtain the text content of all controls in the root node view, which can reduce the number of subsequent methods of obtaining text content, save time, and improve processing efficiency.
[0013] In combination with the first aspect, in some implementations, before the electronic device displays a sub-interface of the first interface, the method also includes: the electronic device obtains second characteristic information of each service among all services supported by the electronic device, the second characteristic information includes a second method, and the second method includes: a method for obtaining text content and a usage order; the electronic device obtains the text content of all sub-interfaces contained in the first interface, specifically including: the electronic device uses the second method to obtain the text content of all sub-interfaces contained in the first interface.
[0014] In some implementations, the method for obtaining text content specifically includes: view.getTooltipText() method; view.getContentDescription() method; getText() method; obtaining the value of a field whose attribute is a string through the declaredFields of the view; and a parsing method for the webview type.
[0015] In this way, the above method can be used to ensure that the text content in the first interface can be obtained.
[0016] In combination with the first aspect, in some implementations, before the electronic device displays the sub-interface of the first interface, the method also includes: the electronic device obtains second characteristic information of each service among all services supported by the electronic device, the second characteristic information includes a third method, the third method includes: whether to use the getLocalVisibleRect function and the text content of the getLocalVisibleRect function for the node; the electronic device determines the text content of the first sub-interface from the text content of all sub-interfaces contained in the first interface, specifically including: the electronic device uses the third method to determine the text content of the first sub-interface from the text content of all sub-interfaces contained in the first interface.
[0017] In some implementations, since the electronic device screens out multiple services to be matched, the electronic device can integrate the second feature information of the multiple services to be matched. Specifically, the electronic device can perform relevant processing such as intersection, union, and / or on the fields in each method of the second feature information of the multiple services to be matched.
[0018] In this way, it is not necessary to detect the visibility of all the text contents of the first interface, which can reduce the number of visibility detections, save time, and improve processing efficiency. In addition, the visible text contents obtained can be used in subsequent content to determine the services provided by the first sub-interface.
[0019] In combination with the first aspect, in some implementations, before the electronic device displays the sub-interface of the first interface, the method also includes: the electronic device obtains third characteristic information of each service among all services supported by the electronic device, and the third characteristic information includes text content that distinguishes the service from other services; the electronic device determines, from the services to be matched, that the service provided by the first sub-interface specifically includes: the electronic device determines, from the services to be matched, a service that contains text content that distinguishes it from other services and is the text content of the first sub-interface as the service provided by the first sub-interface.
[0020] In this way, the services provided by the first sub-interface can be determined based on the visible text content.
[0021] In combination with the first aspect, in some implementations, before the electronic device displays a sub-interface of the first interface, the method further includes: the electronic device obtains third characteristic information of each service among all services supported by the electronic device, and the third characteristic information includes: text content that distinguishes the service from other services, and confidence. The electronic device determines the service provided by the first sub-interface from the services to be matched based on the text content of the first sub-interface, specifically including: the electronic device determines the service provided by the first sub-interface as the service provided by the first sub-interface, which contains text content that distinguishes it from other services and the text content of the first sub-interface that meets the corresponding confidence.
[0022] In this way, the method for detecting page services provided in the embodiment of the present application can be applied when the versions provided by the application are different and the users use different languages.
[0023] In conjunction with the first aspect, in some implementations, after the electronic device determines, from the to-be-matched services, the service provided by the first sub-interface according to the text content of the first sub-interface, the method further includes:
[0024] The electronic device recommends first information related to the service provided by the first sub-interface.
[0025] In this way, the electronic device can determine the service currently used by the user and recommend information related to the service, which can save the user's time and energy in searching and finding information related to the service, making it more convenient for the user to use the electronic device.
[0026] In a second aspect, the present application provides an electronic device, comprising one or more processors and one or more memories. The one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program codes, and the computer program codes include computer instructions, and when the one or more processors execute the computer instructions, the electronic device executes the method of the first aspect or any implementation of the first aspect.
[0027] In a third aspect, an embodiment of the present application provides a chip, comprising one or more processors. When the one or more processors execute computer instructions, the electronic device executes the method of the first aspect or any implementation of the first aspect.
[0028] In a fourth aspect, an embodiment of the present application provides a computer storage medium, including computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes the method of the above-mentioned first aspect or any implementation method of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS
[0029] Figure 1A-Figure 1C Various user interfaces displayed by an electronic device provided in an implementation example of the present application.
[0030] Figure 2 A flowchart of a method for obtaining text content provided for an implementation example of the present application.
[0031] Figure 3 A schematic diagram of a node tree provided in an embodiment of the present application.
[0032] Figure 4 A schematic diagram of the software architecture of an electronic device provided in an embodiment of the present application.
[0033] Figure 5 A schematic diagram of the hardware architecture of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0034] The technical solutions in the embodiments of the present application will be described clearly and in detail below in conjunction with the accompanying drawings. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in the text is only a description of the association relationship of the associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.
[0035] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features, and in the description of the embodiments of the present application, unless otherwise specified, "plurality" means two or more.
[0036] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments.
[0037] The term "user interface (UI)" in the following embodiments of the present application refers to a medium interface for interaction and information exchange between an application or operating system and a user, which realizes the conversion between the internal form of information and the form acceptable to the user. The user interface is a source code written in a specific computer language such as Java and extensible markup language (XML). The interface source code is parsed and rendered on an electronic device, and finally presented as content that can be recognized by the user. The commonly used form of user interface is a graphical user interface (GUI), which refers to a user interface related to computer operation displayed in a graphical manner. It can be a visual interface element such as text, icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets, etc. displayed on the display screen of an electronic device.
[0038] When a user uses a service of an electronic device, he or she expects to directly obtain recommended information related to the service. This can save the user time and energy in searching and finding information related to the service. For example, if the electronic device detects that the user is using the bus or subway service, then the user may expect the electronic device to recommend relevant preferential travel information or surrounding entertainment information; if the electronic device detects that the user is using the takeaway service, then the user may expect the electronic device to recommend takeaway coupon information; if the electronic device detects that the user is using the hotel reservation service, then the user may expect the electronic device to recommend catering and entertainment discount information near the browsed hotel, etc. The above-mentioned various services can be understood as the functions that the electronic device can provide in the displayed page, that is, the services supported by the electronic device.
[0039] In some implementations, the electronic device can obtain the services included in the current page by identifying the information of the page. Specifically, first the electronic device can obtain the activity of the page, thereby obtaining the root node view of the page. Then the electronic device obtains all the contents of the page (such as all text content) by traversing the root node view. The electronic device then determines the services included in the current page based on all the contents. However, with the development of applications, a single page provided by an application may contain multiple services. The electronic device may obtain multiple services through the information about the page provided by the application, rather than just obtaining a service currently used by the user.
[0040] This may cause the electronic device to be unable to determine the service currently used by the user, and thus unable to accurately recommend relevant information for the service currently used by the user.
[0041] In order to solve the above problems, an embodiment of the present application provides a method and an electronic device for detecting page services. In the method, first, the electronic device displays a first interface, and the first interface includes a first sub-interface. The electronic device obtains the user interface information of the first interface and the text content in the first interface. Then, the electronic device determines the services that can be supported by all sub-interfaces contained in the first interface from all services supported by the electronic device based on the user interface information of the first interface. Then, based on the obtained text information (such as the visible text content), the services supported by the first sub-interface in the first interface are determined from the services that can be supported by the first interface.
[0042] In some implementations, before the electronic device displays the first interface, the electronic device obtains the characteristic information of each service among all the supported services. The characteristic information may include, but is not limited to: the activity name of the user interface where each service is located, the method strategy for obtaining the text information of the user interface to which each service belongs, the information that is unique to each service and distinguishes it from other services, etc. For example, the unique information contained in the user interface where the bus service is located may refer to the text content "Switch bus card". In the embodiment of the present application, the characteristic information preset by the electronic device may also include more information, which is not limited to this. The specific description of the characteristic information here can refer to the relevant description of the following process embodiment, which will not be expanded here.
[0043] In some implementations, the electronic device may obtain the root node view of the first interface before the electronic device obtains the text content of the first interface. The electronic device obtains all the contents (e.g., all the text content) in the page by traversing the root node view. In addition, the electronic device performs a visibility attribute detection on the obtained text content to determine the text content displayed in the current page.
[0044] In this way, the electronic device determines the service currently used by the user from among the multiple services, and can also recommend information related to the service currently used by the user.
[0045] The following introduces the application scenarios of the method for detecting page service information provided by the embodiments of the present application.
[0046] In some implementations, the method for detecting page service information provided in the embodiments of the present application can be applied to scenarios where users use various services, such as using bus services or subway services, ordering takeout or buying medicine outside, etc. The electronic device can identify the service used by the user in the current interface and recommend corresponding information.
[0047] Next, the method provided by the embodiment of the present application is introduced in combination with the UI embodiment. The following is an exemplary introduction using the subway service as an example of the service currently used by the user.
[0048] Figure 1A The user interface 21 of the electronic device in the travel scenario is shown. In some implementations, the travel scenario can be set in an application with a payment function, and other types of applications can also set the travel scenario, which is not limited. The electronic device can display the user interface 21 by entering the travel scenario in a payment application. The user can take the bus using the bus service provided by the user interface 21. The user interface 21 may include, but is not limited to: function options 211, bus QR code 212, bus card switching control 213, auxiliary function control 214, etc.
[0049] Among them, function option 211 may include: bus control 211A, subway control 211B, taxi control 211C and air ticket control 211D. Function option 211 can be used to respond to user operations to switch to different function scenarios. Bus QR code 212 can be used to pay fares when taking a bus. When riding, the user can aim the bus QR code 212 displayed by the electronic device at the barcode scanner on the bus to realize the payment function. Switch bus card control 213 can be used to switch different bus cards. For example, different cities can use different bus cards. Figure 1A As shown, the bus card currently used is the bus card of Shenzhen. The electronic device can respond to the click operation for switching bus card control 213 to switch bus cards of different cities. The auxiliary function control 214 can include a route planning control, a travel assistant control, and a ride record control.
[0050] The electronic device can respond to the user's click operation on the subway control 211B and display the following Figure 1B The user interface 22 is shown. The user can take the subway using the subway service provided by the user interface 22. The user interface 22 may include, but is not limited to: function options 221, subway QR code 222, subway card switching control 223, and auxiliary function control 224.
[0051] Among them, function option 221 may include: bus control 221A, subway control 221B, taxi control 221C and air ticket control 221D. This function option 221 is consistent with the above function option 211. Subway QR code 222 can be used to pay the fare when taking the subway. When riding, the user can aim the subway QR code 222 displayed by the electronic device at the code scanner on the subway gate to realize the payment function. Switch subway card control 223 can be used to switch different subway cards. For example, different cities can use different subway cards. Figure 1BAs shown, the subway card currently used is Shenzhen. The electronic device can switch subway cards of different cities in response to a click operation for switching subway card control 223. Auxiliary function control 224 can include route planning control, travel assistant control and ride record control.
[0052] Exemplarily, the electronic device may display the user interface 21 for a very short time (e.g., 1 second), and then immediately switch to display the user interface 22, and display the user interface 22 for a longer period of time (e.g., greater than 10 seconds). In this way, the electronic device can detect that the user interface 21 may not be the interface for the user to obtain the service, and can also determine that the service currently used by the user is a subway service. In the embodiment of the present application, not all displayed user interfaces need to determine the service currently used. Regarding when the electronic device needs to detect the service currently used by the user, please refer to the relevant description in the following process embodiment, which is first explained here.
[0053] like Figure 1C As shown, the electronic device may display the user interface 23 after displaying the user interface 22 for more than 10 seconds. The user interface 23 may include a pop-up window 231, etc. The pop-up window 231 is used to recommend relevant information for the service currently used by the user. For example, if the electronic device detects that the service currently used by the user is a subway service, it recommends information related to the subway service. The pop-up window 231 may include, but is not limited to: a recommended title 232, recommended content 233, a cancel control 234, and a receive control 235.
[0054] Among them, the recommended title 232 may include the text content "Subway Coupon", and the recommended content 233 may include the text content "Multiple 20% off, 5 yuan coupons". The recommended title 232 and the recommended content 233 are used to prompt the user that they can receive coupons for taking the subway and the discount level of the coupons. The electronic device can respond to the user's click operation on the cancel control 234, cancel the coupon collection, and return to the display user interface 22. At this time, the user does not enjoy the discount when taking the subway. The electronic device can respond to the user's click operation on the collection control 235, confirm the coupon collection, and return to the display user interface 22. At this time, the user enjoys a discount when taking the subway, such as a 20% discount or a 5 yuan discount.
[0055] In the embodiment of the present application, the form and specific content of the information recommended by the electronic device for the service currently used by the user are not limited, and there may be more content forms. For example, the user interface 23 may also display the recommended title "subway line", the recommended content "Line 3", the cancel control and the view control, etc.
[0056] As can be seen from the above embodiment, the difference between the text content displayed in the user interface 21 and the user interface 22 is that the text content "Switch bus card" is displayed in the user interface 21, and the text content "Switch subway card" is displayed in the user interface 22. In some implementations, the electronic device can determine the service used by the current user based on the visibility attribute of the text content. In this way, the user's needs can be understood more accurately and corresponding information can be recommended.
[0057] Based on the user interface introduced above, the method flow for detecting page service information provided in an embodiment of the present application is introduced below.
[0058] Figure 2 The process of the method for detecting page service information provided in an embodiment of the present application is exemplified.
[0059] like Figure 2 As shown, the method flow specifically includes:
[0060] S201. The electronic device obtains feature information of all supported services.
[0061] In some implementations, to implement a specific service, the electronic device needs to configure various information for the service, including UI information, logic code information, etc. After the electronic device completes the settings for the service, the electronic device can obtain the characteristic information of the service from the various configuration information of the service. Different services have different characteristic information. In other words, one service corresponds to a set of characteristic information.
[0062] In some implementations, the above-mentioned service may refer to a service that can be supported by the system of the electronic device. The service may include services supported by system applications, and may also include services supported by all or part of third-party applications. For example, the service may refer to a service that can be supported by a system application (such as a mobile assistant) in an electronic device, and the mobile assistant can obtain services supported by all or part of third-party applications.
[0063] In some implementations, after the electronic device obtains the characteristic information of all supported services, the characteristic information can be stored in a cloud server, which can save storage space in the electronic device; the characteristic information can also be directly stored in the memory of the electronic device, which can facilitate reading by the electronic device.
[0064] In an embodiment of the present application, the characteristic information of all services supported by the electronic device can be used in subsequent processes to obtain information on the page to which the service currently used by the user belongs, and to determine the service currently used by the user, which will not be explained in detail here.
[0065] The following specifically introduces the contents included in the feature information of all services supported by the above electronic devices.
[0066] In some implementations, among all services supported by the electronic device, each service may include feature information, and the feature information of each service may include first feature information, second feature information, and third feature information.
[0067] The first characteristic information may include: information about the user interface where the service is located. Exemplarily, the first characteristic information may include, but is not limited to: the page name of the user interface where the service is located, etc. Optionally, the first characteristic information may also include: package name, title, whether to ignore jumps, etc. For example, the package name (pkgname) in the first characteristic information may be set to ""pkgname":"com.XXX.testkits"", and the page activity name in the first characteristic information may be set to ""name":"com.XXX.testkits.XXXXActivity"". Optionally, the first characteristic information may also include ""ignoreActivity":"com.XXX.testkits.XXXXActivity1"", which indicates that the Activity jump is ignored and it is considered that it is still in the previous Activity. Multiple Activity jumps can also be ignored here; it may also include ""label":"Travel"", which indicates that the title of the page where the service is located is further determined to be "Travel". In the embodiment of the present application, the above specific description of the first characteristic information is only for exemplary explanation and is not limited to this.
[0068] The second characteristic information may include: a method or strategy for obtaining text information in the user interface to which the service belongs. The text information may include: text content, and visibility of the text content. Visibility may refer to whether the text content can be displayed in the interface and whether the user can directly observe the text content in the interface.
[0069] The second characteristic information actually includes some method characteristics, which are used to indicate the strategy by which the electronic device can obtain the text content in the user interface to which the service belongs. Specifically, the second characteristic information may include the method and strategy of filtering nodes during traversal, the method and strategy of obtaining text content, and the method and strategy of detecting the visibility attribute of text content.
[0070] Since the electronic device has completed the design of a certain service, the electronic device can obtain a strategy to obtain information specific to a certain service in a more resource-saving and time-saving manner. Exemplarily, the second characteristic information may include: a method for filtering nodes, a method for obtaining text content, a specific method for obtaining text content, a maximum number of traversals, a method for detecting visibility, text content that needs to detect visibility, and the like.
[0071] Among them, the node filtering method can be used to filter out meaningless nodes, so as to reduce the number of traversals. Furthermore, the filtering nodes are filtered as much as possible in the upper layer of the node tree (i.e., closer to the root node of the node tree). For example, when the display form in interface A is a list, and the rightmost branch set in the node tree is the content the user wants. Then, every time you enter the interface A, you need to first access the left branch of the node tree and then access the right branch to get the content the user wants. This will result in a long access time and may also cause jams. Therefore, in an embodiment of the present application, the electronic device can directly filter out node trees that are meaningless and do not affect subsequent judgments in the upper layer of the node tree. This can save power consumption and improve performance.
[0072] For example, the node filtering method in the second feature information can be set to "glnode":"0:1:0:1:2"", which can indicate the nodes that need to be filtered out when traversing the nodes in the process of obtaining the text content. The above "0:1:0:1:2" represents a node in the fifth layer. This means that after traversing to the fourth layer of the node tree, the node "0:1:0:1:2" and its child nodes will no longer be traversed. For details on how to traverse the nodes, please refer to the relevant description in the following process, which will not be explained in detail here.
[0073] The method of obtaining text content in the second characteristic information can be set to ""Method of obtaining":7", which can indicate the method of obtaining text content. Bit operation can be used to represent it here. Among them, the first position indicates that the electronic device can obtain the text content through the view.getTooltipText() method, the second position indicates that the electronic device can obtain the text content through the view.getContentDescription() method, the third position indicates that the electronic device can obtain the text content through the getText() method for the textview type, the fourth position indicates that the electronic device can obtain the value of the field with the attribute of the string through the declaredFields of the view to obtain the text content, and the fifth position indicates that the electronic device can obtain the text content through the parsing method for the webview type. Since the above-mentioned Method of obtaining method adopts bit operation, when the field value of the method of obtaining text content is "7", and the sum of 2 to the power of 0 plus 2 to the power of 1 plus 2 to the power of 2 is 7, the method represented by the first position, the second position, and the third position can be used in turn in the example to obtain the text content of the node traversed by the electronic device.
[0074] The specific method for obtaining text content in the second characteristic information can be set to ""Analyticalmethods":""", which can indicate the specific function method used by the method for obtaining text content represented by the fourth position and the fifth position. Since the method represented by the fourth and fifth positions is used in the above, the Analytical methods here can be left blank.
[0075] The maximum number of traversal times in the second characteristic information may be set to "times": 200. This information may indicate the maximum number of traversal times that the electronic device traverses the node. Here, the value is 200. In the embodiment of the present application, the value of the maximum number of traversal times is not limited.
[0076] The method of detecting visibility in the second characteristic information can be set to "Method of visibility":2". This information can indicate the method in which the electronic device performs visibility detection on the acquired text content. A value of "0" indicates that visibility detection is not required; a value of "1" indicates that visibility detection is required for all acquired text content; a value of "2" indicates that visibility detection is performed on specified acquired text content.
[0077] The text content that needs to be detected for visibility in the second feature information can be set to ""visibletext":"Switch subway card"". Combined with the above-mentioned method of visibility value of 2, this information can indicate that the text content "Switch subway card" obtained by the electronic device needs to be detected for visibility, and other text content obtained by the electronic device does not need to be detected for visibility.
[0078] In the embodiment of the present application, the above-mentioned examples of various methods in the second characteristic information are only for illustrative purposes, wherein the method field may also have other forms of expression, and the values of various methods may also exist in other ranges, which are not limited to this.
[0079] The third characteristic information can be used in subsequent processes to determine the service used by the current user.
[0080] Third characteristic information: information that distinguishes a service from other services. Specifically, it may include text content that distinguishes a service from other services. Optionally, it may also include matching confidence.
[0081] The third characteristic information may include, but is not limited to: information that is unique to a service and distinguishes it from other services. Exemplarily, the third characteristic information may include: characteristic text content to be matched, matching confidence, and the like.
[0082] For example, the feature text content to be matched in the third feature information can be set to "Feature Text":"Subway network map; Switch subway card; Nanjing subway electronic card;", which indicates that the three text contents "Subway network map", "Switch subway card", and "Nanjing subway electronic card" all need to be matched. The matching confidence in the third feature information can be set to "Confidence level":100". For example, when the confidence level is 100%, it means that all three text contents contained in the feature text content to be matched need to be matched successfully; when the confidence level is 66%, it means that at least two of the three text contents contained in the feature text content to be matched need to be matched successfully.
[0083] In the embodiment of the present application, the above-mentioned examples described in the third characteristic information are only for illustrative purposes, wherein the fields may also have other expressions, and the values of various fields may also exist in other ranges, which are not limited to this.
[0084] S202: The electronic device displays a first sub-interface of the first interface.
[0085] In some implementations, the first interface may refer to a user interface provided by a system of the electronic device, or may refer to a user interface provided by a third-party application, without limitation.
[0086] In view of the processing power and efficiency of the electronic device, the electronic device may not detect the services used by the current user for all displayed user interfaces. Therefore, the above-mentioned first interface may refer to a user interface whose display time exceeds a threshold (e.g., 10s). In the embodiment of the present application, determining which user interface can be used as the first interface may also include more determination methods. The above-mentioned display time is only an exemplary description and is not limited to this.
[0087] Optionally, the electronic device may also detect the services used by the current user on all displayed user interfaces.
[0088] In some implementations, the first interface may also refer to a user interface including a sub-interface. Specifically, the electronic device may display a first interface including a sub-interface, or may display a first interface including another sub-interface.
[0089] Exemplarily, the first interface displaying the first sub-interface may refer to the user interface 22. The first interface displaying other sub-interfaces may refer to the user interface 21. In the embodiment of the present application, the sub-interfaces of the first interface are only exemplified and are not limited thereto.
[0090] S203, the electronic device obtains the root node view (DecorView) of the first interface and the user interface information of the first interface.
[0091] In some implementations, after the electronic device displays the first sub-interface of the first interface, the electronic device obtains the root node view (DecorView) of the first interface and the user interface information of the first interface. Specifically, after the electronic device responds to the user operation, it can obtain the root node view (DecorView) of the first interface and the user interface information of the first interface. The above-mentioned user operations may include but are not limited to: touch operations, click operations, two-finger press operations, long press operations, sliding operations, voice commands, gesture commands, etc. In a possible implementation, the above-mentioned user operation may refer to a user operation of switching from other interfaces to the first interface, or it may refer to a user operation acting on the first interface. The user operation is actually used to indicate that after the electronic device responds to the user operation, the user can start using the services supported by the first interface.
[0092] Exemplarily, the user operation may be a click operation of a pointer on the subway control 221B. In the embodiment of the present application, the user operation is only exemplified and is not limited thereto.
[0093] In some implementations, after the electronic device responds to the above-mentioned user operation, it obtains the DecorView of the first interface. Among them, DecorView, as a subclass of frame layout (FrameLayout), can be understood as the root node view of the Android view tree. DecorView is the top-level view (view) in the window (window). The electronic device can obtain DecorView from the window of the first interface. For example, the electronic device can obtain decorView through the method of getWindow().getDecorView(). In the embodiments of the present application, there may be more ways to obtain the decorView of the first interface, which is not limited to this.
[0094] DecorView has only one child View, which is a vertical linear layout (LinearLayout). It contains two child elements, one of which is the title view (TitleView), which is the container of the action bar (ActionBar), and the other is the content view (ContentView), which is the container of the content in the window.
[0095] Optionally, since the text information acquired by the electronic device in the subsequent process is located in the non-status bar area of the first interface, the electronic device can also obtain the area displayed on the first interface through the getWindowVisibleDisplayFrame method. The area may include the title bar and the content bar, but does not include the status bar.
[0096] In some implementations, after the electronic device responds to the above user operation, it can also obtain the user interface information of the first interface. The user interface information may include the page name of the user interface. For example, activity name information, etc. Exemplarily, when the electronic device displays the user interface 22, the electronic device can obtain the name of the activity of the first interface and the title name "Travel to Shenzhen".
[0097] In some implementations, the electronic device may obtain feature information of supported services stored in a cloud server after being powered on, and store the feature information in a database in a memory of the electronic device.
[0098] Optionally, if the electronic device stores the characteristic information of each service in the cloud server in the above S201, after the electronic device displays the first interface, it sends a request to the cloud server. Afterwards, the electronic device can obtain the characteristic information of each service sent by the cloud server.
[0099] S204: The electronic device filters out services to be matched from services supported by the electronic device based on the user interface information of the first interface.
[0100] In some implementations, after the electronic device obtains the relevant information of the page of the first interface in response to the above-mentioned user operation, it can filter out services whose specific information in the first characteristic information is consistent with the relevant information of the page in the first interface from the services supported by the electronic device as services to be matched. Specifically, the electronic device can filter out services whose names and title names of the activities in the first characteristic information are consistent with those in the first interface from various services as services to be matched. Exemplarily, when the electronic device displays the user interface 22, based on whether the activity name is consistent with the tag name, the electronic device can obtain that the services to be matched may include: bus service, subway service, taxi service, and air ticket service. This is equivalent to the electronic device determining one or more services that the first interface supports or can provide.
[0101] In some implementations, the first interface may include multiple sub-interfaces, and the above-mentioned services to be matched may refer to services that can be supported by all sub-interfaces included in the first interface.
[0102] In some implementations, when the number of services to be matched screened out by the electronic device is one, that is, the electronic device includes only one sub-interface, the only screened service to be matched can be determined as the service currently used by the user. The electronic device does not need to perform the following S205-S207.
[0103] In this way, the electronic device can reduce the number of multiple services that are matched with the service currently used by the user in the subsequent process, saving resources and time and being more efficient.
[0104] S205: The electronic device traverses the DecorView of the first interface according to the second characteristic information of the service to be matched, to obtain text information in the first interface.
[0105] In some implementations, the purpose of the electronic device obtaining the DecorView of the first interface is to obtain the text information of the first interface. Therefore, after the electronic device obtains the DecorView of the first interface, it can traverse all the controls in the DecorView. Specifically, the electronic device can use the depth-first search (DFS) method to traverse the DecorView in the first interface. DFS is actually a recursive process, and can also be understood as a pre-order traversal of a tree, which can ensure that no controls in the DecorView are missed. In an embodiment of the present application, other traversal methods can also be used to traverse the DecorView, which is not limited. In an embodiment of the present application, the controls in the DecorView can also be referred to as nodes.
[0106] In some implementations, when the electronic device traverses all controls in DecorView, the electronic device may filter out controls that do not need to obtain text content based on the second feature information of the service to be matched.
[0107] Combine the following Figure 3 The node tree diagram shown shows how to use the filtering node method.
[0108] like Figure 3 As shown, the node tree A includes 18 nodes. Each node has a corresponding representation. For example, node 6 can be represented by "0:1:0". The root node is generally represented by "0", the left child node of the node is generally represented by "0", and the right child node of the node is generally represented by "1". In the embodiment of the present application, the above-mentioned node tree A is only used for exemplary explanation, and the number of nodes contained therein and the position of the nodes are not limited.
[0109] Exemplarily, the node filtering method can be set to "glnode":"0:1:0:1:2"". The "0:1:0:1:2" represents a node, namely, node 15 in node tree A. The node filtering method illustrated in the above example may mean that the electronic device can obtain text content for each node (including: node 1, node 2, node 4, node 5, node 3, node 6, node 8, node 9, node 13, node 14, node 7, node 10, node 11, node 16, node 17) in node tree A except node 15 and its child nodes, and filter out node 15 and its child nodes (namely node 18 and node 19), that is, not obtain the text content located at node 15, node 18, and node 19.
[0110] In the embodiment of the present application, the nodes set in the above filtering node method are only for exemplary explanation, and the number of the set filtering nodes can be one or more, which is not limited.
[0111] This is because when the electronic device obtains the characteristic information of each service in S201, it has determined the specific location of the unique information in each service in DecorView. In this way, it is not necessary to obtain the text content of all controls in DecorView, which can reduce the number of times the method of obtaining text content is used, save time, and improve processing efficiency.
[0112] In another possible implementation, the electronic device can also traverse the DecorView of the first interface based on the second characteristic information of the service to be matched. In fact, it means to traverse the DecorView by using the method of filtering nodes in the second characteristic information. Referring to the relevant description of the second characteristic information in S201 above, it can be seen that the traversal described here does not mean traversing all the controls in DecorView once. Exemplarily, when the node filtering method is set to "glnode":"0:1:0:1:2", it means that when the electronic device traverses to the node before the node "0:1:0:1:2", the node "0:1:0:1:2" and its child nodes in the DecorView node tree are filtered out and no longer traversed. This is because in an embodiment of the present application, the electronic device can use the DFS method to traverse the DecorView in the first interface. That is to say, the electronic device traverses to all nodes in DecorView except the node "0:1:0:1:2" and its child nodes. The method of filtering nodes set by the electronic device can ensure that the nodes to which the text content to be obtained in the subsequent process belongs are traversed, and meaningless nodes are filtered out. In the embodiment of the present application, other traversal methods may also be used to traverse DecorView, which is not limited to this.
[0113] As can be seen from the above content, in the embodiment of the present application, the method of filtering nodes in the second characteristic information can be used in the process of traversing DecorView, and can also be used in the process of selecting a control for obtaining text content after traversing DecorView. There is no limitation on this.
[0114] In some implementations, since the electronic device screens out multiple services to be matched in S204, the electronic device can integrate the second feature information of the multiple services to be matched. Specifically, the electronic device can perform relevant processing such as intersection, union, and or the like on the fields in each method of the second feature information of the multiple services to be matched. Exemplarily, the electronic device obtains the services to be matched: service 1 and service 2. Among them, the filtering node method in the second feature information of service 1 is "glnode":"0:1:0:2"", and the filtering node method in the second feature information of service 2 is "glnode":"0:1:0:2:2", and the electronic device takes the intersection of the filtering nodes to implement the filtering node method, that is, when traversing the first interface, DecorView filters out the node "0:1:0:2:2" and the child nodes of the node "0:1:0:2:2". In the second characteristic information of service 1, "Method of obtaining":"3"", and in the second characteristic information of service 2, "Method of obtaining":"7"". The electronic device obtains the text content by taking the union of different methods of obtaining text content, that is, the electronic device uses method one, method two, and method three when obtaining the text content. Similarly, the electronic device detects the visibility of the text content by taking the union of text content that needs to be detected for visibility. In the embodiment of the present application, the above-mentioned processing of taking the intersection, union, and AND / OR of the fields in each method of the second characteristic information of multiple services to be matched is only for illustrative purposes and is not limited to this.
[0115] Optionally, the electronic device may also be based on the second characteristic information of one of the services to be matched. Exemplarily, in S204, when the electronic device displays the user interface 22, it determines that the services to be matched are: bus service, subway service, taxi service, and air ticket service. Then the electronic device may traverse the DecorView of the first interface based on the method of filtering nodes in the second characteristic information of the bus service.
[0116] In some implementations, the number of times the electronic device traverses DecorView is limited. The number of traversals should be less than or equal to the maximum number of traversals set in the second feature information. In this way, the traversal efficiency can be improved to prevent errors in the traversal and fall into a loop.
[0117] Then, the electronic device can obtain the text content of the first interface based on the second feature information. In fact, it can refer to obtaining the text content of each control after filtering. Specifically, the electronic device can obtain the text content in the first interface according to the method of obtaining text content set in the second feature information.
[0118] Exemplarily, the following five methods for obtaining text content can be set in the second characteristic information: Method 1 refers to the view.getTooltipText() method, Method 2 refers to the view.getContentDescription() method, Method 3 refers to the getText() method, Method 4 refers to obtaining the value of a field whose attribute is a string through the view's declaredFields, and Method 5 is a pointer parsing method for the webview type. In order to obtain the text content, the electronic device can adopt one or more methods for obtaining text content. Whether to adopt one or more methods for obtaining text content, and which specific methods to adopt, can refer to the following content.
[0119] Referring to the relevant description of the second characteristic information in S201 above, it can be known that the function of the method of obtaining the text content adopts bit operation, and the above five methods correspond to the five positions of the bit operation respectively. The field of the method of obtaining the text content can be expressed as "Method of obtaining", and the field value of the method of obtaining the text content and the method of obtaining the text content represented by each value can refer to the following Table 1. Among them, in Table 1, m1 represents method one, m2 represents method two, m3 represents method three, m4 represents method four, and m5 represents method five.
[0120] Table 1
[0121] As can be seen from Table 1, when five methods for obtaining text content are set in the electronic device, the methods for obtaining text content set in the second characteristic information can be divided into 31 cases. For example, when the value of Method of obtaining is 7, the electronic device can sequentially adopt Method 1, Method 2, and Method 3 for node a to obtain text content. The above sequential adoption means that the electronic device first adopts Method 1 to obtain the text content of node a. If the text content is obtained, the acquisition is terminated, and the next node (such as node b) to be traversed is sequentially adopted Method 1, Method 2, and Method 3 to obtain the text content; if the text content is not obtained, Method 2 is adopted to obtain the text content of node a. If the text content is still not obtained, Method 3 is adopted to obtain the text content of node a. If the text content is still not obtained, it means that node a does not contain text content, and then the electronic device sequentially adopts Method 1, Method 2, and Method 3 for the next node (such as node b) to be traversed to obtain the text content.
[0122] Furthermore, when the method used by the electronic device to obtain text content includes method four or method five, the electronic device may also obtain the text content according to the method function used by the specific method four or method five set in the second characteristic information.
[0123] For example, after the electronic device switches from user interface 21 to user interface 22, since not all the information of user interface 22 is re-rendered and updated relative to user interface 21, when the electronic device obtains the text content in user interface 22 by traversing DecorView, it can simultaneously obtain the text content of "Switch bus card" and "Switch subway card". If the electronic device relies solely on the text content, it may not be able to determine the services supported by user interface 22. Therefore, in an embodiment of the present application, the electronic device can also detect the visibility attribute of the text content.
[0124] In some implementations, after the electronic device obtains the text content of the first interface, it can perform visibility detection on all or part of the text content in the obtained first interface according to the text content whose visibility attributes need to be detected and the way of detecting visibility set in the second characteristic information. This is because when the electronic device obtains the characteristic information of each service in S201, the unique information in each service (such as unique text content) has been determined. In this way, there is no need to detect the visibility of all the text content of the obtained first interface, which can reduce the number of visibility detections, save time, and improve processing efficiency. Optionally, there are certain application scenarios that require the detection of visibility attributes of all text content in the obtained first interface.
[0125] For example, the method for detecting visibility can be set to ""Method of visibility":2", and the text content that needs to detect the visibility attribute can be set to ""visibletext":"Switch subway card"". This means that the electronic device can detect the visibility attribute of the text content "Switch subway card" in all the text contents in the first interface obtained. The method for detecting visibility can also be set to ""Method of visibility":1", and the text content that needs to detect the visibility attribute can be set to ""visibletext":""". This means that the electronic device can detect the visibility attribute of all the text contents in the first interface obtained. In the embodiments of the present application, the above-mentioned text content and detection method for visibility detection are only for illustrative purposes, and each service may have different settings, which are not limited to this.
[0126] In some implementations, the electronic device can determine the visibility of a control through the view.getVisibiity method and the view.isShown method. However, the above two method functions are suitable for detecting whether a control can be viewed by the user in a certain scenario, and cannot indicate that the control is currently displayed in the interface and that the user can currently see the control. Therefore, the getLocalVisibleRect method is generally used in the embodiments of the present application to detect the visibility attribute of the control.
[0127] In some implementations, the text content with the visibility attribute of visible in the first interface acquired by the electronic device is text content in the first sub-interface of the first interface.
[0128] Exemplarily, when the electronic device displays the user interface 22, the electronic device detects the visibility of the text content according to the above method and can obtain that the text content "Switch subway card" is visible.
[0129] Optionally, the electronic device may also obtain the text position of the text content in the first interface so as to subsequently further determine the services provided by the first interface.
[0130] S206: The electronic device determines, from the services to be matched, the services provided by the first sub-interface of the current first interface based on the third characteristic information of the services to be matched and the text information in the first interface obtained above.
[0131] The electronic device may determine the service provided by the first sub-interface in the current first interface based on the third characteristic information of the service to be matched.
[0132] In some implementations, the electronic device may obtain text information based on the second characteristic information after integrating the second characteristic information of multiple services to be matched, and then determine whether the text information matches the information set in the third characteristic information of one of the services to be matched.
[0133] Optionally, the electronic device may also determine whether the text information obtained according to the second characteristic information of multiple services to be matched matches the information set in the third characteristic information of one of the services to be matched. If not, then repeat the operation of obtaining the text content according to the second characteristic information of another service to be matched in S205, and then determine whether it matches the information set in the third characteristic information of another service to be matched. This continues until there is a matching service to be matched or all services to be matched have been judged once.
[0134] Exemplarily, in S205, the electronic device obtains text information X through the second characteristic information of service A, and the text information X includes visible text content Y. Then the electronic device determines that the visible text content Y does not belong to the characteristic text content to be matched set in the third characteristic information of service A. Next, the electronic device repeats S205 and obtains text information M through the second characteristic information of service B, and the text information M includes visible text content M. Then the electronic device repeats S206 to determine whether the visible text content M belongs to the characteristic text content to be matched set in the third characteristic information of service B. In the embodiment of the present application, the above is only an exemplary description and is not limited to this.
[0135] In some implementation methods, the above-mentioned determination of the services provided by the current first interface may include: the electronic device may determine whether the visible text content obtained in S205 belongs to the characteristic text content to be matched set in the third characteristic information to determine the services provided by the sub-interface of the first interface. From the description of the third characteristic information in S201, it can be seen that the third characteristic information may include text content that is different from other services, that is, the characteristic text content to be matched.
[0136] For example, the feature text content to be matched in the third feature information of the first service is set to ""Feature Text":"Switch Metro Card"", and the electronic device obtains the visible text content "Switch Metro Card" according to the second feature information of the first service in S205. It can be known that the visible text content obtained by the electronic device belongs to the text content that is unique to the first service and distinguishes it from other services. Therefore, the electronic device can determine that the service provided by the first interface is the first service.
[0137] Furthermore, since the version of the application in the electronic device will be updated, the text content in the user interface may change. In addition, the language of the electronic device is different based on the user's settings (for example, Chinese or English). Therefore, the third feature information of each service can also be set with a confidence level. Exemplarily, the feature text content to be matched in the third feature information of the first service is set to "Feature Text": "Switching subway cards", and the confidence level is set to "Confidence level": 50". The confidence level can represent 50%. In other words, as long as the electronic device obtains at least one of the visible text contents "Switching subway cards" and "Switching subway cards", it can determine that the service provided by the first interface is the first service.
[0138] Optionally, the electronic device may also determine the service provided by the first interface by determining whether the position of the optional text content matches the position range of the feature text content to be matched set by the third feature information.
[0139] In some implementations, determining the service provided by the current first interface from the services to be matched may refer to: determining the service provided by the first sub-interface of the current first interface.
[0140] S207. The electronic device recommends information related to the service currently used by the user.
[0141] In some implementations, after the electronic device determines the service provided by the first sub-interface of the first interface, that is, after determining the service currently used by the user, it can obtain a variety of information related to the service currently used by the user. The information may refer to information related to the service that the user may expect to obtain. Specifically, the electronic device can summarize the information related to the service that can be provided by various applications through a system application (such as a mobile phone assistant). The above summary may refer to the electronic device querying and summarizing through keywords. For example, information related to subway services can be queried through the keyword "subway". Summarization can also be performed in other ways, which are not limited to this.
[0142] Then the electronic device can recommend the acquired relevant information to the user. Specifically, the electronic device can make recommendations in different forms, such as displaying a pop-up window, voice broadcast, etc.
[0143] For example, when the electronic device determines that the service currently used by the user is a subway service, the electronic device can obtain information related to the subway service, such as: preferential information for taking the subway, subway lines, and some surrounding service information. Figure 1C The pop-up window 231 shown recommends information related to subway services.
[0144] When the electronic device determines that the service currently used by the user is a map service, the electronic device can recommend entertainment information near the searched location by way of a pop-up window; when the electronic device determines that the service currently used by the user is a takeout and medicine purchase service, the electronic device can recommend coupon information for takeout and medicine purchase by way of a pop-up window or prompt window; when the electronic device determines that the service currently used by the user is a hotel reservation service, the electronic device can recommend dining and entertainment discount information near the browsed hotel by way of a pop-up window, etc.
[0145] In the embodiment of the present application, the above-mentioned recommended information related to the service currently used by the user is only for illustrative purposes and is not intended to be limiting.
[0146] In this way, the electronic device can determine the service currently used by the user and recommend information related to the service, which can save the user's time and energy in searching and finding information related to the service, making it more convenient for the user to use the electronic device.
[0147] Figure 4 A schematic structural diagram of an electronic device 100 is shown.
[0148] The electronic device 100 may be equipped with Or a portable terminal device of other operating systems, the electronic device 100 can be a mobile phone, a tablet computer, a desktop computer, a laptop computer, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, and a cellular phone, a personal digital assistant (PDA), an augmented reality (AR) device, a virtual reality (VR) device, an artificial intelligence (AI) device, a wearable device, a vehicle, a vehicle-mounted device, a smart home device and / or a smart city device, but not limited thereto, the electronic device 100 can also include a laptop computer with a touch-sensitive surface or a touch panel, a desktop computer with a touch-sensitive surface or a touch panel, and other non-portable terminal devices, etc. The embodiment of the present application does not impose any special restrictions on the specific type of the electronic device.
[0149] The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.
[0150] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or separate some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.
[0151] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.
[0152] In some implementations, the processor 110 may obtain feature information of supported services. The processor 110 may also be used to obtain the DecorView of the first interface and the user interface information of the first interface. The processor 110 may also be used to obtain the text content of the first interface according to the above five methods for obtaining text content. The processor 110 may also be used to detect the visibility attribute of the text content, and may also be used to determine the text content in the first sub-interface. The processor 110 may also be used to determine the services supported by the first sub-interface of the first interface based on the text content of the first sub-interface. The processor 110 may also be used to recommend information related to the services supported by the first sub-interface.
[0153] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.
[0154] In some implementations, the memory may be used to store feature information of all services supported by the electronic device 100. The feature information includes first feature information, second feature information, and third feature information.
[0155] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.
[0156] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple groups of I2C buses. The processor 110 may be coupled to the touch sensor 180K, the charger, the flash, the camera 193, etc. through different I2C bus interfaces. For example: the processor 110 may be coupled to the touch sensor 180K through the I2C interface, so that the processor 110 communicates with the touch sensor 180K through the I2C bus interface, thereby realizing the touch function of the electronic device 100.
[0157] The I2S interface can be used for audio communication. In some embodiments, the processor 110 can include multiple I2S buses. The processor 110 can be coupled to the audio module 170 via the I2S bus to achieve communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 can transmit an audio signal to the wireless communication module 160 via the I2S interface to achieve the function of answering a call through a Bluetooth headset.
[0158] The PCM interface can also be used for audio communication, sampling, quantizing and encoding analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 can be coupled via a PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 via the PCM interface to realize the function of answering calls via a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.
[0159] The UART interface is a universal serial data bus for asynchronous communication. The bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, the UART interface is generally used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 through the UART interface to implement the Bluetooth function. In some embodiments, the audio module 170 can transmit an audio signal to the wireless communication module 160 through the UART interface to implement the function of playing music through a Bluetooth headset.
[0160] The MIPI interface can be used to connect the processor 110 with peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a camera serial interface (CSI), a display serial interface (DSI), etc. In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to implement the shooting function of the electronic device 100. The processor 110 and the display screen 194 communicate via the DSI interface to implement the display function of the electronic device 100.
[0161] The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 with the camera 193, the display 194, the wireless communication module 160, the audio module 170, the sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.
[0162] The USB interface 130 is an interface that complies with the USB standard specification, and specifically can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect a charger to charge the electronic device 100, and can also be used to transfer data between the electronic device 100 and a peripheral device. It can also be used to connect headphones to play audio through the headphones. The interface can also be used to connect other electronic devices, such as AR devices, etc.
[0163] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present invention is only a schematic illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.
[0164] The charging management module 140 is used to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from a wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 may receive wireless charging input through a wireless charging coil of the electronic device 100. While the charging management module 140 is charging the battery 142, it may also power the electronic device through the power management module 141.
[0165] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the display screen 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle number, battery health status (leakage, impedance), etc. In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.
[0166] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.
[0167] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.
[0168] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.
[0169] The modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 170A, a receiver 170B, etc.), or displays an image or video through a display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.
[0170] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, demodulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.
[0171] In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).
[0172] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.
[0173] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.
[0174] The electronic device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.
[0175] In the embodiment of the present application, the electronic device can display, for example, the following information provided in the above UI embodiment through the display screen 194: Figure 1A-Figure 1C The user interface shown and so on will not be described in detail here.
[0176] ISP is used to process the data fed back by camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to ISP for processing and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, ISP can be set in camera 193.
[0177] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.
[0178] The digital signal processor is used to process digital signals, and can process not only digital image signals but also other digital signals. For example, when the electronic device 100 is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.
[0179] Video codecs are used to compress or decompress digital videos. The electronic device 100 may support one or more video codecs. Thus, the electronic device 100 may play or record videos in a variety of coding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.
[0180] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn. Through NPU, applications such as intelligent cognition of electronic device 100 can be realized, such as image recognition, face recognition, voice recognition, text understanding, etc.
[0181] The internal memory 121 may include one or more random access memories (RAM) and one or more non-volatile memories (NVM).
[0182] Random access memory may include static random-access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM, for example, the fifth generation DDR SDRAM is generally referred to as DDR5 SDRAM), etc.; non-volatile memory may include disk storage devices and flash memory.
[0183] Flash memory can be divided into NOR FLASH, NAND FLASH, 3D NAND FLASH, etc. according to the operating principle; single-level cell (SLC), multi-level cell (MLC), triple-level cell (TLC), quad-level cell (QLC), etc. according to the storage unit potential level; universal flash storage (UFS), embedded multi media card (eMMC), etc. according to the storage specification.
[0184] The random access memory can be directly read and written by the processor 110, and can be used to store executable programs (such as machine instructions) of the operating system or other running programs, and can also be used to store user and application data.
[0185] The non-volatile memory may also store executable programs and user and application data, etc., and may be loaded into the random access memory in advance for direct reading and writing by the processor 110 .
[0186] The external memory interface 120 can be used to connect to an external non-volatile memory to expand the storage capacity of the electronic device 100. The external non-volatile memory communicates with the processor 110 through the external memory interface 120 to implement a data storage function. For example, files such as music and videos are stored in the external non-volatile memory.
[0187] The electronic device 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.
[0188] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be arranged in the processor 110, or some functional modules of the audio module 170 can be arranged in the processor 110.
[0189] The speaker 170A, also called a "speaker", is used to convert an audio electrical signal into a sound signal. The electronic device 100 can listen to music or listen to a hands-free call through the speaker 170A.
[0190] The receiver 170B, also called a "earpiece", is used to convert audio electrical signals into sound signals. When the electronic device 100 receives a call or voice message, the voice can be received by placing the receiver 170B close to the human ear.
[0191] Microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak by putting their mouth close to microphone 170C to input the sound signal into microphone 170C. The electronic device 100 can be provided with at least one microphone 170C. In other embodiments, the electronic device 100 can be provided with two microphones 170C, which can not only collect sound signals but also realize noise reduction function. In other embodiments, the electronic device 100 can also be provided with three, four or more microphones 170C to collect sound signals, reduce noise, identify the sound source, realize directional recording function, etc.
[0192] The earphone interface 170D is used to connect a wired earphone and can be a USB interface 130 or a 3.5 mm open mobile terminal platform (OMTP) standard interface or a cellular telecommunications industry association of the USA (CTIA) standard interface.
[0193] The pressure sensor 180A is used to sense the pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A can be set on the display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. The capacitive pressure sensor can be a parallel plate including at least two conductive materials. When a force acts on the pressure sensor 180A, the capacitance between the electrodes changes. The electronic device 100 determines the intensity of the pressure according to the change in capacitance. When a touch operation acts on the display screen 194, the electronic device 100 detects the touch operation intensity according to the pressure sensor 180A. The electronic device 100 can also calculate the touch position according to the detection signal of the pressure sensor 180A. In some embodiments, touch operations acting on the same touch position but with different touch operation intensities can correspond to different operation instructions. For example: when a touch operation with a touch operation intensity less than the first pressure threshold acts on the short message application icon, an instruction to view the short message is executed. When a touch operation with a touch operation intensity greater than or equal to the first pressure threshold acts on the short message application icon, an instruction to create a new short message is executed.
[0194] The gyro sensor 180B can be used to determine the motion posture of the electronic device 100. In some embodiments, the angular velocity of the electronic device 100 around three axes (i.e., x, y, and z axes) can be determined by the gyro sensor 180B. The gyro sensor 180B can be used for anti-shake shooting. For example, when the shutter is pressed, the gyro sensor 180B detects the angle of the electronic device 100 shaking, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to offset the shaking of the electronic device 100 through reverse movement to achieve anti-shake. The gyro sensor 180B can also be used for navigation and somatosensory game scenes.
[0195] The air pressure sensor 180C is used to measure air pressure. In some embodiments, the electronic device 100 calculates the altitude through the air pressure value measured by the air pressure sensor 180C to assist positioning and navigation.
[0196] The magnetic sensor 180D includes a Hall sensor. The electronic device 100 can use the magnetic sensor 180D to detect the opening and closing of the flip leather case. In some embodiments, when the electronic device 100 is a flip phone, the electronic device 100 can detect the opening and closing of the flip cover according to the magnetic sensor 180D. Then, according to the detected opening and closing state of the leather case or the opening and closing state of the flip cover, the flip cover can be automatically unlocked.
[0197] The acceleration sensor 180E can detect the magnitude of the acceleration of the electronic device 100 in all directions (generally three axes). When the electronic device 100 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the posture of the electronic device and is applied to applications such as horizontal and vertical screen switching and pedometers.
[0198] The distance sensor 180F is used to measure the distance. The electronic device 100 can measure the distance by infrared or laser. In some embodiments, when shooting a scene, the electronic device 100 can use the distance sensor 180F to measure the distance to achieve fast focusing.
[0199] The proximity light sensor 180G may include, for example, a light emitting diode (LED) and a light detector, such as a photodiode. The light emitting diode may be an infrared light emitting diode. The electronic device 100 emits infrared light outward through the light emitting diode. The electronic device 100 uses a photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the electronic device 100. When insufficient reflected light is detected, the electronic device 100 can determine that there is no object near the electronic device 100. The electronic device 100 can use the proximity light sensor 180G to detect that the user holds the electronic device 100 close to the ear to talk, so as to automatically turn off the screen to save power. The proximity light sensor 180G can also be used in leather case mode and pocket mode to automatically unlock and lock the screen.
[0200] The ambient light sensor 180L is used to sense the ambient light brightness. The electronic device 100 can adaptively adjust the brightness of the display screen 194 according to the perceived ambient light brightness. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the electronic device 100 is in a pocket to prevent accidental touches.
[0201] The fingerprint sensor 180H is used to collect fingerprints. The electronic device 100 can use the collected fingerprint characteristics to implement fingerprint unlocking, access application locks, fingerprint photography, fingerprint call answering, etc.
[0202] The temperature sensor 180J is used to detect temperature. In some embodiments, the electronic device 100 uses the temperature detected by the temperature sensor 180J to execute a temperature processing strategy. For example, when the temperature reported by the temperature sensor 180J exceeds a threshold, the electronic device 100 reduces the performance of a processor located near the temperature sensor 180J to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is lower than another threshold, the electronic device 100 heats the battery 142 to avoid abnormal shutdown of the electronic device 100 due to low temperature. In other embodiments, when the temperature is lower than another threshold, the electronic device 100 performs a boost on the output voltage of the battery 142 to avoid abnormal shutdown caused by low temperature.
[0203] The touch sensor 180K is also called a "touch control device". The touch sensor 180K can be set on the display screen 194. The touch sensor 180K and the display screen 194 form a touch screen, also called a "touch control screen". The touch sensor 180K is used to detect touch operations acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In other embodiments, the touch sensor 180K can also be set on the surface of the electronic device 100, which is different from the position of the display screen 194.
[0204] The bone conduction sensor 180M can obtain a vibration signal. In some embodiments, the bone conduction sensor 180M can obtain a vibration signal of a vibrating bone block of the vocal part of the human body. The bone conduction sensor 180M can also contact the human pulse to receive a blood pressure beat signal. In some embodiments, the bone conduction sensor 180M can also be set in an earphone and combined into a bone conduction earphone. The audio module 170 can parse out a voice signal based on the vibration signal of the vibrating bone block of the vocal part obtained by the bone conduction sensor 180M to realize a voice function. The application processor can parse the heart rate information based on the blood pressure beat signal obtained by the bone conduction sensor 180M to realize a heart rate detection function.
[0205] The key 190 includes a power key, a volume key, etc. The key 190 may be a mechanical key or a touch key. The electronic device 100 may receive key input and generate key signal input related to user settings and function control of the electronic device 100.
[0206] Motor 191 can generate vibration prompts. Motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, audio playback, etc.) can correspond to different vibration feedback effects. For touch operations acting on different areas of the display screen 194, motor 191 can also correspond to different vibration feedback effects. Different application scenarios (for example: time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.
[0207] Indicator 192 may be an indicator light, which may be used to indicate charging status, power changes, messages, missed calls, notifications, etc.
[0208] The SIM card interface 195 is used to connect a SIM card.
[0209] The software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. In the embodiment of the present invention, the Android system of the layered architecture is taken as an example to exemplify the software structure of the electronic device 100.
[0210] Figure 5 1 is a software structure block diagram of the electronic device 100 according to an embodiment of the present invention.
[0211] The layered architecture divides the software into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: the application layer, the application framework layer, the Android runtime and system library, and the kernel layer.
[0212] The application layer can include a series of application packages.
[0213] like Figure 5 As shown, the application package may include camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message and other applications.
[0214] The application framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer includes some predefined functions.
[0215] like Figure 5 As shown, the application framework layer may include an input manager (InputManager), a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, and the like.
[0216] Among them, InputManager can be used to detect user input operations.
[0217] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.
[0218] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.
[0219] The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images.
[0220] The phone manager is used to provide communication functions of the electronic device 100, such as management of call status (including connecting, hanging up, etc.).
[0221] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.
[0222] The notification manager enables applications to display notification information in the status bar. It can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as notifications of applications running in the background, or a notification that appears on the screen in the form of a dialog window. For example, a text message is displayed in the status bar, a prompt sound is emitted, an electronic device vibrates, an indicator light flashes, etc.
[0223] In some implementations, the application framework diagram may further include: a view service (ViewService), a view system (ViewSystem) and a view server (ViewServer). Among them, ViewServer includes a view manager (SurfaceFlinger) and an algorithm module.
[0224] Among them, ViewService is a system-level view service, which is used to provide display interface related services for various applications. Among them, ViewSystem is equivalent to the inner class of ViewService, which serves as a bridge for interaction between the Java layer and the native layer and can be called by ViewService.
[0225] Android Runtime includes core libraries and virtual machines. Android runtime is responsible for scheduling and management of the Android system.
[0226] The core library consists of two parts: one part is the function that needs to be called by the Java language, and the other part is the Android core library.
[0227] The application layer and the application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.
[0228] The system library may include multiple functional modules, such as surface manager, media library, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.
[0229] The surface manager is used to manage the display subsystem and provide the fusion of 2D and 3D layers for multiple applications.
[0230] The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.
[0231] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.
[0232] A 2D graphics engine is a drawing engine for 2D drawings.
[0233] The media library can include functional modules such as ViewSystem and ViewService corresponding to the Framework layer.
[0234] Most of the classes in the Framework layer are just "intermediaries" for applications to use library files in the system library. Because the upper-level applications are written in Java, they need the most direct support of Java interfaces, and the system library supports the operation of another language (such as C++). Therefore, the Framework layer acts as an intermediary between the application layer and the system library. Each module in the Framework layer does not really implement specific functions, or only implements part of the functions, and focuses on the core library to complete. For example, ViewSystem and ViewService in the Framework layer also have corresponding classes in the system library. The difference is that the Framework layer is mostly written in Java, and most of the system library is written in C++.
[0235] The kernel layer is the layer between hardware and software. The kernel layer contains at least display driver, camera driver, audio driver, and sensor driver.
[0236] Understandably, Figure 5 The illustrated structure does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.
[0237] It should be understood that each step in the above method embodiment provided by the present application can be completed by an integrated logic circuit of hardware in a processor or by instructions in the form of software. The method steps disclosed in the embodiments of the present application can be directly embodied as being executed by a hardware processor, or by a combination of hardware and software modules in a processor.
[0238] The present application also provides an electronic device, which may include: a memory and a processor, wherein the memory may be used to store a computer program; and the processor may be used to call the computer program in the memory so that the electronic device executes the method in any one of the above embodiments.
[0239] The present application also provides a chip system, which includes at least one processor for implementing the functions involved in the method executed by the electronic device in any of the above embodiments.
[0240] In one possible design, the chip system also includes a memory, which is used to store program instructions and data, and the memory is located inside or outside the processor.
[0241] The chip system may be composed of the chip, or may include the chip and other discrete devices.
[0242] Optionally, the processor in the chip system may be one or more. The processor may be implemented by hardware or by software. When implemented by hardware, the processor may be a logic circuit, an integrated circuit, etc. When implemented by software, the processor may be a general-purpose processor implemented by reading software code stored in a memory.
[0243] Optionally, the memory in the chip system may also be one or more. The memory may be integrated with the processor or may be separately arranged with the processor, which is not limited in the embodiments of the present application. Exemplarily, the memory may be a non-transient processor, such as a read-only memory ROM, which may be integrated with the processor on the same chip or may be arranged on different chips respectively. The embodiments of the present application do not specifically limit the type of memory and the arrangement of the memory and the processor.
[0244] Exemplarily, the chip system can be a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on chip (SoC), a central processor unit (CPU), a network processor (NP), a digital signal processor (DSP), a microcontroller unit (MCU), a programmable logic device (PLD) or other integrated chips.
[0245] The present application also provides a computer program product, which includes: a computer program (also referred to as code, or instruction), which enables a computer to execute the method executed by the electronic device in any of the above embodiments when the computer program is executed.
[0246] The present application also provides a computer-readable storage medium, which stores a computer program (also referred to as code or instruction). When the computer program is executed, the computer executes the method executed by the electronic device in any of the above embodiments.
[0247] The various implementation modes of the present application can be combined arbitrarily to achieve different technical effects.
[0248] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state disk (SSD)), etc.
[0249] Those skilled in the art can understand that to implement all or part of the processes in the above-mentioned embodiments, the processes can be completed by computer programs to instruct related hardware, and the programs can be stored in computer-readable storage media. When the programs are executed, they can include the processes of the above-mentioned method embodiments. The aforementioned storage media include: ROM or random access memory RAM, magnetic disk or optical disk and other media that can store program codes.
[0250] In short, the above description is only an embodiment of the technical solution of the present invention, and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made according to the disclosure of the present invention shall be included in the protection scope of the present invention.
Claims
1. A method for detecting a page service, characterized in that: The method comprises: The electronic device displays a first sub-interface of the first interface; The electronic device determines, from all services supported by the electronic device, services to be matched, where the services to be matched include services supported by all sub-interfaces included in the first interface; The electronic device acquires text content of the first sub-interface; The electronic device determines, based on the text content of the first sub-interface, from the services to be matched, a service provided by the first sub-interface.
2. The method according to claim 1, characterized in that Before the electronic device displays the first sub-interface of the first interface, the method further includes: The electronic device acquires first characteristic information of each service among all services supported by the electronic device, where the first characteristic information includes an activity name of a user interface where the service is located; The to-be-matched service includes: among all services supported by the electronic device, a service whose activity name in the user interface is the same as the activity name in the first interface.
3. The method according to claim 1 or 2, characterized in that: The electronic device obtains the text content of the first sub-interface, specifically including: The electronic device traverses the root node view of the first interface and obtains text contents of all sub-interfaces contained in the first interface; The electronic device determines the text content of the first sub-interface from the text content of all sub-interfaces included in the first interface.
4. The method according to claim 3, characterized in that Before the electronic device displays the sub-interface of the first interface, the method further includes: The electronic device acquires second characteristic information of each service among all services supported by the electronic device, where the second characteristic information includes a first method, the first method is a filtering node method, and the first method indicates a filtering node and a subnode of the filtering node; The electronic device traverses the root node view of the first interface, specifically including: The electronic device traverses the root node view of the first interface in a manner of filtering out the filtering node indicated by the first method and the child nodes of the filtering node.
5. The method according to claim 3 or 4, characterized in that: Before the electronic device displays the sub-interface of the first interface, the method further includes: The electronic device acquires second characteristic information of each service among all services supported by the electronic device, wherein the second characteristic information includes a second method, and the second method includes: a method for acquiring text content and a usage order; The electronic device obtains text content of all sub-interfaces included in the first interface, specifically including: The electronic device uses the second method to obtain text contents of all sub-interfaces contained in the first interface.
6. The method according to any one of claims 3 to 5, characterized in that: Before the electronic device displays the sub-interface of the first interface, the method further includes: The electronic device obtains second characteristic information of each service among all services supported by the electronic device, where the second characteristic information includes a third method, and the third method includes: whether to use a getLocalVisibleRect function and text content of using the getLocalVisibleRect function for a node; The electronic device determines the text content of the first sub-interface from the text content of all sub-interfaces included in the first interface, specifically including: The electronic device uses the third method to determine the text content of the first sub-interface from the text content of all sub-interfaces included in the first interface.
7. The method according to any one of claims 2 to 6, characterized in that: Before the electronic device displays the sub-interface of the first interface, the method further includes: The electronic device obtains third characteristic information of each service among all services supported by the electronic device, wherein the third characteristic information includes text content that distinguishes the service from other services; The electronic device determines, according to the text content of the first sub-interface, from the services to be matched, that the services provided by the first sub-interface specifically include: The electronic device determines, from the services to be matched, a service whose text content that is different from other services and is the text content of the first sub-interface as a service provided by the first sub-interface.
8. The method according to any one of claims 2 to 6, characterized in that: Before the electronic device displays the sub-interface of the first interface, the method further includes: The electronic device acquires third characteristic information of each service among all services supported by the electronic device, wherein the third characteristic information includes: text content that distinguishes the service from other services, and confidence; The electronic device determines, according to the text content of the first sub-interface, from the services to be matched, that the services provided by the first sub-interface specifically include: The electronic device determines, from the services to be matched, a service whose text content that is different from other services and that meets the corresponding confidence level with the text content of the first sub-interface as a service provided by the first sub-interface.
9. The method according to any one of claims 1 to 8, characterized in that After the electronic device determines, from the to-be-matched services, the service provided by the first sub-interface according to the text content of the first sub-interface, the method further includes: The electronic device recommends first information related to the service provided by the first sub-interface.
10. An electronic device, characterized in that: The electronic device includes a memory and one or more processors; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the electronic device to execute the method as described in any one of claims 1-9.
11. A chip, which is applied to electronic equipment, characterized in that: The chip includes one or more processors, and the processor is used to call computer instructions to enable the electronic device to execute the method as described in any one of claims 1-9.
12. A computer-readable storage medium comprising instructions, characterized in that: When the instructions are executed on an electronic device, the electronic device is caused to execute the method as claimed in any one of claims 1 to 9.
