System control operation method and electronic equipment

Abstract

The invention provides a system control operation method and electronic equipment. An operating system of the electronic equipment can provide intelligent capabilities such as content search capability, content generation capability and task identification capability for an upper-layer application, so that the upper-layer application can provide corresponding intelligent services. Label attributes are newly added to a system control provided by the operating system. Different attribute values of the tag attributes may correspond to different intelligence capabilities. A developer of any application can set the label attribute for the system control, the application can be developed into the intelligent application capable of providing the intelligent service by means of the intelligent capability integrated by the operating system, and the intelligent capability does not need to be constructed by itself with time and energy. Therefore, the workload of developers can be greatly reduced and the development efficiency of the developers can be improved while the developers are helped to develop applications meeting diversified requirements of users.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to a system control operation method and an electronic device. Background Art

[0002] Nowadays, when using electronic devices, users often need to search for information and edit text in the electronic devices. The above information search may include searching for relevant information in data stored in the electronic device and / or data stored in the cloud. The above text editing may include rewriting, expanding, summarizing, and other re-creation processing of the target text.

[0003] In order to meet the above-mentioned needs of users for using electronic devices, developers often need to obtain permissions from multiple parties and collect information from multiple parties when developing applications (such as collecting various files and information stored locally or in the cloud by users, the user's text editing habits and preferences, etc.), and also need to perform complex business logic processing on the collected information to provide functions that meet user needs. This results in a large workload for developers. Summary of the invention

[0004] The present application provides a system control operation method and an electronic device. Based on the intelligent capabilities of the operating system, such as content search capability, content generation capability, and task recognition capability, the method can assist APP developers in developing APPs that can provide intelligent services, reduce the development workload of developers, and improve the development efficiency of developers.

[0005] In a first aspect, the present application provides a system control operation method. The method can be applied to an electronic device including a first application and a second application. In response to a first operation on a first system control included in the first application, the electronic device implements a first function, wherein the first system control has a first attribute value, the first attribute value corresponds to a first function, and the first function includes a content search function, or a task identification function, or a content generation function; in response to a second operation on a second system control included in the second application, the electronic device implements the first function, wherein the second system control has the first attribute value.

[0006] The first system control and the second system control can both be one of system controls such as a text box, an edit box, a button, etc. For example, the first system control and the second system control can refer to the present application Figure 4A edit box 411 as shown, or Figure 5A The smart write control 513A shown, or Figure 5B The controls in the task options box 514 shown, or Figure 6BThe AI ​​edit box 612 is shown. The first attribute value may be AI search, or AI generate, or AI action. The first attribute value may be an attribute value corresponding to the flag attribute in the system control.

[0007] It can be seen that system controls in different applications can provide intelligent services (i.e., the first function) when they all have the first attribute value. Developers of any application can set the attribute value corresponding to the intelligent service (such as the first attribute value) for the system control during development, so that the system control can provide the corresponding intelligent service. The above development operation of setting attribute values ​​for the system control is simple and convenient. This can greatly reduce the workload of developers and improve their development efficiency while helping them develop applications that meet the diverse needs of users (such as information search needs, text editing needs, etc.).

[0008] In combination with the first aspect, in some embodiments, the operating system of the electronic device has a first capability, the first capability is used to implement a first function, the first attribute value is associated with the first capability, and the electronic device can call the first capability to implement the first function.

[0009] The first capability may be an intelligent capability, for example, a content search capability, or a content generation capability, or a task recognition capability. The operating system of the electronic device may provide an interface for calling intelligent capabilities such as the first capability to the upper-level application. When a first operation acting on the above-mentioned first system control is detected, the first application in the electronic device may call the first capability through the corresponding interface to implement the first function. When a second operation acting on the above-mentioned second system control is detected, the second application in the electronic device may call the first capability through the corresponding interface to implement the first function.

[0010] It can be seen that when the system control has the attribute value associated with the intelligent capability, the application can call the corresponding intelligent capability according to the operation acting on the system control and provide the corresponding intelligent service. Therefore, the developer of any application can set the attribute value associated with the intelligent capability for the system control during development, so that the system control can provide the corresponding intelligent service based on the intelligent capability of the operating system. The application developer does not need to spend a lot of time and energy to build the above-mentioned intelligent capabilities by himself. He only needs to use the intelligent capabilities integrated by the operating system to develop the application he develops into an intelligent application that can provide intelligent services. The above method can help developers develop applications that meet the diverse needs of users (such as information search needs, text editing needs, etc.), while greatly reducing the workload of developers and improving the development efficiency of developers.

[0011] In combination with the first aspect, in some embodiments, the first system control has a second attribute value, the second attribute value corresponds to a second function, the second function is different from the first function, and the second function includes a content search function, or a task identification function, or a content generation function; in response to the second operation on the first system control, the electronic device implements the second function.

[0012] It can be seen that a system control can be set with attribute values ​​corresponding to multiple intelligent services. That is, a system control can provide users with multiple intelligent services. For example, if a system control has the attribute values: AI search and AIgenerate, then this system control can provide both content search and content generation functions. This system control can effectively improve the user experience of text editing or information search.

[0013] In combination with the first aspect, in some embodiments, the operating system of the electronic device has a second capability, the second capability is used to implement a second function, the second attribute value is associated with the second capability, the electronic device implements the second function, and the electronic device calls the second capability to implement the second function.

[0014] The second capability may be an intelligent capability, such as content search capability, content generation capability, or task recognition capability. The operating system of the electronic device may provide an interface for invoking the intelligent capability such as the second capability to the upper layer application.

[0015] It can be seen that when the system control has the attribute value associated with the intelligent capability, the application can call the corresponding intelligent capability according to the operation acting on the system control and provide the corresponding intelligent service. Therefore, the developer of any application can set the attribute value associated with the intelligent capability for the system control during development, so that the system control can provide the corresponding intelligent service based on the intelligent capability of the operating system. The application developer does not need to spend a lot of time and energy to build the above-mentioned intelligent capabilities by himself. He only needs to use the intelligent capabilities integrated by the operating system to develop the application he develops into an intelligent application that can provide intelligent services. The above method can help developers develop applications that meet the diverse needs of users (such as information search needs, text editing needs, etc.), while greatly reducing the workload of developers and improving the development efficiency of developers.

[0016] In combination with the first aspect, in some embodiments, the first function is a content search function; in response to the first operation, the electronic device determines a query statement, and searches for the first content according to the query statement; the electronic device displays the first content.

[0017] In the case where the first function is a content search function, the first operation can refer to the present application Figure 4AThe operation for triggering a query (such as inputting a preset symbol 412 in the edit box 411) and the operation for inputting a query statement (such as inputting text 413 after the preset symbol 412) are shown. Figure 4A Text 413 shown.

[0018] The electronic device may use a language model to determine the semantic vector of the query statement, and search a semantic vector database for one or more semantic vectors that are most similar to the semantic vector of the query statement based on the semantic vector of the query statement. The first content may be a file or text content corresponding to the one or more most similar semantic vectors.

[0019] The electronic device displays the first content, which can refer to this application Figure 4A The search results are shown displayed in list box 414.

[0020] It can be seen that when the first system control is able to provide a content search function, the user can enter a query statement in the first system control at any time during text editing to query the information he needs, without having to exit the text editing interface and then search for information. In addition, the electronic device can insert the queried content into the first system control. The above content search function can help users quickly find the required information when editing text, and reduce the situation where the user's text editing ideas are interrupted due to information search. The above method can effectively enhance the user's text editing experience and improve the efficiency of user text editing.

[0021] In combination with the first aspect, in some embodiments, the first function is a task identification function. In response to the first operation, the electronic device obtains the first text and determines one or more task types for processing the first text. The task types include one or more of the following: expansion, rewriting, summarizing, translation, insertion, and meeting minutes. The electronic device displays options corresponding to one or more task types.

[0022] In the case where the first function is a task identification function, the first operation can refer to the present application Figure 5A The operation of the smart writing control 513A is shown. The first text can refer to the application Figure 5A The text 512 shown. The first text may be selected by the user, or may be intelligently selected by the electronic device.

[0023] The electronic device may use the language model to determine one or more task types suitable for processing the first text.

[0024] The electronic device displays one or more options corresponding to the task type, which can refer to this application Figure 5B The controls displayed in task options box 514 are shown.

[0025] It can be seen that the electronic device can estimate the user's need to process the first text based on the first text, and then provide the user with a suitable task type for processing the first text. In this way, when the user needs to process the first text during text editing, the user can quickly find the required task type to process the first text without having to search for the required task type one by one among a large number of task types. This can improve the user's experience of text editing.

[0026] In combination with the first aspect, in some embodiments, the first function is a content generation function, the first system control indicates a first task type, the first task type includes expansion, or rewriting, or summarizing, or translating, or meeting minutes, and in response to the first operation, the electronic device obtains the second text, and processes the second text according to the first task type to generate a third text; the electronic device displays the third text.

[0027] For example, if the first task type is expansion, the first system control can refer to the present application Figure 5B The expansion control 514A shown. If the first task type is rewriting, the first system control can refer to this application Figure 5B The rewrite control 514B shown. In the case where the first function is a content generation function, the first operation can refer to the present application Figure 5B Operation of the expand control 514A is shown.

[0028] The second text can refer to this application Figure 5A The text 512 shown, or Figure 5C The text in the edit box 515 is shown. The second text may be selected by the user, or may be intelligently selected by the electronic device.

[0029] The electronic device may use the language model to process the second text according to the first task type.

[0030] It can be seen that during the text editing process, the user can select the corresponding text at any time to expand, rewrite, summarize or translate, etc., without exiting the text editing interface and re-creating the existing text. This can improve the efficiency of the user's text editing.

[0031] In a second aspect, the present application provides an electronic device, which may include a memory and a processor. The memory may be used to store a computer program. The processor may be used to call the computer program so that the electronic device executes any possible implementation method in the first aspect.

[0032] In a third aspect, the present application provides a computer-readable storage medium, comprising instructions, which, when executed on an electronic device, enable the electronic device to execute any possible implementation method in the first aspect.

[0033] In a fourth aspect, the present application provides a computer program product, which may include computer instructions. When the computer instructions are run on an electronic device, the electronic device executes any possible implementation method in the first aspect.

[0034] In a fifth aspect, the present application provides a chip, which is applied to an electronic device, and the chip includes one or more processors, and the processor is used to call computer instructions to enable the electronic device to execute any possible implementation method as in the first aspect.

[0035] It can be understood that the electronic device provided in the second aspect, the computer-readable storage medium provided in the third aspect, the computer program product provided in the fourth aspect, and the chip provided in the fifth aspect are all used to execute the method provided in the embodiment of the present application. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method, which will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 is a schematic diagram of the hardware structure of an electronic device 100 provided in an embodiment of the present application;

[0037] Figure 2 is a software structure block diagram of an electronic device 100 provided in an embodiment of the present application;

[0038] Figure 3 is a schematic diagram of setting system control properties provided by an embodiment of the present application;

[0039] Figure 4A and Figure 4B is a schematic diagram of a scenario for performing content search provided in an embodiment of the present application;

[0040] Figure 5A to Figure 5C This is a schematic diagram of a scenario for performing task identification and content generation provided in an embodiment of the present application;

[0041] Figure 6A to Figure 6F It is a schematic diagram of a scenario for performing content search, content generation, and task identification provided by an embodiment of the present application;

[0042] Figure 7 is a flow chart of a method for content search provided by an embodiment of the present application;

[0043] Figure 8 This is a flow chart of a method for task identification and content generation provided in an embodiment of the present application. DETAILED DESCRIPTION

[0044] The technical solutions in the embodiments of the present application are described below in conjunction with the drawings in the embodiments of the present application. Among them, in the description of the embodiments of the present application, the terms used in the following embodiments are only for the purpose of describing specific embodiments, and are not intended to be used as limitations on the present application. As used in the specification and the appended claims of the present application, the singular expressions "a", "said", "above", "the" and "this" are intended to also include expressions such as "one or more", unless there is a clear indication to the contrary in the context. It should also be understood that in the following embodiments of the present application, "at least one", "one or more" refer to one or more (including two). The term "and / or" is used to describe the association relationship of associated objects, indicating that three relationships can exist; for example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the associated objects before and after are in a "or" relationship.

[0045] References to "one embodiment" or "some embodiments" etc. described in this specification mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Thus, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways. The term "connection" includes direct connection and indirect connection, unless otherwise specified. "First" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated.

[0046] In the embodiments of the present application, the words "exemplarily" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplarily" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplarily" or "for example" is intended to present related concepts in a specific way.

[0047] The term "user interface (UI)" in the following embodiments of the present application refers to the medium interface for interaction and information exchange between an application (APP) or an operating system (OS) and a user, which realizes the conversion between the internal form of information and the form acceptable to the user. The user interface is a source code written in a specific computer language such as Java and extensible markup language (XML). The interface source code is parsed and rendered on an electronic device and finally presented as content that can be recognized by the user. The commonly used form of user interface is the graphical user interface (GUI), which refers to a user interface related to computer operation displayed in a graphical manner. It can be a visual interface element such as text, icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets, etc. displayed on the display screen of an electronic device.

[0048] To facilitate understanding, the concept of "system control" involved in this application is first introduced here.

[0049] An operating system is installed in an electronic device. An operating system is a set of interrelated system software programs that manage and control the operation of electronic devices, use and run hardware and software resources, and provide public services to organize user interactions. An operating system is an interface between users and electronic devices. Apps in electronic devices run based on an operating system.

[0050] System controls can be pre-packaged interface controls provided by the operating system to developers. System controls are helpful for developers to develop quickly and keep the interface of the APP in the operating system consistent. For example, system controls can include but are not limited to: text boxes (TextView), edit boxes (EditView), buttons (Button), list controls (ListView), etc.

[0051] System controls can be declared and used in layout files. When declaring, you also need to set the properties of the system controls to control their display effects in the interface. For example, developers can declare system controls in layout files and set one or more of the following properties of the system controls: width, height, color, display position, etc. In this way, developers can set the layout of the application interface and the display style of each system control in the application interface according to the development requirements of the APP.

[0052] In some embodiments, the system control can also be used to implement one or more specific functions. For example, monitoring preset input, content search, content generation, task identification and other functions. The above-mentioned monitoring preset input function can represent the function of monitoring preset operations (such as click operations, long press operations, etc.) acting on the system control, or monitoring the function of inputting preset characters or text and other contents on the system control. The above-mentioned content search function can represent the function of searching for relevant content locally and / or in the cloud of the electronic device based on the query statement input by the user. The above-mentioned content generation function can represent the function of generating a new text of a specified task type based on the target text. The above-mentioned task identification function can represent the function of determining one or more task types suitable for processing the target text based on the target text, so that the user can select the corresponding task type to process the target text. The above-mentioned task types may include but are not limited to: expansion, rewriting, summarization, translation, meeting minutes, etc.

[0053] If you want to use system controls to implement certain specific functions, you can set the properties of the system controls in the layout file. Furthermore, you can add the implementation methods of the properties corresponding to the above functions in the code. For example, In the operating system, developers can add implementation methods for attributes corresponding to functions in activities. The process of using system controls to implement specific functions may include: (1) setting attributes corresponding to specific functions for system controls in the layout file; (2) using the super.setContentView() method to load the layout file; (3) using the super.findViewById() method to obtain a system control reference; (4) using the system control reference to operate the control, such as adding a listener, calling an interface that implements a content search function, calling an interface that implements a content generation function, calling an interface that implements a task identification function, and so on. The above method of using system controls to implement specific functions is only an exemplary description of this application and should not constitute a limitation on this application.

[0054] The present application provides a system control operation method, which can assist APP developers in developing APPs that can provide intelligent services, reduce the development workload of developers, and improve the development efficiency of developers. The above-mentioned intelligent services may include one or more of the following: content search service (or content search function), content generation service (or content generation function), task identification service (or task identification function).

[0055] Specifically, the operating system of the electronic device has one or more of the following intelligent capabilities: content search capability, content generation capability, and task identification capability. The above-mentioned content search capability can be used to implement the above-mentioned content search function. The above-mentioned content generation capability can be used to implement the content generation function. The above-mentioned task identification capability can be used to implement the task identification function. The operating system can provide an interface for calling these intelligent capabilities to the upper-layer application so that the upper-layer application can provide corresponding intelligent services. The system control provided by the operating system has added a tag (flag) attribute, so that the system control can implement the above-mentioned intelligent service. The flag attribute can include one or more of the following attribute values: AI search, AI generate, AI action, etc. Different attribute values ​​may correspond to different intelligent capabilities. For example, the attribute value AI search may correspond to content search capability. The attribute value AIgenerate may correspond to content generation capability. The attribute value AI action may correspond to task identification capability. If the system control is set with one or more attribute values ​​in the above-mentioned flag attribute, the system control can provide corresponding intelligent services.

[0056] Therefore, any APP developer can set the above flag attribute for the system control during development, so that the system control can provide corresponding intelligent services based on the intelligent capabilities of the operating system. APP developers do not need to spend a lot of time and energy to build the above intelligent capabilities by themselves. They can develop their developed APPs into intelligent applications that can provide intelligent services with the help of the intelligent capabilities integrated by the operating system. The above method can help developers develop APPs that meet the diverse needs of users (such as information search needs, text editing needs, etc.), while greatly reducing the workload of developers and improving their development efficiency.

[0057] The structure of the electronic device 100 involved in this application is introduced below.

[0058] The electronic device 100 may be a mobile phone, a tablet computer, a laptop computer, a smart watch, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), etc. The specific type of the electronic device 100 is not limited in the embodiment of the present application.

[0059] Figure 1 The schematic diagram of the hardware structure of the electronic device 100 is exemplarily shown.

[0060] like Figure 1As shown, the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc.

[0061] It is to be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0062] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0063] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0064] The processor 110 may also be provided with a memory for storing instructions and data. In some examples, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0065] In the present application, a computer program may be stored in the memory, which is used to enable the controller or processor to implement the system control operation method of the present application through an interface or protocol.

[0066] Exemplarily, the computer program stored in the memory can be used to: obtain a query statement, determine the semantics of the query statement, search for relevant files from the local and / or cloud of the electronic device 100 based on the semantics of the query statement, and extract the information to be queried by the query statement from the relevant files, identify the purpose of the information to be queried by the query statement, and provide corresponding functional controls for the user to use the above-mentioned queried information.

[0067] The USB interface 130 is an interface that complies with USB standard specifications, and may be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 may be used to connect a charger to charge the electronic device 100, or may be used to transmit data between the electronic device 100 and a peripheral device. It may also be used to connect headphones to play audio through the headphones.

[0068] The charging management module 140 is used to receive charging input from a charger. The charger can be a wireless charger or a wired charger. While the charging management module 140 charges the battery 142, it can also power the electronic device through the power management module 141.

[0069] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140 to power the processor 110, the internal memory 121, the external memory, the display screen 194, the camera 193, and the wireless communication module 160.

[0070] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0071] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0072] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, etc. the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1.

[0073] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0074] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which is connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering.

[0075] The display screen 194 is used to display images, videos, etc. In some embodiments, the electronic device 100 may include 1 or N display screens 194 , where N is a positive integer greater than 1.

[0076] The electronic device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0077] The ISP is used to process data fed back by the camera 193. For example, when taking a photo, the shutter is opened, light is transmitted to the camera photosensitive element through the lens, and the light signal is converted into an electrical signal. The camera photosensitive element transmits the electrical signal to the ISP for processing and converts it into an image visible to the naked eye.

[0078] The camera 193 is used to capture still images or videos. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0079] The digital signal processor is used to process digital signals, and can process not only digital image signals but also other digital signals. For example, when the electronic device 100 is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0080] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn. NPU can realize applications such as intelligent cognition of electronic device 100, such as image recognition, face recognition, voice recognition, text understanding, text generation, etc.

[0081] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function, such as storing music, video and other files in the external memory card.

[0082] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a content search function, a text editing function, a language instruction recognition function, etc.), etc. In some embodiments, the program storage area may also store a program for implementing a language model, and a program for implementing the above-mentioned intelligent capabilities. The data storage area may store data (such as audio data, a phone book, etc.) created during the use of the electronic device 100. In some embodiments, the data storage area may also store a semantic vector database. The semantic vector database may store semantic vectors associated with files locally and / or in the cloud of the electronic device 100. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0083] The electronic device 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0084] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some examples, the audio module 170 can be arranged in the processor 110, or some functional modules of the audio module 170 can be arranged in the processor 110. The speaker 170A, also known as the "speaker", is used to convert audio electrical signals into sound signals. The receiver 170B, also known as the "earpiece", is used to convert audio electrical signals into sound signals. The microphone 170C, also known as the "microphone" or "microphone", is used to convert sound signals into electrical signals. The headphone jack 170D is used to connect wired headphones.

[0085] The sensor module 180 may include a pressure sensor, a gyro sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a gravity sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, and the like.

[0086] The buttons 190 include a power button, a volume button, etc. The motor 191 can generate a vibration prompt. The indicator 192 can be an indicator light, which can be used to indicate the charging status, power change, messages, missed calls, notifications, etc.

[0087] The SIM card interface 195 is used to connect a SIM card. The SIM card can be connected to and separated from the electronic device 100 by inserting it into the SIM card interface 195 or removing it from the SIM card interface 195. The electronic device 100 can support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The electronic device 100 interacts with the network through the SIM card to implement functions such as calls and data communications. In some examples, the electronic device 100 uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100.

[0088] The software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. Taking the system as an example, the software structure of the electronic device 100 is exemplified.

[0089] Figure 2 The software structure block diagram of the electronic device 100 is exemplarily shown.

[0090] The layered architecture divides the software into several layers, each with a clear role and division of labor. The layers communicate with each other through software interfaces. The system is divided into four layers, from top to bottom: application layer, application framework layer, Android Runtime and system library, and kernel layer.

[0091] The application layer can include a series of application packages.

[0092] like Figure 2 As shown, the application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, text editing, short message, etc. Among them, the text editing application can perform text editing and creation.

[0093] In some embodiments, the text editing application may provide a content search function. In this way, when the user performs text editing in the text editing application, the user can search the local and / or cloud content of the electronic device 100 at any time to find the information needed for text editing. The above-mentioned search for the local content of the electronic device 100 may represent the search for the content stored in the electronic device 100, for example, pictures in the gallery application of the electronic device 100, memo information in the memo application, documents in the file application, etc. The above-mentioned search for the content in the cloud may include searching for the content in the cloud storage space associated with the electronic device 100. For example, a device account is logged in to the electronic device 100. The electronic device 100 can search the cloud storage space of the device account. For another example, a cloud disk application account is logged in to the cloud disk application of the electronic device 100. The electronic device 100 can search the cloud storage space of the cloud disk application account. The above-mentioned cloud storage space may also be referred to as a cloud disk, a network disk, etc. Optionally, the above-mentioned search for the content in the cloud may also include searching for web page content. The above-mentioned web page content may represent content on the Internet, for example, news, goods, addresses, papers, etc. on the Internet. The embodiment of the present application does not limit the scope that the electronic device 100 can search.

[0094] In some embodiments, the text editing application can also provide a content generation function. In this way, when the user is editing the text, the target text can be processed accordingly. For example, the target text can be expanded, rewritten, summarized, translated, etc.

[0095] In some embodiments, the text editing application can also provide a task identification function to identify the user's need for text editing, and then provide the user with a suitable task type for processing the target text. In this way, when the user needs to process the target text during the text editing process, the user can quickly find the required task type to process the target text without having to search for the required task type one by one among a large number of task types. This can improve the user's text editing experience.

[0096] Not limited to text editing applications, the electronic device 100 may also include more other APPs that can provide one or more of the above-mentioned intelligent services.

[0097] The application framework layer provides API and programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0098] like Figure 2 As shown, the application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, an activity manager, a content search module, a content generation module, a task identification module, and the like.

[0099] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.

[0100] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0101] The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images.

[0102] The phone manager is used to provide communication functions of the electronic device 100, such as management of call status (including connecting, hanging up, etc.).

[0103] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0104] The notification manager enables applications to display notification information in the status bar (such as the pull-down notification bar). It can be used to convey notification-type messages and can automatically disappear after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as notifications of applications running in the background, or a notification that appears on the screen in the form of a dialog window. For example, a text message is displayed in the status bar, a prompt sound is emitted, an electronic device vibrates, an indicator light flashes, etc.

[0105] The activity manager is responsible for managing activities, starting, switching, and scheduling components in the system, as well as managing and scheduling applications. The activity manager can be called by upper-level applications to open corresponding activities.

[0106] The content search module can be used to provide an interface for calling content search capabilities to upper-layer applications. Applications can call the content search capabilities of the operating system through this interface, thereby providing content search functions for users.

[0107] The content generation module can be used to provide an interface for calling content generation capabilities to upper-level applications. Applications can call the content generation capabilities of the operating system through the interface, thereby providing users with content generation functions.

[0108] The task identification module can be used to provide an interface for calling the task identification capability to the upper-level application. The application can call the task identification capability of the operating system through the interface, thereby providing the task identification function for the user.

[0109] Android Runtime includes core libraries and virtual machines. Android Runtime is responsible for scheduling and management of the Android system.

[0110] The core library consists of two parts: one is the function that needs to be called by the Java language, and the other is the Android core library.

[0111] The application layer and the application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0112] The system library may include multiple functional modules, such as surface manager, media library, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0113] The surface manager is used to manage the display subsystem and provide the fusion of 2D and 3D layers for multiple applications.

[0114] The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0115] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0116] A 2D graphics engine is a drawing engine for 2D drawings.

[0117] The kernel layer is the layer between hardware and software. The kernel layer contains at least display driver, camera driver, audio driver, and sensor driver.

[0118] Figure 3 A schematic diagram of setting system control properties based on the intelligent capabilities provided by the operating system provided in an embodiment of the present application is exemplified.

[0119] like Figure 3As shown, the interface of APP1 may include one or more of the following system controls: text box, edit box, button, etc. For example, APP1 may be a text editing application, or a social communication application, or a browser application, etc. The embodiment of the present application does not limit the type of APP1.

[0120] During the development process, the developer of APP1 can set the properties of each system control in APP1. In addition to setting the properties related to the display style of the system control (such as width, height, color, position, etc.), the developer can also set the properties corresponding to the functions that the system control needs to implement.

[0121] For example, if the developer wants a system control in APP1 to provide the above-mentioned intelligent service, the developer can set a flag attribute for the system control and set the attribute value of the flag attribute of the system control to the attribute value corresponding to the intelligent service that needs to be provided. Furthermore, the developer can add the implementation method of the corresponding attribute value of the intelligent service in the activity so that the system control can call the interface of the intelligent capability to provide the corresponding intelligent service.

[0122] The attribute values ​​of the flag attribute may include AI search, AI generate, and AI action. Among them, the operating system of the electronic device 100 has intelligent capabilities such as content search capability, content generation capability, and task recognition capability. The attribute value AI search may correspond to content search capability. The attribute value AI generate may correspond to content generation capability. The attribute value AI action may correspond to task recognition capability. The names of the above attribute values ​​are only exemplary descriptions of this application and should not constitute a limitation on this application.

[0123] For example, the developer sets the attribute value of the flag attribute of a button to AI action. Then, this button can provide a task identification function. In response to an operation on this button, such as a click operation, the electronic device 100 can call the task identification capability, determine one or more task types suitable for processing the target text based on the target text, and provide corresponding task options for the user to select.

[0124] In some embodiments, developers can set flag attributes for system controls in the layout file of system controls. For example, developers can add "android:flag="AI search"" in the layout file of system controls to add a flag attribute with an attribute value of AI search to the system control. Developers can add "android:flag="AI generate"" in the layout file of system controls to add a flag attribute with an attribute value of AI generate to the system control. Developers can add "android:flag="AI action"" in the layout file of system controls to add a flag attribute with an attribute value of AI action to the system control.

[0125] In some embodiments, a system control can be set with multiple attribute values ​​in the above flag attribute. That is, a system control can provide multiple intelligent services for users. For example, if a developer adds flag attributes with attribute values ​​of AI search and AI generate to a system control, then this system control can provide both content search function and content generation function.

[0126] The implementation methods of the above-mentioned content search capability, content generation capability, and task identification capability will be introduced in subsequent embodiments, but will not be expanded here.

[0127] like Figure 3 As shown, intelligent capability implementation resources may be stored in the electronic device 100. The intelligent capability of the operating system of the electronic device 100 may be implemented based on the intelligent capability implementation resources. The intelligent capability implementation resources may include a language model and a semantic vector database.

[0128] The language model may refer to an artificial intelligence (AI) model for processing language (or symbol system), which can be used to generate text that meets the requirements according to the prompt information (prompt), and ensure that the output result is related to the prompt. The language model may include a language model based on probability theory and statistics (i.e., a statistical language model), a language model based on deep learning and neural network implementation (i.e., a neural network language model), and the like. The embodiment of the present application does not limit the implementation method of the language model. Exemplarily, the electronic device 100 may implement the system control operation method of the present application through a large language model (largelanguagemodel, LLM). Among them, LLM may be a neural network model that can learn the grammar and semantics of natural language to generate human-readable text. Among them, LLM may be obtained by training using a large-scale corpus. The above-mentioned corpus may include a large amount of text data. LLM can handle various natural language processing tasks, such as natural language generation, text error correction, text classification, text summarization, machine translation, speech recognition, and the like.

[0129] Among them, the above prompt can be a natural language that humans can understand (i.e., a hard prompt (hard prompt / discrete prompt)). For example, the prompt can be a text or sentence composed of natural language. Alternatively, the prompt can be machine-readable content, which belongs to a non-natural language (i.e., a soft prompt (soft prompt / continuous prompt)). For example, a soft prompt can be an embedding, which is a word converted into a meaning vector representation processed by a computer. Hard prompts are more interpretable. Soft prompts can more flexibly eliminate the parameter restrictions of the language model. Prompt can be used to start and guide the language model to generate output of a specified type, theme or format. Exemplarily, in the field of natural language processing, a prompt can include a set of questions and answers or task descriptions. For example, the prompt is: "Give your pet three names", "Translate this English sentence into French", etc. For another example, the prompt can also include question and answer examples. The prompt for the task description "Give your pet three names" may include question-and-answer examples: "Pet type: cat, pet name: name 1, name 2, name 3", "Pet type: dog, pet name: name 4, name 5, name 6". It can be understood that the question-and-answer examples included in the above prompt can be used to guide the language model to answer the user's questions in the manner of the question-and-answer examples. The language model can generate text that meets the requirements based on the prompt and ensure that the output result is related to the prompt.

[0130] The semantic vector database may store semantic vectors associated with files locally in the electronic device 100 and / or in the cloud. A file may be associated with one or more semantic vectors.

[0131] In some embodiments, the electronic device 100 may establish a semantic vector database. Specifically, the electronic device 100 may obtain files locally on the electronic device 100 and / or in the cloud, and extract semantic vectors from the obtained files using a semantic vector extraction model. The electronic device 100 may store the extracted semantic vectors in the semantic vector database.

[0132] The above-mentioned semantic vector extraction model can be used to extract the semantic vector of given content. The above-mentioned given content may include but is not limited to data of types such as text, tables, and pictures. The semantic vector extraction model may be a model based on a neural network. The embodiment of the present application does not limit the implementation method of the semantic vector extraction model. The electronic device 100 inputs a section of content into the semantic vector extraction model, and the semantic vector of this section of content can be obtained through the semantic vector extraction model. The semantic vector of this section of content can represent the semantics of this section of content. For example, the semantics of this section of content may include an overview of the information described in this section of content (the theme or main idea of ​​this section of content). For example, the electronic device 100 inputs a section of text introducing cats into the semantic vector extraction model, and the semantic vector of this section of text obtained can be used to indicate that this section of text is a text introducing cats.

[0133] In some embodiments, a file may contain multiple contents (or multiple data fragments). The electronic device 100 can split the contents in a file and determine the semantic vectors associated with each data fragment after the split. In this way, a file can be associated with multiple semantic vectors. For example, a text file may contain multiple paragraphs of text. The electronic device 100 can split the text file according to paragraphs and extract the semantic vector associated with each paragraph through a semantic vector extraction model. It can be understood that the themes to be expressed in different paragraphs or chapters in a text file may be different. For example, the first paragraph of a text file introduces the types of cats, and the second paragraph introduces the living habits of cats. The above-mentioned splitting of the file and then extracting the semantic vectors of the split data fragments can facilitate the electronic device 100 to more accurately find the content that the user wants to search when providing a content search function.

[0134] The present application embodiment does not limit the granularity of the above-mentioned split file. For example, the granularity of splitting the file may include but is not limited to: by text sentence, by text paragraph, by text chapter, by file page, by table row, by table column, etc.

[0135] Optionally, the electronic device 100 may also treat a file as a whole and extract a semantic vector associated with the file through a semantic vector extraction model, that is, a file may be associated with a semantic vector.

[0136] Optionally, the semantic vector database may also be established by other devices other than the electronic device 100, such as a cloud server. After establishing the semantic vector database, the cloud server may send the semantic vector database to the electronic device 100.

[0137] In some embodiments, when the files stored locally on the electronic device 100 change, or when the files stored in the cloud storage space associated with the electronic device 100 change, the electronic device 100 can update the semantic vector database. The file changes may include but are not limited to: newly added files, changes in the content of the files, deletion of existing files, etc.

[0138] Depend on Figure 3 It can be seen that the electronic device 100 stores intelligent capability implementation resources. Based on the intelligent capability implementation resources, the operating system in the electronic device 100 can provide intelligent capabilities to upper-level applications. The developer of any APP can set the above flag attribute for the system control during development, so that the system control can provide corresponding intelligent services based on the intelligent capabilities of the operating system. The intelligent capabilities integrated in the operating system can help developers develop APPs that meet the diverse needs of users (such as information search needs, text editing needs, etc.), while greatly reducing the workload of developers and improving the development efficiency of developers.

[0139] The following introduces some scenarios of using intelligent services based on system controls with flag attributes added, provided in embodiments of the present application.

[0140] Here we take the system controls in the Memo app as an example. For system controls with flag attributes added to other apps, refer to the system controls in the Memo app.

[0141] Figure 4A and Figure 4B A schematic diagram of a scenario for performing content search is exemplified.

[0142] like Figure 4A As shown, the electronic device 100 may display a user interface 410 of a memo application. The system controls in the user interface 410 may include an edit box 411. The edit box 411 may be used for text input and editing. The edit box 411 may be added with a flag attribute with an attribute value of "AI search". That is, the edit box 411 may provide a content search function.

[0143] In some embodiments, the user can trigger the content search function by entering a preset symbol in the edit box 411. For example, the user enters the preset symbol 412 in the edit box 411. The preset symbol 412 can be " / @". The user can enter a query statement after the preset symbol 412 to search for the content they want to find. When the preset symbol 412 is detected in the edit box 411, the electronic device 100 can detect whether the edit box 411 has a flag attribute with an attribute value of "AI search". If the edit box 411 has a flag attribute with an attribute value of "AI search", the electronic device 100 can determine the text content after the preset symbol 412 as a query statement.

[0144] like Figure 4A As shown, the text content after the preset symbol 412 in the edit box 411 is text 413. The content of text 413 is "resume". The electronic device 100 can use text 413 as a query statement, call the content search capability of the operating system, and search for files or texts and other contents associated with the query statement locally on the electronic device 100 and / or in the cloud.

[0145] The electronic device 100 can display the search results on the screen. For example, using text 413 as a query statement, the electronic device 100 can search for Figure 4A The electronic device 100 may display a list box 414. The list box 414 may include option 414A, option 414B, and option 414C. Different options in the list box 414 may correspond to different files searched by the electronic device 100. Not limited to the files corresponding to the above options 414A to 414C, the electronic device 100 may also display more or less searched content in the list box 414. For example, in response to an operation of sliding up or down in the list box 414, the electronic device 100 may display options corresponding to other search results in the list box 414.

[0146] The user can select the search results he needs in the list box 414. If the search results currently displayed in the list box 414 do not contain the content the user wants, the user can adjust the text after the preset symbol 412 in real time to change the query sentence until the desired content is found.

[0147] like Figure 4A As shown, in response to an operation on option 414A, such as a click operation or a touch operation, the electronic device 100 may display Figure 4B The user interface 410 shown. Among them, the electronic device 100 can delete Figure 4A The preset symbol 412 and text 413 are shown, and text 415 is displayed at the position where the preset symbol 412 and text 413 are displayed. The content of text 415 can be Figure 4A The name of the file corresponding to option 414A selected by the user is shown, for example, "Resume-V1.pdf".

[0148] In some embodiments, in response to an operation on the text 415, such as a click operation, the electronic device 100 can open the file "Resume-V1.pdf" and display the content of the file.

[0149] It should be noted that the above-mentioned preset symbols for triggering the content search function are only exemplary illustrations of the present application and should not be construed as limiting the present application. The content of the above-mentioned preset symbols may also be other. The above-mentioned preset symbols may be single symbols, for example, " / " or "@" or "#", etc. The above-mentioned preset symbols may also be combined symbols, for example, " / #" or "@#", etc. Not limited to the above-mentioned preset symbols, the electronic device 100 may also determine the query statement for search when other preset operations are detected. For example, the above-mentioned preset operation may be an operation on a preset shortcut key. In response to the operation on the above-mentioned preset shortcut key, the electronic device 100 may search with part or all of the text in the edit box 411 as a query statement.

[0150] By the above Figure 4A and Figure 4B As can be seen from the scenario shown, when the edit box 411 can provide a content search function, the user can enter a query statement in the edit box 411 at any time during the text editing process to query the information he needs, without having to exit the text editing interface and then search for information. In addition, the electronic device 100 can insert the queried content into the edit box 411. The above content search function can help users quickly find the required information when editing text, and reduce the situation where the user's text editing ideas are interrupted due to information search. The above method can effectively enhance the user's text editing experience and improve the efficiency of user text editing.

[0151] Figure 5A to Figure 5C A schematic diagram of a scenario for performing task identification and content generation is shown as an example.

[0152] like Figure 5A As shown, the electronic device 100 can display a user interface 510 of a memo application. The user interface 510 may include an edit box 511. A text 512 may be entered in the edit box 511. For example, the content of the text 512 may be "cat and mouse". The content of the text 512 is only an exemplary description of the present application. In response to the operation of selecting the text 512, the electronic device 100 may display a function option box 513. The function option box 513 may include system controls: a smart write control 513A, a copy control 513B, and a paste control 513C.

[0153] The smart writing control 513A may be added with a flag attribute with an attribute value of "AI action", that is, the smart writing control 513A may provide a task identification function.

[0154] In some embodiments, in response to an operation on the smart writing control 513A, such as a click operation, the electronic device 100 may use the selected text 512 as the target text, call the task identification capability of the operating system, and determine one or more task types suitable for processing the text 512. The electronic device 100 may display options corresponding to the determined task types on the screen.

[0155] For example, the electronic device 100 determines that the task types suitable for processing the text 512 include: expansion and rewriting. The electronic device 100 may display Figure 5B Task options box 514 is shown. Task options box 514 may include an expand control 514A and an overwrite control 514B.

[0156] The expansion control 514A can be used to expand the text based on the text 512 to generate a new expanded text. The expansion control 514A can be added with a flag attribute with an attribute value of "AI generate". That is, the expansion control 514A can provide a content generation function.

[0157] The rewrite control 514B can be used to modify the expression of the text 512 to generate a rewritten new text. The rewrite control 514B can be added with a flag attribute with an attribute value of "AI generate". That is, the rewrite control 514B can provide a content generation function.

[0158] In some embodiments, in response to Figure 5B The electronic device 100 can use the selected text 512 as the target text, call the content generation capability of the operating system, and generate the expanded text of the text 512 by operating the expansion control 514A, such as clicking. The electronic device 100 can display the expanded text on the screen.

[0159] For example, Figure 5C As shown, in response to Figure 5B As shown in the operation of the expansion control 514A, the electronic device 100 can create a new edit box 515 on the user interface 510 and display the expanded text of the above text 512 in the edit box 515.

[0160] In some embodiments, the edit box 515 may be added with a flag attribute with an attribute value of "AI action". That is, the edit box 515 may provide a task identification function. In particular, when text is input in the edit box 515, the electronic device 100 may use the text in the edit box 515 as the target text, call the task identification capability of the operating system, and determine one or more task types suitable for processing the text in the edit box 515.

[0161] For example, the electronic device 100 determines that the task types suitable for processing the text in the edit box 515 include: rewriting, summarizing, and inserting. The electronic device 100 may display Figure 5C The task options box 516 is shown. The task options box 516 may include an overwrite control 516A, a summary control 516B, and an insert control 516C.

[0162] Rewriting control 516A can refer to the above Figure 5B Overwrite control 514B is shown.

[0163] The summary control 516B can be used to summarize the text in the edit box 515 and generate a new text after the summary. The summary control 516B can be added with a flag attribute with an attribute value of "AI generate". That is, the summary control 516B can provide a content generation function.

[0164] The electronic device 100 may use the summarized new text or the rewritten new text to replace the original text in the display edit box 515. Alternatively, the electronic device 100 may display the summarized new text or the rewritten new text after the original text in the edit box 515. The embodiment of the present application does not limit the display method of the rewritten new text and the summarized new text.

[0165] The insert control 516C may be used to insert the text in the edit box 515 into the edit box 511. For example, in response to an operation on the insert control 516C, the electronic device 100 may insert the text in the edit box 515 after the text 512.

[0166] In some embodiments, the target text for content regeneration in the edit box 515 may be selected by the user. For example, the user may select a portion of the text in the edit box 515. Then, the electronic device 100 may use the selected portion of the text as the target text, call the task identification capability of the operating system, and determine one or more task types suitable for processing the target text.

[0167] contrast Figure 5B and Figure 5CIt can be seen that when the target text is different, the task type determined by the electronic device 100 to be applicable to the target text may be different. For example, when the target text has less content, the user may need to expand the target text. When the target text is long, the user may need to summarize the target text. When the target text is in a foreign language (such as English, etc.), the user may need to translate the target text.

[0168] It should be noted that the above Figure 5B The task options box 514 and Figure 5C The task types corresponding to the controls in the task option box 516 are merely exemplary descriptions of the present application and should not constitute limitations on the present application.

[0169] Depend on Figure 5A to Figure 5C As can be seen from the scenario shown, the electronic device 100 can estimate the user's need to process the target text based on the target text, and then provide the user with a suitable task type for processing the target text. In this way, when the user needs to process the target text during text editing, the user can quickly find the required task type to process the target text without having to search for the required task type one by one among a large number of task types. This can improve the user's experience of text editing.

[0170] Furthermore, during the text editing process in the editing box 511, the user can select the corresponding text at any time for expansion, rewriting, summary or translation, etc., without exiting the text editing interface and re-creating the existing text. This can improve the efficiency of the user's text editing.

[0171] In some embodiments, a system control may be set with multiple attribute values ​​in the flag attribute. That is, a system control may provide multiple intelligent services. The following describes the scenario of using intelligent services based on a system control with multiple flag attribute values ​​added.

[0172] Figure 6A to Figure 6F A schematic diagram of a scenario for performing content search, content generation, and task identification is exemplified.

[0173] like Fig. 6A As shown, the electronic device 100 can display a user interface 610 of a memo application. The system controls in the user interface 610 may include: an AI control 611, a style control, a picture control, etc. The style control can be used to select a display style such as a font and a size of the input text. The picture control can be used to insert a picture when editing text. The AI ​​control 611 can be used to call up an AI edit box to input a language instruction instructing the electronic device 100 to execute in the AI ​​edit box. In response to Fig. 6A As shown in FIG. 6 , for example, when operating the AI ​​control 611, the electronic device 100 may display Figure 6B The AI ​​edit box 612 is shown. The user can enter a language instruction in the AI ​​edit box 612 that the electronic device 100 is expected to execute.

[0174] like Figure 6B As shown, the AI ​​edit box 612 may include a send control 612A. When a language instruction is input into the AI ​​edit box 612, in response to an operation on the send control 612A, the electronic device 100 may recognize and execute the language instruction in the AI ​​edit box 612.

[0175] In some embodiments, the AI ​​edit box 612 may be added with a flag attribute whose attribute value is “AI search”, that is, the AI ​​edit box 612 may provide a content search function.

[0176] like Figure 6C As shown, the user enters the text "Summarize this article" in the AI ​​edit box 612, and further enters the preset symbol 612B. The preset symbol 612B can be " / @". The preset symbol 612B can be used to trigger the content search function. The user can enter a query statement after the preset symbol 612B to search for the content they want to find. When the preset symbol 612B is detected in the AI ​​edit box 612, the electronic device 100 can detect whether the AI ​​edit box 612 has a flag attribute with an attribute value of "AI search". If the AI ​​edit box 612 has a flag attribute with an attribute value of "AI search", the electronic device 100 can determine the text content after the preset symbol 612B as a query statement. The preset symbol 612B can refer to the aforementioned Figure 4A The preset symbol 412 is shown.

[0177] like Figure 6C As shown, the text content after the preset symbol 612B in the AI ​​editing box 612 is text 612C. The content of text 612C is "prose". The electronic device 100 can use text 612C as a query statement, call the content search capability of the operating system, and search for files or texts and other contents associated with the query statement locally on the electronic device 100 and / or in the cloud.

[0178] The electronic device 100 may display the search results on the screen. For example, using the text 612C as a query statement, the electronic device 100 may search for Figure 6C The electronic device 100 may display a list box 613. The list box 613 may include an option 613A, an option 613B, and an option 613C. Different options in the list box 613 may correspond to different files searched by the electronic device 100.

[0179] In response to Figure 6CAs shown, when the option 613B is operated, such as a click operation or a touch operation, the electronic device 100 may display Fig.6D The user interface 610 shown in FIG. 100 can be deleted. Figure 6C The preset symbol 612B and text 612C are shown, and text 612D is displayed at the position of the preset symbol 612B and text 612C. The content of text 612D can be Figure 6C The name of the file corresponding to option 613B selected by the user is shown, for example, "Prose 2".

[0180] In some embodiments, the AI ​​edit box 612 may be added with a flag attribute whose attribute value is “AI generate”, that is, the AI ​​edit box 612 may provide a content generation function.

[0181] like Fig.6D As shown, in response to the operation of the send control 612A, the electronic device 100 can use the text in the AI ​​edit box 612 (e.g., "Summarize this article "Prose 2") as a language instruction, recognize and execute the language instruction. Among them, the electronic device 100 can use a language model to recognize the language instruction. The embodiment of the present application does not limit the method for the electronic device 100 to recognize the language instruction.

[0182] The electronic device 100 can recognize that the language instruction in the AI ​​edit box 612 is a language instruction for re-creating the target text, and the target text is the text recorded in the file named "Prose 2". Further, the electronic device 100 can detect whether the AI ​​edit box 612 has a flag attribute with an attribute value of "AI generate". If the AI ​​edit box 612 has a flag attribute with an attribute value of "AI generate", the electronic device 100 can call the content generation capability to summarize the file content of "Prose 2" and generate a summary text. The electronic device 100 can display the above summary text on the screen.

[0183] For example, Fig. 6E As shown, in response to Fig.6D The electronic device 100 can create a new edit box 614 on the user interface 610 by operating the send control 612A. In the process of generating the summary text, the electronic device 100 can generate and display the summary text word by word in the edit box 614. Optionally, the electronic device 100 can also display the summary text in the AI ​​edit box 612. Fig. 6E The prompt text “AI is modifying the content for you…” is shown to prompt the user that the electronic device 100 is executing the user's language instructions to generate a summary text of “Essay 2”.

[0184] like Fig. 6FAs shown, when the summary text is generated, the electronic device 100 can display the complete summary text in the editing box 614.

[0185] In some embodiments, the edit box 614 may be added with a flag attribute whose attribute value is "AI action". That is, the edit box 614 can provide a task identification function. Among them, when text is entered in the edit box 614, the electronic device 100 can use the text in the edit box 614 as the target text, call the task identification capability of the operating system, and determine one or more task types suitable for processing the text in the edit box 614. The scenario in which the edit box 614 implements the task identification function can refer to the aforementioned Figure 5C The edit box 515 shown implements a scenario of the task identification function.

[0186] For example, the electronic device 100 determines that the task types suitable for processing the text in the edit box 614 include: rewriting, summarizing, and inserting. The electronic device 100 may display Fig. 6F The task options box 615 is shown. The task options box 615 may include an overwrite control 615A, a summary control 615B, and an insert control 615C.

[0187] By the above Figure 6A to Figure 6F As can be seen from the scenario shown, during the text editing process, the user can call up the AI ​​editing box 612 in the text editing interface at any time, and enter language instructions in the AI ​​editing box 612 to instruct the electronic device 100 to re-create the target text. In addition, when the user enters the language instruction in the AI ​​editing box 612, he can also query the information he needs through the query statement. This can improve the efficiency of the user inputting language instructions. The above-mentioned system controls (such as the AI ​​editing box 612) with multiple flag attribute values ​​can provide users with a variety of intelligent services, effectively improving the user's text editing experience.

[0188] Please refer to Figure 7 , Figure 7 A flow chart of a method for content search provided in an embodiment of the present application is exemplified.

[0189] like Figure 7 As shown, the method may include steps S711 to S715.

[0190] S711. Detecting operation 1 for triggering a query on control 1 having an AI search attribute value.

[0191] Control 1 can refer to the above Figure 4A The edit box 411 shown, or the aforementioned Figure 6C The AI ​​edit box 612 is shown. Operation 1 can refer to Figure 4AThe operation of inputting a preset symbol 412 in the edit box 411 is shown.

[0192] S712: Determine a query statement according to operation 1.

[0193] For example, operation 1 is the above Figure 4A The operation of inputting a preset symbol 412 in the edit box 411 is shown. The electronic device 100 can determine the text input after the preset symbol 412 as a query statement. In some embodiments, the electronic device 100 can also determine the text before the preset symbol 412 as a query statement. Alternatively, the electronic device 100 can prompt the user to select text as a query statement. The embodiment of the present application does not limit the method for the electronic device 100 to determine the query statement.

[0194] S713: Use the language model to determine the semantic vector 1 of the query statement, where the semantic vector 1 can represent the semantics of the query statement.

[0195] In some embodiments, the electronic device 100 can use a language model to determine file content information from a query statement. The file content information can represent the content described by the file containing the information to be queried by the query statement. The above-mentioned file content information determined by the query statement can represent the semantic limitation of the content to be searched by the query statement in the query statement. In other words, the file content information can be a condition that limits the semantics or meaning of the search results, and can be used by the electronic device 100 to search for the search results required by the query statement according to the semantics. Among them, the file content information of a file can be determined from the content described by this file. The file content information of a file can represent the semantics of this file or part of the content in this file. The file content information of a file can be seen after opening this file.

[0196] Specifically, the input of the language model may include a query statement and a prompt. Among them, the prompt can be used to instruct the language model to determine the file content information from the query statement. For example, the prompt can be: Please summarize the given content, as briefly as possible but keep the meaning of the given content description. The language model can output the file content information of the query statement. Then, the electronic device 100 can use the above file content information as the input of the semantic vector extraction model, and use the semantic vector extraction model to obtain the semantic vector 1 of the query statement. The semantic vector extraction model can refer to the introduction of the aforementioned embodiment.

[0197] The above method is only an example of a method for determining the semantic vector of a query statement in the present application, and should not be construed as limiting the present application. The electronic device 100 may also determine the semantic vector of a query statement by other methods.

[0198] S714. Search, based on semantic vector 1, from a semantic vector database for one or more semantic vectors that are most similar to semantic vector 1.

[0199] The electronic device 100 can calculate the vector similarity and determine one or more semantic vectors in the semantic vector database that are most similar to the semantic vector 1. The embodiment of the present application does not limit the method by which the electronic device 100 calculates the vector similarity. For example, the electronic device 100 can use methods such as cosine similarity, or Euclidean distance, or Pearson correlation coefficient to calculate the similarity between semantic vectors, thereby finding the semantic vector that is most similar to the semantic vector 1.

[0200] S715: Provide on the screen the file or text content corresponding to the most similar one or more semantic vectors.

[0201] The one or more semantic vectors most similar to the semantic vector 1 may include: semantic vectors corresponding to a single file, and / or semantic vectors corresponding to one or more text contents in a file. The electronic device 100 may provide files or text contents corresponding to the one or more semantic vectors most similar to the above on the screen. The electronic device 100 may display option controls corresponding to these files or text contents. For details, please refer to the aforementioned Figure 4A The electronic device 100 is shown displaying a list box 414 .

[0202] By the above Figure 7 It can be seen from the method shown that the system control with the AI ​​search attribute value can provide a content search function. The user can instruct the electronic device 100 to search for the information he needs by acting on the system control to trigger the query. The above-mentioned content search function can improve the user's experience of searching for information in the electronic device 100 and help the user quickly find the required information. Among them, when the system control in the text editing application (such as the memo application) can provide a content search function, the system control can help the user quickly find the required information when editing the text, and reduce the situation where the user's text editing ideas are interrupted due to information search. The above method can effectively enhance the user's experience of text editing and improve the efficiency of user text editing.

[0203] Please refer to Figure 8 , Figure 8 A flowchart of a method for task identification and content generation provided in an embodiment of the present application is exemplified.

[0204] like Figure 8 As shown, the method may include steps S811 to S816.

[0205] 1. (S811-S813) Task identification.

[0206] S811. Operation 2 acting on control 2 having an AI action attribute value is detected.

[0207] Control 2 can refer to the above Figure 5A The smart writing control 513A shown, or the aforementioned Figure 5C The edit box 515 is shown. Operation 2 can refer to the above Figure 5A The click operation on the smart writing control 513A is shown, or the aforementioned Figure 5C The operation of entering text in the edit box 515 is shown.

[0208] That is, if the control 2 is a system control of a button type, the operation 2 may include a click operation or a touch operation on the control 2. If the control 2 is a system control of an edit box type, the operation 2 may include an operation of inputting text in the control 2.

[0209] S812. Obtain a target text according to operation 2, input the target text into a language model, and use the language model to determine one or more task types suitable for processing the target text.

[0210] In some embodiments, if the control 2 is a system control of the button type, the electronic device 100 can display the selected text (eg Figure 5A If the control 2 is a system control of the edit box type, the electronic device 100 can use the entire text or the selected part of the text in the control 2 as the target text. The embodiment of the present application does not limit the above method for obtaining the target text.

[0211] The electronic device 100 can input the target text into the language model. In addition, the input of the language model can also include multiple task types and prompts. Among them, the task types input into the language model can include but are not limited to: expansion, rewriting, summarization, translation, insertion, etc. The prompt can be: given the target text and some task types, please determine which task types the target text is most likely to belong to, the answer given is no more than 3 task types, and the answer does not require punctuation. In this way, the language model can filter out the task types that the user is most likely to process relative to the target text from the above given task types according to the prompt, and the number of filtered task types will not be more than 3. The content of the above prompt is only an exemplary description of the present application and should not constitute a limitation on the present application.

[0212] S813: providing options corresponding to one or more task types applicable to processing the target text on the screen.

[0213] The electronic device 100 provides options corresponding to the task type on the screen, which can refer to the aforementioned Figure 5B The task options box 514 shown, or the aforementioned Figure 5C Task options box 516 is shown.

[0214] 2. (S814-S816) Content generation.

[0215] S814: Operation 3 on an option corresponding to task type 1 in the one or more task types is detected, the option corresponding to task type 1 has an AI generate attribute value, and task type 1 is expansion, rewriting, summary or translation, etc.

[0216] For the options corresponding to the above task type 1, please refer to the above Figure 5B The expansion control 514A and the rewrite control 514B in the task option box 514 shown, or the aforementioned Figure 5C The rewrite control 516A and the summary control 516B in the task option box 515 are shown. Figure 5B A click operation is shown acting on the expand control 514A.

[0217] S815 . In response to operation 3 , the target text is processed using the language model according to task type 1 to generate new text content.

[0218] For example, if the task type 1 is expansion, the electronic device 100 may use the language model to expand the target text, and the generated new text content is the expanded text of the target text.

[0219] For another example, if the task type 1 is rewriting, the electronic device 100 may use the language model to rewrite the target text, and the generated new text content is the rewritten text of the target text.

[0220] For another example, if the task type 1 is summarizing, the electronic device 100 may summarize the target text using a language model, and the generated new text content is a summary text of the target text.

[0221] For another example, if task type 1 is translation, the electronic device 100 may use a language model to translate the target text, and the generated new text content is the translated text of the target text (eg, translating the target text in Chinese into English).

[0222] Among them, the electronic device 100 can determine the prompt based on the above-mentioned task type 1. For example, when task type 1 is expansion, the content of the prompt can be: Please expand the given target text. When task type 1 is rewriting, the content of the prompt can be: Please rewrite the given target text. When task type 1 is summary, the content of the prompt can be: Please summarize the given target text. When task type 1 is translation (such as Chinese to English), the content of the prompt can be: Please translate the given target text into English. The prompts corresponding to the above-mentioned different task types are only exemplary descriptions of this application and should not constitute a limitation on this application.

[0223] The electronic device 100 may input the prompt and the target text corresponding to the task type 1 into the language model. Then, the language model may process the target text according to the task type 1 to generate new text content.

[0224] S816: Display the new text content on the screen.

[0225] In some embodiments, the electronic device 100 may create a new edit box on the screen (see the aforementioned Figure 5C The editing box 515 shown in the figure is opened, and the new text content is displayed in the editing box. The embodiment of the present application does not limit the method for the electronic device 100 to display the new text content.

[0226] By the above Figure 8 It can be seen from the method shown that the system control with the AI ​​action attribute value can provide a task identification function. The user can instruct the electronic device 100 to filter and provide the task type that the user is most likely to want to process the target text through operations acting on the system control. In this way, when the user needs to process the target text during the text editing process, the required task type can be quickly found to process the target text without having to search for the task type they need one by one in a large number of task types. This can improve the user experience of text editing.

[0227] The system control with the AI ​​generate attribute value can provide a content generation function. The user can instruct the electronic device 100 to process the target text through operations acting on the system control to achieve intelligent output of personalized content. In this way, during the text editing process, the user can conveniently re-create the target text through the content generation function provided by the system control, thereby improving the efficiency of text editing.

[0228] It is understandable that the various user interfaces described in the embodiments of the present application are only example interfaces and do not limit the present application scheme. In other embodiments, the user interface may adopt a different interface layout, may include more or fewer controls, and may increase or decrease other functional options, as long as they are based on the same inventive concept provided by the present application, they are all within the scope of protection of the present application.

[0229] It should be noted that, without causing any contradiction or conflict, any feature in any embodiment of the present application, or any part of any feature, can be combined, and the combined technical solution is also within the scope of the embodiments of the present application.

[0230] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A system control operation method, characterized in that: The method is applied to an electronic device including a first application and a second application, and the method includes: In response to a first operation on a first system control included in the first application, the electronic device implements a first function, wherein the first system control has a first attribute value, the first attribute value corresponds to the first function, and the first function includes a content search function, a task identification function, or a content generation function; In response to a second operation on a second system control included in the second application, the electronic device implements the first function, wherein the second system control has the first attribute value.

2. The method according to claim 1, characterized in that: The operating system of the electronic device has a first capability, the first capability is used to implement the first function, the first attribute value is associated with the first capability, and the electronic device implements the first function, specifically including: The electronic device calls the first capability to implement the first function.

3. The method according to claim 1 or 2, characterized in that: The first system control has a second attribute value, the second attribute value corresponds to a second function, the second function is different from the first function, and the second function includes the content search function, or the task identification function, or the content generation function; The method further comprises: In response to a second operation on the first system control, the electronic device implements the second function.

4. The method according to claim 3, characterized in that The operating system of the electronic device has a second capability, the second capability is used to implement the second function, the second attribute value is associated with the second capability, and the electronic device implements the second function, specifically including: The electronic device calls the second capability to implement the second function.

5. The method according to any one of claims 1 to 4, characterized in that The first function is the content search function; and in response to the first operation on the first system control included in the first application, the electronic device implements the first function, specifically including: In response to the first operation, the electronic device determines a query statement, and searches for first content according to the query statement; The electronic device displays the first content.

6. The method according to any one of claims 1 to 4, characterized in that The first function is the task identification function, and in response to the first operation on the first system control included in the first application, the electronic device implements the first function, specifically including: In response to the first operation, the electronic device obtains a first text and determines one or more task types for processing the first text, the task types comprising one or more of the following: expansion, rewriting, summarization, translation, insertion, and meeting minutes; The electronic device displays options corresponding to the one or more task types.

7. The method according to any one of claims 1 to 4, characterized in that The first function is the content generation function, the first system control indicates a first task type, the first task type includes expansion, or rewriting, or summarizing, or translating, or meeting minutes, and the electronic device implements the first function in response to the first operation on the first system control included in the first application, specifically including: In response to the first operation, the electronic device acquires a second text, and processes the second text according to the first task type to generate a third text; The electronic device displays the third text.

8. An electronic device, characterized in that: The electronic device comprises a memory and a processor, wherein the memory is used to store a computer program; and the processor is used to call the computer program so that the electronic device executes the method according to any one of claims 1 to 7.

9. A computer-readable storage medium comprising instructions, characterized in that: When the instructions are executed on an electronic device, the electronic device executes the method according to any one of claims 1 to 7.

10. A computer program product, characterized in that The computer program product comprises computer instructions, and when the computer instructions are run on an electronic device, the electronic device executes the method according to any one of claims 1 to 7.

Images