System control operation method and electronic device

Abstract

Provided in the present application are a system control operation method and an electronic device. An operating system for the electronic device may provide an upper-layer application with intelligence capabilities such as a content search capability, a content generation capability and a task recognition capability, such that the upper-layer application can provide corresponding intelligence services. Label attributes are newly added to a system control provided by the operating system. Different attribute values of the label attributes may correspond to different intelligence capabilities. A developer of any application may set label attributes for the system control, and develop, by means of the intelligence capabilities integrated in the operating system, the application to be an intelligent application which can provide the intelligence services, without spending time and effort constructing the intelligence capabilities by the developer himself / herself. This may help developers develop applications that meet the diverse requirements of users while greatly reducing the workload of the developers and improving the development efficiency of the developers.

Description

System control operation method and electronic equipment

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office of China on September 22, 2023, with application number 202311240687.1, and priority to the Chinese patent application entitled “System Control Operation Method and Electronic Device”, all contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of terminal technology, and in particular to a system control operation method and an electronic device. Background Art

[0003] Nowadays, users often need to search for information and edit text in electronic devices while using them. This information search can include searching for relevant information in data stored on the electronic device and / or in cloud storage. This text editing can include rewriting, expanding, summarizing, and other re-creation processes of the target text.

[0004] To meet these user needs, developers often need to obtain permissions from multiple parties and collect information from multiple sources (such as various files stored locally or in the cloud, and users' text editing habits and preferences). They also need to perform complex business logic processing on this collected information to provide features that meet user needs. This results in a heavy workload for developers.

[0005] Summary of the Invention

[0006] This application provides a system control operation method and electronic device. Based on the operating system's intelligent capabilities, such as content search, content generation, and task recognition, this method can assist app developers in developing apps that provide intelligent services, reducing their development workload and improving their development efficiency.

[0007] In a first aspect, the present application provides a method for operating a system control. The method can be applied to an electronic device that includes a first application and a second application. In response to a first operation on a first system control included in the first application, the electronic device implements a first function, wherein the first system control has a first attribute value, and the first attribute value corresponds to a first function, and the first function includes a content search function, a task identification function, or a content generation function; in response to a second operation on a second system control included in the second application, the electronic device implements the first function, wherein the second system control has the first attribute value.

[0008] Both the first system control and the second system control can be one of the system controls such as a text box, an edit box, or a button. For example, the first system control and the second system control can refer to the edit box 411 shown in Figure 4A of the present application, or the smart write control 513A shown in Figure 5A, or the control in the task option box 514 shown in Figure 5B, or the AI ​​edit box 612 shown in Figure 6B. The first attribute value can be AI search, or AI generate, or AI action. The first attribute value can be the attribute value corresponding to the flag attribute in the system control.

[0009] It can be seen that system controls in different applications can all provide intelligent services (i.e., the first function) when they all have the first attribute value. Developers of any application can set the attribute value corresponding to the intelligent service (such as the first attribute value) for the system control during development, so that the system control can provide the corresponding intelligent service. The above-mentioned development operation of setting attribute values ​​for the system control is simple and convenient. This can greatly reduce the developer's workload and improve the developer's development efficiency while helping developers develop applications that meet the diverse needs of users (such as information search needs, text editing needs, etc.).

[0010] In combination with the first aspect, in some embodiments, the operating system of the electronic device has a first capability, the first capability is used to implement a first function, the first attribute value is associated with the first capability, and the electronic device can call the first capability to implement the first function.

[0011] The first capability may be an intelligent capability, such as content search capability, content generation capability, or task recognition capability. The operating system of the electronic device may provide an interface for calling intelligent capabilities such as the first capability to the upper-layer application. When a first operation acting on the above-mentioned first system control is detected, the first application in the electronic device may call the first capability through the corresponding interface to implement the first function. When a second operation acting on the above-mentioned second system control is detected, the second application in the electronic device may call the first capability through the corresponding interface to implement the first function.

[0012] It can be seen that when the system control has attribute values ​​associated with intelligent capabilities, the application can call the corresponding intelligent capabilities according to the operations acting on the system control and provide corresponding intelligent services. Therefore, the developer of any application can set the attribute values ​​associated with intelligent capabilities for the system control during development, so that the system control can provide corresponding intelligent services based on the intelligent capabilities of the operating system. The application developer does not need to spend a lot of time and energy to build the above-mentioned intelligent capabilities by themselves. They can develop the applications they develop into intelligent applications that can provide intelligent services with the help of the intelligent capabilities integrated by the operating system. The above method can help developers develop applications that meet the diverse needs of users (such as information search needs, text editing needs, etc.) while greatly reducing the workload of developers and improving the development efficiency of developers.

[0013] In combination with the first aspect, in some embodiments, the first system control has a second attribute value, the second attribute value corresponds to a second function, the second function is different from the first function, and the second function includes a content search function, or a task identification function, or a content generation function; in response to the second operation on the first system control, the electronic device implements the second function.

[0014] As you can see, a single system control can have attribute values ​​corresponding to multiple intelligent services. This means a single system control can provide users with multiple intelligent services. For example, if a system control has the attribute values ​​"AI search" and "AI generate," it can provide both content search and content generation. This single system control can effectively enhance the user experience when editing text or searching for information.

[0015] In combination with the first aspect, in some embodiments, the operating system of the electronic device has a second capability, the second capability is used to implement a second function, the second attribute value is associated with the second capability, the electronic device implements the second function, and the electronic device calls the second capability to implement the second function.

[0016] The second capability may be an intelligent capability, such as content search capability, content generation capability, or task recognition capability. The operating system of the electronic device may provide an interface for invoking the intelligent capability such as the second capability to upper layer applications.

[0017] It can be seen that when the system control has attribute values ​​associated with intelligent capabilities, the application can call the corresponding intelligent capabilities according to the operations acting on the system control and provide corresponding intelligent services. Therefore, the developer of any application can set the attribute values ​​associated with intelligent capabilities for the system control during development, so that the system control can provide corresponding intelligent services based on the intelligent capabilities of the operating system. The application developer does not need to spend a lot of time and energy to build the above-mentioned intelligent capabilities by themselves. They can develop the applications they develop into intelligent applications that can provide intelligent services with the help of the intelligent capabilities integrated by the operating system. The above method can help developers develop applications that meet the diverse needs of users (such as information search needs, text editing needs, etc.) while greatly reducing the workload of developers and improving the development efficiency of developers.

[0018] In combination with the first aspect, in some embodiments, the first function is a content search function; in response to the first operation, the electronic device determines a query statement and searches for first content according to the query statement; and the electronic device displays the first content.

[0019] In the case where the first function is a content search function, the first operation may refer to the operation for triggering a query (e.g., the operation of inputting a preset symbol 412 in an edit box 411) and the operation of inputting a query statement (e.g., the operation of inputting text 413 after the preset symbol 412) shown in FIG4A of the present application. The query statement may refer to the text 413 shown in FIG4A.

[0020] The electronic device may use a language model to determine a semantic vector of the query statement and, based on the semantic vector of the query statement, search a semantic vector database for one or more semantic vectors that are most similar to the semantic vector of the query statement. The first content may be a file or text content corresponding to the one or more most similar semantic vectors.

[0021] The electronic device may display the first content by displaying the search results in the list box 414 as shown in FIG. 4A of the present application.

[0022] It can be seen that when the first system control is able to provide a content search function, the user can enter a query statement in the first system control at any time during the text editing process to query the information he needs, without having to exit the text editing interface and then search for information. In addition, the electronic device can insert the queried content into the first system control. The above content search function can help users quickly find the required information when editing text, reducing the situation where the user's text editing ideas are interrupted due to information search. The above method can effectively enhance the user's text editing experience and improve the user's text editing efficiency.

[0023] In combination with the first aspect, in some embodiments, the first function is a task identification function. In response to the first operation, the electronic device obtains the first text and determines one or more task types for processing the first text. The task types include one or more of the following: expansion, rewriting, summarizing, translation, insertion, and meeting minutes; the electronic device displays options corresponding to one or more task types.

[0024] When the first function is a task identification function, the first operation can refer to the operation of the smart writing control 513A shown in Figure 5A of this application. The first text can refer to the text 512 shown in Figure 5A of this application. The first text can be selected by the user or intelligently selected by the electronic device.

[0025] The electronic device may use a language model to determine one or more task types suitable for processing the first text.

[0026] The electronic device may display options corresponding to one or more task types, for example, by referring to the controls displayed in the task option box 514 shown in FIG. 5B of the present application.

[0027] As can be seen, the electronic device can estimate the user's need to process the first text based on the first text, and then provide the user with an appropriate task type for processing the first text. In this way, when the user needs to process the first text during text editing, they can quickly find the required task type to process the first text without having to search through a large number of task types one by one. This can improve the user's text editing experience.

[0028] In combination with the first aspect, in some embodiments, the first function is a content generation function, the first system control indicates a first task type, the first task type includes expansion, or rewriting, or summarizing, or translating, or meeting minutes, and in response to the first operation, the electronic device obtains the second text and processes the second text according to the first task type to generate a third text; the electronic device displays the third text.

[0029] For example, if the first task type is expansion, the first system control can refer to the expansion control 514A shown in Figure 5B of this application. If the first task type is rewriting, the first system control can refer to the rewriting control 514B shown in Figure 5B of this application. If the first function is content generation, the first operation can refer to the operation of the expansion control 514A shown in Figure 5B of this application.

[0030] The second text may refer to the text 512 shown in FIG5A of the present application, or the text in the edit box 515 shown in FIG5C . The second text may be selected by the user, or may be intelligently selected by the electronic device.

[0031] The electronic device may use the language model to process the second text according to the first task type.

[0032] As can be seen, during the text editing process, users can select the corresponding text at any time to expand, rewrite, summarize or translate, without having to exit the text editing interface and re-create the existing text. This can improve the efficiency of user text editing.

[0033] In a second aspect, the present application provides an electronic device that may include a memory and a processor. The memory may be used to store a computer program. The processor may be used to invoke the computer program, causing the electronic device to execute any possible implementation method described in the first aspect.

[0034] In a third aspect, the present application provides a computer-readable storage medium comprising instructions, which, when executed on an electronic device, enables the electronic device to execute any possible implementation method in the first aspect.

[0035] In a fourth aspect, the present application provides a computer program product, which may include computer instructions. When the computer instructions are run on an electronic device, the electronic device executes any possible implementation method as in the first aspect.

[0036] In a fifth aspect, the present application provides a chip, which is applied to an electronic device. The chip includes one or more processors, and the processor is used to call computer instructions to enable the electronic device to execute any possible implementation method as in the first aspect.

[0037] It is understandable that the electronic device provided in the second aspect, the computer-readable storage medium provided in the third aspect, the computer program product provided in the fourth aspect, and the chip provided in the fifth aspect are all used to execute the methods provided in the embodiments of the present application. Therefore, the beneficial effects that can be achieved can be referred to the beneficial effects of the corresponding methods and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] FIG1 is a schematic diagram of the hardware structure of an electronic device 100 provided in an embodiment of the present application;

[0039] FIG2 is a software structure block diagram of an electronic device 100 provided in an embodiment of the present application;

[0040] FIG3 is a schematic diagram of setting system control properties provided by an embodiment of the present application;

[0041] 4A and 4B are schematic diagrams of scenarios for performing content search according to an embodiment of the present application;

[0042] 5A to 5C are schematic diagrams of scenarios for performing task identification and content generation according to an embodiment of the present application;

[0043] 6A to 6F are schematic diagrams of scenarios for performing content search, content generation, and task identification according to an embodiment of the present application;

[0044] FIG7 is a flow chart of a method for content search provided by an embodiment of the present application;

[0045] FIG8 is a flow chart of a method for task identification and content generation provided in an embodiment of the present application. DETAILED DESCRIPTION

[0046] The technical solutions in the embodiments of the present application are described below in conjunction with the drawings in the embodiments of the present application. Among them, in the description of the embodiments of the present application, the terms used in the following embodiments are only for the purpose of describing specific embodiments, and are not intended to be used as limitations on the present application. As used in the specification and claims of the present application, the singular expressions "a", "said", "above", "the" and "this" are intended to also include expressions such as "one or more", unless there is a clear contrary indication in the context. It should also be understood that in the following embodiments of the present application, "at least one", "one or more" refer to one or more (including two). The term "and / or" is used to describe the association relationship of associated objects, indicating that three relationships can exist; for example, A and / or B can represent: the existence of A alone, the existence of A and B at the same time, and the existence of B alone, where A and B can be singular or plural. The character " / " generally indicates that the associated objects before and after are in an "or" relationship.

[0047] References to "one embodiment" or "some embodiments" etc. described in this specification mean that the specific features, structures or characteristics described in conjunction with the embodiment are included in one or more embodiments of the present application. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. appearing in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in another way. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in another way. The term "connected" includes direct and indirect connections, unless otherwise stated. "First" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated.

[0048] In the embodiments of this application, words such as "exemplarily" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplarily" or "for example" in the embodiments of this application should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplarily" or "for example" is intended to present the relevant concepts in a concrete manner.

[0049] The term "user interface (UI)" in the following embodiments of this application refers to the medium interface for interaction and information exchange between an application (APP) or an operating system (OS) and a user, which realizes the conversion between the internal form of information and the form acceptable to the user. The user interface is a source code written in a specific computer language such as Java and extensible markup language (XML). The interface source code is parsed and rendered on an electronic device and finally presented as content that the user can recognize. The commonly used form of user interface is graphical user interface (GUI), which refers to a user interface related to computer operation that is displayed in a graphical manner. It can be a visual interface element such as text, icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets, etc. displayed on the display screen of an electronic device.

[0050] To facilitate understanding, the concept of "system control" involved in this application is first introduced here.

[0051] Electronic devices are installed with an operating system. An operating system is a set of interrelated system software programs that manage and control the operation of electronic devices, utilize and run hardware and software resources, and provide common services to organize user interactions. The operating system is the interface between the user and the electronic device. Apps in electronic devices run based on the operating system.

[0052] System controls are pre-packaged interface controls provided by the operating system to developers. They help developers quickly develop apps and maintain consistency across the app interface. For example, system controls include, but are not limited to, text boxes (TextView), edit boxes (EditView), buttons (Button), and list controls (ListView).

[0053] System controls can be declared and used in layout files. When declaring a system control, you must also set its properties to control its display within the interface. For example, developers can declare a system control in a layout file and set one or more of the following properties: width, height, color, display position, and so on. This allows developers to customize the layout of the application interface and the display style of each system control within the application interface based on their app development requirements.

[0054] In some embodiments, the system control can also be used to implement one or more specific functions. For example, monitoring preset input, content search, content generation, task identification and other functions. The above-mentioned monitoring preset input function can represent the function of monitoring preset operations (such as click operations, long press operations, etc.) on the system control, or monitoring the function of inputting preset characters or text and other content on the system control. The above-mentioned content search function can represent the function of searching for relevant content locally and / or in the cloud on the electronic device based on the query statement input by the user. The above-mentioned content generation function can represent the function of generating a new text of a specified task type based on the target text. The above-mentioned task identification function can represent the function of determining one or more task types suitable for processing the target text based on the target text, so that the user can select the corresponding task type to process the target text. The above-mentioned task types may include but are not limited to: expansion, rewriting, summarizing, translation, meeting minutes, etc.

[0055] If you want to use system controls to implement certain specific functions, developers can set the properties corresponding to the functions they need to implement for the system controls in the layout file. Furthermore, developers can add implementation methods for the properties corresponding to the above functions in the code. For example, in In an operating system, developers can add implementation methods for attributes corresponding to functions in an activity. The process of using system controls to implement specific functions may include: (1) setting attributes corresponding to specific functions for system controls in a layout file; (2) using the super.setContentView() method to load the layout file; (3) using the super.findViewById() method to obtain a reference to a system control; (4) using the system control reference to operate the control, such as adding a listener, calling an interface that implements a content search function, calling an interface that implements a content generation function, calling an interface that implements a task identification function, and so on. The above-mentioned method of using system controls to implement specific functions is merely an exemplary description of this application and should not constitute a limitation on this application.

[0056] This application provides a system control operation method that can assist APP developers in developing APPs that can provide intelligent services, reduce the developer's development workload, and improve the developer's development efficiency. The above-mentioned intelligent services may include one or more of the following: content search service (or content search function), content generation service (or content generation function), and task identification service (or task identification function).

[0057] Specifically, the operating system of the electronic device has one or more of the following intelligent capabilities: content search capability, content generation capability, and task identification capability. The above-mentioned content search capability can be used to implement the above-mentioned content search function. The above-mentioned content generation capability can be used to implement the content generation function. The above-mentioned task identification capability can be used to implement the task identification function. The operating system can provide an interface for calling these intelligent capabilities to the upper-layer application so that the upper-layer application can provide corresponding intelligent services. The system control provided by the operating system has added a flag attribute, so that the system control can implement the above-mentioned intelligent services. The flag attribute can include one or more of the following attribute values: AI search, AI generate, AI action, etc. Different attribute values ​​can correspond to different intelligent capabilities. For example, the attribute value AI search can correspond to content search capability. The attribute value AI generate can correspond to content generation capability. The attribute value AI action can correspond to task identification capability. If the system control is set with one or more attribute values ​​in the above-mentioned flag attribute, the system control can provide corresponding intelligent services.

[0058] Therefore, any APP developer can set the above flag attribute for the system control during development, and then the system control can provide corresponding intelligent services based on the intelligent capabilities of the operating system. APP developers do not need to spend a lot of time and energy to build the above intelligent capabilities themselves. They can develop their APP into an intelligent application that can provide intelligent services by simply leveraging the intelligent capabilities integrated into the operating system. The above method can help developers develop APPs that meet the diverse needs of users (such as information search needs, text editing needs, etc.) while greatly reducing the developer's workload and improving the developer's development efficiency.

[0059] The structure of the electronic device 100 involved in this application is introduced below.

[0060] The electronic device 100 may be a mobile phone, a tablet computer, a laptop computer, a smart watch, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), etc. The embodiment of the present application does not limit the specific type of the electronic device 100.

[0061] FIG1 exemplarily shows a schematic diagram of the hardware structure of an electronic device 100 .

[0062] As shown in Figure 1, the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc.

[0063] It should be understood that the structures illustrated in the embodiments of the present application do not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0064] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors.

[0065] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0066] Processor 110 may also include a memory for storing instructions and data. In some examples, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the instruction or data again, it can directly retrieve it from the memory. This avoids repeated accesses, reduces processor 110 latency, and thus improves system efficiency.

[0067] In the present application, a computer program may be stored in the memory, which is used to enable the controller or processor to implement the system control operation method of the present application through an interface or protocol.

[0068] Exemplarily, the computer program stored in the memory can be used to: obtain a query statement, determine the semantics of the query statement, search for relevant files from the local and / or cloud of the electronic device 100 based on the semantics of the query statement, and extract the information to be queried by the query statement from the relevant files, identify the purpose of the information to be queried by the query statement, and provide corresponding functional controls for the user to use the above-mentioned queried information.

[0069] USB interface 130 is an interface that complies with USB standards and may be a Mini USB interface, a Micro USB interface, a USB Type-C interface, or the like. USB interface 130 can be used to connect a charger to charge electronic device 100, transfer data between electronic device 100 and peripheral devices, or connect headphones to play audio.

[0070] The charging management module 140 is used to receive charging input from a charger. The charger can be a wireless charger or a wired charger. While charging the battery 142, the charging management module 140 can also power the electronic device through the power management module 141.

[0071] The power management module 141 is used to connect the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140 to provide power to the processor 110, the internal memory 121, the external memory, the display 194, the camera 193, and the wireless communication module 160.

[0072] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0073] Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In other embodiments, the antennas can be used in conjunction with a tuning switch.

[0074] The mobile communication module 150 can provide wireless communication solutions, including 2G / 3G / 4G / 5G, for the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low-noise amplifier (LNA), and the like. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, filter and amplify the received electromagnetic waves, and transmit them to the modem processor for demodulation. The mobile communication module 150 can also amplify the signals modulated by the modem processor and convert them into electromagnetic waves for radiation via the antenna 1.

[0075] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices that integrate at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, frequency modulate it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0076] The electronic device 100 implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing that connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering.

[0077] The display screen 194 is used to display images, videos, etc. In some embodiments, the electronic device 100 may include 1 or N display screens 194 , where N is a positive integer greater than 1.

[0078] The electronic device 100 can implement a shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, and an application processor.

[0079] The ISP is used to process data fed back by the camera 193. For example, when taking a photo, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, which is then transmitted to the ISP for processing and converted into an image visible to the naked eye.

[0080] The camera 193 is used to capture still images or videos. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0081] The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0082] The NPU is a neural network (NN) computing processor. Drawing on the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it rapidly processes input information and can continuously self-learn. The NPU can enable intelligent cognitive applications in electronic device 100, such as image recognition, face recognition, speech recognition, text comprehension, and text generation.

[0083] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 via the external memory interface 120 to implement data storage functions. For example, files such as music and videos can be stored on the external memory card.

[0084] The internal memory 121 can be used to store computer executable program code, which includes instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system, an application required for at least one function (such as a content search function, a text editing function, a language instruction recognition function, etc.), etc. In some embodiments, the program storage area may also store a program for implementing a language model, and a program for implementing the above-mentioned intelligent capabilities. The data storage area may store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.). In some embodiments, the data storage area may also store a semantic vector database. The semantic vector database may store semantic vectors associated with files locally and / or in the cloud of the electronic device 100. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0085] The electronic device 100 can implement audio functions such as music playback and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0086] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some examples, the audio module 170 can be set in the processor 110, or some functional modules of the audio module 170 can be set in the processor 110. The speaker 170A, also known as the "speaker", is used to convert audio electrical signals into sound signals. The receiver 170B, also known as the "earpiece", is used to convert audio electrical signals into sound signals. The microphone 170C, also known as the "microphone" or "microphone", is used to convert sound signals into electrical signals. The headphone jack 170D is used to connect wired headphones.

[0087] The sensor module 180 may include a pressure sensor, a gyro sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a gravity sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, and the like.

[0088] Buttons 190 include a power button, a volume button, etc. Motor 191 can generate vibration prompts. Indicator 192 can be an indicator light that can be used to indicate charging status, power changes, messages, missed calls, notifications, etc.

[0089] The SIM card interface 195 is used to connect a SIM card. A SIM card can be connected to and disconnected from the electronic device 100 by inserting or removing it from the SIM card interface 195. The electronic device 100 may support one or N SIM card interfaces, where N is a positive integer greater than 1. The electronic device 100 interacts with the network through the SIM card to implement functions such as calls and data communications. In some examples, the electronic device 100 uses an eSIM, or embedded SIM card. The eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100.

[0090] The software system of the electronic device 100 can adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture. Taking the system as an example, the software structure of the electronic device 100 is exemplarily described.

[0091] FIG. 2 exemplarily shows a software structure block diagram of the electronic device 100 .

[0092] The layered architecture divides the software into several layers, each with a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, The system is divided into four layers, from top to bottom: application layer, application framework layer, Android runtime (Android Runtime) and system library, and kernel layer.

[0093] The application layer can include a series of application packages.

[0094] As shown in FIG2 , the application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, text editing, and short message. Among them, the text editing application can be used for text editing and creation.

[0095] In some embodiments, a text editing application may provide a content search function. This allows users, while editing text in the text editing application, to search for content locally on the electronic device 100 and / or in the cloud at any time to find the materials needed for editing. Searching for content locally on the electronic device 100 may mean searching for content stored on the electronic device 100, such as images in the gallery application, memos in the memo application, documents in the file application, and so on. Searching for content in the cloud may include searching for content in cloud storage associated with the electronic device 100. For example, if a device account is logged into the electronic device 100, the electronic device 100 may search the cloud storage associated with that device account. For another example, if a cloud disk application account is logged into the cloud disk application of the electronic device 100, the electronic device 100 may search the cloud storage associated with that cloud disk application account. These cloud storage spaces may also be referred to as cloud disks, online disks, and other names. Optionally, searching for content in the cloud may also include searching for web content. These web content may represent content on the internet, such as news, products, addresses, and papers. This embodiment of the present application does not limit the scope of searches that the electronic device 100 can perform.

[0096] In some embodiments, the text editing application may also provide a content generation function. In this way, when the user is editing the text, he or she can perform corresponding processing on the target text. For example, the target text can be expanded, rewritten, summarized, translated, etc.

[0097] In some embodiments, the text editing application can also provide a task recognition function to identify the user's need for text editing and then provide the user with an appropriate task type for processing the target text. In this way, when the user needs to process the target text during the text editing process, they can quickly find the required task type to process the target text without having to search for the required task type one by one among a large number of task types. This can improve the user's text editing experience.

[0098] Not limited to text editing applications, the electronic device 100 may also include more other APPs that can provide one or more of the above-mentioned smart services.

[0099] The application framework layer provides APIs and programming frameworks for applications in the application layer. The application framework layer includes some predefined functions.

[0100] As shown in FIG2 , the application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, an activity manager, a content search module, a content generation module, a task identification module, and the like.

[0101] The window manager is used to manage window programs. The window manager can obtain the display size, determine whether there is a status bar, lock the screen, take screenshots, etc.

[0102] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0103] The view system includes visual controls, such as those for displaying text and images. The view system is used to build applications. A display interface can consist of one or more views. For example, a display interface containing a text notification icon might include a view for displaying text and a view for displaying images.

[0104] The phone manager is used to provide communication functions of the electronic device 100, such as management of call status (including answering, hanging up, etc.).

[0105] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0106] The Notification Manager allows applications to display notification information in the status bar (such as the pull-down notification bar). It can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the Notification Manager is used to notify the completion of downloads, message reminders, etc. The Notification Manager can also be used to display notifications in the form of icons or scrolling text in the status bar at the top of the system, such as notifications from applications running in the background, or notifications that appear on the screen in the form of dialog windows. For example, text messages can be displayed in the status bar, prompts can be sounded, electronic devices can vibrate, indicator lights can flash, etc.

[0107] The Activity Manager is responsible for managing activities, starting, switching, and scheduling components in the system, as well as managing and scheduling applications. The Activity Manager can be called by upper-level applications to open corresponding activities.

[0108] The content search module can be used to provide an interface for calling content search capabilities to upper-level applications. Applications can use this interface to call the content search capabilities of the operating system, thereby providing content search functions for users.

[0109] The content generation module can be used to provide an interface for calling content generation capabilities to upper-level applications. Applications can use this interface to call the content generation capabilities of the operating system, thereby providing users with content generation functions.

[0110] The task identification module can be used to provide an interface for upper-level applications to call task identification capabilities. Applications can use this interface to call the task identification capabilities of the operating system, thereby providing task identification functions for users.

[0111] Android Runtime includes core libraries and a virtual machine. Android Runtime is responsible for scheduling and management of the Android system.

[0112] The core library consists of two parts: one is the function that needs to be called by the Java language, and the other is the Android core library.

[0113] The application layer and application framework layer run in a virtual machine. The virtual machine executes Java files in the application layer and application framework layer as binary files. The virtual machine manages object lifecycles, stack management, thread management, security and exception management, and garbage collection.

[0114] The system library can include multiple functional modules, such as surface manager, media library, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0115] The surface manager is used to manage the display subsystem and provide fusion of 2D and 3D layers for multiple applications.

[0116] The media library supports playback and recording of a variety of common audio and video formats, as well as static image files. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0117] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0118] A 2D graphics engine is a drawing engine for 2D drawings.

[0119] The kernel layer is the layer between hardware and software. The kernel layer includes at least display driver, camera driver, audio driver, and sensor driver.

[0120] FIG3 exemplarily shows a schematic diagram of setting system control properties based on the intelligent capabilities provided by the operating system according to an embodiment of the present application.

[0121] As shown in Figure 3, the interface of APP1 may include one or more of the following system controls: text boxes, edit boxes, buttons, etc. For example, APP1 may be a text editing application, a social communication application, or a browser application. This embodiment of the application does not limit the type of APP1.

[0122] During the development process, the developer of APP1 can set the properties of each system control in APP1. In addition to setting properties related to the display style of the system control (such as width, height, color, position, etc.), the developer can also set properties corresponding to the functions that the system control needs to implement.

[0123] For example, if a developer wants a system control in App 1 to provide the aforementioned intelligent services, they can set a flag attribute for the control and set the value of the flag attribute to the property value corresponding to the intelligent service they want to provide. Furthermore, the developer can add an implementation method for the corresponding property value of the intelligent service in the activity so that the system control can call the intelligent capability interface to provide the corresponding intelligent service.

[0124] The attribute values ​​of the flag attribute may include AI search, AI generate, and AI action. The operating system of the electronic device 100 has intelligent capabilities such as content search capability, content generation capability, and task recognition capability. The attribute value AI search may correspond to content search capability. The attribute value AI generate may correspond to content generation capability. The attribute value AI action may correspond to task recognition capability. The names of the above attribute values ​​are merely illustrative of this application and should not constitute a limitation on this application.

[0125] For example, a developer can set the flag attribute value of a button to AI action. This button can then provide task recognition functionality. In response to an action on this button, such as a click, the electronic device 100 can invoke task recognition capabilities, determine one or more task types suitable for processing the target text based on the target text, and provide corresponding task options for the user to select.

[0126] In some embodiments, developers can set flag attributes for system controls in the layout file of the system control. For example, developers can add "android:flag="AI search"" to the layout file of the system control to add a flag attribute with an attribute value of AI search to the system control. Developers can add "android:flag="AI generate"" to the layout file of the system control to add a flag attribute with an attribute value of AI generate to the system control. Developers can add "android:flag="AI action"" to the layout file of the system control to add a flag attribute with an attribute value of AI action to the system control.

[0127] In some embodiments, a system control can be configured with multiple attribute values ​​in the aforementioned flag attribute. This means that a system control can provide users with multiple intelligent services. For example, if a developer adds flag attributes with attribute values ​​of "AI search" and "AI generate" to a system control, this system control can provide both content search and content generation functions.

[0128] The implementation methods of the above-mentioned content search capability, content generation capability, and task identification capability will be introduced in subsequent embodiments and will not be expanded here.

[0129] As shown in Figure 3, intelligent capability implementation resources may be stored in the electronic device 100. The intelligent capability of the operating system of the electronic device 100 may be implemented based on the intelligent capability implementation resources. The intelligent capability implementation resources may include a language model and a semantic vector database.

[0130] A language model may refer to an artificial intelligence (AI) model for processing language (or symbology), which can be used to generate text that meets the requirements based on a prompt and ensure that the output result is relevant to the prompt. Language models may include language models based on probability theory and statistics (i.e., statistical language models), language models based on deep learning and neural networks (i.e., neural network language models), and so on. The embodiments of the present application do not limit the implementation method of the language model. For example, the electronic device 100 may implement the system control operation method of the present application using a large language model (LLM). The LLM may be a neural network model that can learn the grammar and semantics of natural language to generate human-readable text. The LLM may be trained using a large corpus. The corpus may include a large amount of text data. The LLM can handle various natural language processing tasks, such as natural language generation, text error correction, text classification, text summarization, machine translation, speech recognition, and so on.

[0131] Among them, the above prompt can be a natural language that humans can understand (i.e., a hard prompt / discrete prompt). For example, the prompt can be a text or sentence composed of natural language. Alternatively, the prompt can be machine-readable content that belongs to a non-natural language (i.e., a soft prompt / continuous prompt). For example, a soft prompt can be an embedding, which is a word converted into a meaning vector representation processed by a computer. Hard prompts are more interpretable. Soft prompts can more flexibly eliminate the parameter restrictions of the language model. Prompt can be used to start and guide the language model to generate output of a specified type, topic or format. Exemplarily, in the field of natural language processing, a prompt can include a set of questions and answers or task descriptions. For example, the prompt is: "Give your pet three names", "Translate this English sentence into French", etc. For another example, the prompt can also include question and answer examples. The prompt for the task "Give your pet three names" might include question-and-answer examples: "Pet type: cat, pet names: Name 1, Name 2, Name 3," and "Pet type: dog, pet names: Name 4, Name 5, Name 6." These question-and-answer examples can be used to guide the language model in answering the user's question using the same format as the question-and-answer examples. The language model can then generate text that meets the requirements based on the prompt and ensure that the output is relevant to the prompt.

[0132] The semantic vector database may store semantic vectors associated with files locally stored in the electronic device 100 and / or in the cloud. A file may be associated with one or more semantic vectors.

[0133] In some embodiments, the electronic device 100 may establish a semantic vector database. Specifically, the electronic device 100 may obtain files locally on the electronic device 100 and / or in the cloud, and extract semantic vectors from the obtained files using a semantic vector extraction model. The electronic device 100 may store the extracted semantic vectors in the semantic vector database.

[0134] The above-mentioned semantic vector extraction model can be used to extract the semantic vector of given content. The above-mentioned given content may include but is not limited to data of types such as text, tables, and pictures. The semantic vector extraction model may be a model based on a neural network. The embodiment of the present application does not limit the implementation method of the semantic vector extraction model. The electronic device 100 inputs a section of content into the semantic vector extraction model, and the semantic vector of this section of content can be obtained through the semantic vector extraction model. The semantic vector of this section of content can represent the semantics of this section of content. For example, the semantics of this section of content may include an overview of the information described in this section of content (the theme or main idea of ​​this section of content). For example, the electronic device 100 inputs a section of text introducing cats into the semantic vector extraction model, and the semantic vector of this section of text obtained can be used to indicate that this section of text is a text introducing cats.

[0135] In some embodiments, a file may contain multiple contents (or multiple data segments). The electronic device 100 can split the contents in a file and determine the semantic vectors associated with each data segment after the split. In this way, a file can be associated with multiple semantic vectors. For example, a text file may contain multiple paragraphs of text. The electronic device 100 can split the text file according to paragraphs and extract the semantic vector associated with each paragraph through a semantic vector extraction model. It can be understood that the themes to be expressed in different paragraphs or different chapters in a text file may be different. For example, the first paragraph of a text file introduces the types of cats, and the second paragraph introduces the living habits of cats. The above-mentioned splitting of the file and then extracting the semantic vectors of the split data segments can facilitate the electronic device 100 to more accurately find the content that the user wants to search for when providing a content search function.

[0136] The present application embodiment does not limit the granularity of the above-mentioned file splitting. For example, the granularity of the file splitting may include but is not limited to: by text sentence, by text paragraph, by text chapter, by file page, by table row, by table column, etc.

[0137] Optionally, the electronic device 100 may also treat a file as a whole and extract a semantic vector associated with the file using a semantic vector extraction model, that is, a file may be associated with one semantic vector.

[0138] Optionally, the semantic vector database may be established by another device other than the electronic device 100, such as a cloud server. After establishing the semantic vector database, the cloud server may send the semantic vector database to the electronic device 100.

[0139] In some embodiments, electronic device 100 may update the semantic vector database when files stored locally on electronic device 100 change, or when files stored in a cloud storage space associated with electronic device 100 change. Such file changes may include, but are not limited to, newly added files, changes to file content, and deletion of existing files.

[0140] As can be seen from Figure 3, the electronic device 100 stores intelligent capability implementation resources. Based on the intelligent capability implementation resources, the operating system in the electronic device 100 can provide intelligent capabilities to upper-level applications. When developing, the developer of any APP can set the above-mentioned flag attribute for the system control, so that the system control can provide corresponding intelligent services based on the intelligent capabilities of the operating system. The intelligent capabilities integrated in the operating system can help developers develop APPs that meet the diverse needs of users (such as information search needs, text editing needs, etc.), while greatly reducing the developer's workload and improving the developer's development efficiency.

[0141] The following describes some scenarios for using intelligent services based on system controls with flag attributes added, provided by embodiments of the present application.

[0142] This article specifically introduces the system controls in the Memo app as an example. For system controls in other apps that have flag attributes, refer to the system controls in the Memo app.

[0143] 4A and 4B are schematic diagrams of exemplary scenarios for performing content search.

[0144] As shown in FIG4A , electronic device 100 may display user interface 410 of a memo application. System controls in user interface 410 may include an edit box 411. Edit box 411 may be used for text input and editing. Edit box 411 may also include a flag attribute with a value of "AI search." This means that edit box 411 may provide a content search function.

[0145] In some embodiments, the user can trigger the content search function by entering a preset symbol in the edit box 411. For example, the user enters the preset symbol 412 in the edit box 411. The preset symbol 412 can be " / @". The user can enter a query statement after the preset symbol 412 to search for the content they want to find. When the preset symbol 412 is detected in the edit box 411, the electronic device 100 can detect whether the edit box 411 has a flag attribute with an attribute value of "AI search". If the edit box 411 has a flag attribute with an attribute value of "AI search", the electronic device 100 can determine the text content after the preset symbol 412 as a query statement.

[0146] As shown in FIG4A , the text content following the preset symbol 412 in the edit box 411 is text 413. The content of text 413 is "resume." The electronic device 100 can use text 413 as a query statement to invoke the content search capability of the operating system and search for files, text, or other content associated with the query statement locally on the electronic device 100 and / or in the cloud.

[0147] The electronic device 100 can display the search results on the screen. For example, using text 413 as a query statement, the electronic device 100 can search for files related to the resume shown in Figure 4A. The electronic device 100 can display a list box 414. List box 414 may include option 414A, option 414B, and option 414C. Different options in list box 414 may correspond to different files searched by the electronic device 100. Not limited to the files corresponding to the above options 414A to 414C, the electronic device 100 can also display more or less searched content in list box 414. For example, in response to an operation of sliding up or down in list box 414, the electronic device 100 can display options corresponding to other search results in list box 414.

[0148] The user can select the search results they need in the list box 414. If the search results currently displayed in the list box 414 do not contain the content the user wants, the user can adjust the text after the preset symbol 412 in real time to change the query until the desired content is found.

[0149] As shown in FIG4A , in response to an operation on option 414A, such as a click operation or a touch operation, the electronic device 100 may display the user interface 410 shown in FIG4B . The electronic device 100 may delete the preset symbol 412 and text 413 shown in FIG4A , and display text 415 in the place where the preset symbol 412 and text 413 were displayed. The content of text 415 may be the name of the file corresponding to option 414A selected by the user shown in FIG4A , such as "Resume-V1.pdf".

[0150] In some embodiments, in response to an operation on the text 415, such as a click operation, the electronic device 100 can open the file "Resume-V1.pdf" and display the content of the file.

[0151] It should be noted that the above-mentioned preset symbols for triggering the content search function are only exemplary illustrations of the present application and should not constitute a limitation to the present application. The content of the above-mentioned preset symbols can also be other. The above-mentioned preset symbols can be single symbols, for example, " / " or "@" or "#", etc. The above-mentioned preset symbols can also be combined symbols, for example, " / #" or "@#", etc. Not limited to the above-mentioned preset symbols, the electronic device 100 can also determine the query statement for search when detecting other preset operations. For example, the above-mentioned preset operation can be an operation on a preset shortcut key. In response to the operation on the above-mentioned preset shortcut key, the electronic device 100 can search with part or all of the text in the edit box 411 as a query statement.

[0152] As can be seen from the scenarios shown in Figures 4A and 4B above, when the edit box 411 is able to provide a content search function, the user can enter a query statement in the edit box 411 at any time during the text editing process to query the information they need, without having to exit the text editing interface and then search for information. In addition, the electronic device 100 can insert the queried content into the edit box 411. The above-mentioned content search function can help users quickly find the required information when editing text, and reduce the situation where the user's text editing ideas are interrupted due to information search. The above-mentioned method can effectively enhance the user's text editing experience and improve the efficiency of user text editing.

[0153] 5A to 5C are schematic diagrams of scenarios for performing task identification and content generation.

[0154] As shown in Figure 5A, electronic device 100 may display user interface 510 of the memo application. User interface 510 may include an edit box 511. Text 512 may be entered in edit box 511. For example, the content of text 512 may be "cat and mouse". The content of text 512 is merely an example of the present application. In response to the operation of selecting text 512, electronic device 100 may display function option box 513. Function option box 513 may include system controls: smart write control 513A, copy control 513B, and paste control 513C.

[0155] The smart writing control 513A may be added with a flag attribute whose attribute value is "AI action", that is, the smart writing control 513A may provide a task recognition function.

[0156] In some embodiments, in response to an operation on the smart writing control 513A, such as a click operation, the electronic device 100 may use the selected text 512 as the target text, invoke the task identification capability of the operating system, and determine one or more task types suitable for processing the text 512. The electronic device 100 may display options corresponding to the determined task types on the screen.

[0157] For example, the electronic device 100 determines that the task types applicable to processing the text 512 include: expansion and rewriting. The electronic device 100 may display a task option box 514 as shown in FIG5B. The task option box 514 may include an expansion control 514A and a rewriting control 514B.

[0158] The expansion control 514A can be used to expand the text 512 to generate a new expanded text. The expansion control 514A can be added with a flag attribute with an attribute value of "AI generate". In other words, the expansion control 514A can provide a content generation function.

[0159] Rewrite control 514B can be used to modify the expression of text 512 to generate a rewritten new text. Rewrite control 514B can be added with a flag attribute with an attribute value of "AI generate". That is, rewrite control 514B can provide content generation function.

[0160] In some embodiments, in response to an operation on the expansion control 514A shown in FIG5B , such as a click operation, the electronic device 100 may use the selected text 512 as the target text, invoke the content generation capability of the operating system, and generate expanded text of the text 512. The electronic device 100 may display the expanded text on the screen.

[0161] Exemplarily, as shown in FIG. 5C , in response to the operation on the expansion control 514A shown in FIG. 5B , the electronic device 100 may create a new edit box 515 on the user interface 510 and display the expanded text of the above text 512 in the edit box 515 .

[0162] In some embodiments, edit box 515 may include a flag attribute with the attribute value "AI action." That is, edit box 515 may provide a task identification function. When text is entered into edit box 515, electronic device 100 may use the text in edit box 515 as the target text, invoke the operating system's task identification capabilities, and determine one or more task types applicable to processing the text in edit box 515.

[0163] For example, the electronic device 100 determines that the task types applicable to processing the text in the edit box 515 include: rewrite, summarize, and insert. The electronic device 100 may display the task option box 516 shown in Figure 5C. The task option box 516 may include an rewrite control 516A, a summarize control 516B, and an insert control 516C.

[0164] The rewrite control 516A may refer to the rewrite control 514B shown in FIG. 5B .

[0165] Summary control 516B can be used to summarize the text in edit box 515 and generate new summarized text. Summary control 516B can be added with a flag attribute with an attribute value of "AI generate". That is, summary control 516B can provide content generation functionality.

[0166] The electronic device 100 may use the summarized new text or the rewritten new text to replace the original text in the display edit box 515. Alternatively, the electronic device 100 may display the summarized new text or the rewritten new text after the original text in the edit box 515. The embodiment of the present application does not limit the display method of the rewritten new text and the summarized new text.

[0167] The insert control 516C can be used to insert the text in the edit box 515 into the edit box 511. For example, in response to an operation on the insert control 516C, the electronic device 100 can insert the text in the edit box 515 after the text 512.

[0168] In some embodiments, the target text for content regeneration in edit box 515 can be user-selected. For example, the user can select a portion of the text in edit box 515. The electronic device 100 can then use the selected portion of the text as the target text, invoke the task recognition capability of the operating system, and determine one or more task types suitable for processing the target text.

[0169] Comparing FIG5B and FIG5C , it can be seen that when the target text is different, the task type determined by the electronic device 100 to be applicable to the target text may be different. For example, when the target text has less content, the user may need to expand the target text. When the target text is longer, the user may need to summarize the target text. When the target text is in a foreign language (such as English), the user may need to translate the target text.

[0170] It should be noted that the task types corresponding to the controls in the task option box 514 shown in FIG. 5B and the task option box 516 shown in FIG. 5C are merely exemplary descriptions of the present application and should not constitute a limitation to the present application.

[0171] As can be seen from the scenarios shown in Figures 5A to 5C, the electronic device 100 can estimate the user's need to process the target text based on the target text, and then provide the user with an appropriate task type for processing the target text. In this way, when the user needs to process the target text during text editing, they can quickly find the required task type to process the target text without having to search through a large number of task types one by one. This can improve the user's text editing experience.

[0172] Furthermore, when editing text in the editing box 511, the user can select the corresponding text at any time to expand, rewrite, summarize or translate it without exiting the text editing interface and re-creating the existing text. This can improve the efficiency of the user's text editing.

[0173] In some embodiments, a system control can be set with multiple attribute values ​​in the flag attribute. That is, a system control can provide multiple intelligent services. The following describes scenarios for using intelligent services based on a system control with multiple flag attribute values.

[0174] 6A to 6F are schematic diagrams illustrating scenarios of content search, content generation, and task identification.

[0175] As shown in Figure 6A, the electronic device 100 can display a user interface 610 of the memo application. The system controls in the user interface 610 may include: AI control 611, style control, picture control, etc. The style control can be used to select the display style of the input text, such as the font and font size. The picture control can be used to insert a picture when editing text. The AI ​​control 611 can be used to call up the AI ​​edit box to enter a language instruction instructing the electronic device 100 to execute in the AI ​​edit box. In response to the operation of the AI ​​control 611 shown in Figure 6A, such as a click operation, the electronic device 100 can display the AI ​​edit box 612 shown in Figure 6B. The user can enter the language instruction that the electronic device 100 wants to execute in the AI ​​edit box 612.

[0176] 6B , the AI ​​edit box 612 may include a send control 612A. When a language instruction is input into the AI ​​edit box 612 , the electronic device 100 may recognize and execute the language instruction in the AI ​​edit box 612 in response to an operation on the send control 612A.

[0177] In some embodiments, the AI ​​edit box 612 may be added with a flag attribute whose attribute value is “AI search”, that is, the AI ​​edit box 612 may provide a content search function.

[0178] As shown in Figure 6C, the user enters the text "Summarize this article" in the AI ​​editing box 612, and further enters the preset symbol 612B. The preset symbol 612B can be " / @". The preset symbol 612B can be used to trigger the content search function. The user can enter a query statement after the preset symbol 612B to search for the content they want to find. When the preset symbol 612B is detected in the AI ​​editing box 612, the electronic device 100 can detect whether the AI ​​editing box 612 has a flag attribute with an attribute value of "AI search". If the AI ​​editing box 612 has a flag attribute with an attribute value of "AI search", the electronic device 100 can determine the text content after the preset symbol 612B as a query statement. The preset symbol 612B can refer to the preset symbol 412 shown in the aforementioned Figure 4A.

[0179] As shown in FIG6C , the text following the preset symbol 612B in the AI ​​edit box 612 is text 612C. The content of text 612C is "prose." The electronic device 100 can use text 612C as a query statement to invoke the content search capability of the operating system and search for files, text, and other content associated with the query statement locally on the electronic device 100 and / or in the cloud.

[0180] Electronic device 100 may display the search results on the screen. For example, using text 612C as a query statement, electronic device 100 may search for files related to the essay shown in FIG6C . Electronic device 100 may display list box 613. List box 613 may include option 613A, option 613B, and option 613C. Different options in list box 613 may correspond to different files found by electronic device 100.

[0181] In response to an operation on option 613B shown in FIG6C , such as a click operation or a touch operation, electronic device 100 may display user interface 610 shown in FIG6D . Electronic device 100 may delete preset symbol 612B and text 612C shown in FIG6C and display text 612D in the place of preset symbol 612B and text 612C. Text 612D may include the name of the file corresponding to option 613B selected by the user in FIG6C , such as "Prose 2."

[0182] In some embodiments, the AI ​​edit box 612 may be added with a flag attribute whose attribute value is “AI generate”, that is, the AI ​​edit box 612 may provide a content generation function.

[0183] As shown in FIG6D , in response to the operation of the send control 612A, the electronic device 100 can use the text in the AI ​​edit box 612 (e.g., "Summarize this article "Essay 2") as a language instruction, recognize and execute the language instruction. The electronic device 100 can use a language model to recognize the language instruction. The embodiment of the present application does not limit the method by which the electronic device 100 recognizes the language instruction.

[0184] The electronic device 100 can recognize that the language instruction in the AI ​​edit box 612 is a language instruction for re-creating the target text, and the target text is the text recorded in the file named "Prose 2". Furthermore, the electronic device 100 can detect whether the AI ​​edit box 612 has a flag attribute with an attribute value of "AI generate". If the AI ​​edit box 612 has a flag attribute with an attribute value of "AI generate", the electronic device 100 can call the content generation capability to summarize the file content of "Prose 2" and generate a summary text. The electronic device 100 can display the above summary text on the screen.

[0185] For example, as shown in FIG6E , in response to the operation of the send control 612A shown in FIG6D , the electronic device 100 may create a new edit box 614 on the user interface 610. In the process of generating the summary text, the electronic device 100 may generate and display the summary text word by word in the edit box 614. Optionally, the electronic device 100 may also display the prompt text "AI is modifying the content for you..." shown in FIG6E in the AI ​​edit box 612 to prompt the user that the electronic device 100 is executing the user's language instructions to generate the summary text of "Essay 2."

[0186] As shown in FIG. 6F , when the summary text is generated, the electronic device 100 may display the complete summary text in the editing box 614 .

[0187] In some embodiments, the edit box 614 may be added with a flag attribute whose attribute value is "AI action." That is, the edit box 614 may provide a task identification function. When text is entered in the edit box 614, the electronic device 100 may use the text in the edit box 614 as the target text, invoke the operating system's task identification capabilities, and determine one or more task types suitable for processing the text in the edit box 614. The scenario in which the edit box 614 implements the task identification function can refer to the scenario in which the edit box 515 implements the task identification function as shown in FIG. 5C .

[0188] For example, the electronic device 100 determines that the task types applicable to processing the text in the edit box 614 include: rewrite, summarize, and insert. The electronic device 100 may display the task option box 615 shown in Figure 6F. The task option box 615 may include an rewrite control 615A, a summarize control 615B, and an insert control 615C.

[0189] As can be seen from the scenarios shown in Figures 6A to 6F above, during the text editing process, the user can call out the AI ​​editing box 612 in the text editing interface at any time, and enter language instructions in the AI ​​editing box 612 to instruct the electronic device 100 to re-create the target text. In addition, when the user enters language instructions in the AI ​​editing box 612, he can also query the information he needs through query statements. This can improve the efficiency of the user inputting language instructions. The above-mentioned system controls (such as the AI ​​editing box 612) with multiple flag attribute values ​​can provide users with a variety of intelligent services, effectively improving the user's text editing experience.

[0190] Please refer to FIG. 7 , which exemplarily shows a flow chart of a method for content search provided in an embodiment of the present application.

[0191] As shown in FIG. 7 , the method may include steps S711 to S715 .

[0192] S711. Detecting operation 1 for triggering a query on control 1 having an AI search attribute value.

[0193] Control 1 may refer to the edit box 411 shown in FIG4A , or the AI ​​edit box 612 shown in FIG6C . Operation 1 may refer to the operation of inputting a preset symbol 412 in the edit box 411 shown in FIG4A .

[0194] S712: Determine a query statement according to operation 1.

[0195] For example, operation 1 is the operation of entering a preset symbol 412 in the edit box 411 shown in FIG. 4A . The electronic device 100 may determine the text entered after the preset symbol 412 as a query statement. In some embodiments, the electronic device 100 may also determine the text before the preset symbol 412 as a query statement. Alternatively, the electronic device 100 may prompt the user to select text as a query statement. The embodiments of the present application do not limit the method by which the electronic device 100 determines a query statement.

[0196] S713: Use the language model to determine the semantic vector 1 of the query statement, where the semantic vector 1 can represent the semantics of the query statement.

[0197] In some embodiments, the electronic device 100 can use a language model to determine file content information from a query statement. The file content information can represent the content described by the file containing the information to be queried by the query statement. The file content information determined by the query statement can represent the semantic limitation of the content to be searched by the query statement in the query statement. In other words, the file content information can be a condition that limits the semantics or meaning of the search results, which can be used by the electronic device 100 to search for the search results required by the query statement based on the semantics. Among them, the file content information of a file can be determined from the content described by this file. The file content information of a file can represent the semantics of this file or part of the content in this file. The file content information of a file can be seen after opening this file.

[0198] Specifically, the input of the language model may include a query statement and a prompt. Among them, the prompt can be used to instruct the language model to determine the file content information from the query statement. For example, the prompt can be: Please summarize the given content as briefly as possible while maintaining the meaning of the given content description. The language model can output the file content information of the query statement. Then, the electronic device 100 can use the above-mentioned file content information as the input of the semantic vector extraction model, and use the semantic vector extraction model to obtain the semantic vector 1 of the query statement. The semantic vector extraction model can refer to the introduction of the aforementioned embodiment.

[0199] The above method is only an example of a method for determining the semantic vector of a query statement in the present application and should not be construed as limiting the present application. The electronic device 100 may also determine the semantic vector of a query statement by other methods.

[0200] S714 . Search, based on the semantic vector 1 , the semantic vector database for one or more semantic vectors that are most similar to the semantic vector 1 .

[0201] The electronic device 100 can calculate vector similarity to determine one or more semantic vectors in the semantic vector database that are most similar to the semantic vector 1. The present embodiment of the application does not limit the method by which the electronic device 100 calculates vector similarity. For example, the electronic device 100 can use methods such as cosine similarity, Euclidean distance, or Pearson correlation coefficient to calculate the similarity between semantic vectors, thereby finding the semantic vector that is most similar to the semantic vector 1.

[0202] S715: Provide on the screen the file or text content corresponding to the most similar one or more semantic vectors.

[0203] The one or more semantic vectors most similar to semantic vector 1 may include: semantic vectors corresponding to a single file, and / or semantic vectors corresponding to one or more textual contents within a file. The electronic device 100 may display the files or textual contents corresponding to the one or more semantic vectors most similar to the above-mentioned one or more semantic vectors on the screen. The electronic device 100 may display option controls corresponding to these files or textual contents. For details, please refer to the list box 414 displayed by the electronic device 100 shown in FIG. 4A .

[0204] It can be seen from the method shown in Figure 7 above that the system control with the AI ​​search attribute value can provide a content search function. The user can instruct the electronic device 100 to search for the information he needs by acting on the system control to trigger the query. The above-mentioned content search function can improve the user's experience of searching for information in the electronic device 100 and help the user quickly find the required information. Among them, when the system control in the text editing application (such as the memo application) is able to provide a content search function, the system control can help the user quickly find the required information when editing the text, and reduce the situation where the user's text editing ideas are interrupted due to information search. The above method can effectively enhance the user's experience in text editing and improve the efficiency of user text editing.

[0205] Please refer to FIG8 , which exemplarily shows a flow chart of a method for task identification and content generation provided in an embodiment of the present application.

[0206] As shown in FIG. 8 , the method may include steps S811 to S816 .

[0207] 1. (S811-S813) Task identification.

[0208] S811: Detect operation 2 acting on control 2 having an AI action attribute value.

[0209] Control 2 can refer to the smart writing control 513A shown in Figure 5A above, or the edit box 515 shown in Figure 5C above. Operation 2 can refer to the click operation on the smart writing control 513A shown in Figure 5A above, or the operation of entering text in the edit box 515 shown in Figure 5C above.

[0210] That is, if control 2 is a system control of the button type, operation 2 may include a click operation or a touch operation on control 2. If control 2 is a system control of the edit box type, operation 2 may include an operation of entering text in control 2.

[0211] S812: Obtain a target text according to operation 2, input the target text into a language model, and use the language model to determine one or more task types suitable for processing the target text.

[0212] In some embodiments, if control 2 is a system control of the button type, the electronic device 100 may use the text selected on the current interface (e.g., the selected text 512 shown in FIG5A ) as the target text. If control 2 is a system control of the edit box type, the electronic device 100 may use the entire text or the selected portion of the text in control 2 as the target text. This embodiment of the application does not limit the above method for obtaining the target text.

[0213] The electronic device 100 can input the target text into the language model. In addition, the input of the language model can also include a variety of task types and prompts. Among them, the task types input to the language model can include but are not limited to: expansion, rewriting, summarizing, translation, insertion, etc. The prompt can be: given the target text and some task types, please judge which task types the target text is most likely to belong to, and the answer given is no more than 3 task types, and the content of the answer does not require punctuation. In this way, the language model can filter out the task types that the user is most likely to process relative to the target text from the above-mentioned given task types according to the prompt, and the number of filtered task types will not be more than 3. The content of the above prompt is only an illustrative description of the present application and should not constitute a limitation on the present application.

[0214] S813: providing options corresponding to one or more task types applicable to processing the target text on the screen.

[0215] The electronic device 100 may provide options corresponding to the task type on the screen with reference to the task option box 514 shown in FIG. 5B or the task option box 516 shown in FIG. 5C .

[0216] 2. (S814-S816) Content generation.

[0217] S814: Detecting operation 3 on an option corresponding to task type 1 among the one or more task types, where the option corresponding to task type 1 has an AI generate attribute value, and task type 1 is expansion, rewriting, summarization, or translation.

[0218] The options corresponding to the aforementioned task type 1 can refer to the expand control 514A and the rewrite control 514B in the task option box 514 shown in FIG5B , or the rewrite control 516A and the summary control 516B in the task option box 515 shown in FIG5C . The aforementioned operation 3 can refer to the click operation on the expand control 514A shown in FIG5B .

[0219] S815 . In response to operation 3 , the target text is processed using the language model according to task type 1 to generate new text content.

[0220] For example, if the task type 1 is expansion, the electronic device 100 may expand the target text using a language model, and the generated new text content is the expanded text of the target text.

[0221] For another example, if task type 1 is rewriting, the electronic device 100 may use the language model to rewrite the target text, and the generated new text content is the rewritten text of the target text.

[0222] For another example, if task type 1 is summarizing, the electronic device 100 may summarize the target text using a language model, and the generated new text content is a summary text of the target text.

[0223] For another example, if task type 1 is translation, the electronic device 100 may use a language model to translate the target text, and the generated new text content is the translated text of the target text (eg, translating the target text in Chinese into English).

[0224] Among them, the electronic device 100 can determine the prompt based on the above-mentioned task type 1. For example, when task type 1 is expansion, the content of the prompt can be: Please expand the given target text. When task type 1 is rewriting, the content of the prompt can be: Please rewrite the given target text. When task type 1 is summary, the content of the prompt can be: Please summarize the given target text. When task type 1 is translation (such as Chinese to English), the content of the prompt can be: Please translate the given target text into English. The prompts corresponding to the above-mentioned different task types are only illustrative of this application and should not constitute a limitation to this application.

[0225] The electronic device 100 may input the prompt and target text corresponding to task type 1 into the language model. Then, the language model may process the target text according to task type 1 to generate new text content.

[0226] S816: Display the new text content on the screen.

[0227] In some embodiments, the electronic device 100 may create a new edit box on the screen (refer to the edit box 515 shown in FIG5C ) and display the new text content in the edit box. The embodiment of the present application does not limit the method by which the electronic device 100 displays the new text content.

[0228] As can be seen from the method shown in FIG8 above, a system control with an AI action attribute value can provide a task identification function. The user can instruct the electronic device 100 to filter and provide the task type that the user is most likely to want to process the target text through operations acting on the system control. In this way, when the user needs to process the target text during the text editing process, the user can quickly find the required task type to process the target text without having to search for the required task type one by one among a large number of task types. This can improve the user experience of text editing.

[0229] A system control with the AI ​​generate attribute value can provide content generation functionality. Users can instruct electronic device 100 to process target text by operating on the system control, achieving intelligent output of personalized content. This allows users to conveniently recreate target text using the content generation functionality provided by the system control, improving text editing efficiency.

[0230] It is understood that the various user interfaces described in the embodiments of this application are merely exemplary interfaces and do not limit the scope of this application. In other embodiments, the user interface may adopt a different interface layout, include more or fewer controls, and add or remove other functional options. As long as they are based on the same inventive concept provided by this application, they are all within the scope of protection of this application.

[0231] It should be noted that, without causing any contradiction or conflict, any feature in any embodiment of the present application, or any part of any feature, can be combined, and the combined technical solution is also within the scope of the embodiments of the present application.

[0232] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A system control operation method, characterized in that: The method is applied to an electronic device including a first application and a second application, and the method includes: In response to a first operation on a first system control included in the first application, the electronic device implements a first function, wherein the first system control has a first attribute value, the first attribute value corresponds to the first function, and the first function includes a content search function, a task identification function, or a content generation function; In response to a second operation on a second system control included in the second application, the electronic device implements the first function, wherein the second system control has the first attribute value.

2. The method according to claim 1, characterized in that The operating system of the electronic device has a first capability, the first capability is used to implement the first function, the first attribute value is associated with the first capability, and the electronic device implements the first function, specifically including: The electronic device calls the first capability to implement the first function.

3. The method according to claim 1 or 2, characterized in that: The first system control has a second attribute value, the second attribute value corresponds to a second function, the second function is different from the first function, and the second function includes the content search function, or the task identification function, or the content generation function; The method further comprises: In response to a second operation on the first system control, the electronic device implements the second function.

4. The method according to claim 3, characterized in that The operating system of the electronic device has a second capability, the second capability is used to implement the second function, the second attribute value is associated with the second capability, and the electronic device implements the second function, specifically including: The electronic device calls the second capability to implement the second function.

5. The method according to any one of claims 1 to 4, characterized in that The first function is the content search function; and in response to the first operation on the first system control included in the first application, the electronic device implements the first function, specifically including: In response to the first operation, the electronic device determines a query statement, and searches for first content according to the query statement; The electronic device displays the first content.

6. The method according to any one of claims 1 to 4, characterized in that The first function is the task identification function, and in response to the first operation on the first system control included in the first application, the electronic device implements the first function, specifically including: In response to the first operation, the electronic device obtains a first text and determines one or more task types for processing the first text, the task types comprising one or more of the following: expansion, rewriting, summarization, translation, insertion, and meeting minutes; The electronic device displays options corresponding to the one or more task types.

7. The method according to any one of claims 1 to 4, characterized in that The first function is the content generation function, the first system control indicates a first task type, the first task type includes expansion, or rewriting, or summarizing, or translating, or meeting minutes, and the electronic device implements the first function in response to the first operation on the first system control included in the first application, specifically including: In response to the first operation, the electronic device acquires a second text, and processes the second text according to the first task type to generate a third text; The electronic device displays the third text.

8. An electronic device, characterized in that: The electronic device comprises a memory and a processor, wherein the memory is used to store a computer program; and the processor is used to call the computer program so that the electronic device executes the method according to any one of claims 1 to 7.

9. A computer-readable storage medium comprising instructions, characterized in that: When the instructions are executed on an electronic device, the electronic device executes the method according to any one of claims 1 to 7.

10. A computer program product, characterized in that The computer program product comprises computer instructions, and when the computer instructions are run on an electronic device, the electronic device executes the method according to any one of claims 1 to 7.

Images