Task execution method and electronic device

Abstract

The application provides a task execution method and an electronic device. The method comprises: executing a first task based on a first setting; when a first condition is met, switching the first setting to a second setting and / or outputting first prompt information, and continuing to execute the first task; wherein the first condition is a condition for determining that there is a conversation intention between a first user and a second user, the first user is a user of the electronic device, and the second user is any user other than the first user; the first setting and the second setting are different settings of the same setting item associated with the first task, and the immersion degree of the second setting is lower than that of the first setting; and the first prompt information is used to indicate the conversation intention. The method can improve the timely and effective perception of the user to the external conversation during task execution, and has high practicability.

Description

Technical Field

[0001] This application relates to the field of electronic equipment technology, and in particular to a task execution method and an electronic device. Background Technology

[0002] Current electronic devices emphasize user immersion when performing many tasks. By using certain technologies to isolate or reduce the user's perception of the outside world, users can become more immersed in the tasks performed by the electronic device, thereby providing immersive services and enhancing the user experience. For example, when an electronic device plays audio or video, connecting noise-canceling headphones can reduce external noise, thus highlighting the audio being played and improving the user's listening experience. Similarly, virtual reality or augmented reality devices can enhance the user experience by disabling ambient light transmission to highlight media content displayed to the user. Furthermore, smart cockpits in cars can provide an immersive environment for users by adjusting variable transparency glass and window sunshades, enhancing the immersive experience when browsing media on the in-vehicle system.

[0003] However, immersive services provided by electronic devices can diminish a user's perception of the outside world, potentially affecting their interaction with it. For example, while electronic devices are providing immersive services, users may not be able to notice or participate in conversations initiated by other users, impacting their social efficiency and even posing a risk of privacy breaches. Therefore, the practicality of immersive services provided by electronic devices is currently limited. Summary of the Invention

[0004] This application provides a task execution method and an electronic device to improve the practicality of electronic devices in providing immersive services.

[0005] In a first aspect, embodiments of this application provide a task execution method applied to an electronic device. The method includes: executing a first task based on a first setting; when a first condition is met, switching the first setting to a second setting and / or outputting a first prompt message, and continuing to execute the first task; wherein, the first condition is a condition for determining that there is a dialogue intention between a first user and a second user, the first user is a user of the electronic device, and the second user is any user other than the first user; the first setting and the second setting are different settings of the same setting item associated with the first task, and the immersion level of the second setting is lower than that of the first setting; the first prompt message is used to indicate the dialogue intention.

[0006] In this method, the first task performed by the electronic device can be a task for providing immersive services. Based on this method, when the electronic device detects a conversational intent between the user and other users during task execution, it can adjust the relevant settings of the electronic device and / or display conversational prompts. This reduces the user's immersion in the task while it continues, thereby increasing the user's awareness of the outside world and helping them to perceive conversational intents accurately and promptly and participate in the conversation. If the user participates in the conversation, adjusting the relevant settings can also reduce interference during task execution, improving conversational efficiency. Therefore, this method for providing immersive services is highly practical.

[0007] In one possible design, satisfying the first condition includes: detecting that the distance between the first user and the second user is less than or equal to a first preset threshold, and detecting that the first user is looking at the second user or that the second user is looking at the electronic device or the first user; and / or detecting that footsteps are present in the collected ambient audio and that the audio energy of the footsteps changes according to a preset trend, the preset trend including: first increasing to exceed a second preset threshold, and then decreasing to below a third preset threshold; wherein the third preset threshold is less than the second preset threshold.

[0008] Based on this method, electronic devices can detect scenarios where other users approach the user, or where the user is looking at the user, or vice versa. They can also detect scenarios where other users approach and stop. These scenarios represent potential dialogue opportunities. Therefore, this method can detect dialogue intent with relatively high accuracy.

[0009] In one possible design, the first prompt also indicates the location of the second user. This method helps users of electronic devices more quickly identify another user they might be conversing with, thus enabling them to initiate a conversation more efficiently.

[0010] In one possible design, switching the first setting to the second setting includes at least one of the following: reducing the first audio playback volume included in the first setting to the second audio playback volume included in the second setting; when the electronic device is connected to noise-canceling headphones, reducing the noise reduction level of the noise-canceling headphones from the first noise reduction level included in the first setting to the second noise reduction level included in the second setting; when the electronic device is an extended reality device, switching the ambient spectral transmission mode of the extended reality device from the off state included in the first setting to the on state included in the second setting; when the electronic device is located in a vehicle, switching the sunshade of the vehicle from the off state included in the first setting to the on state included in the second setting, and / or reducing the transparency of the privacy glass of the vehicle from the first transparency included in the first setting to the second transparency included in the second setting.

[0011] The above methods can reduce the level of immersion during task execution, thereby improving the user's perception of the outside world and making it easier for the user to quickly and efficiently participate in dialogue with the outside world.

[0012] In one possible design, the method further includes: when the first condition is met, if the electronic device is located in the vehicle, playing audio collected by an audio device outside the vehicle's cabin via an audio device inside the vehicle's cabin. This method helps users inside the vehicle's cabin to more accurately perceive the conversational intentions of users outside the vehicle's cabin, thereby enabling them to participate in conversations more efficiently.

[0013] In one possible design, after switching the first setting to the second setting and / or outputting the first prompt information and continuing to execute the first task, the method further includes: stopping the execution of the first task when a second condition is met, or performing setting processing; wherein the second condition is a condition for determining that a dialogue has occurred between the first user and the second user; the setting processing includes at least one of the following: turning off the audio input and output of the electronic device; turning off the camera of the electronic device; turning off the display function of the electronic device; generating a summary of the execution of the first task; collecting audio or video content generated by executing the first task; and amplifying human voice through the noise-canceling headphones when the electronic device is connected.

[0014] In this method, electronic devices can pause tasks or perform pre-defined processes to reduce interference with user operations when a conversation between users is detected, enabling users to conduct conversations more efficiently.

[0015] In one possible design, before stopping the execution of the first task, the method further includes: determining that the first task is not a task executed based on a real-time media stream; or, before performing the setting process, the method further includes: determining that the first task is a task based on a real-time media stream.

[0016] Based on this method, when an electronic device detects a conversation, it can directly stop tasks that support pausing, and reduce the interference of task execution on the user's conversation by executing set processing for tasks that do not support pausing. This method is highly practical.

[0017] In one possible design, satisfying the second condition includes at least one of the following: receiving a first instruction to indicate the start of a dialogue; detecting that the collected ambient audio includes human voices; detecting that the first user is looking at the second user and the second user is looking at the first user or the electronic device.

[0018] Based on this method, electronic devices can accurately detect when a conversation occurs. Combined with the above method, electronic devices can automatically perform processing that facilitates efficient user interaction when a conversation is detected, minimizing manual user intervention and improving user experience.

[0019] In one possible design, after stopping the execution of the first task, or after performing setting processing, the method further includes: continuing to execute the first task based on the first setting when a third condition is met; wherein the third condition is a condition used to determine that the first user and the second user have ended their conversation.

[0020] Based on this method, when an electronic device detects that the dialogue has ended, it can resume the initial task execution mode, thereby restoring the normal execution of the task.

[0021] In one possible design, after performing the setting process, the method further includes: when the setting process includes generating a summary of the execution of the first task, outputting the summary; and / or, when the setting process includes acquiring audio or video content generated by executing the first task, outputting the audio or video content.

[0022] Based on this method, electronic devices can provide feedback to users on the relevant content of task execution during the conversation after it ends, making it easier for users to understand the task execution status during the conversation and thus improving the user experience.

[0023] In one possible design, satisfying the third condition includes at least one of the following: receiving a second instruction to indicate the end of the conversation; not detecting human voices in the collected ambient audio for a duration exceeding a set duration; detecting the first user looking at the electronic device.

[0024] Based on this method, electronic devices can accurately detect the end of a conversation. Combined with the above method, electronic devices can resume normal task execution upon detecting the end of a conversation, minimizing manual user intervention and improving user experience.

[0025] In a second aspect, this application provides an electronic device, the electronic device including a memory and one or more processors; wherein the memory is used to store computer program code, the computer program code including computer instructions; when the computer instructions are executed by the one or more processors, the electronic device causes the electronic device to perform the method described in the first aspect or any possible design of the first aspect.

[0026] Thirdly, this application provides a computer-readable storage medium storing a computer program that, when run on an electronic device, causes the electronic device to perform the method described in the first aspect or any possible design of the first aspect.

[0027] Fourthly, this application provides a computer program product comprising a computer program or instructions that, when executed on an electronic device, cause the electronic device to perform the method described in the first aspect or any possible design of the first aspect.

[0028] Fifthly, this application provides a chip system including a processor and a memory, wherein the memory stores instructions; when the instructions are executed by the processor, they implement the methods described in the first aspect or any possible design of the first aspect. The chip system may be composed of chips or may include chips and other discrete devices.

[0029] The beneficial effects of the second to fifth aspects mentioned above can be found in the beneficial effects of the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0030] Figure 1 A schematic diagram of the hardware architecture of an electronic device provided in an embodiment of this application;

[0031] Figure 2 A schematic diagram of the software architecture of an electronic device provided in an embodiment of this application;

[0032] Figure 3A schematic diagram illustrating a task execution method provided in an embodiment of this application;

[0033] Figure 4 A schematic diagram of a device control scenario provided in an embodiment of this application;

[0034] Figure 5 A flowchart illustrating a dialogue intent recognition method provided in an embodiment of this application;

[0035] Figure 6 A flowchart illustrating a dialogue intent recognition method provided in an embodiment of this application;

[0036] Figure 7 This is a schematic diagram of a dialog prompt message provided in an embodiment of this application;

[0037] Figure 8 A schematic diagram of a dialog end control interface provided in an embodiment of this application;

[0038] Figure 9 A schematic diagram of an interface for returning video conference minutes, provided as an embodiment of this application;

[0039] Figure 10 A schematic diagram illustrating a task execution method provided in an embodiment of this application;

[0040] Figure 11 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the embodiments of this application will be further described in detail below with reference to the accompanying drawings.

[0042] In the description of the embodiments of this application, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined with "first" and "second" may explicitly or implicitly include one or more of that feature.

[0043] It should be understood that in the embodiments of this application, "at least one" means one or more, and "more than one" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, a and b, a and c, b and c, or a, b, and c, where a, b, and c can be single or multiple.

[0044] To improve the practicality of immersive services provided by electronic devices, this application provides a task execution method and an electronic device. In this method, when the electronic device is providing immersive services (i.e., performing immersive tasks) to a user, it can automatically adjust relevant settings to enhance the user's environmental awareness when it detects a dialogue intent between another user and the user. This allows the user to quickly and effectively perceive and participate in the dialogue. The electronic device can also automatically pause or take over the immersive task when a dialogue is detected to have begun, thus preventing the execution of the immersive task from affecting the user's participation in the dialogue. This method enables users to interact quickly and efficiently with the outside world while the electronic device is providing immersive services, reducing or avoiding cumbersome manual control processes. Therefore, the process of the electronic device providing immersive services does not affect the efficiency and user experience of the user's interaction with the outside world, making it highly practical.

[0045] The technical solutions provided in this application can be executed by one or more electronic devices. In some embodiments of this application, the electronic devices can be portable devices, such as mobile phones, tablets, wearable devices with wireless communication capabilities (e.g., smart glasses), in-vehicle terminal devices, augmented reality (AR) / virtual reality (VR) devices, laptops, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), smart home devices (e.g., smart TVs), smart robots, workshop equipment, wireless terminals in self-driving vehicles, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, or wireless terminals in smart homes, and flying devices (e.g., smart robots, drones, airplanes). Among these, wearable devices are portable devices that users can wear directly on their bodies or integrate into their clothing or accessories.

[0046] In some embodiments of this application, the electronic device may also be a portable terminal device that includes other functions. Exemplary embodiments of the portable terminal device include, but are not limited to, carrying... Alternatively, it could be a portable terminal device with another operating system. The aforementioned portable terminal device could also be other portable terminal devices, such as a laptop computer with a touch-sensitive surface (e.g., a touch panel). It should also be understood that in some other embodiments of this application, the aforementioned electronic device may not be a portable terminal device, but rather a desktop computer with a touch-sensitive surface (e.g., a touch panel).

[0047] See below. Figure 1 The structure of the electronic device to which the method provided in the embodiments of this application is applicable will be described.

[0048] like Figure 1As shown, the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a USB interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a SIM card interface 195, etc.

[0049] The sensor module 180 may include a gyroscope sensor, an accelerometer, a proximity sensor, a fingerprint sensor, a touch sensor, a temperature sensor, a pressure sensor, a distance sensor, a magnetic sensor, an ambient light sensor, a barometric pressure sensor, a bone conduction sensor, etc.

[0050] Understandable, Figure 1 The electronic device 100 shown is merely an example and does not constitute a limitation on the electronic device. The electronic device may have more or fewer components than those shown in the figure, may combine two or more components, or may have different component configurations. Figure 1 The various components shown can be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application-specific integrated circuits.

[0051] Processor 110 may include one or more processing units, such as application processors (APs), modem processors, graphics processing units (GPUs), image signal processors (ISPs), controllers, memory, video codecs, digital signal processors (DSPs), baseband processors, and / or neural network processing units (NPUs). Different processing units may be independent devices or integrated into one or more processors. The controller may serve as the central nervous system and command center of the electronic device 100. The controller can generate operation control signals based on instruction opcodes and timing signals to control instruction fetching and execution.

[0052] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0053] The execution of the task execution method provided in this application embodiment can be controlled by the processor 110 or by calling other components. For example, it can call the processing program of this application embodiment stored in the internal memory 121, or call the processing program of this application embodiment stored in a third-party device through the external memory interface 120 to control the wireless communication module 160 to perform data communication with other devices, thereby improving the intelligence and convenience of the electronic device 100 and enhancing the user experience. The processor 110 may include different devices. For example, when integrating a CPU and a GPU, the CPU and GPU can cooperate to execute the task execution method provided in this application embodiment.

[0054] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Miniled LED, a MicroLED, a Micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, electronic device 100 may include one or N displays screens 194, where N is a positive integer greater than 1. Display screen 194 can be used to display information input by the user or information provided to the user, as well as various graphical user interfaces (GUIs). For example, display screen 194 can display a conference interface, a video interface, etc.

[0055] In this embodiment of the application, the display screen 194 can be a single flexible display screen, or it can be a splicing display screen composed of two rigid screens and a flexible screen located between the two rigid screens.

[0056] Camera 193 (a front-facing camera or a rear-facing camera, or a single camera that can function as both) is used to capture still images or videos. Typically, camera 193 may include a photosensitive element such as a lens assembly and an image sensor. The lens assembly includes multiple lenses (convex or concave lenses) for collecting light signals reflected from the object being photographed and transmitting these signals to the image sensor. The image sensor generates a raw image of the object being photographed based on the light signals.

[0057] The internal memory 121 can be used to store computer executable program code, which includes instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. The program storage area can store the operating system, application program code (such as the functions corresponding to the scheme in this application), etc. The data storage area can store data created during the use of the electronic device 100.

[0058] The internal memory 121 may also store one or more computer programs corresponding to the algorithm of this application. The one or more computer programs are stored in the internal memory 121 and configured to be executed by one or more processors 110. The one or more computer programs include instructions that can be used to perform the various steps in the following embodiments.

[0059] In addition, the internal memory 121 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0060] Of course, the algorithm code of the embodiment of this application can also be stored in external memory. In this case, the processor 110 can run the algorithm code of the embodiment of this application stored in external memory through the external memory interface 120.

[0061] A touch sensor, also known as a "touch panel," can be located on the display screen 194. The touch sensor and display screen 194 together form a touch display screen, also called a "touch screen." The touch sensor detects touch operations applied to or near it. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In other embodiments, the touch sensor may also be located on the surface of the electronic device 100, in a different position than the display screen 194.

[0062] The wireless communication function of electronic device 100 can be realized through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor, etc.

[0063] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover one or more communication frequency bands. Different antennas can also be multiplexed to improve antenna utilization. For example, antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with tuning switches.

[0064] The mobile communication module 150 can provide solutions for wireless communication, including 2G / 3G / 4G / 5G, applied to the electronic device 100. The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves via antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via antenna 1. In some embodiments, at least some functional modules of the mobile communication module 150 may be housed in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 may be housed in the same device. In this embodiment, the mobile communication module 150 can also be used for information interaction with other devices.

[0065] The modem processor may include a modulator and a demodulator. The modulator modulates the low-frequency baseband signal to be transmitted into a mid-to-high frequency signal. The demodulator demodulates the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After processing by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs sound signals through an audio device (not limited to speaker 170A, receiver 170B, etc.) or displays images or videos through the display screen 194. In some embodiments, the modem processor may be a separate device. In other embodiments, the modem processor may be independent of the processor 110 and may be housed in the same device as the mobile communication module 150 or other functional modules.

[0066] The wireless communication module 160 can provide solutions for wireless communication applications on the electronic device 100, including wireless local area networks (WLANs) (such as wireless fidelity (WiFi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via antenna 2, performs frequency modulation and filtering of the electromagnetic wave signals, and sends the processed signal to processor 110. The wireless communication module 160 can also receive signals to be transmitted from processor 110, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via antenna 2. In this embodiment, the wireless communication module 160 can be used to establish connections with other electronic devices for data interaction. Alternatively, the wireless communication module 160 can be used to access access point devices, send control commands to other electronic devices, or receive data from other electronic devices.

[0067] In addition, the electronic device 100 can implement audio functions through an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, and an application processor, such as music playback and recording. The electronic device 100 can receive input from buttons 190, generating key signal inputs related to user settings and function control. The electronic device 100 can use a motor 191 to generate vibration alerts (such as vibration alerts for incoming calls). The indicator 192 in the electronic device 100 can be an indicator light, used to indicate charging status, battery level changes, messages, missed calls, notifications, etc. The SIM card interface 195 in the electronic device 100 is used to connect a SIM card. The SIM card can be inserted into or removed from the SIM card interface 195 to achieve contact and separation with the electronic device 100.

[0068] It should be understood that, in practical applications, electronic device 100 may include more than Figure 1The number of more or fewer components shown is not limited in the embodiments of this application. The illustrated electronic device 100 is merely an example, and the electronic device 100 may have more or fewer components than shown in the figure, may combine two or more components, or may have different component configurations. The various components shown in the figure may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application-specific integrated circuits.

[0069] The software system of electronic device 100 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. A layered architecture divides the software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. For example, ... Figure 2 As shown, the software architecture can be divided into four layers, from top to bottom: the application layer, the application framework layer (framework, FWK), the runtime and system libraries, and the (Linux) kernel layer.

[0070] The application layer is the top layer of the operating system and includes native operating system applications such as camera, gallery, calendar, Bluetooth, music, video, and messaging applications, as well as third-party applications. The applications discussed in this application embodiment are referred to as applications (APPs), which are software programs capable of performing one or more specific functions. Typically, multiple applications can be installed on an electronic device, such as camera applications and email applications. The applications mentioned below can be system applications pre-installed on the electronic device at the factory, or third-party applications downloaded by the user from the network or obtained from other electronic devices during the use of the electronic device.

[0071] Of course, for developers, they can write applications and install them into this layer. In one possible implementation, the application can be developed using the Java language, by calling the application programming interface (API) provided by the application framework layer. Developers can then interact with the underlying operating system (such as the kernel layer) through the application framework to develop their own applications.

[0072] The application framework layer provides the API and programming framework for the application layer. It can include predefined functions. The application framework layer may include a window manager, content provider, view system, phone manager, resource manager, notification manager, etc.

[0073] The window manager is used to manage window programs. The window manager can obtain the screen size, determine if a status bar is present, lock the screen (or display), and capture the screen, among other things.

[0074] Content providers are used to store and retrieve data, making that data accessible to applications. This data may include files (e.g., documents, videos, images, audio), text, and other information.

[0075] A view system includes visual controls, such as controls that display text, images, documents, and other content. View systems can be used to build applications. An interface in a display window can consist of one or more views. For example, a display interface including a text message notification icon could include a view that displays text and a view that displays images.

[0076] The phone manager provides communication functionality for electronic devices. The notification manager allows applications to display notification information in the status bar; it can be used to convey informative messages and can disappear automatically after a short pause without user interaction.

[0077] The runtime includes the core libraries and the virtual machine. The runtime is responsible for system scheduling and management.

[0078] The system's core library consists of two parts: one part contains the functionalities that the Java language needs to call, and the other part is the system's core library. The application layer and application framework layer run in a virtual machine. Taking Java as an example, the virtual machine executes the Java files in the application layer and application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0079] The system library can include multiple functional modules. Examples include: a surface manager, a media library, a 3D graphics processing library (e.g., OpenGL ES), a 2D graphics engine (e.g., SGL), and an image processing library. The surface manager manages the display subsystem and provides fusion of 2D and 3D layers for multiple applications. The media library supports playback and recording of various common audio and video formats, as well as still image files. It supports multiple audio and video encoding formats, such as MPEG4, H.564, MP3, AAC, AMR, JPG, and PNG. The 3D graphics processing library implements 3D graphics drawing, image rendering, compositing, and layer processing. The 2D graphics engine is the drawing engine for 2D graphics.

[0080] The kernel layer provides core system services for the operating system, such as security, memory management, process management, network protocol stack, and driver models, all of which are implemented based on the kernel layer. The kernel layer also serves as an abstraction layer between the hardware and software stacks. This layer contains many drivers related to electronic devices, including: display drivers; keyboard drivers as input devices; Flash drivers for memory-based devices; camera drivers; audio drivers; Bluetooth drivers; and WiFi drivers.

[0081] It is important to understand that the functional services described above are just an example. In practical applications, electronic devices may be divided into more or fewer functional services based on other factors, or the functions of each service may be divided in other ways, or they may not be divided into functional services but work as a whole.

[0082] The solutions provided in this application can be applied to a system composed of at least one electronic device. For example, the system can be a mobile phone, tablet, computer, headphones with a camera, an extended reality device with ambient light transmission capabilities (e.g., AR, VR, or mixed reality (MR) devices), a car with a smart cockpit, etc. As another example, the system can be a system composed of headphones and terminal devices (e.g., mobile phones, tablets, computers, smart screens, etc.) connected wirelessly.

[0083] The aforementioned system includes an electronic device that can control the task execution process of the system based on the method provided in this application embodiment, thereby adapting to different user needs. Different electronic devices in the system can communicate with each other via wireless or wired communication. For example, wireless communication methods may include short-range communication technologies such as Bluetooth and WiFi, while wired communication methods may include communication via cable connections. This application embodiment does not impose specific limitations on the specific communication methods.

[0084] Reference Figure 3 The implementation process of the solution provided in this application embodiment may include the following multiple stages.

[0085] Phase 1: Immersive Task Execution Phase

[0086] During this phase, electronic devices can control a system consisting of at least one electronic device, including this device, to perform immersive tasks.

[0087] In this embodiment, the immersive task can be any multimedia task requiring user participation. The user participation method can include at least one of listening, viewing, and operating. For example, immersive tasks can include various types of multimedia tasks such as audio and video playback, audio and video conferencing, live streaming, and games; this embodiment does not impose specific limitations.

[0088] When the system performs an immersive task, the electronic device can control the settings related to the immersive task either by default or based on user instructions. For example, the electronic device can control the settings related to the immersive task to put the system into immersive mode, thereby providing the user with a better immersive experience. Immersive mode can be a mode in which the user is highly immersed in the task and less aware of the external environment.

[0089] Optionally, the settings related to immersive tasks, and the specific settings corresponding to these settings in different modes, can be preset. For example, settings related to immersive tasks involving audio playback may include the volume of the audio device, audio playback mode, noise reduction level, etc. As another example, settings related to immersive tasks in a vehicle environment may include media volume, cabin light transmittance, and the transparency of privacy glass, etc.

[0090] As an alternative implementation, the electronic device can configure settings related to the immersive task in response to user instructions. Alternatively, the specific settings or configuration methods for the immersive task-related settings in immersive mode can be pre-defined. The electronic device can adjust the immersive task-related settings to correspond to the immersive mode when the immersive task begins.

[0091] The above method will be illustrated by example 1 below.

[0092] Example 1: In one example, consider a scenario where a user uses noise-canceling headphones to connect to an electronic device to participate in a video conference. Figure 4 As shown in diagram (a), user A can access a video conference via an electronic device. The electronic device can display the video conference interface, and user A can wear noise-canceling headphones wirelessly connected to the electronic device to send and receive audio. The electronic device can set the noise-canceling headphones to noise-canceling mode, thereby reducing environmental noise interference with the user's participation in the video conference and helping user A to immerse themselves in the meeting. In this scenario, the electronic device and the noise-canceling headphones can form a system for performing an immersive task, which is the video conference itself. Settings related to this task can include the noise-canceling level of the headphones. By setting the headphones to noise-canceling mode, the noise-canceling level can be set to a higher level, thus putting the system in immersive mode.

[0093] Phase 2: Dialogue Preparation Phase

[0094] During immersive task execution, when the electronic device detects an intention to converse between the user participating in the immersive task and other users, it can adjust the system to a high-awareness mode by modifying the relevant settings. This enhances the user's awareness of the external environment, allowing them to promptly perceive potential conversations. Optionally, the user participating in the immersive task can be a user associated with the electronic device or system, i.e., the user of the electronic device or system. The high-awareness mode can be a mode where the user's task immersion is low, but their awareness of the external environment is high. The user's task immersion in the high-awareness mode is lower than their task immersion in the immersive mode, and their environmental awareness in the high-awareness mode is lower than their environmental awareness in the immersive mode.

[0095] Optionally, the specific method by which the electronic device adjusts settings related to immersive tasks when adjusting the system to high-awareness mode can be preset. The method of adjusting the system to high-awareness mode can vary depending on the scenario.

[0096] For ease of understanding and description, the following embodiments will use the user participating in the immersive task as the first user and the other users as the second user as an example.

[0097] In some embodiments of this application, the electronic device may detect a dialogue intent between a first user and a second user in a manner that includes the following methods 1 and / or 2.

[0098] Method 1: The electronic device can determine that a dialogue intention exists between a first user and a second user when a first preset condition is met. The first preset condition includes:

[0099] 1) There is a second user in the environment where the electronic device is located. The second user is any user other than the first user.

[0100] 2) The distance between the first user and the second user is less than or equal to the set distance threshold.

[0101] 3) The second user is looking at the electronic device or the first user.

[0102] Optionally, the second user's gaze at the electronic device can specifically be a display screen (e.g., the screen of an electronic device) used to display an interface related to the immersive task. When the second user's gaze at the electronic device or the first user is detected, it can be assumed that the second user intends to engage in dialogue with the first user.

[0103] Optionally, the condition described above, where the second user is looking at the electronic device or the first user, can be replaced by the first user looking at the second user. Based on this condition, when the first user is detected looking at the second user, it can be assumed that the first user intends to engage in conversation with the second user.

[0104] In some embodiments of this application, the electronic device can use a camera with facial recognition capabilities to capture and identify the first user currently participating in the immersive task, and detect whether a second user exists in the environment. The electronic device can use a distance sensor (e.g., a time-of-flight (TOF) sensor, a depth camera, a proximity sensor, etc.) to detect the distance between the first and second users. The electronic device can use a camera with gaze detection capabilities to detect whether the second user is looking at the electronic device or the first user. Of course, the electronic device can also perform the above detections using other devices or other methods; no specific limitations are imposed in the embodiments of this application.

[0105] Optionally, Method 1 can be applied in scenarios where the second user is within the field of view (FOV) of the aforementioned camera.

[0106] For example, refer to Figure 5 One possible process for an electronic device to identify dialogue intent according to method 1 may include:

[0107] S501: Electronic devices acquire images of their surroundings in real time.

[0108] The electronic device can begin executing step S501 when the immersive task is started. After starting the immersive task, the electronic device can also detect and recognize the face of the first user participating in the immersive task.

[0109] S502: The electronic device detects the presence of a second user whose face is different from that of the first user participating in the immersive task based on the acquired image; if so, proceed to step S503; otherwise, continue to step S501.

[0110] Optionally, when the electronic device detects a user appearing within the field of view of the camera, it can detect whether the user's face is different from that of the first user. If so, it can confirm that the user is a second user whose face is different from that of the first user.

[0111] S503: The electronic device detects the location of the second user and the distance between the second user and the first user. Then it executes step S504.

[0112] S504: The electronic device determines whether the distance between the second user and the first user is less than or equal to the set distance threshold; if yes, proceed to step S505; otherwise, continue to step S501.

[0113] S505: The electronic device detects whether the second user is looking at the electronic device or the first user; if yes, proceed to step S506; otherwise, continue to step S501.

[0114] S506: The electronic device determines that the second user has the intention to communicate with the first user.

[0115] The above method 1 will be illustrated by example 2 below.

[0116] Example 2: In one example, within the scenario of Example 1, during user A's participation in a video conference, such as... Figure 4 As shown in diagram (b), when user B appears within the field of view of the electronic device's camera, the electronic device can determine through facial recognition that user B is not the user participating in the immersive task (i.e., user A), and can detect the distance between user B and user A through a distance sensor. When user B moves closer to user A, i.e., when the electronic device detects that the distance between user B and user A is less than or equal to a set distance threshold, the electronic device can continue to detect whether user B is looking at the electronic device's display screen or user A. When it detects that user B is looking at the electronic device's display screen or user A, the electronic device can confirm that user B has the intention to converse with user A.

[0117] Method 2: The electronic device can determine that a dialogue intention exists between the first user and the second user when a second preset condition is met. The second preset condition includes:

[0118] 1) Footsteps are present in the environment where the electronic device is located.

[0119] 2) The trend of the change in the audio energy of the footsteps is as follows: first increasing to above the first preset threshold, and then decreasing to below the second preset threshold. Wherein, the second preset threshold is less than the first preset threshold.

[0120] In some embodiments of this application, the electronic device can collect audio from the surrounding environment using an audio device (such as a microphone), and then identify the presence of footsteps based on the audio. If footsteps are present, they are considered to belong to a second user. Based on the changing trend of the audio energy of the footsteps, it can be determined that the second user gradually approaches the first user and stops, thus suggesting that the second user intends to communicate with the first user. Of course, the electronic device can also perform the above detection using other devices or other methods, and no specific limitations are imposed in the embodiments of this application.

[0121] Optionally, method 2 can be applied in scenarios where the second user is outside the field of view (FOV) of the camera on the electronic device.

[0122] For example, refer to Figure 6 One possible process for an electronic device to identify dialogue intent according to method 2 may include:

[0123] S601: Electronic devices collect audio from their environment in real time.

[0124] The electronic device can begin executing step S601 after initiating the immersive task.

[0125] S602: The electronic device detects whether there are footsteps in the environment based on the collected audio; if so, proceed to step S603; otherwise, proceed to step S601.

[0126] S603: The electronic device detects whether the audio energy of the footsteps gradually increases to above the first preset threshold; if so, proceed to step S604, otherwise proceed to step S601.

[0127] S604: The electronic device detects whether the audio energy of the footsteps gradually decreases below the second preset threshold; if so, proceed to step S605, otherwise proceed to step S601.

[0128] S605: The electronic device determines that the second user has the intention to communicate with the first user.

[0129] The above method 2 will be illustrated by example 3 below.

[0130] Example 3: In one example, within the scenario of Example 1, during user A's participation in a video conference, such as... Figure 4 As shown in diagram (b), when user B appears within the field of view of the electronic device's camera, the electronic device can collect audio from the surrounding environment and identify footsteps within the collected audio. The electronic device can continuously detect the audio energy of the footsteps, and when the detected audio energy of the footsteps gradually increases and exceeds a first preset threshold, and then decreases to below a second preset threshold (e.g., decreasing to below the second preset threshold within a set short period of time), it confirms that user B has the intention to converse with user A.

[0131] In some embodiments of this application, the specific settings or methods for immersive task-related settings in the high-awareness environment mode can be preset. When the electronic device detects the aforementioned dialogue intent, it can adjust the immersive task-related settings to correspond to the high-awareness environment mode. For example, if the volume setting in the high-awareness environment mode is set to low volume, the electronic device can lower the volume of the playing audio when it detects the aforementioned dialogue intent.

[0132] Example 4: In one example, within the scenario of Example 2 or Example 3 above, after detecting the intention of user B to converse with user A, the electronic device adjusts the immersive task-related settings to correspond to the high-awareness mode. This adjustment may include: reducing the audio playback volume of the audio device, and / or reducing the audio noise reduction level of the audio device. The audio device can be an audio playback device such as a speaker.

[0133] In some embodiments of this application, after detecting the dialogue intention between the first user and the second user, the electronic device can also display or play a prompt message. This prompt message can be used to indicate the location of the dialogue intention to the user, i.e., the location of the second user relative to the first user. For example, in the scenarios of Example 2 or Example 3 above, after detecting the dialogue intention between user B and user A, the electronic device can also display, as shown below... Figure 7 The prompt information shown can be used to indicate to user A the location of user B who intends to talk to user A.

[0134] Based on the above method, electronic devices can automatically prepare for possible conversations when they detect dialogue intent. By adjusting to a high-awareness mode and providing prompts to users participating in immersive tasks, users can switch from participating in immersive tasks to participating in conversations with the outside world in a timely and efficient manner.

[0135] Phase 3: Dialogue Phase

[0136] In some embodiments of this application, after adjusting the system to a high-awareness mode, the electronic device can pause or take over the immersive task upon receiving a dialogue instruction from the user or upon detecting a dialogue, thereby adjusting the system to dialogue mode. Optionally, for immersive tasks that can be paused (e.g., audio / video playback), the electronic device can directly pause the execution of the immersive task. For immersive tasks that cannot be paused (e.g., audio / video conferencing based on real-time media streaming), the electronic device can take over the immersive task. Optionally, the specific method by which the electronic device takes over the immersive task can be preset. The method of taking over the immersive task can be different in different scenarios. For example, taking over the immersive task may include at least one of the following: executing the immersive task in the background, turning off the audio / video output (i.e., not playing the audio / video of the immersive task), and recording the execution process, progress, or minutes of the immersive task during the dialogue.

[0137] As an optional implementation of the aforementioned dialogue instructions, in stage 2, after detecting the dialogue intent between the first user and the second user, the electronic device can also display a control for triggering entry into dialogue mode, through which the user can issue dialogue instructions. For example, such as Figure 7As shown, the electronic device can display control 1 for triggering entry into the dialogue mode, and can confirm entry into the dialogue mode in response to the user's click on control 1 (i.e., the operation of issuing a dialogue instruction). Of course, the electronic device can also receive dialogue instructions issued by the user in other ways, and no specific limitations are made in this embodiment.

[0138] In some embodiments of this application, the electronic device can confirm that a conversation has occurred when a third preset condition is met. The third preset condition may include at least one of the following:

[0139] 1) User voice was detected in the environment.

[0140] Among them, electronic devices can collect audio from the environment through audio devices and identify whether user voice is present in the collected audio.

[0141] 2) It was detected that there was a two-way dialogue intention between the first user and the second user, that is, the first user had the intention to talk to the second user and the second user had the intention to talk to the first user.

[0142] Wherein, based on the first condition mentioned above, when the electronic device determines that it detects that the first user is looking at the second user and the second user is looking at the electronic device or the first user, it can be considered that the first user and the second user have a two-way dialogue intention.

[0143] Example 5: In one example, in the video conferencing scenario described in Example 4, after the electronic device adjusts the system to a high-awareness mode, it receives a dialogue instruction from the user (e.g., receives a click from user A). Figure 7 When operating control 1 as shown, or when a conversation is detected, the system can be switched to conversation mode by performing the following operations: mute the current video conference in both directions (i.e., turn off the audio input and audio output of the audio device), turn off the camera used to participate in the video conference, turn off the noise reduction of the headphones, record the minutes of the video conference, etc.

[0144] In some embodiments of this application, the electronic device may also display a prompt message indicating that it is currently in dialogue mode after being switched to dialogue mode.

[0145] In some embodiments of this application, after the electronic device adjusts the system to the high-awareness mode, if it does not receive a dialogue instruction from the user within a set time and does not detect a dialogue, it can switch back to the immersive mode described in stage 1 and continue to perform the immersive task.

[0146] In the above method, the electronic device adjusts the system to dialogue mode after the dialogue occurs, which can reduce the interference of the current task on the user's participation in the dialogue and facilitate the user's efficient participation in the dialogue.

[0147] Phase 4: Immersive Task Recovery Phase

[0148] In some embodiments of this application, after switching to dialogue mode, the electronic device can continue to perform immersive tasks when it receives a dialogue end instruction from the user or when it detects that the dialogue has ended. For example, it can continue to perform immersive tasks after switching to the immersive mode described in stage 1.

[0149] Regarding the aforementioned dialogue end indication, as an optional implementation, the electronic device can display a control for ending the dialogue after switching to dialogue mode, and can continue performing the immersive task in response to the user's operation on the control. For example, in the scenario of Example 5, the electronic device can also display... Figure 8 The control 2 shown is used to end the conversation, and it can switch to immersive mode in response to user A's click on control 2 (i.e., issuing a conversation end instruction) to continue performing the immersive task. Of course, the electronic device can also receive the conversation end instruction issued by the user in other ways, and no specific limitation is made in this embodiment.

[0150] In some embodiments of this application, the electronic device can confirm the end of the conversation when it detects that a fourth preset condition is met. The fourth condition may include at least one of the following:

[0151] 1) The duration of user voice not detected in the environment exceeds the set duration, that is, the duration of user voice not detected by the electronic device exceeds the set duration.

[0152] 2) The first user was detected to be looking at the electronic device again.

[0153] Specifically, the first user refocusing on the electronic device may be refocusing on the electronic device's display screen.

[0154] Example 6: In one example, in the video conferencing scenario described in Example 5, after the electronic device switches the system to dialogue mode, it receives a dialogue end instruction from the user (e.g., receives a click from user A). Figure 8 When operating control 2 as shown, or when the end of the conversation is detected, the device can switch to immersive mode to continue performing the immersive task. Optionally, switching the electronic device to immersive mode may include: unmuting the current video conference (i.e., turning on the audio input and output of the audio device), turning on the camera used to participate in the video conference, turning on noise cancellation on the headphones, and stopping the recording of video conference minutes, etc. Figure 9 As shown, the electronic device can display a summary of the video conference recorded during the conversation on the screen, so that user A can understand the content of the video conference during the conversation through the summary.

[0155] Based on the method provided in the above embodiments, when performing immersive tasks, electronic devices can identify the dialogue intentions between users before the dialogue occurs. By adjusting the working mode and providing dialogue prompts, they can provide users with an environment that makes it easier for them to perceive and participate in the dialogue, thereby reducing the interference of immersive tasks on the dialogue and ensuring that users can quickly and efficiently perceive and participate in dialogues with the outside world during immersive tasks. After a dialogue occurs, the electronic device can automatically pause or take over the immersive task, improving the user's dialogue efficiency and experience.

[0156] The methods provided in the above embodiments are illustrated below with reference to Examples 7 to 9.

[0157] Example 7: In one example, consider a scenario where user A uses noise-canceling headphones connected to an electronic device to watch a video. The immersive task in this scenario is video playback, which the electronic device can perform as described in stage 1 above. For example, the electronic device can activate the noise-canceling mode of the headphones and play the video at a certain volume; in this case, the electronic device is in immersive mode. While user A is watching the video, when the electronic device detects that user B intends to converse with user A based on the method described in stage 2 above, it can switch to a high-awareness environment mode. When the electronic device switches to high-awareness environment mode, it can perform at least one of the following operations:

[0158] 1) Lower the video playback volume.

[0159] 2) Reduce the noise cancellation level of the noise-canceling headphones.

[0160] 3) Display a prompt message, which indicates that user B intends to have a conversation with user A and the location of user B relative to user A.

[0161] After switching to high-awareness mode, the electronic device can switch to dialogue mode upon confirming the start of a dialogue (i.e., receiving a dialogue instruction or detecting the occurrence of a dialogue) based on the method described in stage 3 above. When the electronic device switches to dialogue mode, it can pause video playback.

[0162] After switching to dialogue mode, the electronic device can switch to immersive mode and continue performing the video playback task when the dialogue ends (i.e., a dialogue end indication is received or the dialogue end is detected) based on the method described in stage 4 above. Optionally, the electronic device can continue performing the video playback task in the same manner and configuration as before switching to the ambient high awareness mode.

[0163] Example 8: In one example, taking a scenario where user A uses a VR device to watch a video, the immersive task in this scenario is video playback. The VR device can perform this task as described in stage 1 above, and at this time, the VR device is in immersive mode. While user A is watching the video, when the VR device detects that user B intends to interact with user A based on the method described in stage 2 above, it can switch to a high-awareness environment mode. When the VR device switches to high-awareness environment mode, it can perform at least one of the following operations:

[0164] 1) Turn on ambient light transmission mode.

[0165] 2) Lower the video playback volume.

[0166] 3) If the VR device is connected to noise-canceling headphones, reduce the noise cancellation level of the noise-canceling headphones.

[0167] After switching to high-awareness mode, the VR device can switch to dialogue mode upon confirming the start of the dialogue based on the method described in stage 3 above. When switching to dialogue mode, the VR device can pause video playback. Since video playback is a pauseable media task not based on real-time media streaming, the VR device can pause video playback after switching to dialogue mode.

[0168] After switching to dialogue mode, and confirming the end of the dialogue based on the method described in stage 4 above, the VR device can switch back to immersion mode and continue video playback. Optionally, the VR device can continue video playback in the same manner and configuration as before switching to high-awareness mode. For example, the VR device can disable ambient occlusion mode and increase the video playback volume. If noise-canceling headphones are connected to the VR device, the noise cancellation level of the headphones can also be improved.

[0169] In another example, if the task of watching a video is replaced by watching a live stream, the method for the VR device to switch from immersive mode to high-awareness mode when it detects a dialogue intent is the same as described above. The VR device's operation of switching to dialogue mode upon confirming the start of a dialogue can include: pausing audio playback and displaying the live stream content; if noise-canceling headphones are connected, disabling their noise-canceling mode; and recording the live stream content or a summary of the dialogue. Subsequently, after confirming the end of the dialogue, the VR device can switch back to immersive mode and continue the live stream task. It can also prompt the user to view the live stream content or summary recorded by the VR device during the dialogue. For example, the VR device can disable ambient occlusion mode, continue playing audio with increased volume, continue displaying the live stream interface, and, if noise-canceling headphones are connected, increase their noise-canceling level.

[0170] Example 9: In one example, taking a scenario where user A is watching a video in a smart cockpit, the immersive task in this scenario is video playback. The smart cockpit can perform this task by referring to the method described in stage 1 above. For example, the smart cockpit can close the sunshade, set the privacy glass to opaque, and play the video at a certain volume. At this time, the smart cockpit is in immersive mode. While user A is watching the video, when the smart cockpit detects that user B has the intention to talk to user A based on the method described in stage 2 above, it can switch to a high-awareness mode. When the smart cockpit switches to the high-awareness mode, it can perform at least one of the following operations:

[0171] 1) Lower the video playback volume.

[0172] 2) Open the sunshade curtain.

[0173] 3) Increase the transparency of the privacy glass or set the privacy glass to transparent.

[0174] 4) Display or play a prompt message, which indicates that user B intends to have a conversation with user A and the location of user B relative to user A.

[0175] 5) Play audio collected by external audio devices through the audio devices inside the smart cockpit.

[0176] After switching to the high-awareness mode, the intelligent cockpit can switch to dialogue mode upon confirming the start of the dialogue based on the method described in stage 3 above. When switching to dialogue mode, the intelligent cockpit can pause video playback.

[0177] After switching to dialogue mode, the intelligent cockpit can switch to immersive mode and continue video playback after confirming the end of the dialogue based on the method described in stage 4 above. Optionally, the intelligent cockpit can continue video playback in the same manner and configuration as before switching to the high-awareness mode.

[0178] It should be understood that the implementation process provided in the above embodiments is merely an example of the method flow applicable to the embodiments of this application. The execution order of each step in each implementation process can be adjusted according to actual needs, and other steps can be added or some steps can be removed. The execution order between steps that are not temporally related in each implementation process can be arbitrary, and this application embodiment does not impose any restrictions on this.

[0179] It should be noted that the application scenarios provided in the above embodiments are merely illustrative examples of the applicable scenarios of the embodiments of this application, and do not limit the applicable scenarios of the solutions of this application. Some methods or the same technical concepts provided in any of the above embodiments can also be applied in other embodiments or other scenarios, or can be combined with the methods provided in other embodiments. Specifically, they can be applied in combination with specific embodiments or specific scenarios, and will not be listed and described one by one in this application.

[0180] Based on the above embodiments and the same technical concept, this application also provides a task execution method, such as... Figure 10 As shown, the method may include:

[0181] S1001: The electronic device performs the first task based on the first setting.

[0182] The first task can be a processing task that includes at least one type of processing, such as audio output (e.g., audio playback), audio input, video output (e.g., content display), and video input. For example, the first task could be a task related to audio / video playback, audio / video calls, audio / video conferencing, or gaming. Here, "audio / video" refers to either audio or video.

[0183] An electronic device can perform the first task on its own, or it can perform the first task collaboratively with multiple electronic devices, including the electronic device itself.

[0184] Optionally, the first task can be the immersive task described in the foregoing embodiments.

[0185] The first setting is a specific setting associated with the first task. For example, if the setting includes audio playback volume, the first setting can be a specific audio playback volume (i.e., an audio playback volume value) set by the electronic device. As another example, if the setting includes the noise cancellation level of noise-canceling headphones connected to the electronic device, the first setting can be a specific noise cancellation level set by the electronic device. Yet another example, if the setting includes the ambient occlusion state of the extended reality device, the first setting can be enabling the ambient occlusion mode of the extended reality device.

[0186] The first setting can be used to control the way an electronic device performs its first task (or configuration method). Based on this setting, the electronic device can provide the user with a certain immersive experience when performing the first task.

[0187] Optionally, the first setting can be preset, for example, it can be pre-configured by the electronic device. The first setting can also be user-defined, for example, the electronic device can determine and apply (i.e. implement) the first setting based on user operations.

[0188] Regarding the method by which the first electronic device performs the first task based on the first setting, the method of the electronic device performing an immersive task in immersive mode as described in stage 1 above can also be referred to, and will not be described in detail here.

[0189] S1002: When the first condition is met, the electronic device switches the first setting to the second setting and / or outputs the first prompt information, and continues to execute the first task; wherein, the first condition is a condition used to determine that there is a dialogue intention between the first user and the second user, the first user is the user of the electronic device, and the second user is any user other than the first user; the first setting and the second setting are different settings of the same setting item associated with the first task, and the immersion level of the second setting is lower than that of the first setting; the first prompt information is used to indicate the dialogue intention.

[0190] Optionally, the first prompt can also be used to indicate the location of the second user.

[0191] The second setting is a different setting from the first setting, which is associated with the first task. The second setting can be used to control how the electronic device performs the first task. By switching from the first setting to the second setting, the immersive experience provided by the electronic device can be adjusted. Immersion level, also known as the degree of immersion or sense of immersion, refers to the level of engagement and focus a user experiences in a specific activity or environment.

[0192] In some embodiments of this application, the second setting may be a setting obtained by adjusting the first setting with the goal of reducing immersion. This will be explained below in conjunction with specific scenarios.

[0193] In the first example, taking a scenario where the first setting includes a first audio playback volume as an example, adjusting the first setting to reduce immersion can be done by lowering the first audio playback volume. Therefore, the second setting can include a second audio playback volume obtained after lowering the first audio playback volume. For example, this method can be applied to the scenario described in Example 4, where the audio playback volume of the audio device in immersive mode is the first audio playback volume, and the lowered audio playback volume is the second audio playback volume.

[0194] In the second example, taking the scenario of an electronic device connecting to noise-canceling headphones as an example, the electronic device can receive and play audio through the noise-canceling headphones. In this scenario, when the electronic device performs a first task, it can activate the noise-canceling mode of the noise-canceling headphones. The first setting can include the first noise-canceling level of the noise-canceling headphones. Adjusting the first setting with the goal of reducing immersion can be achieved by lowering the noise-canceling level of the noise-canceling headphones; therefore, the second setting can include a second noise-canceling level obtained by lowering the first noise-canceling level of the noise-canceling headphones. The method described in Example 4 above can also be referred to.

[0195] In the third example, taking the scenario where the electronic device is an extended reality device as an example, the extended reality device can turn off the ambient spectral mode when performing the first task; that is, the first setting can include setting the ambient spectral mode to be off. Adjusting the first setting to reduce immersion can be achieved by turning on the ambient spectral mode. Therefore, the second setting can include setting the ambient spectral mode to be on. The method described in Example 8 above can also be referred to.

[0196] In the fourth example, taking an electronic device located in a vehicle (e.g., the electronic device being a smart cockpit), when performing the first task, the electronic device can close the vehicle's sunshade and / or set the transparency of the privacy glass to a first transparency. That is, the first setting includes the sunshade being closed and / or the privacy glass being transparent to a first transparency. Adjusting the first setting with the goal of reducing immersion can be achieved by opening the vehicle's sunshade and / or reducing the transparency of the privacy glass. Therefore, the second setting can include the sunshade being open and / or the second transparency obtained after reducing the first transparency of the privacy glass. The method described in Example 9 above can also be referenced. In this example, when the first condition is met, the electronic device can also play audio collected by an audio device outside the vehicle's cabin through the audio device inside the vehicle's cabin, so that the user inside the cabin can hear the sounds from outside the cabin more clearly.

[0197] In the above method, since the immersion level of the second setting is lower than that of the first setting, the electronic device switches the first setting to the second setting and / or outputs the first prompt information when the first condition is met, that is, when it is determined that there is a dialogue intention between the first user and the second user, and continues to perform the first task. This can enhance the user's perception of the external environment while performing the first task, making it easier for the user to perceive and participate in the dialogue with the outside world in a timely manner.

[0198] In some embodiments of this application, satisfying the first condition may include at least one of the following:

[0199] 1) The distance between the first user and the second user is less than or equal to a first set threshold, and the first user is detected looking at the second user or the second user is detected looking at an electronic device or the first user.

[0200] The detection method for this condition can be implemented by referring to the aforementioned method 1, and will not be elaborated here.

[0201] 2) Footsteps are detected in the collected ambient audio and the audio energy of the footsteps changes according to a set trend. The set trend includes: first increasing to exceed the second set threshold, and then decreasing to below the third set threshold; wherein the third set threshold is less than the second set threshold.

[0202] The detection method for this condition can be implemented by referring to method 2 mentioned above, and will not be elaborated here.

[0203] Optionally, step S1002 can also be implemented by referring to the method described in stage 2 above.

[0204] In some embodiments of this application, such as Figure 10 As shown, after step S1002, the following step S1003 may also be included.

[0205] S1003: When the second condition is met, the electronic device stops executing the first task, or executes the setting process; wherein, the second condition is a condition used to determine that a dialogue has occurred between the first user and the second user.

[0206] The electronic device's cessation of the first task can be temporary, and it can resume execution of the first task when the conditions are met later.

[0207] Optionally, if the first task is not based on a real-time media stream, the electronic device may stop executing the first task when the second condition is met. If the first task is based on a real-time media stream, the electronic device may perform the set processing when the second condition is met.

[0208] In some embodiments of this application, the setting process may include at least one of the following: turning off the audio input and output of the electronic device; turning off the camera of the electronic device; turning off the display function of the electronic device; generating a summary of the execution of the first task; acquiring audio or video content generated by executing the first task; and amplifying human voice through noise-canceling headphones when the electronic device is connected. The setting process can also refer to the process for determining the start of a dialogue in the foregoing embodiments, and will not be detailed here.

[0209] In some embodiments of this application, satisfying the second condition may include at least one of the following: receiving a first instruction, the first instruction being used to indicate the start of a dialogue; detecting that the collected ambient audio includes human voices; detecting that the first user is looking at the second user and the second user is looking at the first user or the electronic device (i.e., detecting that the first user and the second user have a two-way dialogue intention).

[0210] Optionally, the first instruction can be an instruction issued by the user through operation of the first electronic device, or it can be an instruction issued by the user via voice. For example, in Figure 4 In the scenario shown, the first user can be user A, the second user can be user B, and the first instruction can be user A clicking... Figure 7 The instructions issued by the control 1 shown are for operation.

[0211] The method for detecting that the electronic device meets the second condition can be implemented by referring to the method for determining the start of the dialogue described in stage 3 above, and will not be described in detail here.

[0212] In some embodiments of this application, such as Figure 10 As shown, after step S1003, the following step S1004 may also be included.

[0213] S1004: When the third condition is met, the electronic device continues to perform the first task based on the first setting; wherein, the third condition is a condition used to determine that the first user and the second user have ended the dialogue.

[0214] In some embodiments of this application, satisfying the third condition may include at least one of the following: receiving a second instruction, the second instruction being used to indicate the end of the dialogue; not detecting human voices in the collected ambient audio for a duration exceeding a set duration; detecting the first user looking at the electronic device.

[0215] Optionally, the second instruction can be an instruction issued by the user through operation of the first electronic device, or it can be an instruction issued by the user via voice. For example, in Figure 4 In the scenario shown, the first user can be user A, the second user can be user B, and the second instruction can be user A clicking... Figure 8 The instructions issued by the control 2 shown in the figure.

[0216] The method for detecting that the electronic device meets the third condition can be implemented by referring to the method for determining the start of the dialogue described in stage 4 above, and will not be described in detail here.

[0217] In some embodiments of this application, when the electronic device performs a setting process upon meeting a second condition, and the setting process includes generating a summary of the execution of the first task, the electronic device may also output (e.g., display) the summary of the execution of the first task upon meeting a third condition, so that the user can confirm the execution status of the first task during the conversation. When the electronic device performs a setting process upon meeting a second condition, and the setting process includes acquiring audio or video content generated by executing the first task, the electronic device may also output (e.g., play) the acquired audio or video content upon meeting a third condition, so that the user can confirm the execution status of the first task during the conversation.

[0218] The specific steps performed by the electronic device in the above method can be found in the relevant descriptions in the foregoing embodiments, and will not be repeated here.

[0219] Based on the above embodiments and the same technical concept, this application also provides an electronic device for implementing the task execution method provided in this application. Figure 11As shown, electronic device 1100 may include: memory 1101, one or more processors 1102, and one or more computer programs (not shown). These devices may be coupled via one or more communication buses 1103. Optionally, electronic device 1100 may also include a display screen 1104.

[0220] The memory 1101 stores one or more computer programs (code), and the one or more computer programs include computer instructions; one or more processors 1102 call the computer instructions stored in the memory 1101, causing the electronic device 1100 to execute the task execution method provided in the above-described embodiments of this application.

[0221] In a specific implementation, memory 1101 may include high-speed random access memory, and may also include non-volatile memory, such as one or more disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. Memory 1101 may store an operating system (hereinafter referred to as the system), such as embedded operating systems like Android, iOS, Windows, or Linux. Memory 1101 can be used to store implementation programs of the embodiments of this application. Memory 1101 may also store network communication programs, which can be used to communicate with one or more additional devices, one or more user devices, or one or more network devices.

[0222] One or more processors 1102 may be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of programs according to the present application.

[0223] Display screen 1104 is used to display application interfaces and other related user interfaces.

[0224] It should be noted that, Figure 11 This is merely one implementation of the electronic device 1100 provided in this application embodiment. In practical applications, the electronic device 1100 may include more or fewer components, as detailed in the following references. Figure 1 The specific structure and description shown are not limited here.

[0225] Based on the above embodiments and the same technical concept, this application also provides a computer-readable storage medium storing a computer program that, when run on a computer, causes the computer to perform the method provided in the above embodiments.

[0226] Based on the above embodiments and the same technical concept, this application also provides a computer program product, which includes a computer program or instructions. When the computer program or instructions are run on a computer, the computer performs the method provided in the above embodiments.

[0227] Based on the above embodiments and the same technical concept, this application also provides a computer program product, which includes a computer program or instructions. When the computer program or instructions are run on a computer, the computer performs the method provided in the above embodiments.

[0228] The methods provided in this application can be implemented, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, they can be implemented, in whole or in part, in the form of a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of the present invention is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user equipment, or other programmable device. Computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. A computer-readable storage medium can be any available medium that a computer can access, or a data storage device such as a server or data center that integrates one or more available media. Available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., digital video discs (DVDs), or semiconductor media (e.g., SSDs), etc.

[0229] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A task execution method, applied to an electronic device, characterized in that, The method includes: Based on the initial settings, execute the first task; When the first condition is met, the first setting is switched to the second setting and / or the first prompt message is output, and the first task continues to be executed; Wherein, the first condition is a condition used to determine that there is a dialogue intention between the first user and the second user, the first user is the user of the electronic device, and the second user is any user other than the first user; The first setting and the second setting are different settings of the same setting item associated with the first task, and the immersion level of the second setting is lower than that of the first setting; The first prompt message is used to indicate the intent of the dialogue.

2. The method as described in claim 1, characterized in that, The first condition being met includes: The system detects that the distance between the first user and the second user is less than or equal to a first preset threshold, and detects that the first user is looking at the second user or that the second user is looking at the electronic device or the first user; and / or Footsteps are detected in the collected ambient audio, and the audio energy of the footsteps changes according to a set trend. The set trend includes: first increasing to exceed a second set threshold, and then decreasing to below a third set threshold; wherein the third set threshold is less than the second set threshold.

3. The method as described in claim 1 or 2, characterized in that, The first prompt message is also used to indicate the location of the second user.

4. The method according to any one of claims 1 to 3, characterized in that, The switching from the first setting to the second setting includes at least one of the following: Reduce the first audio playback volume included in the first setting to the second audio playback volume included in the second setting; When the electronic device is connected to noise-canceling headphones, the noise reduction level of the noise-canceling headphones is reduced from the first noise reduction level included in the first setting to the second noise reduction level included in the second setting; When the electronic device is an extended reality device, the ambient spectral mode of the extended reality device is switched from the off state included in the first setting to the on state included in the second setting; When the electronic device is in the vehicle, it switches the vehicle's sunshade from the closed state included in the first setting to the open state included in the second setting, and / or reduces the transparency of the vehicle's privacy glass from the first transparency included in the first setting to the second transparency included in the second setting.

5. The method as described in claim 4, characterized in that, The method further includes: When the first condition is met, if the electronic device is located in the vehicle, the audio collected by the audio device outside the vehicle's cabin is played through the audio device inside the vehicle's cabin.

6. The method according to any one of claims 1 to 5, characterized in that, After switching the first setting to the second setting and / or outputting the first prompt message, and continuing to execute the first task, the method further includes: If the second condition is met, stop executing the first task, or execute the set processing. The second condition is used to determine that a dialogue has occurred between the first user and the second user. The configuration process includes at least one of the following: Turn off the audio input and output of the electronic device; Turn off the camera of the electronic device; Turn off the display function of the electronic device; Generate a summary of the execution of the first task; Collect audio or video content generated by performing the first task; When the electronic device is connected to noise-canceling headphones, the human voice is amplified through the noise-canceling headphones.

7. The method as described in claim 6, characterized in that, Before stopping the execution of the first task, the method further includes: determining that the first task is not a task executed based on a real-time media stream; or Before performing the set processing, the method further includes: determining that the first task is a task based on a real-time media stream.

8. The method as described in claim 6 or 7, characterized in that, The second condition being met includes at least one of the following: A first instruction is received, which is used to indicate the start of a dialogue; Human voices were detected in the collected ambient audio. The system detects that the first user is looking at the second user and the second user is looking at the first user or the electronic device.

9. The method according to any one of claims 6 to 8, characterized in that, After stopping the execution of the first task, or after performing the setting process, the method further includes: When the third condition is met, the first task continues to be executed based on the first setting; wherein, the third condition is a condition used to determine that the first user and the second user have ended their conversation.

10. The method as described in claim 9, characterized in that, After performing the setting process, the method further includes: When the setting process includes generating a summary of the execution of the first task, output the summary; and / or When the setting process includes collecting audio or video content generated by executing the first task, the audio or video content is output.

11. The method as described in claim 9 or 10, characterized in that, The third condition being met includes at least one of the following: A second instruction has been received, which indicates the end of the dialogue. No human voices were detected in the collected ambient audio for a duration exceeding the set duration; The system detected that the first user was looking at the electronic device.

12. An electronic device, characterized in that, The electronic device includes a memory and one or more processors; The memory is used to store computer program code, which includes computer instructions; when the computer instructions are executed by the one or more processors, the electronic device performs the method as described in any one of claims 1 to 11.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when run on an electronic device, causes the electronic device to perform the method as described in any one of claims 1 to 11.

14. A computer program product, characterized in that, The computer program product includes a computer program or instructions that, when executed on an electronic device, cause the electronic device to perform the method as described in any one of claims 1 to 11.

Images