Display method of digital assistant virtual image and related device

Abstract

The invention discloses a display method of a digital assistant virtual image, which can improve the naturalness of interaction between a user and a digital assistant. According to the method, a virtual image of a digital assistant is set to comprise a plurality of feature bodies, and after the electronic equipment obtains user input, the electronic equipment generates a corresponding signal instruction, so that the form of the virtual image displayed on an interface is triggered to change; therefore, different working states of the digital assistant are represented in different forms, so that a user can confirm the current state of interaction with the digital assistant according to the form change condition of the digital assistant, and the naturalness of interaction between the user and the digital assistant is improved.

Description

[0001] This application is required to be submitted to the State Intellectual Property Office on June 29, 2023, with application number 202310786258.8 and the name of the invention being “A

[0002] The entire contents of the Chinese patent application entitled "Display method and related device for digital assistant" are incorporated into the present application by reference. Technical Field

[0003] The present application relates to the field of human-computer interaction technology, and in particular to a method for displaying a virtual image of a digital assistant and related devices. Background Art

[0004] With the rapid development of smart devices, configuring digital assistants on smart devices has gradually become the mainstream. Using digital assistants to replace manual operations of users and automatically complete a series of operations of smart devices can free users' hands and improve the user experience of smart devices.

[0005] Currently, when using digital assistants, users usually need to wake up the digital assistants through voice commands or manual operations to enter the user interface of the digital assistants. The user interface of the digital assistants will display an assistant image, which is usually a two-dimensional pattern that is static or continuously moves in a certain pattern.

[0006] In the user interface of the digital assistant, after the user further inputs voice commands or gesture commands, it is usually necessary to wait for the digital assistant to output the recognition results before confirming whether the digital assistant has received the voice commands or gesture commands. Later, the digital assistant's form does not change during the process of recognizing or processing commands. Therefore, after the user inputs voice commands or gesture commands, it is usually necessary to wait for the digital assistant to output the recognition results before confirming whether the digital assistant has received the voice commands or gesture commands. That is, when the digital assistant recognizes or processes the commands input by the user, the user cannot get feedback from the digital assistant, and it is difficult to identify the current working status of the digital assistant. It is easy to mistakenly believe that the digital assistant has not received the command, which causes the user to repeat the command. Summary of the invention

[0007] The present application provides a method for displaying a digital assistant virtual image, which can improve the naturalness of the interaction between the user and the digital assistant.

[0008] The first aspect of the present application provides a method for displaying a digital assistant virtual image, which is applied to an electronic device, and the virtual image of the digital assistant includes multiple feature bodies. In this method, the virtual image is first displayed in a first form on a first interface; then, a signal instruction generated by the electronic device is detected, and the virtual image is displayed in a second form. Among them, the signal instruction is generated by the electronic device based on the received first user input, and the user input can be before or after the first form is displayed. That is, after the electronic device receives the first user input, a signal instruction is generated, thereby triggering the virtual image to switch from the first form to the second form, so that the form of the virtual image displayed on the interface of the electronic device changes.

[0009] In addition, the first form and the second form both display at least one of a plurality of characteristic bodies, and the change from the first form to the second form displays that at least one characteristic body undergoes at least one of a change in shape, a change in position, a change in movement pattern, and a change in display area.

[0010] The different forms of the virtual image represent the different working states of the digital assistant, that is, the digital assistant uses the form of the virtual image to indicate the current working state of itself. In this way, when the form of the virtual image displayed on the interface changes, the user can intuitively and clearly understand the change in the working state of the digital assistant, and thus understand the current response of the digital assistant to the user's instructions.

[0011] In this solution, the virtual image of the digital assistant is set to include multiple features, and after the electronic device obtains user input, the electronic device generates a corresponding signal instruction, thereby triggering the change of the form of the virtual image displayed on the interface, thereby achieving the use of different forms to represent the different working states of the digital assistant, so that the user can confirm the current state of interaction with the digital assistant through the changes in the form of the digital assistant, thereby improving the naturalness of the interaction between the user and the digital assistant.

[0012] In a possible implementation, the plurality of feature bodies include a first feature body, the plurality of feature bodies include the first feature body, and the shape change of the first feature body includes at least one of the following changes: enlargement, reduction, stretching, compression, rotation, and switching between a main form, a geometric array, or a strip. That is, the shape of the first feature body includes: a main form, a geometric array, and a strip. Moreover, the shape change of the first feature body includes switching between multiple shapes, and enlarging, reducing, stretching, compressing, or rotating each shape. That is, in addition to switching between different shapes, the shape change of the first feature body may also include enlarging, reducing, stretching, compressing, or rotating under the same shape.

[0013] The main form refers to the form of the first feature body under normal circumstances, while the geometric array and the long strip are the forms of the first feature body under certain specific circumstances.

[0014] In a possible implementation, the plurality of feature bodies further include a second feature body. The main form of the first feature body is a ring, the second feature body is a sphere, and the center of the ring coincides with the center of the sphere. The position change of the first feature body includes at least one of a change in orbital position after flipping around the sphere and a change in position caused by the overall movement or rotation of the virtual image. The motion law of the first feature body includes the ring rotating around the sphere with the center of the ring as the fulcrum according to a preset trajectory. The motion law of the second feature body includes the sphere reciprocating along a straight line passing through the center of the ring and perpendicular to the plane where the ring is located according to a preset law.

[0015] In a possible implementation, when the second user input is detected, a third feature body is displayed on the first feature body, and the third feature body can move on the first feature body and indicate the position of the user.

[0016] In one possible implementation, the virtual image has a breathing state, and the virtual image is displayed in a first form on the first interface, including: if the first interface is an off-screen interface, the first form is a breathing state including a first feature body and a second feature body, and the first feature body is displayed in a main form.

[0017] In one possible implementation, the first interface is a screen-off interface, and the digital assistant in a breathing state moves back and forth on the screen according to a preset pattern.

[0018] In a possible implementation, when the first interface is the screen-off interface and the first state is the breathing state, in response to detecting the second user input, a third feature body is displayed on the first feature body, and the third feature body can move on the first feature body and indicate the user's position. The first feature body is a ring, and the third feature body can be a light effect area on the ring.

[0019] In a possible implementation, the virtual image has a waiting state and a listening state, and the first interface is a lock screen interface or any screen interface after unlocking. Displaying the virtual image in a first form on the first interface specifically includes: displaying the waiting state as the first form on the first interface, the waiting state includes a first feature body, and the first feature body is displayed in a long strip as a navigation prompt line, and the navigation prompt line is used as a reference line for screen gesture navigation; wherein, when no input for waking up the digital assistant is received, the waiting state is permanently displayed on the first interface.

[0020] Alternatively, a third user input is detected and displayed on the first interface in a listening state as the first form, and the third user input is used to wake up the digital assistant. The listening state includes a first feature body and a second feature body, and the first feature body is displayed in the main form.

[0021] In a possible implementation, the first interface is a lock screen interface or any screen interface after unlocking, and the first form is a listening state. Then, displaying the virtual image in the first form on the first interface specifically includes: the first feature body is first displayed in a geometric array; the first feature body is then changed from the geometric array to the main form, and the second feature body is displayed. That is, in the initial stage, only the first feature body is displayed, and the second feature body is not displayed, and the first feature body is initially displayed in a geometric array, covering the entire screen; then the first feature body is changed from the geometric array to the main form through shape display, and the second feature body is displayed.

[0022] That is to say, when waking up the digital assistant, the digital assistant first appears in the form of a geometric array (such as a dot matrix) and gradually fills the screen; then, the digital assistant gradually transforms from the geometric array to the main form of the first feature body (i.e., a ring) and displays the second feature body (i.e., a sphere), thereby realizing the awakening process of the digital assistant in the form of an animation, increasing the intuitiveness of the interaction process, and helping users to identify the current working status of the digital assistant.

[0023] In one possible implementation, the first feature body is first displayed in a geometric array, specifically: a portion of the array of geometric figures is displayed at an initial display position, and then the geometric figures are displayed on the entire screen. That is, the initial display position of the geometric array is first determined; secondly, the portion of the array of geometric figures is displayed at the initial display position, and then the geometric figures are gradually spread over the entire screen. For example, a portion of the dot matrix is ​​first displayed from an initial display position, and then the dot matrix is ​​gradually spread to cover the entire screen. Among them, the user input refers to the input used to wake up the digital assistant.

[0024] In one possible implementation, the initial display position may be near the user input that triggers the digital assistant to wake up. For example, when the user input is long pressing the power button of the smartphone, the initial display position of the geometric array may be on the screen near the power button.

[0025] Furthermore, when no user input is detected, the initial position is a default position, which is an area where the virtual image is displayed, such as the bottom of the screen.

[0026] In this solution, the initial display position of the geometric array is determined by user input, which can make the display process of the digital assistant more closely aligned with the user input, enhance the intuitiveness of the interaction between the user and the digital assistant, and improve the user experience.

[0027] In one possible implementation, the first interface is the lock screen interface or any screen interface after unlocking, the first form is the wake-up state, and the second form is the listening state; the signal instruction is generated based on the user performing a preset gesture on the digital assistant in the first form.

[0028] In a possible implementation, the plurality of feature bodies further include a second feature body; in the second form, the first feature body is in the main form of a ring, the second feature body is a sphere, and the ring is located on the equator of the sphere in a top-down perspective.

[0029] In one possible implementation, the virtual image is displayed in a first form on the first interface, specifically: the virtual image is displayed at a preset position on the first interface; or the virtual image is displayed by overlaying a target content element in the first interface, wherein the target content element is a content element of a preset type.

[0030] That is, the virtual image is covered on the target content element, so that the target content element is no longer displayed on the first interface.

[0031] In one possible implementation, when the virtual image is displayed over the target content element in the first interface, the gesture event interface of the target content element is copied to the virtual image so that performing a gesture on the virtual image can realize the gesture corresponding function of the target content element.

[0032] That is to say, even if the target content element is covered by the virtual image, the virtual image inherits the original function of the target content element, so that when a specific gesture (such as single-click) is performed on the virtual image, the gesture-corresponding function of the target content element can still be executed.

[0033] In a possible implementation, the first interface is a camera interface, and the target content element is a shutter control in the camera interface.

[0034] In one possible implementation, the virtual image also has a task execution state, which includes a first feature body; in at least one process interface during the digital assistant's execution of an automated task, the first feature body in the task execution state corresponds to at least one of a shape change and a position change.

[0035] Specifically, in the process interface of the digital assistant performing automated tasks, the digital assistant is actually always in the task execution state, and the first feature body of the digital assistant in the task execution state can undergo various changes to show the user the automated process of the digital assistant performing tasks, making the automated process of the digital assistant performing tasks more intuitive and clear, making it easier for users to observe whether there are any errors in the process of the digital assistant performing automated tasks.

[0036] In a possible implementation, the first feature body is a geometric array, and the display position of the geometric array matches the target control in the process interface, and the next process interface is entered by simulating the click event of the target control. In other words, in the process of executing the automation task, the digital assistant can use the first feature body as the geometric array to simulate the user's operation of clicking the control on the interface, thereby visualizing the entire process of the digital assistant simulating the user to execute the automation task.

[0037] In one possible implementation, the first form is a listening state, and the second form is a task execution state. Both the listening state and the task execution state include a first feature body, and the first feature body in the listening state and the task execution state has different forms respectively; in the task execution state, the forms of the first feature body include: a geometric array within the interface, an outer frame of a digital assistant dialogue card, an outer frame of an input box when calling a digital assistant input, or an outer frame of a task result page.

[0038] That is, in different scenarios, the first feature body may be represented in different forms so as to be more in line with the content displayed on the current interface.

[0039] In a possible implementation, the first feature body has different morphological changes at different task progress of the automation task. That is, in the process of the digital assistant performing the automation task, at different task progress, the first feature body may change into different forms, so as to ensure that the form of the first feature body always closely fits the current interface.

[0040] In a possible implementation, when the automation task is an intelligent recognition task, the first feature body is changed from a geometric array covering the entire screen to a geometric array displayed only on the recognized executable object.

[0041] In one possible implementation, corresponding to the execution process of the automated task, in the task execution state, the morphological changes of the first feature body include: initially, the first feature body is displayed in the main form of a ring at an initial position relative to the second feature body (for example, the periphery of the second feature body), and the initial position is the same as or different from the position of the first feature body in the listening state; the first feature body is separated from the second feature body by the ring, and is enlarged and displayed as the outer frame of the digital assistant dialogue card; the first feature body changes from the outer frame of the dialogue card to a geometric array in the dialogue bubble of the digital assistant; after the automated task is executed, the virtual image is displayed in the listening state.

[0042] In a possible implementation, the method may further include: the electronic device acquires scene information, determines a target form of the first feature body based on the scene information, and displays the first feature body in the target form. That is, the target form is related to the scene information. When the electronic device acquires specific scene information, the electronic device may display the first feature body in a specific form.

[0043] In one possible implementation, the virtual image has a state to be awakened, which includes at least a first feature body, and the first feature body is displayed in a long strip; in response to detecting that the focus of human eye sight falls on the area where the screen is located, the electronic device displays a second feature body.

[0044] That is to say, in some scenarios, when the digital assistant is in the wake-up state, only a long bar is displayed; when the digital assistant detects that the user's eyes fall on the screen, it can be considered that the user has an interactive tendency, so the second feature body is displayed, thereby indicating that the digital assistant detects that the user has an interactive tendency and is waiting for interaction, so as to improve the naturalness of human-computer interaction.

[0045] In a possible implementation, the method further includes: in response to detecting a human voice, the second feature body moves relative to the first feature body in a direction close to a source of the human voice.

[0046] In a possible implementation, the electronic device is a smart car, and the method further includes: indicating a vehicle function or warning information by deformation or movement of a first feature body among the multiple feature bodies.

[0047] A second aspect of the present application provides a display device for a digital assistant virtual image, which is applied to an electronic device, wherein the virtual image includes a plurality of feature bodies. The device includes:

[0048] A display module, configured to display a virtual image in a first form on a first interface;

[0049] The display module is further used to detect the signal instruction generated by the electronic device and display the virtual image in the second form;

[0050] Among them, the signal instruction is generated based on the first user input, the first form and the second form display at least one of the multiple feature bodies, and the change from the first form to the second form means that at least one feature body undergoes at least one of a shape change, a position change, a motion law change, and a display area change.

[0051] In a possible implementation, the plurality of feature bodies include a first feature body, and the shape change of the first feature body includes at least one of the following changes: enlargement, reduction, stretching, compression, rotation, and switching between a main form, a geometric array, or a strip.

[0052] In a possible implementation, the plurality of feature bodies further include a second feature body, wherein the main shape of the first feature body is a ring, the second feature body is a sphere, and the center of the ring and the center of the sphere are co-pointed;

[0053] The position change of the first feature body includes at least one of a change in orbital position after flipping around the sphere and a change in position caused by overall movement or rotation of the virtual image;

[0054] The motion law of the first characteristic body includes that the ring rotates around the sphere according to a preset trajectory with the center of the circle as the fulcrum;

[0055] The motion law of the second characteristic body includes that the sphere moves back and forth according to a preset law along a straight line passing through the center of the ring and perpendicular to the plane where the ring is located.

[0056] In a possible implementation, the display module is further configured to detect a second user input and display a third feature body on the first feature body, where the third feature body can move on the first feature body and indicate the user's position.

[0057] In a possible implementation, the virtual image has a breathing state, and the virtual image is displayed in a first form on the first interface, including:

[0058] If the first interface is a screen-off interface, the first form is a breathing form including a first feature body and a second feature body, and the first feature body is displayed in a main form.

[0059] In a possible implementation, when the first interface is a screen-off interface, the virtual image in a breathing state reciprocates on the screen according to a preset rule.

[0060] In a possible implementation, when the first interface is a screen-off interface and the first state is a breathing state, the display module is further used to:

[0061] A second user input is detected, and a third feature is displayed on the first feature, the third feature being movable on the first feature and indicating the user's position.

[0062] In a possible implementation, the virtual image has a waiting state and a listening state, and the first interface is a lock screen interface or any screen interface after unlocking;

[0063] Displaying a virtual image in a first form on a first interface includes:

[0064] The first state to be awakened is displayed on the first interface as the first form, the state to be awakened includes a first feature body, and the first feature body is displayed in a long strip as a navigation prompt line, and the navigation prompt line is used as a reference line for screen gesture navigation; wherein, when no input for waking up the digital assistant is received, the state to be awakened is permanently displayed on the first interface;

[0065] or,

[0066] A third user input is detected and displayed on the first interface in a listening state as a first form. The third user input is used to wake up the digital assistant. The listening state includes a first feature body and a second feature body, and the first feature body is displayed in a main form.

[0067] In a possible implementation, the first interface is a lock screen interface or any screen interface after unlocking, and the first state is a listening state;

[0068] Displaying the virtual image in the first form on the first interface specifically includes:

[0069] The first feature is first displayed as a geometric array;

[0070] The first feature body is then transformed from a geometric array into a main form, and a second feature body is displayed.

[0071] In a possible implementation, the first feature body is first displayed in a geometric array, specifically:

[0072] Displays the geometry of a portion of the array at the initial display position, and then displays the geometry on the entire screen.

[0073] In one possible implementation, the initial display position is determined based on user input;

[0074] And when no user input is detected, the initial position is the area where the virtual image is displayed.

[0075] In a possible implementation, the virtual image is displayed in the first form on the first interface, specifically:

[0076] Displaying a virtual image at a preset position on the first interface; or,

[0077] The virtual image is overlaid and displayed on a target content element in the first interface, wherein the target content element is a content element of a preset type.

[0078] In one possible implementation, when the virtual image is displayed over the target content element in the first interface, the gesture event interface of the target content element is copied to the virtual image so that performing a gesture on the virtual image can realize the gesture corresponding function of the target content element.

[0079] In a possible implementation, the first interface is a camera interface, and the target content element is a shutter control in the camera interface.

[0080] In one possible implementation, the first interface is the lock screen interface or any screen interface after unlocking, the first form is the wake-up state, and the second form is the listening state; the signal instruction is generated based on the user performing a preset gesture on the digital assistant in the first form.

[0081] In a possible implementation, the plurality of feature bodies further includes a second feature body;

[0082] In the second form, the first feature body is in the main form of a ring, the second feature body is a sphere, and the ring is located on the equator of the sphere in a top-down perspective.

[0083] In a possible implementation, the virtual image further has a task execution state, and the task execution state includes a first feature body;

[0084] In at least one process interface during which the digital assistant performs an automated task, the first feature body in the task execution state corresponds to at least one of a shape change and a position change.

[0085] In a possible implementation, the first feature body is a geometric array, and the display position of the geometric array matches the target control in the process interface, so as to simulate a click event of the target control to enter the next process interface.

[0086] In a possible implementation, the first state is a listening state, and the second state is a task execution state. Both the listening state and the task execution state include a first feature body, and the first feature bodies in the listening state and the task execution state have different forms respectively.

[0087] In the task execution state, the shape of the first feature body includes: a geometric array within the interface, an outer frame of a dialog card, an outer frame of an input box when calling a digital assistant input, or an outer frame of a task result page.

[0088] In a possible implementation, the first feature body has different morphological changes at different task progresses of the automation task.

[0089] In a possible implementation, when the automation task is an intelligent recognition task, the first feature body is changed from a geometric array covering the entire screen to a geometric array displayed only on the recognized executable object.

[0090] In a possible implementation, in the task execution state, the morphological change of the first feature body includes:

[0091] The first feature body is first displayed in the main form of a ring on the periphery of the second feature body;

[0092] The first feature body is then separated from the second feature body by the ring and is enlarged and displayed to serve as the outer frame of the conversation card;

[0093] The first feature body changes from the outer frame of the dialogue card to the geometric array inside the dialogue bubble;

[0094] After the automated task is completed, the avatar is displayed in a listening state.

[0095] In a possible implementation, the device further includes:

[0096] The acquisition module is used to acquire scene information and display a first feature body among multiple feature bodies in a target form based on the scene information.

[0097] In a possible implementation, the virtual image has a waiting-to-be-awakened state, the waiting-to-be-awakened state includes at least a first feature body, and the first feature body is displayed in a long strip;

[0098] The display module is also used to display a second feature body in response to detecting that the focus of the human eye falls on the area where the screen is located.

[0099] In a possible implementation, the display module is further configured to, in response to detecting a human voice, cause the second feature body to move relative to the first feature body toward a direction close to a source of the human voice.

[0100] In a possible implementation, the electronic device is a smart car, and the display module is further used for:

[0101] The vehicle function or warning information is indicated by the deformation or movement of the first feature body among the multiple feature bodies.

[0102] The third aspect of the present application provides a display device for a digital assistant virtual image, which may include a processor, the processor and a memory are coupled, the memory stores program instructions, and when the program instructions stored in the memory are executed by the processor, the method of the first aspect or any implementation of the first aspect is implemented. For the steps in each possible implementation of the first aspect executed by the processor, please refer to the first aspect for details, and no further description is given here.

[0103] The fourth aspect of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer-readable storage medium is run on a computer, the computer executes the method of any implementation manner of the first aspect.

[0104] A fifth aspect of the present application provides a circuit system, the circuit system includes a processing circuit, and the processing circuit is configured to execute a method in any implementation manner of the above-mentioned first aspect.

[0105] The sixth aspect of the present application provides a computer program product, which, when executed on a computer, enables the computer to execute a method implemented in any one of the first aspects.

[0106] The seventh aspect of the present application provides a chip system, which includes a processor for supporting a server or a threshold value acquisition device to implement the functions involved in any implementation of the first aspect, for example, sending or processing the data and / or information involved in the above method. In one possible design, the chip system also includes a memory, which is used to store program instructions and data necessary for the server or communication device. The chip system can be composed of chips, or it can include chips and other discrete devices.

[0107] The beneficial effects of the second to seventh aspects mentioned above can be referred to the introduction of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0108] Figure 1 A schematic diagram of an application scenario provided in an embodiment of the present application;

[0109] Figure 2 A schematic diagram of the structure of an electronic device 101 provided in an embodiment of the present application;

[0110] Figure 3 A schematic diagram of a flow chart of a method for displaying a digital assistant virtual image provided in an embodiment of the present application;

[0111] Figure 4 A schematic diagram of a main image of a digital assistant provided in an embodiment of the present application;

[0112] Figure 5 A schematic diagram of the movement law of a digital assistant in a breathing state provided in an embodiment of the present application;

[0113] Figure 6 A schematic diagram of a digital assistant switching from a breathing state to a waiting-to-be-awakened state provided in an embodiment of the present application;

[0114] Figure 7 A schematic diagram of a digital assistant provided in an embodiment of the present application in a wake-up state;

[0115] Figure 8 A schematic diagram of a digital assistant provided in an embodiment of the present application in a listening state;

[0116] Fig. 9 A schematic diagram of a digital assistant provided in an embodiment of the present application in a thinking state;

[0117] Fig.10 A schematic diagram of a digital assistant provided in an embodiment of the present application in a task execution state;

[0118] Fig.11 A schematic diagram of various forms of a digital assistant provided in an embodiment of the present application;

[0119] Fig.12 A schematic diagram of a digital assistant in a waiting-to-be-awakened state provided in an embodiment of the present application;

[0120] Fig.13 A schematic diagram of a digital assistant switching from a waiting state to a listening state provided in an embodiment of the present application;

[0121] Fig.14 A schematic diagram of a virtual image overlaid on a camera shutter control provided in an embodiment of the present application;

[0122] Fig.15 A schematic diagram of a virtual image overlaid on an assistant suggestion component provided in an embodiment of the present application;

[0123] Fig.16 A schematic diagram of a process for a digital assistant to perform an automated task provided in an embodiment of the present application;

[0124] Fig.17 A schematic diagram of a digital assistant provided in an embodiment of the present application, in which a circular ring is converted into a dot matrix and laid out on a task execution interface;

[0125] Fig.18 A schematic diagram of using a digital assistant to input text on a desktop provided in an embodiment of the present application;

[0126] Fig.19 A schematic diagram of a ring of a digital assistant provided in an embodiment of the present application deforming and returning to its original shape;

[0127] Fig. 20 A schematic diagram of the morphological changes of a digital assistant in a smart recognition scenario provided by an embodiment of the present application;

[0128] Fig.21 A schematic diagram of a digital assistant appearing in a smart recognition scenario provided by an embodiment of the present application;

[0129] Fig. 22 A schematic diagram of a digital assistant that generates deformation based on scene information provided in an embodiment of the present application;

[0130] Fig.23 A schematic diagram of changes in a digital assistant on a vehicle-mounted terminal provided in an embodiment of the present application;

[0131] Fig.24 A schematic diagram of a flow chart of a display method of a digital assistant provided in an embodiment of the present application;

[0132] Fig.25 A schematic diagram of an application scenario of the display method of a digital assistant provided in an embodiment of the present application;

[0133] Fig.26 A schematic diagram of a digital assistant in a dormant state provided in an embodiment of the present application;

[0134] Fig. 27 A schematic diagram of a region division provided in an embodiment of the present application;

[0135] Fig.28 A schematic diagram of switching the display mode of a digital assistant provided in an embodiment of the present application;

[0136] Fig.29 A schematic diagram of changes in a digital assistant in a wake-up state provided in an embodiment of the present application;

[0137] Fig.30 A schematic diagram of another digital assistant in a wake-up state provided in an embodiment of the present application;

[0138] Fig.31 A schematic diagram of the structure of a display device for a digital assistant virtual image provided in an embodiment of the present application;

[0139] Fig.32 A schematic diagram of the structure of an execution device provided in an embodiment of the present application;

[0140] Fig.33 A schematic diagram of the structure of a computer-readable storage medium provided in an embodiment of the present application. DETAILED DESCRIPTION

[0141] In order to make the purpose, technical solutions and advantages of the present application clearer, the embodiments of the present application are described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only embodiments of a part of the present application, rather than all embodiments. It is known to those of ordinary skill in the art that with the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0142] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the descriptions used in this way can be interchanged where appropriate, so that the embodiments can be implemented in a sequence other than that illustrated or described in the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or modules is not necessarily limited to those steps or modules that are clearly listed, but may include other steps or modules that are not clearly listed or inherent to these processes, methods, products or devices. The naming or numbering of the steps that appear in the present application does not mean that the steps in the method flow must be executed in the time / logical sequence indicated by the naming or numbering. The process steps that have been named or numbered can change the execution order according to the technical purpose to be achieved, as long as the same or similar technical effects can be achieved.

[0143] The division of units in this application is a logical division. There may be other division methods when it is implemented in actual applications. For example, multiple units can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection between units can be electrical or other similar forms, which are not limited in this application. In addition, the units or sub-units described as separate components may or may not be physically separated, may or may not be physical units, or may be distributed in multiple circuit units, and some or all of the units may be selected according to actual needs to achieve the purpose of the present application.

[0144] To facilitate understanding, some technical terms involved in the embodiments of the present application are first introduced below.

[0145] (1) Digital Assistant

[0146] A digital assistant is a virtual assistant that receives user commands to replace manual operations and helps users complete a series of operations on smart devices. Generally speaking, most digital assistants are voice assistants that complete the operation of smart devices by receiving user voice commands. For example, when the smart device is a smartphone or smart TV, the digital assistant can receive user voice commands to open an application or play a song.

[0147] (2) Distance sensor

[0148] A distance sensor, also called a displacement sensor, is a type of sensor that is used to sense the distance between it and an object to complete a preset function and has been widely used.

[0149] Distance sensors can be divided into optical distance sensors, infrared distance sensors, ultrasonic distance sensors and other types according to their working principles. Most distance sensors used on mobile phones are infrared distance sensors, which have an infrared transmitting tube and an infrared receiving tube. When the infrared rays emitted by the transmitting tube are received by the receiving tube, it indicates that the distance is close and the screen needs to be turned off to avoid misoperation. When the receiving tube cannot receive the infrared rays emitted by the transmitting tube, it indicates that the distance is far and there is no need to turn off the screen. The working principles of other types of distance sensors are similar. They also judge the distance through the emission and reception of a certain substance. The emitted substance can be ultrasonic waves, light pulses, etc.

[0150] See also Figure 1 , Figure 1 The following is a schematic diagram of an application scenario provided by an embodiment of the present application. Figure 1 As shown, the application scenario includes a user and an electronic device with a digital assistant. The user can issue instructions to the digital assistant in the electronic device through voice or gestures, so that the digital assistant assists the user to automatically complete a series of operations of the electronic device, such as automatically executing an application in the electronic device.

[0151] The electronic device is equipped with a digital assistant, and the electronic device may be equipped with a command receiving device such as a microphone or a camera, which can receive the user's voice commands or gesture commands, and then automatically complete a series of operations by processing the user's input commands through the digital assistant, and feedback the processing results to the user. The electronic device also has a display screen, which can display the form of the digital assistant on the display screen. When the electronic device obtains the user input, the electronic device generates a signal instruction to trigger the change of the digital assistant's virtual image, so that the process of interaction between the digital assistant and the user is more intuitive. For example, when the electronic device is at a certain distance from the user, the electronic device recognizes the user's spatial interaction behavior by sensing the user's position change or posture change, and displays the morphological changes of the digital assistant in response to the user's spatial interaction behavior on the display screen.

[0152] The above describes that the electronic device adjusts the form of the digital assistant by obtaining the spatial interaction behavior of the user in the real physical world. In addition, the electronic device can also be a virtual reality (VR) device or an augmented reality (AR) device, and the electronic device adjusts the form of the digital assistant by obtaining the spatial interaction behavior of the user in a virtual world such as the VR world or the AR world. Among them, in the virtual world, the user who interacts with the electronic device can be a virtual image representing the real user in the virtual world. The real user performs spatial interaction by controlling the virtual image, thereby changing the form of the digital assistant in the virtual world.

[0153] Exemplarily, electronic devices with digital assistants may be, for example, smartphones, personal computers (PCs), laptops, tablet computers, smart TVs, mobile internet devices (MIDs), display devices in autonomous vehicles, smart speakers, wearable devices, VR devices, AR devices, display devices in industrial control, display devices in remote medical surgery, display devices in smart grids, wireless communication devices in smart cities, display devices in smart homes, etc.

[0154] See also Figure 2 , Figure 2 Schematic diagram of the structure of an electronic device 101 provided in an embodiment of the present application. Figure 2 As shown, the electronic device 101 includes a processor 103, which is coupled to a system bus 105. The processor 103 may be one or more processors, each of which may include one or more processor cores. A display adapter (video adapter) 107, which may drive a display 109, which is coupled to the system bus 105. The system bus 105 is coupled to an input / output (I / O) bus via a bus bridge 111. An I / O interface 115 is coupled to the I / O bus. The I / O interface 115 communicates with a variety of I / O devices, such as an input device 117 (such as a touch screen, etc.), an external memory 121 (for example, a hard disk, a floppy disk, an optical disk or a USB flash drive), a multimedia interface, etc.). A transceiver 123 (which may send and / or receive radio communication signals), a camera 155 (which may capture static and dynamic digital video images), and an external USB port 125. Optionally, the interface connected to the I / O interface 115 may be a USB interface.

[0155] The processor 103 may be any conventional processor, including a reduced instruction set computing (RISC) processor, a complex instruction set computing (CISC) processor, or a combination thereof. Alternatively, the processor may be a dedicated device such as an ASIC.

[0156] The electronic device 101 can communicate with the software deployment server 149 through the network interface 129. Exemplarily, the network interface 129 is a hardware network interface, such as a network card. The network 127 can be an external network, such as the Internet, or an internal network, such as Ethernet or a virtual private network (VPN). Optionally, the network 127 can also be a wireless network, such as a WiFi network, a cellular network, etc.

[0157] The hard disk drive interface 131 is coupled to the system bus 105. The hard disk drive interface is connected to the hard disk drive 133. The internal memory 135 is coupled to the system bus 105. The data running in the internal memory 135 may include an operating system (OS) 137 of the electronic device 101, an application 143, and a scheduler.

[0158] The operating system includes a shell 139 and a kernel 141. Shell 139 is an interface between the user and the kernel of the operating system. The shell is the outermost layer of the operating system. The shell manages the interaction between the user and the operating system: waiting for user input, interpreting user input to the operating system, and processing various operating system output results.

[0159] The kernel 141 consists of those parts of the operating system that manage memory, files, peripherals, and system resources. The kernel 141 interacts directly with the hardware. The operating system kernel usually runs processes and provides communication between processes, provides CPU time slice management, interrupts, memory management, IO management, etc.

[0160] See also Figure 3 , Figure 3 A flowchart of a method for displaying a digital assistant virtual image provided in an embodiment of the present application is shown below. Figure 3 As shown, the method for displaying the virtual image of a digital assistant includes the following steps 301-302, and the method for displaying the virtual image of a digital assistant is applied to an electronic device equipped with a digital assistant.

[0161] Step 301, displaying the virtual image of the digital assistant in a first form on a first interface.

[0162] In this embodiment, the virtual image of the digital assistant may include multiple features. When the virtual image is displayed in different forms, the multiple features included in the virtual image will change. Among them, the first form is one of the multiple forms of the virtual image of the digital assistant, which is displayed on the first interface of the electronic device.

[0163] Step 302, based on the signal instruction generated by the electronic device, display the virtual image in a second form.

[0164] The signal instruction is generated by the electronic device based on the received first user input. After the electronic device receives the first user input, the signal instruction is generated, thereby triggering the virtual image to switch from the first form to the second form, so that the form of the virtual image displayed on the interface of the electronic device changes.

[0165] The virtual image in the first form and the virtual image in the second form each include at least one of the plurality of features. The virtual image changes from the first form to the second form in particular when at least one of the features undergoes at least one of a change in shape, position, motion pattern, or display area.

[0166] It should be noted that the first user input received by the electronic device may be before the electronic device displays the first form of the virtual image, or after the first form of the virtual image is displayed. If the first user input received may be before the electronic device displays the first form of the virtual image, it means that the first user input will prompt the electronic device to display the first form of the virtual image, and change from the first form to the second form, that is, cause at least two form changes. If the first user input received may be after the electronic device displays the first form of the virtual image, it means that the first user input will prompt the digital assistant's virtual image to change from the first form to the first form, that is, cause at least one form change.

[0167] In this embodiment, different forms of the virtual image represent different working states of the digital assistant, that is, the digital assistant uses the form of the virtual image to indicate the current working state of the digital assistant. In this way, when the form of the virtual image displayed on the interface changes, the user can intuitively and clearly understand the change in the working state of the digital assistant, and thus understand the current response of the digital assistant to the user's instructions.

[0168] Optionally, the plurality of feature bodies include a first feature body, and the shape of the first feature body includes: a main form, a geometric array, and a long strip. Furthermore, the shape change of the first feature body includes switching between multiple shapes, and enlarging, reducing, stretching, compressing, or rotating each shape. That is, in addition to switching between different shapes, the shape change of the first feature body may also include enlarging, reducing, stretching, compressing, or rotating the same shape. Among them, the main form refers to the form of the first feature body under normal circumstances, while the geometric array and the long strip are the forms of the first feature body under certain specific circumstances.

[0169] Optionally, the plurality of feature bodies further include a second feature body. Among the plurality of feature bodies, the main form of the first feature body is a ring, and the second feature body is a sphere, and the center of the ring coincides with the center of the sphere. The position change of the first feature body includes at least one of the orbital position change after flipping around the sphere and the position change caused by the overall movement or rotation of the virtual image; the motion law of the first feature body includes the ring rotating around the sphere with the center of the ring as the fulcrum according to a preset trajectory; the motion law of the second feature body includes the sphere reciprocating along a straight line passing through the center of the ring and perpendicular to the plane where the ring is located according to a preset law.

[0170] Exemplarily, corresponding to different processing stages of the task, the digital assistant can have the following six different forms.

[0171] Breathing state: corresponds to the state of the digital assistant when there is no task processing, that is, it is used to indicate that the digital assistant is in a state of no task processing. Among them, the breathing state is the state of the digital assistant in the screen-off interface.

[0172] Waiting state: corresponds to the state of the digital assistant waiting to be awakened (essentially, the state of the digital assistant when there is no task to be processed), that is, it is used to indicate that the digital assistant is in a state of waiting for a wake-up command. Among them, the waiting state is the state of the digital assistant in the bright screen interface, where the bright screen interface includes the lock screen interface or the interface after the screen is unlocked.

[0173] Wake-up state: corresponds to the state when the digital assistant is awakened and ready to receive commands, that is, it is used to indicate that the digital assistant is in the state of waiting to receive commands.

[0174] Listening state: corresponds to the state of the digital assistant during the process of receiving commands, that is, it is used to indicate that the digital assistant is in the command receiving state.

[0175] Thinking state: The state of the digital assistant when receiving and parsing the command, which is used to indicate that the digital assistant is in the command parsing state.

[0176] Task execution state: The state of the task obtained when the digital assistant executes the parsed instruction, which is used to indicate that the digital assistant is in the task execution state.

[0177] From the perspective of the degree of interaction intention, the above-mentioned multiple forms are arranged from low to high according to the degree of interaction intention: breathing state, waiting to be awakened state, awakened state, listening state, thinking state, and task execution state. In addition, in the off-screen interface, the breathing state can be transformed into the awakened state. In the bright-screen interface, the awakened state can be transformed into the listening state instead of the awakened state. When applied to different electronic devices, the forms of digital assistants may also be different. For example, when applied to smartphones, the forms of digital assistants include the above-mentioned breathing state, waiting to be awakened state, listening state, thinking state, and task execution state, and the forms can be switched back and forth in sequence; when applied to smart TVs, the forms of digital assistants include the above-mentioned breathing state, awakened state, listening state, thinking state, and task execution state, and the forms can be switched back and forth in sequence.

[0178] The above introduces the different forms of digital assistants. The following will introduce the main image of digital assistants.

[0179] The main image of the digital assistant includes at least two feature bodies, and the different forms of the digital assistant are distinguished by the changes in the shape, position, and motion law of the feature bodies. Specifically for the main image of a digital assistant in a certain form, each feature body can be dynamically presented to represent the concept of a living body of the digital assistant's activities. Different forms of digital assistants are distinguished by changes in the shape, position, and / or motion law of different feature bodies. Among them, the change in the shape of the feature body may mean that the feature body keeps its current position unchanged, while adjusting the part of the feature body presented on the display screen (for example, adjusting the front of the feature body to the back of the feature body), switching between a plurality of pre-set different shapes (including enlarging, reducing, stretching, compressing or rotating each shape), and adjusting the position or action of certain parts on the feature body (for example, adjusting the arm on the feature body from static to waving). The change in the position of the feature body may mean that the position where the feature body is displayed changes, for example, from being displayed at the bottom of the screen to being displayed at the top of the screen or in the center of the screen. The change in the motion law of the feature body may refer to a change in the motion of the entire feature body on the display screen, for example, the feature body changes from being stationary to moving along a preset trajectory, or from moving along a sinusoidal trajectory to rotating around a certain point.

[0180] See also Figure 4 , Figure 4 A schematic diagram of a main body image of a digital assistant provided in an embodiment of the present application. Figure 4As shown, the main image of the digital assistant includes three characteristic bodies: a ring, a sphere, and an image body. Among them, the image body is wrapped in the sphere; the ring surrounds the sphere, similar to the ring of a planet. Digital assistants in different forms, the ring, sphere, and image body can have different postures and different motion laws. Moreover, between digital assistants in different forms, the posture and / or motion law of at least one characteristic body has changed.

[0181] In addition, to facilitate the description of the motion state changes of each feature body, a reference system is defined for the main image of the digital assistant: in the breathing state, the plane where the ring is located is the xoy plane, and the x-axis points to the screen of the electronic device, and the vertical direction is the z-axis. In simple terms, the width direction of the electronic device is the y-axis, and the height direction of the electronic device is the z-axis.

[0182] The following will introduce in detail the posture and movement rules of the characteristic bodies in each form of the digital assistant with the help of the attached drawings.

[0183] 1. Breathing state

[0184] When the digital assistant is in the breathing state, the ring, sphere and image body all move regularly along the preset trajectory and according to the preset period. During the movement, the relative positions between the ring, sphere and image body remain unchanged, that is, the center points of the ring, sphere and image body always coincide. In addition, the image body and the sphere can rotate around an axis or a point, for example, around the center of the sphere. That is, the image body and the sphere rotate while moving along the preset trajectory (similar to the earth rotating around the sun while rotating itself).

[0185] Exemplarily, when the avatar has a breathing state, the avatar is displayed in the first form on the first interface, which may specifically include: if the first interface is a screen-off interface, the first form is a breathing state. The breathing state includes a first feature body and a second feature body, and the first feature body is displayed in the main form.

[0186] Optionally, when the first interface is a screen-off interface, the digital assistant in a breathing state makes reciprocating motion on the screen according to a preset pattern.

[0187] See also Figure 5 , Figure 5 A schematic diagram of the motion law of a digital assistant in a breathing state provided in an embodiment of the present application. Figure 5 As shown, Figure 5The breathing state image of the digital assistant at time t is shown in FIG. The ring moves back and forth along the z-axis direction, and the function of the moving distance and time conforms to a sine curve. During the movement of the ring, the sphere and the image body rotate around the center of the sphere. In addition, in order to reflect the rotation effect of the sphere, corresponding light and shadow effects can be added to the sphere to indicate the rotation of the sphere.

[0188] 2. Waiting for wake-up

[0189] When the digital assistant is in the wake-up state, the circular ring of the digital assistant (i.e., the first characteristic body mentioned above) closes inward and is displayed in the form of a gesture prompt line, which can specifically be a long strip.

[0190] The gesture prompt line is designed to match the full-screen gestures. The gesture prompt line is a global control that can be displayed at the bottom of the screen to provide a reference for full-screen gestures. For example, swiping up from the bottom usually starts from the position of the gesture prompt line.

[0191] Specifically, the digital assistant only displays the first feature body, and the shape of the first feature body is a long strip. Other feature bodies of the digital assistant (such as the sphere and image body of the digital assistant) are hidden. Generally, gesture prompt lines are usually displayed at the bottom of the screen and have specific functions. Correspondingly, the digital assistant in the wake-up state also has the function of gesture prompt lines, such as swiping left and right to switch applications.

[0192] See also Figure 6 , Figure 6 A schematic diagram of a digital assistant switching from a breathing state to a waiting-to-be-awakened state provided in an embodiment of the present application. Figure 6 As shown in (a), in the off-screen interface, when the digital assistant is in the breathing state, the virtual image of the digital assistant includes a ring, a sphere, and the image itself. Figure 6 As shown in (b), when the user triggers the electronic device to switch from the off-screen interface to the on-screen interface, the digital assistant changes from the breathing state to the waiting-to-wake-up state. In the waiting-to-wake-up state, the ring of the digital assistant changes to a long strip at the bottom of the screen, and the sphere and image of the digital assistant are hidden.

[0193] 3. Awakening state

[0194] When the digital assistant is in the awake state, the ring is fixed on the equatorial plane of the sphere and rotates around the z-axis as the user moves. In addition, in order to indicate the rotation relationship of the ring, part of the area on the ring is set as a light effect area. The brightness of the light effect area on the ring is higher than the non-light effect area on the ring. In addition, the light effect area on the ring points to the user, and when the ring rotates, the light effect area moves and points to the user. Alternatively, when the user moves, the ring may also rotate with the user, so that the light effect area always points to the user's location. Alternatively, when the user approaches or moves away from the electronic device, the ring may change its pitch angle (i.e., rotate around the plane where the display screen of the electronic device is located), so that the ring can simulate the "head up" or "head down" action when seeing the user approaching.

[0195] For example, when the first interface is a screen-off interface and the first form displayed on the first interface is a breathing state, if a second user input is detected, a third feature body is displayed on the first feature body, and the third feature body is used to indicate the user's orientation information and can move on the first feature body. The first feature body is the above-mentioned ring, and the third feature body can be a light effect area on the ring.

[0196] See also Figure 7 , Figure 7 A schematic diagram of a digital assistant in an awake state provided in an embodiment of the present application. Figure 7 As shown, part of the area on the ring close to the screen side is the light effect area, which indicates the user's position relative to the electronic device. When the user's position changes, the ring rotates around the z-axis so that the light effect area always faces the user. In addition, when the distance between the user and the electronic device is less than a preset value, the user continues to approach the TV, and in addition to rotating around the z-axis, the ring will also produce a change in pitch angle around the y-axis. For example, after the ring rotates upward around the y-axis for a certain angle, it gradually stops due to inertia, so that the ring can simulate the "head-up" action when seeing the user approaching.

[0197] 4. Listening attitude

[0198] When the digital assistant is in the listening state, the ring rotates around the y-axis on the xoy plane. After the rotation, the ring is located on the yoz plane and no longer blocks the sphere and the image. In addition, when the ring flips, it can drive the sphere and the image to flip together, so that the bottom or top of the sphere and the image is facing the screen.

[0199] See also Figure 8 , Figure 8 A schematic diagram of a digital assistant in a listening state provided in an embodiment of the present application. Figure 8 As shown, Figure 8The middle ring rotates to the yoz plane, and the sphere and the image body also rotate.

[0200] 5. Thinking attitude

[0201] When the digital assistant is in the thinking state, the ring and the image body are always in a rotating state, and the fulcrum around which the ring and the image body rotate can be the center of the sphere. In addition, the trajectory of the rotation of the ring and the image body can be random or preset. In addition, in the thinking state, in order to indicate the rotation relationship of the ring, part of the area on the ring can also be set as a light effect area, and the light effect area moves with the movement of the ring, or the light effect area continuously flows and moves on the ring. Among them, the method of setting the light effect area on the ring can refer to the description of the above-mentioned awakening state, which will not be repeated here.

[0202] See also Fig. 9 , Fig. 9 A schematic diagram of a digital assistant in a thinking state provided in an embodiment of the present application. Fig. 9 The figure shows two states of the digital assistant during its rotation, and in these two states, the ring and the image are always rotating.

[0203] 6. Task execution state

[0204] When the digital assistant is in the task execution state, the ring of the digital assistant may be transformed into other shapes (such as a geometric array) and separated from the sphere and the image body, and displayed at other locations on the interface. Fig.10 , Fig.10 A schematic diagram of a digital assistant in a task execution state provided in an embodiment of the present application. Fig.10 As shown, the digital assistant's ring is transformed into a geometric array and is separated from the digital assistant's sphere and image body, and is displayed in other locations.

[0205] In general, see Fig.11 , Fig.11 A schematic diagram of various forms of a digital assistant provided in an embodiment of the present application. Fig.11 Demonstrates the multiple forms of digital assistants.

[0206] In combination with the various forms introduced above, the first form and the second form of the virtual image mentioned in this embodiment in various situations will be introduced below.

[0207] Optionally, when the virtual image has a waiting state and a listening state, the first interface showing the first form of the virtual image may be a lock screen interface or any screen interface after unlocking.

[0208] Then, displaying the virtual image in the first form on the first interface may specifically include: displaying the to-be-awakened state as the first form on the first interface. The to-be-awakened state includes a first feature body, and the first feature body is displayed in a long strip as a navigation prompt line. The navigation prompt line is used as a reference line for screen gesture navigation. In addition, when no input to wake up the digital assistant is received, the to-be-awakened state is permanently displayed on the display interface.

[0209] Specifically, after detecting the user input, the electronic device determines a response strategy based on the recognition of the user input and displays the response result on the screen. The response result displayed on the screen by the electronic device may be that the interface remains unchanged or that the interface has changed.

[0210] For example, when the electronic device is a smart phone, the user moves the probe close to the smart phone, and the face recognition hardware on the smart phone recognizes the face, then it can be considered that the smart phone detects the user input. Alternatively, the smart phone detects the approach of a human hand through an infrared sensor, and it can also be considered that the smart phone detects the user input. According to the user input detected by the smart phone, the screen is switched from the off-screen interface to the lock-screen interface (the lock-screen interface is essentially the bright-screen interface). Furthermore, after the face recognition verification on the lock-screen interface is passed, it is unlocked to enter the desktop or the running application interface. For another example, the user directly double-clicks the off-screen interface, and the electronic device switches to the lock-screen interface.

[0211] When the electronic device switches from the off-screen interface to the bright-screen interface and the digital assistant is not awakened, the digital assistant is displayed in the waiting-to-be-awakened state. In the waiting-to-be-awakened state, the ring of the digital assistant closes inward and is displayed in the form of a gesture prompt line, which can be specifically a long strip. The gesture prompt line is a global control and is displayed at the bottom of the screen in any interface. It is used to provide a reference for full-screen gestures. For example, swiping up from the bottom usually swipes up from the position of the gesture prompt line.

[0212] That is to say, from turning the screen off to turning it on, and then to the user actively waking up the digital assistant and interacting with the digital assistant, the digital assistant is always displayed in a location that the user can reach and in different forms.

[0213] In this embodiment, in the lock screen interface after the screen is turned off, and in the desktop or in-application interface when the user does not wake up the digital assistant through a preset operation, the digital assistant can be displayed in a waiting-to-be-awakened state. That is, the ring of the digital assistant closes to form a long strip as a system gesture prompt line (usually displayed at the bottom of the screen), and as the ring closes, the image body and the sphere in the ring disappear (which may be accompanied by a relevant disappearance animation).

[0214] For example, see Fig.12 , Fig.12A schematic diagram of a digital assistant in a waiting state provided in an embodiment of the present application. Fig.12 As shown in (a), in the off-screen interface, the digital assistant is displayed in a breathing state and can move in the off-screen interface according to a preset pattern. Fig.12 As shown in (b), when the user's input causes the electronic device to enter the lock screen interface from the off screen interface, the digital assistant moves to the bottom, the ring shrinks inward, and the image body and the sphere are squeezed to be completely hidden (this process is reflected in animation, and the animation can also be hidden). At this time, the digital assistant becomes a bar and is displayed at the bottom of the screen as a system gesture prompt line. Fig.12 As shown in (c) and (d), after the screen is unlocked, the user clicks the gallery application icon (or any other application icon) to enter the gallery interface. During the above series of operations, the user has not awakened the digital assistant, so the digital assistant is always displayed in the awakening state, which is displayed on the interface in the form of a gesture prompt line.

[0215] When the first state is the wake-up state, the digital assistant switches from the first state to the second state, which can be specifically switching from the wake-up state to the listening state.

[0216] Exemplarily, when the first interface is a lock screen interface or any screen interface after unlocking, the first form is a waiting state, and the second form displayed based on the signal instruction generated by the electronic device is a listening state. Among them, the signal instruction that triggers the display of the virtual image in the second form can be generated after the user performs a preset gesture on the digital assistant in the first form. For example, the user performs a long press gesture or a double-click gesture on the digital assistant in the first form. This embodiment does not specifically limit the preset gesture.

[0217] Optionally, the plurality of features include a first feature and a second feature. In the second form, the first feature is in the main form of a ring, the second feature is a sphere, and the ring is located on the equator of the sphere in a top-down perspective.

[0218] The above describes an embodiment in which the first state is the waiting state, and the following describes an embodiment in which the first state is the listening state.

[0219] Optionally, when a third user input is detected, the virtual image of the digital assistant is displayed on the first interface in a listening state as a first form, wherein the third user input is used to wake up the digital assistant. Exemplarily, the third user input may be, for example, a user's voice input, an input from an external input device (such as a remote control or a mouse), a touch input generated by a user's direct interaction with an electronic device screen, or an input from a user's interaction with a physical button, a back panel gesture, or other shortcut operations.

[0220] The listening state of the virtual image includes a first characteristic body (such as the aforementioned ring) and a second characteristic body (such as the aforementioned sphere), and the first characteristic body is displayed in the main state.

[0221] Specifically, on the screen-lit interface, tap the gesture prompt line to wake up the digital assistant, which puts the digital assistant into listening mode. Fig.13 , Fig.13 A schematic diagram of a digital assistant switching from a waiting state to a listening state provided in an embodiment of the present application. Fig.13 As shown in (a), the user interacts with the gesture prompt line where the digital assistant is located by single-clicking, double-clicking, or long-pressing to trigger the digital assistant to wake up. Fig.13 As shown in (b), after the digital assistant is awakened, the digital assistant is displayed in the current interface in a listening state.

[0222] In some scenarios, specific content may be displayed on the interface of the electronic device, and there may be less effective space on the interface to display the digital assistant. In this case, the virtual image of the digital assistant can be integrated with the specific content on the interface.

[0223] Exemplarily, the above-mentioned display of the virtual image in the first form on the first interface may specifically include: displaying the virtual image at a preset position on the first interface; or overlaying and displaying the virtual image on a target content element in the first interface, wherein the target content element is a content element of a preset type. That is, the virtual image is overlaid on the target content element, so that the target content element is no longer displayed on the first interface. In this way, by overlaying and displaying the virtual image on the target content element, it is possible to avoid adding an additional interface for displaying the virtual image to the original interface and blocking excessive content on the original interface (such as Fig.13 The floating window for displaying the virtual image of the digital assistant shown on the right blocks part of the content at the bottom of the screen), thereby improving the integration of the digital assistant with the interface.

[0224] The target content element is, for example, a control or icon of a preset type predefined by the system. For example, a control or icon of a preset type may refer to a space or icon having a circular shape.

[0225] Optionally, when the avatar is displayed over the target content element in the first interface, the gesture event interface of the target content element can be copied to the avatar, so that the gesture-corresponding function of the target content element can be realized by performing a gesture on the avatar. In other words, even if the target content element is covered by the avatar, the avatar inherits the original function of the target content element, so that when a specific gesture (such as single-click) is performed on the avatar, the gesture-corresponding function of the target content element can still be executed.

[0226] For example, when the first interface is a camera interface, the target content element is a shutter control in the camera interface. The virtual image is displayed overlaid on the shutter control of the camera interface, and clicking the virtual image can still realize the shooting of photos or videos.

[0227] That is to say, after the virtual image of the digital assistant is overlaid on the target content element, when the display area of ​​the virtual image of the digital assistant receives a gesture input for the original target content element, the gesture can be used to implement the function of the original control. Among them, the display area of ​​the virtual image implements the function of the original content element when receiving the gesture input, and the mapping relationship between the gesture and function of the target content element can be associated with the digital assistant, so that the digital assistant triggers the corresponding function after receiving the gesture input. Taking the target content element as an example, when the control is interacted, the coordinates corresponding to the display area are touched as the trigger of the control, and the corresponding instructions are generated. Therefore, when the digital assistant integrates the original function of the control, it only needs to synchronize the trigger interface of the touch instruction to the digital assistant, so that when the area where the digital assistant is located is touched, the corresponding touch instruction is generated to realize the function of the original control.

[0228] Alternatively, as long as the display area corresponding to the control is touched by a preset gesture, the function corresponding to the control is executed according to the gesture type, regardless of whether there are other layers displayed above the control. In addition, to implement the digital assistant function itself, it is only necessary to recognize the gesture corresponding to the digital assistant and then trigger the function corresponding to the gesture.

[0229] See also Fig.14 , Fig.14 A schematic diagram of a virtual image overlaid on a camera shutter control provided in an embodiment of the present application. Fig.14 As shown in , in one of the initial states, the digital assistant is not awakened and is displayed on the interface in a state to be awakened as a navigation prompt line. When the user interacts with the digital assistant in the state to be awakened (i.e., the navigation prompt line), such as long pressing the navigation prompt line or long pressing the power button to wake up the digital assistant, the digital assistant is in a listening state. At this time, when the system displays the digital assistant in the listening state, it first checks the type of control in the current interface. Since the camera shutter control in the current interface is a preset type of control, the digital assistant is displayed in a fusion with the camera shutter control. After the fusion display, the digital assistant is displayed in a listening state overlaying the original camera shutter control.

[0230] In addition, in another initial state, the digital assistant is not displayed in any state on the current interface. After the digital assistant is called out by long pressing a control (such as the power button) or by voice, the digital assistant can also be displayed over a preset type of control or icon on the current interface.

[0231] After the digital assistant's avatar is overlaid on the original camera shutter control, clicking the display area of ​​the digital assistant's avatar (i.e., the display area of ​​the original camera shutter control) can also trigger taking a photo. This can be achieved by the digital assistant passing instructions to the camera application to control taking photos, or by the camera application directly recognizing gestures.

[0232] In some cases, the navigation prompt line is permanently displayed in the system globally. Then, when the digital assistant is in the wake-up state, it can also be understood that the virtual image hides the sphere, changes the circle, and merges it with the navigation prompt line, so that the navigation prompt line not only has the function of a navigation prompt line, but also serves as the permanent state of the digital assistant virtual image.

[0233] In addition, when there are multiple controls or icons that can be overlaid in the interface, the digital assistant can switch between different controls or icons. Moreover, the digital assistant can be cut and merged from the controls of the first displayed interface to the controls or icons in the latest interface in the order of display time.

[0234] See also Fig.15 , Fig.15 A schematic diagram of a virtual image overlaid on an assistant suggestion component provided in an embodiment of the present application. Fig.15 As shown in (a), the digital assistant is displayed in the waiting state. In this state, the digital assistant integrates the navigation prompt line, and the deformed ring overlaps the navigation prompt line. Fig.15 As shown in (b), when switching to another home screen page, the home screen page contains an assistant suggestion widget, and the assistant suggestion widget displays an icon of a virtual image, then the digital assistant can flow from the navigation prompt line to the virtual icon in the assistant suggestion widget. And, when the navigation prompt line is globally resident, it is Fig.15 The solution shown in (b); when the navigation prompt line is only a deformed circle, the navigation prompt line will no longer be displayed after the digital assistant flows from the navigation prompt line to the virtual icon in the assistant suggestion widget.

[0235] The above introduces in detail the waiting state and listening state of the virtual image. The following will continue to introduce the task execution state of the virtual image.

[0236] Optionally, the virtual image of the digital assistant also has a task execution state, which includes a first feature body. Corresponding to at least one process interface in the process of the digital assistant executing the automated task, the first feature body in the task execution state generates at least one of a shape change and a position change.

[0237] Specifically, when the digital assistant performs an automated task, a series of changes will occur on the display interface of the electronic device. For example, when the digital assistant performs an automated task of calling a contact, the electronic device will first open the contact interface, then select a specific contact in the contact interface, and then enter the phone interface for calling the specific contact. Then, in the process interface during the digital assistant's automated task execution, the digital assistant is actually always in the task execution state, and the first feature body of the digital assistant in the task execution state can undergo various changes to show the user the automated process of the digital assistant performing the task, making the automated process of the digital assistant performing the task more intuitive and clear, and making it easier for the user to observe whether there are any errors in the process of the digital assistant performing the automated task.

[0238] In one possible example, the first feature body is displayed in the form of a geometric array, and the display position of the geometric array is adapted to the target control in the process interface. The geometric array enters the next process interface by simulating the click event of the target control. In other words, in the process of performing the automation task, the digital assistant can simulate the operation of the user clicking the control on the interface with the first feature body as the geometric array, thereby visualizing the entire process of the digital assistant simulating the user to perform the automation task.

[0239] Simply put, if executing an automated task requires opening an application, and in the automated step of opening the application, you need to click the application icon on the desktop, then the application icon is the execution target of this step. When opening the application, the dot matrix that changes to a circle can be displayed on the application icon.

[0240] For example, see Fig.16 , Fig.16 A schematic diagram of a process for a digital assistant to perform an automated task provided in an embodiment of the present application. Fig.16 As shown, first, the digital assistant receives the user's voice command "Share this photo to WeChat Moments". Then, the digital assistant displays the dot matrix on the "Send" control of the current interface to simulate the user's operation of clicking the "Send" control. Secondly, in response to the digital assistant simulating the user's operation of clicking the "Send" control, the interface changes. The current interface changes to display applications that can send photos, and these applications include the "WeChat" application; in the changed interface, the dot matrix is ​​displayed on the icon of the "WeChat" application to simulate the user's operation of clicking the "WeChat" application icon. Finally, the interface changes again, changing to an interface for editing a picture to be published in the "WeChat" application's Moments. At this time, the dot matrix is ​​displayed on the "Publish" control to simulate the user's operation of clicking the "Publish" control.

[0241] Optionally, when the first state is the listening state and the second state is the task execution state, both the listening state and the task execution state include a first feature body, and the first feature bodies in the listening state and the task execution state have different forms.

[0242] Specifically, in the task execution state, the first feature body includes: a geometric array in the interface, an outer frame of a digital assistant dialog card, an outer frame of an input box when calling a digital assistant input, or an outer frame of a task result page. That is, in different scenarios, the first feature body can be represented in different forms to better fit the content displayed on the current interface.

[0243] Optionally, the first feature body has different morphological changes corresponding to the task progress of the automated task. That is, in the process of the digital assistant performing the automated task, at different task progresses, the first feature body may change into different forms, so that the form of the first feature body always closely fits the current interface.

[0244] Exemplarily, corresponding to the execution process of the automated task, in the task execution state, the morphological change of the first feature body includes the following process: initially, the first feature body is displayed in the main form of a ring at an initial position relative to the second feature body, and the initial position is the same as or different from the position of the first feature body in the listening state; then, the first feature body is separated from the second feature body, and after being enlarged and deformed, it serves as the outer frame of the digital assistant dialogue card; secondly, the first feature body is changed from the outer frame of the dialogue card to a geometric array within the dialogue bubble of the dialogue card; finally, after the automated task is executed, the virtual image is displayed in the listening state.

[0245] In general, in the task execution state, for the first feature body whose main form is a ring, the ring separates from the sphere, and after the shape change, it is displayed in the task execution interface in the changed feature state. The task execution interface refers to the interface when the digital assistant receives the task instruction and transmits the instruction to the local application for execution based on the analysis of the task instruction, or the execution result interface. Therefore, the task execution interface can be part of the interface or the entire interface of the execution application.

[0246] Among them, the morphological changes of the ring include: the ring concentric with the image body and the sphere disappears and transforms into a geometric array (such as a dot matrix) of the task execution interface on the screen, or an outer frame surrounding the task execution interface. In addition, the process of transforming from a ring to a geometric array or an outer frame can be accompanied by animation. For example, the ring is decomposed into several points, and then laid on the task execution interface to form a dot matrix; the ring is deformed and elongated, transformed into a frame, and enlarged into the outer frame of the task execution interface.

[0247] The following will take several application scenarios of digital assistants as examples to introduce the process of morphological changes of digital assistants during task execution.

[0248] See also Fig.17 , Fig.17 A schematic diagram of a digital assistant provided in an embodiment of the present application, wherein a circular ring is converted into a dot matrix and laid on a task execution interface. Fig.17 As shown, Fig.17 (a) corresponds to the listening state of the digital assistant, and the language command input by the user at this time is "help me order a cup of Xinba coffee..."

[0249] Fig.17 (b) corresponds to the thinking state of the digital assistant (where the thinking state is a dynamic image, the ring keeps rotating according to a certain rule, Fig.17 (b) is a certain instantaneous state). Specifically, the thinking state corresponds to the digital assistant performing the task analysis process. At this time, the interface for the user to interact with the digital assistant displays the process of the digital assistant analyzing the task "recommending you with the historical orders of Xinba".

[0250] After completing the task analysis process, the task execution phase begins, and the digital assistant is in the task execution state. Fig.17 (c). Fig.17 In (c), the circular body of the digital assistant icon at the bottom disappears, and the task card obtained by executing the task is displayed above (i.e., a conversation bubble, but presented in the form of a card). At the same time, a number of dots are displayed on the task card, which are transformed from the original circular body of the digital assistant.

[0251] It is understandable that in order to avoid interference with the task execution interface, the dot matrix may disappear after being displayed for a period of time. Fig.17 In (c), the dots of the upper task card have disappeared, and the lower task card shows a momentary state of the disappearance of the dots (the dots and interface content are displayed at the same time, and both are displayed semi-transparently). When the card interface is fully displayed, the dots disappear. That is, for Fig.17 In the task execution interface, the display process is: only the dot matrix is ​​displayed → the dot matrix gradually disappears, and the interface content is gradually displayed (the dot matrix and the interface exist at the same time) - only the interface content is displayed.

[0252] In another application scenario, when a digital assistant is called to perform voice-to-text input, the task execution interface can be an input box or a pop-up card interface. These cards can be components or subroutines provided by locally installed applications.

[0253] See also Fig.18 , Fig.18 This is a schematic diagram of using a digital assistant to input text on a desktop provided by an embodiment of the present application. Fig.18As shown in (a), the user clicks the search widget on the desktop, and the search widget enters the edit mode. Fig.18 As shown in (b), the digital assistant is enabled by long pressing the power button. At this time, the digital assistant is in a listening state and uses the microphone to obtain the user's voice input. Fig.18 As shown in (c), the digital assistant receives voice input, converts the voice into text and fills it into the input box; at this time, the digital assistant is in the task execution state, the ring disappears from the outside of the sphere, and is enlarged, deformed and transferred to the outside of the search box, forming an outer frame surrounding the search box. Fig.18 As shown in (d), after the input is completed, the search widget expands into a search card, and the outer frame of the circular transformation can be expanded and surround the search card; of course, the outer frame of the circular transformation can also disappear after the search card is presented, or disappear after a preset time.

[0254] When the deformed body of the ring-shaped deformation disappears, or within a preset time after disappearance, the ring in the initial state (undeformed) can be displayed outside the sphere.

[0255] For example, see Fig.19 , Fig.19 A schematic diagram of a ring of a digital assistant provided in an embodiment of the present application deforming and returning to its original shape. Fig.19 As shown in (a), after the digital assistant is initially called out, the digital assistant is displayed in a listening state, and the ring on the digital assistant is displayed outside the sphere.

[0256] like Fig.19 As shown in (b), after the user completes the input, the thinking state ( Fig.19 (not shown in the figure), the digital assistant is displayed in the task execution state. In the initial stage of the task execution state, the ring of the digital assistant deforms outward, overlaps with the border position of the digital assistant's dialogue card, and is displayed on the border or outside the border, or displayed on the upper layer of the border. Visually, the ring of the digital assistant becomes the outer frame of the dialogue card.

[0257] like Fig.19 As shown in (c), when the card is no longer expanded, the ring disappears from the card frame and is displayed again as a ring of the initial size outside the sphere.

[0258] like Fig.19 As shown in (d), when the result is presented in a dialogue bubble, the ring on the digital assistant is converted into a dot matrix again and displayed on the dialogue bubble. It can be understood that Fig.19 The intermediate state shown in (c) can be omitted, and the state change of the ring can be directly obtained by Fig.19 The card frame shown in (b) is transformed to Fig.19 The dot matrix shown in (d).

[0259] like Fig.19 As shown in (e), the result of the dialogue card is fully displayed, the dot matrix disappears, and the ring is displayed outside the sphere again as a ring of the initial size.

[0260] In another application scenario, when calling a digital assistant for intelligent recognition or other automatic tasks, the task execution interface is the application interface currently being operated. For intelligent recognition, a smart recognition result is usually generated, which is one or more executable objects obtained by recognition (such as smart removal of people / smart removal of objects in album editing), then the task execution interface can be the area where the executable object is located. In the process of executing intelligent recognition, the entire interface is recognized in real time, and finally the executable object is recognized. Corresponding to this task execution process, the dot matrix corresponding to the circle can also be adapted to the task execution process, that is, in the recognition process stage, the dot matrix is ​​spread all over the interface, and after the executable object is recognized, the dot matrix is ​​displayed only in the area of ​​the executable object.

[0261] For example, see Fig. 20 , Fig. 20 A schematic diagram of the morphological changes of a digital assistant in a smart recognition scenario provided by an embodiment of the present application. Fig. 20 In the process, the user calls up the digital assistant and uses the intelligent recognition function. Fig. 20 As shown in (a), a picture is displayed on the interface of a smart phone. The picture is displayed in full screen. Then, the user long presses anywhere on the interface to trigger the smart recognition function. In addition, the icon of the digital assistant is displayed on the current interface. When the digital assistant first appears, the digital assistant is in a listening state. Fig. 20 As shown in (b), the digital assistant performs intelligent recognition on the image. At this time, the ring of the digital assistant disappears from the outside of the sphere and is transformed into a dot matrix displayed on the interface, covering the entire screen. Fig. 20 As shown in (c), after the image recognition is completed, at least one executable object is recognized, and the dot matrix is ​​only displayed in the display area where the executable object is located. At this time, the points of the dot matrix may not be a regular dot matrix, but may move dynamically in the display area where the executable object is located, so that the dot matrix becomes a plurality of irregular discrete points.

[0262] It should be noted that in the above Fig.19 and Fig. 20 In order to fully present the switching between the digital assistant forms, the established steps are used to illustrate. When actually operating an electronic device, the electronic device supports calling up the digital assistant to perform preset functions through shortcut operations, such as Fig. 20 At this time, the digital assistant can directly display the corresponding status, such as directly displaying the task execution state, or displaying the listening state at the moment of calling, and then immediately switching to the task execution state.

[0263] Taking the smart recognition function as an example, suppose the system defines long pressing the power button as a shortcut operation for smart recognition. When the user long presses the power button, the operating system displays the digital assistant icon on the interface and directly displays the task execution state, that is, long pressing the power button directly enters the task execution state. Fig. 20 (b) Display interface.

[0264] In some designs, in the task execution state, the image of the digital assistant and the sphere may not be displayed on the interface, but only the changed shape of the ring body may be displayed, such as Fig. 20 In (c), only the dot matrix can be displayed without showing the sphere and the image body below.

[0265] The above introduces the changing process of the digital assistant in the task execution state. In actual applications, when the digital assistant initially appears on the interface, it may also appear in a certain pattern.

[0266] Optionally, when the first interface is a lock screen interface or any screen interface after unlocking, and the first form displayed on the first interface is a listening state, then, displaying the virtual image in the first form on the first interface specifically includes: in the initial stage, only the first feature body is displayed, and the second feature body is not displayed, and the first feature body is initially displayed in a geometric array, covering the entire screen; then, the first feature body is changed from the geometric array to be displayed in the main form, and the second feature body is displayed.

[0267] That is to say, when waking up the digital assistant, the digital assistant first appears in the form of a geometric array (such as a dot matrix) and gradually fills the screen; then, the digital assistant gradually transforms from the geometric array to the main form of the first feature body (i.e., a ring) and displays the second feature body (i.e., a sphere), thereby realizing the awakening process of the digital assistant in the form of an animation, increasing the intuitiveness of the interaction process, and helping users to identify the current working status of the digital assistant.

[0268] The geometric array may be, for example, a dot matrix, that is, an array formed by a plurality of dots arranged in a regular pattern. The geometric array may also be an array formed by other geometric shapes (such as polygons such as triangles and rectangles) arranged in a regular pattern.

[0269] Optionally, the first feature body is initially displayed in a geometric array, which may specifically include: first determining an initial display position of the geometric array; secondly, displaying a portion of the geometric pattern of the array at the initial display position, and then gradually spreading the geometric pattern to cover the entire screen. For example, first displaying a portion of the dot matrix from an initial display position, and then gradually spreading the dot matrix to cover the entire screen.

[0270] Optionally, the initial display position of the geometric array is determined based on user input. The user input refers to the input used to wake up the digital assistant. The initial display position may be near the user input that triggers the waking up of the digital assistant. For example, when the user input is a long press of the power button of a smartphone, the initial display position of the geometric array may be on the screen near the power button.

[0271] Furthermore, when no user input is detected, the initial position is a default position, which is an area where the virtual image is displayed, such as a preset area at the bottom of the screen.

[0272] In this solution, the initial display position of the geometric array is determined by user input, which can make the display process of the digital assistant more closely aligned with the user input, enhance the intuitiveness of the interaction between the user and the digital assistant, and improve the user experience.

[0273] In general, in the process of calling up the digital assistant, the appearance of the dot matrix can have a preset pattern. When the user's operation is detected (touching or pressing a physical button), the initial display position of the digital assistant is determined based on the user's operation, and the dot matrix is ​​expanded from the initial display position to the full image display. When no user operation is detected, the display position of the digital assistant's virtual image is used as the initial display position, and the dot matrix is ​​expanded from the initial position to the full image display.

[0274] For example, see Fig.21 , Fig.21 This is a schematic diagram of a digital assistant appearing in a smart recognition scenario provided by an embodiment of the present application. Fig.21 As shown in the figure, when the user activates the digital assistant by long pressing the power button on the right side of the smartphone, the digital assistant is initially displayed in the form of a dot matrix on the screen near the power button, and then gradually spreads to the entire screen. When the user activates the digital assistant by long pressing the digital assistant at the bottom of the screen, the digital assistant is initially displayed in the form of a dot matrix near the area where the digital assistant's virtual image is located, and then gradually spreads to the entire screen.

[0275] In some scenarios, the morphological changes of the digital assistant may also be related to the scene information, so that some information in the current usage scenario can be accurately represented through the digital assistant's virtual image.

[0276] Optionally, in some embodiments, the electronic device may obtain scene information, and determine the target form of the first feature body of the digital assistant based on the scene information, and display the first feature body in the target form. That is, the target form is related to the scene information. When the electronic device obtains specific scene information, the electronic device may display the first feature body in a specific form.

[0277] The scene information may include external environment information and system event information. External environment information is a signal obtained by a sensor to indicate the external environment, which may be obtained through a sensor on an electronic device or a network, such as temperature, humidity, weather and other external environment information. System event information is an event obtained from the system of an electronic device and occurring within a preset time, such as music being played, an alarm notification within a preset time, searching for a file and other event information.

[0278] When the first feature body is a ring, the ring can be deformed based on the external environment information. For example, on a rainy day, the ring changes into an umbrella shape. Fig. 22 , Fig. 22 A schematic diagram of a digital assistant that generates deformation based on scene information provided in an embodiment of the present application. Fig. 22 As shown in the figure, after calling up the digital assistant, if you maintain the current interface, the digital assistant will always be displayed in the listening state. Fig. 22 After a preset time in the listening state, the ring of the digital assistant changes shape based on the scene information. Figure 2 In the picture, the current weather is light rain, so the ring changes into an umbrella shape.

[0279] In addition, the ring can also be deformed based on system event information. For example, when the screen is turned off to play music, the ring of the digital assistant changes to the shape of headphones; when there is an alarm notification within the preset time, the ring of the digital assistant changes to the shape of a physical alarm clock. For another example, when the user calls the digital assistant to search for a file, the ring of the digital assistant changes to a magnifying glass.

[0280] Of course, in addition to the first feature body that can be deformed, other feature bodies of the digital assistant can also be deformed based on scene information, which will not be repeated here.

[0281] Optionally, when the virtual image is in a state to be awakened, the state to be awakened includes at least a first feature body, and the first feature body is currently displayed in a long strip. In response to the electronic device detecting that the focus of the human eye falls on the area where the screen is located, the electronic device displays a second feature body.

[0282] That is to say, in some scenarios, when the digital assistant is in the wake-up state, only a long bar is displayed; when the digital assistant detects that the user's eyes fall on the screen, it can be considered that the user has an interactive tendency, so the second feature body is displayed, thereby indicating that the digital assistant detects that the user has an interactive tendency and is waiting for interaction, so as to improve the naturalness of human-computer interaction.

[0283] Optionally, in response to detecting a human voice, the second feature body may move relative to the first feature body in a direction closer to the source of the human voice. Specifically, when the second feature body just appears, its position is relatively fixed, for example, located directly above or below the first feature body. When the electronic device detects a human voice, the electronic device may move the second feature body according to the direction of the human voice, so that the second feature body moves with the human voice to improve the naturalness of the interaction.

[0284] Specifically, in another common terminal, the in-vehicle terminal, the digital assistant, as the main method of interaction, can also be resident on the display screen of the vehicle (such as the central control screen) and interact with the user visually through at least one deformation in a ring or a sphere, thereby keeping the user's attention on the digital assistant and enhancing the sense of conversation.

[0285] In the vehicle-mounted terminal environment, scene information mainly includes human information, system information and external environment information.

[0286] The human information may come from at least one of the driver and the passenger, including head orientation, eye contact, gestures, posture, and voice, and may be distinguished as the main driver or the front passenger (it is usually difficult for the back seat to interact with the central control screen).

[0287] System information may refer to changes in manual or adaptive adjustments of vehicle control functions, such as driving mode, air conditioning settings, window defogger, seat belt status changes, etc., as well as some vehicle condition change information, such as changes in vehicle speed, driving direction, vehicle bumps, special traffic conditions, etc. In addition, system information may also include details such as the type of the aforementioned running application.

[0288] External environmental information usually includes changes in weather, light, temperature, etc.

[0289] Based on these scene information, virtual images corresponding to different states can trigger different deformations of the virtual form, such as the deformation of the above-mentioned ring, or the deformation of the sphere.

[0290] In addition, the difference between the vehicle terminal and the handheld terminal is that the central control screen basically does not have an off state, so the digital assistant on the vehicle terminal is displayed in the wake-up state after it is turned on. In this way, in the wake-up state, the digital assistant has different deformation states according to the different scene conditions detected.

[0291] For example, see Fig.23 , Fig.23 A schematic diagram of a change in a digital assistant on a vehicle-mounted terminal provided in an embodiment of the present application. Fig.23In the initial wake-up state of the digital assistant, only the deformed circle is displayed. When it detects that the driver or co-driver is looking at the central control screen, part of the sphere is displayed. If a human voice is further detected on this basis, more spherical area will be displayed, such as a half sphere.

[0292] Furthermore, corresponding to the different voices of the driver and the co-driver, the ring can be elongated, and the sphere is located on one side of the ring in the direction of the voice source. Fig.23 In the embodiment, when the driver's voice is detected, the hemisphere is located on the side of the elongated circular body close to the driver's side, that is, the hemisphere moves toward the driver's side; when the passenger's voice is detected, the hemisphere is located on the side of the elongated circular body close to the passenger's side, that is, the hemisphere moves toward the passenger's side.

[0293] Optionally, in the case where the electronic device is a smart car, a digital assistant can be displayed on the screen of the electronic device based on the activation or alarm of the vehicle-mounted function, and the corresponding function or alarm information can be indicated by the deformation or movement of the first feature body.

[0294] That is to say, on the car computer, a digital assistant can also be used to prompt vehicle condition information. For example, when it is detected that the vehicle is in a bumpy road environment, the awakened digital assistant is displayed, and the sphere vibrates up and down along the screen where the ring is located. Alternatively, the sphere and the ring vibrate up and down together, and the vibration frequency of the two can be consistent or inconsistent. For another example, when it is detected that the vehicle starts the window defogger function, the awakened digital assistant is displayed, and the sphere is rendered in white, and then the ring uses the x-axis as the fixed axis to produce an up and down swing in the z-axis direction, and as it swings, the transparency of the white rendered on the sphere becomes lower until it is completely transparent and the ring stops swinging. For another example, when it is detected that the seat belt valve is not locked or the locking progress is not enough (that is, the seat belt is not worn), the awakened digital assistant is displayed, and after the ring body is tilted at a preset angle, a part of the ring body is broken in the middle and becomes an arc, and the arc gradually closes to form a ring.

[0295] The above mainly introduces the process of the digital assistant's form changing when the electronic device is a smartphone or a car terminal. The following will introduce the process of the digital assistant's form changing when the electronic device is a smart TV.

[0296] See also Fig.24 , Fig.24 A flowchart of a display method of a digital assistant provided in an embodiment of the present application. Fig.24 As shown, the display method of the digital assistant includes the following steps 2401-2403, and the display method of the digital assistant is applied to an electronic device configured with a digital assistant.

[0297] Step 2401, obtain the user's location and display the first form of the digital assistant based on the user's location.

[0298] In this embodiment, the electronic device configured with the digital assistant may be deployed with a sensor, such as an infrared sensor, an ultrasonic sensor, or an optical distance sensor, so that the electronic device can detect the user's position based on the deployed sensor. For example, when the electronic device is deployed with an infrared sensor, the electronic device emits infrared light through the infrared sensor and receives infrared light returned by the user's body, and then calculates the direction and distance of the user relative to the electronic device, and then determines the user's position.

[0299] Optionally, the electronic device may also obtain the user's location in other ways, such as obtaining the user's location from other devices. For example, when the electronic device is a smart TV or smart speaker in a smart home, the electronic device can connect to other smart devices (such as home cameras, sweeping robots, or smart routers) through the network, so that after the other smart devices determine the user's location through their own sensors or positioning modules, the electronic device can obtain the user's location from the other smart devices.

[0300] When the electronic device obtains the user's location, the first form of the digital assistant can be displayed on the display screen of the electronic device based on the user's location, that is, the first form of the digital assistant displayed by the electronic device is related to the user's location. Simply put, the farther the user's location is from the electronic device, the lower the user's tendency to interact with the electronic device, so the electronic device can display a digital assistant form with a lower degree of interaction intention; the closer the user's location is to the electronic device, the higher the user's tendency to interact with the electronic device, so the electronic device can display a digital assistant form with a higher degree of interaction intention.

[0301] Step 2402, obtaining the user's spatial interaction behavior, where the spatial interaction behavior is used to indicate the user's activities in the space.

[0302] After determining the user's position and displaying the first form of the digital assistant, the electronic device can further capture the user's spatial interaction behavior, that is, the user's activities in the space, through sensors, and then determine the user's further interaction tendencies.

[0303] Exemplarily, the user's spatial interaction behavior may include, but is not limited to, any of the following behaviors: approaching the electronic device, moving away from the electronic device, moving to the left, moving to the right, pointing a finger at the electronic device, pointing an arm at the electronic device, facing the face toward the electronic device, and a preset gesture. In general, the user's spatial interaction behavior can be used to indicate the interaction tendency between the user and the digital assistant of the electronic device. For example, when the user's spatial interaction behavior is approaching the electronic device, it means that the user expects to conduct further interaction with the digital assistant of the electronic device; when the user's spatial interaction behavior is moving away from the electronic device, it means that the user does not expect to conduct further interaction with the digital assistant of the electronic device.

[0304] Step 2403, in response to the spatial interaction behavior, display the second form of the digital assistant.

[0305] In this embodiment, after acquiring the user's spatial interaction behavior, the electronic device switches from the first form of the digital assistant to displaying the second form of the digital assistant. The first form of the digital assistant and the second form of the digital assistant are used to represent different states of the digital assistant and the user's interaction process, thereby facilitating the user to identify the current interaction state of the digital assistant and issue corresponding interaction instructions.

[0306] Optionally, the second form of the digital assistant displayed by the electronic device may be related only to the spatial interaction behavior. That is, after the electronic device obtains the user's spatial interaction behavior, no matter what the first form of the digital assistant currently displayed is, it determines to display the corresponding form of the digital assistant (i.e., the second form mentioned above) according to the pre-established mapping relationship between the spatial interaction behavior and the form of the digital assistant.

[0307] In other embodiments, the second form of the digital assistant displayed by the electronic device may also be related to the first form of the digital assistant and the spatial interaction behavior that has been displayed by the electronic device. That is to say, the electronic device actually determines the second form of the digital assistant that needs to be displayed based on the currently displayed first form and the user's spatial interaction behavior. For example, the digital assistant can have multiple forms, and different forms correspond to different degrees of interaction intention between the user and the electronic device judged by the electronic device. When the electronic device judges that the user expects the degree of interaction intention with the electronic device to increase based on the acquired spatial interaction behavior, the electronic device can determine to display a second form with a higher degree of interaction intention than the first form based on the first form of the digital assistant currently displayed, and display the second form; conversely, when the electronic device judges that the degree of interaction intention between the user and the electronic device is reduced based on the acquired spatial interaction behavior, the electronic device can determine to display another second form with a lower degree of interaction intention than the first form based on the first form of the digital assistant currently displayed, and display the other second form.

[0308] Optionally, in some embodiments, the electronic device may divide the area near the electronic device into multiple areas based on the location of the electronic device, and different areas are not the same relative to the location of the electronic device. For example, the electronic device can be divided into three areas, namely the first area, the second area and the third area. Among them, the electronic device is located in the first area, the second area is located outside the first area relative to the electronic device and surrounds the first area, and the third area is located outside the second area relative to the electronic device and surrounds the second area. For example, the first area, the second area and the third area may be areas divided with the electronic device as the center, and the plane shape of the first area is a circle or an ellipse, the plane shape of the second area is a ring surrounded outside the first area, and the plane shape of the third area is a ring surrounded outside the second area. In this way, when the electronic device recognizes that the user's spatial interaction behavior is moving between different areas, it can accurately determine the user's interaction tendency with the electronic device, and then adjust the form of the displayed digital assistant.

[0309] For example, assuming that the user's position acquired by the electronic device in the above step 2401 is in the first target area, and the spatial interaction behavior acquired by the electronic device in the above step 2402 is that the user moves from the first target area to the second target area, the electronic device switches the first form of the digital assistant to the second form. The first target area belongs to any one of the above-mentioned first area, second area and third area, and the second target area can be an area adjacent to the first target area.

[0310] Optionally, in response to the user moving from the first target area to the second target area, the electronic device may display a second form of the digital assistant; wherein the second form is the form corresponding to the second target area. In other words, each area divided around the electronic device corresponds to a form of the digital assistant, and when the user moves from one area to another, the form of the digital assistant is switched so that the currently displayed form is the form corresponding to the area where the user moves.

[0311] Specifically, when the first form is used to indicate that the digital assistant is in a non-task processing state and the first target area is outside the second target area, the second form is used to indicate that the digital assistant is in a state of waiting to receive instructions. Specifically, when the first target area is outside the second target area, the user moves from the first area to the second area, which means that the user moves from an area farther away from the electronic device to an area closer to the electronic device, that is, the user is approaching the electronic device, so the electronic device can recognize that the user has a further tendency to interact, thereby switching the form of the digital assistant to show the user that the digital assistant has switched from a non-task processing state to a state of waiting to receive instructions. Among them, the non-task processing state may be a state indicating that the digital assistant has no tasks to be processed and is in a dormant state; the state of waiting to receive instructions may refer to a state in which the digital assistant has been awakened and is ready to receive instructions at any time. The state of waiting to receive instructions of the digital assistant is a state with a higher degree of interaction intention than the state of non-task processing.

[0312] Alternatively, when the first form is used to indicate that the digital assistant is in a state to be received, and the first target area is outside the second target area, the second form is used to indicate that the digital assistant is in a state to be received. Similarly, when the first form is used to indicate that the digital assistant is in a state to be received, the user moves from the first target area to the second target area, which means that the user has a further tendency to interact, thereby switching the form of the digital assistant to show the user that the digital assistant has switched from a state to be received to a state to be received. Among them, the instruction receiving state may refer to a state in which the digital assistant has received instructions issued by the user through the corresponding hardware on the electronic device, that is, the hardware receiving the instructions is already in a working state, for example, the digital assistant receives voice instructions issued by the user through the microphone on the electronic device. The instruction receiving state of the digital assistant is a state with a higher degree of interaction intention than the state to be received.

[0313] Alternatively, when the first form is used to indicate that the digital assistant is in a state of waiting to receive instructions, and the second target area is outside the first target area, the second form is used to indicate that the digital assistant is in a state of no task processing. It is understandable that when the second target area is outside the first target area, the user moves from the first target area to the second target area, which means that the user is moving away from the electronic device, so the electronic device can recognize that the user has no further interaction tendency, and thus switches from the state of waiting to receive instructions to the state of no task processing with a lower degree of interaction intention.

[0314] In this solution, the user's interaction tendency with the digital assistant is identified by obtaining the user's movement between areas with different distances from the electronic device, and then the morphology of the digital assistant is adjusted. This can well fit the relationship between social status and distance between humans and improve the naturalness of the morphology change of the digital assistant.

[0315] Optionally, before switching the first form of the digital assistant to the second form, the electronic device may also obtain the user's spatial activity behavior, which is used to indicate the user's activity in space. The spatial activity behavior is the behavior performed by the user without the intention to interact with the digital assistant, that is, the spatial interaction behavior can be used to indicate that the user does not intend to interact with the digital assistant.

[0316] Then, in response to the spatial activity behavior, the electronic device displays the adjusted first form of the digital assistant; wherein the adjusted first form is obtained by adjusting at least one feature of the first form, and the adjusted first form and the first form are used to represent the same state in the process of interaction between the digital assistant and the user. That is, in the case where the user does not have an intention to interact, the spatial activity behavior performed by the user will change the features of the form of the digital assistant, but will not change the form of the digital assistant, that is, will not cause the interactive state of the digital assistant to change.

[0317] The features adjusted in the first form may include, but are not limited to, the following features of the feature body used to represent the digital assistant: the posture of the feature body, the movement law of the feature body, the light effect of the feature body, the material of the feature body, the color of the feature body, the brightness of the feature body, etc. For example, when the user walks in a certain divided area instead of moving to another area, the feature body of the digital assistant (such as a ring body) may rotate with the user; for another example, when the user points his finger in different directions, the feature body of the digital assistant may rotate or rotate with the position pointed by the user's finger; for another example, when the user gradually approaches the display screen of the electronic device, the ring body of the digital assistant may rotate and change the pitch angle.

[0318] The above introduces the process of switching the form of the digital assistant based on the user's spatial interaction behavior. The following will introduce the specific form of the digital assistant in detail.

[0319] Exemplarily, the first form and the second form of the digital assistant both include multiple features, and compared with the first form, the posture and / or motion law of at least one feature in the second form is changed. That is, the different forms of the digital assistant are all composed of the same multiple features, and the difference between the different forms is that the posture and / or motion law of at least one feature has changed.

[0320] In general, by using multiple feature bodies to construct the image of a digital assistant, different feature bodies can independently adjust posture and movement rules. In this way, by adjusting the changes in the feature bodies, the digital assistant in different states can be represented, forming a unified style of feedback method that responds to spatial interactive behavior input, which helps users understand the feedback of the digital assistant.

[0321] In one possible implementation, the multiple feature bodies used to represent the image of the digital assistant include a torus, a sphere, and an image body. The image body is wrapped in the sphere, and the torus surrounds the sphere. That is, in the image of the digital assistant, from the inside to the outside, they are the image body-the sphere-the torus. In addition, the sphere may be transparent or transparent so that the user can see through the sphere to see the image body wrapped in the sphere. The image body may be an object of any shape designed for the digital assistant, such as an object similar to the shape of the human body, an object of irregular shape, etc.

[0322] In this solution, by constructing the image of the digital assistant based on the characteristic bodies of a torus, a sphere and an image body, it is possible to achieve a seamless and smooth transformation when switching the form of the digital assistant by adjusting the posture or movement pattern of the characteristic bodies, making the form change process of the digital assistant more natural and in line with human cognition, making it easier for users to understand the current interaction status with the digital assistant.

[0323] Exemplarily, when the first form or the second form is used to indicate that the digital assistant is in a non-task processing state, the first form or the second form is that multiple feature bodies move along a preset trajectory, and the sphere and the image body rotate during the movement.

[0324] Simply put, when the digital assistant is in a non-task processing state, it means that the user has no tendency to interact with the digital assistant. Therefore, the form of the digital assistant can be manifested as moving according to predetermined rules without any interactive intention, that is, multiple feature bodies move along preset trajectories and the sphere and the image body rotate during the movement.

[0325] Exemplarily, when the first form or the second form is used to indicate that the digital assistant is in a state of waiting to receive commands, the first form or the second form is that a partial area in the ring body has a light effect, and the partial area is facing the user's position.

[0326] When the digital assistant is in the state of waiting to receive instructions, it means that the user has the tendency to interact with the digital assistant, and the digital assistant is ready to receive the user's instructions at any time. Therefore, the digital assistant can be manifested as always paying attention to the user (that is, the area with light effects is facing the user's position) and waiting for the user to issue instructions.

[0327] Exemplarily, when the second form is used to indicate that the digital assistant is in a command receiving state, the second form is that the top or bottom of the multiple features are facing the user. That is, when the digital assistant is in a command receiving state, it means that the digital assistant is receiving the user's command, so the form of the digital assistant can be expressed as the state in which the digital assistant is concentrating on receiving the command (that is, the top or bottom of the multiple features are facing the user, indicating listening to the user's command).

[0328] In another possible implementation, the first form and the second form are both forms of a virtual human body, and the different forms of the virtual human body differ in any one or more of the following features: the expression, movement, and size of the virtual human body. In simple terms, the form of the digital assistant is also composed of multiple feature bodies, and the multiple feature bodies can be divided according to human body parts, that is, different feature bodies correspond to different parts of the virtual human body. For example, the multiple feature bodies can be divided into head feature bodies, eye feature bodies, mouth feature bodies, left hand feature bodies, right hand feature bodies, etc.

[0329] In this solution, by setting the form of the digital assistant to that of a virtual human body, it is possible to achieve human-like form changes when switching the form of the digital assistant in response to the user's spatial interaction input, making the form changes of the digital assistant more natural and easier for users to understand the current interaction status with the digital assistant.

[0330] To facilitate understanding, the execution process of the display method of the digital assistant provided in the embodiment of the present application will be described in detail below with reference to specific examples.

[0331] See also Fig.25 , Fig.25 A schematic diagram of an application scenario of the display method of a digital assistant provided in an embodiment of the present application. Fig.25 The application scenario shown includes a user 401 and an electronic device 402. The electronic device 402 may be, for example, a smart TV in the user's home, and a digital assistant is deployed on the smart TV.

[0332] During the startup of the electronic device 402 , the user 401 can control the electronic device 402 through voice commands, so that the electronic device 402 automatically executes certain applications, such as automatically opening an application for playing movies and TV shows.

[0333] The electronic device 402 is configured with hardware such as a distance sensor and a microphone, wherein the distance sensor is used to detect the user's position and the microphone is used to receive the user's voice command. The electronic device 402 is equipped with a digital assistant in software to process the user's voice command, thereby automatically executing certain applications according to the voice command.

[0334] For the sake of ease of description, the following will take a smart TV as an example of an electronic device that executes the display method of a digital assistant, and will describe in detail the process of a smart TV executing the display method of a digital assistant.

[0335] See also Fig.26 , Fig.26 A schematic diagram of a digital assistant in a dormant state provided in an embodiment of the present application. Fig.26As shown, when the smart TV is not used for a long time and there is no user around the smart TV, the smart TV enters the lock screen standby state and displays the lock screen content on the display screen. At this time, when the smart TV enters the lock screen standby state, the digital assistant is not displayed on the display screen of the smart TV, and the digital assistant runs in the background, waiting to be awakened.

[0336] Optionally, the lock screen content displayed on the display screen may be content unrelated to the digital assistant. Alternatively, the lock screen content displayed on the display screen may also be a certain form of the digital assistant, such as the breathing form of the digital assistant.

[0337] During the startup of the smart TV, when the smart TV detects a user through a sensor, it can first determine the user's location based on the sensor, and then determine the area where the user is located based on the user's location, and determine the initial display form of the digital assistant (i.e., the first form mentioned above) according to the area where the user is located.

[0338] In actual applications, a preset logic can be set in the smart TV to divide the space around the smart TV into multiple areas. When the user is in different areas, different forms of digital assistants are displayed accordingly. Fig. 27 , Fig. 27 A schematic diagram of a region division provided in an embodiment of the present application. Fig. 27 As shown, in this embodiment, with the smart TV as the center, the space around the smart TV is divided into four areas according to different radii, from outside to inside: non-perception area A1, presence perception area A2, natural behavior area A3 and typical task area A4. Specifically, with the smart TV as the center, three radii are defined: R1, R2 and R3, where R1>R2>R3. Then, the distance between the non-perception area A1 and the smart TV is greater than R1, the distance between the presence perception area A2 and the smart TV is between R1 and R2, the distance between the natural behavior area A3 and the smart TV is between R2 and R3, and the distance between the typical task area A4 and the smart TV is less than R3. Among them, the non-perception area corresponds to the breathing state, the presence perception area corresponds to the awakening state, and the natural behavior area and the typical task area correspond to the listening state.

[0339] When the digital assistant of the smart TV is activated, for example, the user activates the digital assistant through the wake-up word / remote control, the smart TV will determine an initial form of the digital assistant from the breathing state, the wake-up state and the thinking state according to the user's position, and display it on the screen.

[0340] For example, when a user activates the digital assistant of a smart TV using a voice command on a remote control in a non-sensing area, the initial state corresponding to the non-sensing area is a breathing state, and the smart TV activates the main program of the digital assistant and displays the breathing state of the digital assistant. When a user activates the digital assistant of a smart TV using a voice command on a remote control in a presence sensing area, the awakening state of the digital assistant is displayed.

[0341] After the smart TV displays the initial form of the digital assistant based on the user's location, the smart TV will also capture the user's activities in the space through the sensors deployed by itself, and parse the sensor data according to preset logic to identify whether there is spatial interaction behavior and the type of spatial interaction behavior from the captured user spatial activities.

[0342] For example, a smart TV uses a distance sensor to sense the user's position and distance relative to the TV, and determines whether the user is walking towards the TV based on the changes in position and distance over a period of time. If it is detected that the distance between the user and the TV continues to decrease over a period of time, and the real-time distance between the user and the TV is less than a preset value, it is considered that a walking input is detected. Similarly, when the user moves left or right relative to the TV, the user's horizontal left / right movement input can be detected.

[0343] After detecting the user's spatial interaction behavior, the smart TV can adjust at least one feature of the initial form of the digital assistant according to the type of spatial interaction behavior, thereby achieving form switching of the digital assistant.

[0344] See also Fig.28 , Fig.28 A schematic diagram of a switching display mode of a digital assistant provided in an embodiment of the present application. Fig.28 As shown in (a), when the digital assistant in the smart TV is activated, the user is in the non-sensing area A1, and the smart TV displays the breathing state of the digital assistant, that is, the ring moves regularly up and down along the z-axis. When the user keeps walking in the non-sensing area A1, the smart TV will not perceive the spatial input of the non-sensing area A1, so the user's walking will not affect the form of the digital assistant, that is, the smart TV always displays the breathing state of the digital assistant.

[0345] like Fig.28 As shown in (b), as the user moves from the non-sensing area A1 to the presence sensing area A2, the smart TV detects that the user's spatial interaction behavior is approaching the smart TV, and the user is in the presence sensing area A2, so the smart TV switches the digital assistant to the awake state. When the digital assistant is in the awake state, the ring body is fixed on the equatorial plane of the sphere and rotates around the z-axis following the user's movement. In addition, the light effect area on the ring body points to the user.

[0346] In addition, when the user moves in the presence sensing area A2 to adjust the orientation relative to the TV, the circular ring in the digital assistant rotates with the rotation of the human body and always turns the light effect area toward the user. Fig.29 , Fig.29 A schematic diagram of a digital assistant in a wake-up state provided in an embodiment of the present application. Fig.29 As shown in (a) in FIG. 1 , when the user is in the presence sensing area A2, the digital assistant is in the awake state, and the light effect area on the ring body in the digital assistant points to the user; Fig.29 As shown in (b), when the user walks in the presence perception area A2 and moves from the left to the right, the digital assistant is still in the awake state, and the ring in the digital assistant follows the user and makes the light effect area on the ring always point to the user, that is, the light effect area on the ring also moves from the left to the right. In addition, when the user continues to approach the smart TV, the smart TV detects the user's direction input, and the ring can also rotate around the y-axis to produce a change in the pitch direction. When the user always moves in the presence perception area A2, the movement rules of the sphere and the image body may not change.

[0347] When the user enters the natural behavior area A3 (including the natural behavior area A3 and the typical task area A4) from the presence perception area A2, the digital assistant switches from the awake state to the listening state, and the position and movement pattern of the ring body change. First, the ring body rotates 90 degrees around the y-axis from the equatorial plane of the sphere to the yoz plane. At the same time, the light effect can change, and the entire ring body is filled with light effects. The movement patterns of the sphere and the image body change. For example, after the sphere and the image body rotate 90 degrees around the y-axis once, they rotate continuously around the z-axis. Fig.30 As shown, when the user enters the natural behavior area A3 from the presence perception area A2, the digital assistant displayed on the smart TV switches from the wake-up state to the listening state, indicating that the digital assistant is receiving the voice commands issued by the user.

[0348] When the digital assistant is in the listening state, when the user inputs a voice command, for example, the voice command is "What's the weather like today?", the digital assistant will parse the voice command and perform the corresponding operation. During the process of the digital assistant parsing the voice command and performing the corresponding operation, the smart TV may display the thinking state of the digital assistant, and the display duration of the thinking state is positively correlated with the length of time the digital assistant takes to parse the voice command and perform the corresponding operation.

[0349] In addition, when the user is in the natural behavior area A3, the digital assistant is in a listening state. When the user inputs a voice command to the digital assistant, the user often needs to first input a voice command as a wake-up command, such as "Hello, Xiaoyi" or "Xiaoyi Xiaoyi", so that the digital assistant can receive and parse the voice command after the wake-up command. However, when the user enters the typical task area A4, the digital assistant is still in a listening state, but the user can directly input a voice command to the digital assistant without having to input a voice command as a wake-up command.

[0350] It should be noted that after the digital assistant receives and parses the user's voice command, the digital assistant can change its own display form according to the content of the user's voice command. For example, when the digital assistant parses the voice command to play music, the digital assistant's form can be to dynamically change color or rotation rhythm to follow the rhythm of the music, that is, the digital assistant will also adjust its own display form accordingly when performing music playback operations. For another example, when the digital assistant parses the voice command to turn on the lights, turn off the lights, or report the weather, the digital assistant's form can change in brightness, color, or material following the action of turning on the lights, turning off the lights, or the type of weather being reported.

[0351] In addition, when the digital assistant parses voice commands and performs corresponding tasks based on the content of the voice commands, the digital assistant can change its own display form according to the progress of its own task execution. For example, the digital assistant can gradually change the material of the ring body, change the area of ​​the ring body filled with specific materials, etc. according to the progress of the task.

[0352] The above introduces the changes in the form of the digital assistant displayed on the smart TV when the user moves from an area farther away from the smart TV to an area closer to the smart TV. In fact, the user may also move from an area closer to the smart TV to an area farther away from the smart TV. In this case, the smart TV can decide the form of the digital assistant to be displayed based on the user's spatial interaction behavior and the area where the user is located after moving, thereby achieving the form switching of the digital assistant. For example, when the user moves from the natural behavior area A3 to the presence perception area A2, the digital assistant displayed on the smart TV switches from the listening state to the awake state; when the user moves from the presence perception area A2 to the non-perception area A1, the digital assistant displayed on the smart TV switches from the awake state to the breathing state.

[0353] The above describes the morphological change process of the digital assistant when the main body image of the digital assistant is composed of a torus, a sphere, and an image body. The following describes the morphological change process of the digital assistant when the main body image of the digital assistant is composed of a virtual human body.

[0354] Specifically, the main body image of the digital assistant can be a virtual human body, and the shape of the digital assistant is also composed of multiple feature bodies. The multiple feature bodies can be divided according to human body parts, that is, different feature bodies correspond to different parts of the virtual human body. For example, the multiple feature bodies can be divided into head feature bodies, eye feature bodies, mouth feature bodies, left hand feature bodies, right hand feature bodies, etc.

[0355] When the main image of the digital assistant is a virtual human body, the smart TV can also divide the space near the smart TV into multiple areas with the location of the smart TV as the center of the circle. In this way, after the digital assistant in the smart TV is activated, the smart TV can display the corresponding form of the digital assistant according to the area where the user is located in the space and the user's spatial interaction behavior.

[0356] Specifically, in this embodiment, the space near the smart TV can be divided into three areas, area 1 is an area 1.2-3.7 meters away from the smart TV, area 2 is an area 0.5-1.2 meters away from the smart TV, and area 3 is an area less than 0.5 meters away from the smart TV.

[0357] When the smart TV recognizes that the user is in area 1, the smart TV can consider the distance between the user and the digital assistant to be an identification distance, and the spatial interaction behaviors of the user obtained by the smart TV include: the user faces the digital assistant, the user's gaze is on the digital assistant, and the user gazes at the digital assistant for more than 3 seconds, then the smart TV can display the form of the digital assistant as a form with expressions and actions. Specifically, the form of the digital assistant can be a virtual human body representing the digital assistant looking at the user, blinking and smiling at the user, and the body and head swaying slightly, indicating that the digital assistant is willing to interact with the user. In addition, if there are multiple users in area 1, and multiple users have the above spatial interaction behaviors, 1.2 meters can be used as the dividing distance to control the virtual human body representing the digital assistant to look in the direction of different users in turn.

[0358] When the smart TV recognizes that the user is in area 2, the smart TV can consider the distance between the user and the digital assistant to be a greeting distance, and the spatial interaction behavior of the user obtained by the smart TV includes: the user faces the digital assistant, the user's gaze point is on the digital assistant, and the user gazes at the digital assistant for more than 3 seconds, then the smart TV can display the form of the digital assistant as a form with expressions and actions, and the smart TV outputs the corresponding voice of the digital assistant. Specifically, the form of the digital assistant can be a virtual human body representing the digital assistant waving to the user and greeting the user; in addition, the voice of the digital assistant output by the smart TV can be to inquire about the user's needs, for example, the voice of the digital assistant can be specifically "Excuse me, how can I help you?"

[0359] When the smart TV recognizes that the user is in area 3, the smart TV can consider that the distance between the user and the digital assistant is a manual touch screen operation distance, and the spatial interaction behavior of the user obtained by the smart TV includes: the user faces the digital assistant, the user's gaze point is on the digital assistant, and the user gazes at the digital assistant for more than 3 seconds, then the smart TV can display the task information that requires touch screen operation on the display screen, and proportionally reduce the main image of the digital assistant. Specifically, the projected height of the main image of the digital assistant on the display screen can be H = the actual height of the main image * 0.5 / 1.2. Among them, H is the projected height of the main image of the digital assistant on the screen, and the actual height of the main image of the digital assistant is the real height of the virtual human body representing the digital assistant. The real height of the male virtual human body can be, for example, 174cm, and the real height of the female virtual human body can be, for example, 161cm. The specific value can be adjusted according to the actual situation, and this embodiment does not make specific limitations on this.

[0360] The method provided in the embodiment of the present application is described in detail above. Next, the device provided in the embodiment of the present application for executing the above method will be introduced.

[0361] See also Fig.31 , Fig.31 A schematic diagram of the structure of a display device for a digital assistant virtual image provided in an embodiment of the present application. Fig.31 As shown, the display device of the digital assistant virtual image is applied to an electronic device, and the virtual image includes multiple feature bodies. The device includes:

[0362] Display module 3101, used for displaying a virtual image in a first form on a first interface;

[0363] The display module 3101 is further used to detect the signal instruction generated by the electronic device and display the virtual image in the second form;

[0364] Among them, the signal instruction is generated based on the first user input, the first form and the second form display at least one of the multiple feature bodies, and the change from the first form to the second form means that at least one feature body undergoes at least one of a shape change, a position change, a motion law change, and a display area change.

[0365] In a possible implementation, the plurality of feature bodies include a first feature body, and the shape change of the first feature body includes at least one of the following changes: enlargement, reduction, stretching, compression, rotation, and switching between a main form, a geometric array, or a strip.

[0366] In a possible implementation, the plurality of feature bodies further include a second feature body, wherein the main shape of the first feature body is a ring, the second feature body is a sphere, and the center of the ring and the center of the sphere are co-pointed;

[0367] The position change of the first feature body includes at least one of a change in orbital position after flipping around the sphere and a change in position caused by overall movement or rotation of the virtual image;

[0368] The motion law of the first characteristic body includes that the ring rotates around the sphere according to a preset trajectory with the center of the circle as the fulcrum;

[0369] The motion law of the second characteristic body includes that the sphere moves back and forth according to a preset law along a straight line passing through the center of the ring and perpendicular to the plane where the ring is located.

[0370] In a possible implementation, the display module 3101 is further configured to detect a second user input and display a third feature body on the first feature body, where the third feature body can move on the first feature body and indicate the user's position.

[0371] In a possible implementation, the virtual image has a breathing state, and the virtual image is displayed in a first form on the first interface, including:

[0372] If the first interface is a screen-off interface, the first form is a breathing form including a first feature body and a second feature body, and the first feature body is displayed in a main form.

[0373] In a possible implementation, when the first interface is a screen-off interface, the virtual image in a breathing state reciprocates on the screen according to a preset rule.

[0374] In a possible implementation, when the first interface is the screen-off interface and the first state is the breathing state, the display module 3101 is further used to:

[0375] A second user input is detected, and a third feature is displayed on the first feature, the third feature being movable on the first feature and indicating the user's position.

[0376] In a possible implementation, the virtual image has a waiting state and a listening state, and the first interface is a lock screen interface or any screen interface after unlocking;

[0377] Displaying a virtual image in a first form on a first interface includes:

[0378] The first state to be awakened is displayed on the first interface as the first form, the state to be awakened includes a first feature body, and the first feature body is displayed in a long strip as a navigation prompt line, and the navigation prompt line is used as a reference line for screen gesture navigation; wherein, when no input for waking up the digital assistant is received, the state to be awakened is permanently displayed on the first interface;

[0379] or,

[0380] A third user input is detected and displayed on the first interface in a listening state as a first form. The third user input is used to wake up the digital assistant. The listening state includes a first feature body and a second feature body, and the first feature body is displayed in a main form.

[0381] In a possible implementation, the first interface is a lock screen interface or any screen interface after unlocking, and the first state is a listening state;

[0382] Displaying the virtual image in the first form on the first interface specifically includes:

[0383] The first feature is first displayed as a geometric array;

[0384] The first feature body is then transformed from a geometric array into a main form, and a second feature body is displayed.

[0385] In a possible implementation, the first feature body is first displayed in a geometric array, specifically:

[0386] Displays the geometry of a portion of the array at the initial display position, and then displays the geometry on the entire screen.

[0387] In one possible implementation, the initial display position is determined based on user input;

[0388] And when no user input is detected, the initial position is the area where the virtual image is displayed.

[0389] In a possible implementation, the virtual image is displayed in the first form on the first interface, specifically:

[0390] Displaying a virtual image at a preset position on the first interface; or,

[0391] The virtual image is overlaid and displayed on a target content element in the first interface, wherein the target content element is a content element of a preset type.

[0392] In one possible implementation, when the virtual image is displayed over the target content element in the first interface, the gesture event interface of the target content element is copied to the virtual image so that performing a gesture on the virtual image can realize the gesture corresponding function of the target content element.

[0393] In a possible implementation, the first interface is a camera interface, and the target content element is a shutter control in the camera interface.

[0394] In one possible implementation, the first interface is the lock screen interface or any screen interface after unlocking, the first form is the wake-up state, and the second form is the listening state; the signal instruction is generated based on the user performing a preset gesture on the digital assistant in the first form.

[0395] In a possible implementation, the plurality of feature bodies further includes a second feature body;

[0396] In the second form, the first feature body is in the main form of a ring, the second feature body is a sphere, and the ring is located on the equator of the sphere in a top-down perspective.

[0397] In a possible implementation, the virtual image further has a task execution state, and the task execution state includes a first feature body;

[0398] In at least one process interface during which the digital assistant performs an automated task, the first feature body in the task execution state corresponds to at least one of a shape change and a position change.

[0399] In a possible implementation, the first feature body is a geometric array, and the display position of the geometric array matches the target control in the process interface, so as to simulate a click event of the target control to enter the next process interface.

[0400] In a possible implementation, the first state is a listening state, and the second state is a task execution state. Both the listening state and the task execution state include a first feature body, and the first feature bodies in the listening state and the task execution state have different forms respectively.

[0401] In the task execution state, the shape of the first feature body includes: a geometric array within the interface, an outer frame of a dialog card, an outer frame of an input box when calling a digital assistant input, or an outer frame of a task result page.

[0402] In a possible implementation, the first feature body has different morphological changes at different task progresses of the automation task.

[0403] In a possible implementation, when the automation task is an intelligent recognition task, the first feature body is changed from a geometric array covering the entire screen to a geometric array displayed only on the recognized executable object.

[0404] In a possible implementation, in the task execution state, the morphological change of the first feature body includes:

[0405] The first feature body is first displayed in the main form of a ring on the periphery of the second feature body;

[0406] The first feature body is then separated from the second feature body by the ring and is enlarged and displayed to serve as the outer frame of the conversation card;

[0407] The first feature body changes from the outer frame of the dialogue card to the geometric array inside the dialogue bubble;

[0408] After the automated task is completed, the avatar is displayed in a listening state.

[0409] In a possible implementation, the device further includes:

[0410] The acquisition module 3102 is used to acquire scene information and display a first feature body among multiple feature bodies in a target form based on the scene information.

[0411] In a possible implementation, the virtual image has a waiting-to-be-awakened state, the waiting-to-be-awakened state includes at least a first feature body, and the first feature body is displayed in a long strip;

[0412] The display module 3101 is further configured to display a second feature body in response to detecting that the focus of the human eye falls on the area where the screen is located.

[0413] In a possible implementation, the display module 3101 is further configured to, in response to detecting a human voice, cause the second feature body to move relative to the first feature body toward a direction close to a source of the human voice.

[0414] In a possible implementation, the electronic device is a smart car, and the display module 3101 is further used for:

[0415] The vehicle function or warning information is indicated by the deformation or movement of the first feature body among the multiple feature bodies.

[0416] See also Fig.32 , Fig.32 This is a schematic diagram of the structure of an execution device provided in an embodiment of the present application. The execution device 3200 can be specifically a mobile phone, a tablet, a laptop, a smart wearable device, a server, etc., which is not limited here. Specifically, the execution device 3200 includes: a receiver 3201, a transmitter 3202, a processor 3203 and a memory 3204 (wherein the number of processors 3203 in the execution device 3200 can be one or more, Fig.32 In the example of FIG. 3 , a processor 3203 may include an application processor 32031 and a communication processor 32032. In some embodiments of the present application, the receiver 3201, the transmitter 3202, the processor 3203 and the memory 3204 may be connected via a bus or other means.

[0417] The memory 3204 may include a read-only memory and a random access memory, and provides instructions and data to the processor 3203. A portion of the memory 3204 may also include a non-volatile random access memory (NVRAM). The memory 3204 stores processor and operation instructions, executable modules or data structures, or subsets thereof, or extended sets thereof, wherein the operation instructions may include various operation instructions for implementing various operations.

[0418] The processor 3203 controls the operation of the execution device. In a specific application, the various components of the execution device are coupled together through a bus system, wherein the bus system includes not only a data bus but also a power bus, a control bus, and a status signal bus, etc. However, for the sake of clarity, various buses are referred to as bus systems in the figure.

[0419] The method disclosed in the above embodiment of the present application can be applied to the processor 3203, or implemented by the processor 3203. The processor 3203 can be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor 3203 or an instruction in the form of software. The above processor 3203 can be a general-purpose processor, a digital signal processor (digital signal processing, DSP), a microprocessor or a microcontroller, and can further include an application specific integrated circuit (ASIC), a field-programmable gate array (field-programmable gate array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.

[0420] The processor 3203 can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software modules in the decoding processor for execution. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 3204, and the processor 3203 reads the information in the memory 3204 and completes the steps of the above method in combination with its hardware.

[0421] The receiver 3201 can be used to receive input digital or character information and generate signal input related to the relevant settings and function control of the execution device. The transmitter 3202 can be used to output digital or character information through the first interface; the transmitter 3202 can also be used to send instructions to the disk group through the first interface to modify the data in the disk group; the transmitter 3202 can also include a display device such as a display screen.

[0422] The electronic device provided in the embodiment of the present application may specifically be a chip, and the chip includes: a processing unit and a communication unit, the processing unit may be, for example, a processor, and the communication unit may be, for example, an input / output interface, a pin or a circuit, etc. The processing unit may execute the computer execution instructions stored in the storage unit so that the chip in the execution device executes the method described in the above embodiment. Optionally, the storage unit is a storage unit in the chip, such as a register, a cache, etc. The storage unit may also be a storage unit located outside the chip in the wireless access device, such as a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM), etc.

[0423] See also Fig.33 , Fig.33 This is a schematic diagram of the structure of a computer-readable storage medium provided in an embodiment of the present application. The present application also provides a computer-readable storage medium. In some embodiments, the above Figure 3 The disclosed methods may be implemented as computer program instructions encoded in a machine-readable format on a computer-readable storage medium or on other non-transitory media or articles of manufacture.

[0424] Fig.33 Schematically illustrates a conceptual partial view of an example computer-readable storage medium including a computer program for executing a computer process on a computing device, arranged in accordance with at least some embodiments presented herein.

[0425] In one embodiment, the computer readable storage medium 3300 is provided using a signal bearing medium 3301. The signal bearing medium 3301 may include one or more program instructions 3302, which when executed by one or more processors may provide the above Figure 3 Describes the functionality or part of the functionality.

[0426] In some examples, the signal bearing medium 3301 may include a computer readable medium 3303 such as, but not limited to, a hard drive, a compact disk (CD), a digital video disk (DVD), a digital tape, a memory, a ROM or RAM, and the like.

[0427] In some embodiments, the signal bearing medium 3301 may include a computer recordable medium 3304, such as, but not limited to, a memory, a read / write (R / W) CD, a R / W DVD, etc. In some embodiments, the signal bearing medium 3301 may include a communication medium 3305, such as, but not limited to, a digital and / or analog communication medium (e.g., a fiber optic cable, a waveguide, a wired communication link, a wireless communication link, etc.). Thus, for example, the signal bearing medium 3301 may be communicated by a wireless form of the communication medium 3305 (e.g., a wireless communication medium that complies with the IEEE 802.X standard or other transmission protocol).

[0428] The one or more program instructions 3302 may be, for example, computer executable instructions or logic implementing instructions. In some examples, the computing device of the computing device may be configured to provide various operations, functions, or actions in response to the program instructions 3302 communicated to the computing device via one or more of the computer readable medium 3303, the computer recordable medium 3304, and / or the communication medium 3305.

[0429] It should also be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

[0430] Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a training device, or a network device, etc.) to execute the method of each embodiment of the present application.

[0431] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0432] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, a computer, a training device or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, training device or data center. The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that contains one or more available media integration. Available media can be magnetic media, (e.g., floppy disk, hard disk, tape), optical media (e.g., DVD), or semiconductor media (e.g., solid-state hard disk (SSD)), etc.

Claims

1. A method for displaying a digital assistant virtual image, applied to an electronic device, characterized in that: The virtual image includes a plurality of features, and the method includes: Displaying the virtual image in a first form on a first interface; Detecting a signal instruction generated by the electronic device, and displaying the virtual image in a second form; Among them, the signal instruction is generated based on the first user input, the first form and the second form display at least one of the multiple feature bodies, and the change from the first form to the second form means that at least one feature body undergoes at least one of a shape change, a position change, a motion law change, and a display area change.

2. The method according to claim 1, characterized in that The plurality of features include a first feature, and the shape change of the first feature includes at least one of the following changes: enlargement, reduction, stretching, compression, rotation, and switching between a main form, a geometric array, or a strip.

3. The method according to claim 2, characterized in that The plurality of feature bodies further include a second feature body, wherein the main shape of the first feature body is a ring, the second feature body is a sphere, and the center of the ring and the center of the sphere are co-pointed; The position change of the first feature body includes at least one of a change in orbital position after flipping around the sphere and a position change caused by overall movement or rotation of the virtual image; The movement law of the first characteristic body includes that the ring rotates around the sphere according to a preset trajectory with the center of the circle as the fulcrum; The motion law of the second characteristic body includes that the sphere reciprocates along a straight line passing through the center of the circle and perpendicular to the plane where the circle is located according to a preset law.

4. The method according to claim 3, characterized in that The method further comprises: A second user input is detected, and a third feature is displayed on the first feature, wherein the third feature is movable on the first feature and indicates the user's position.

5. The method according to any one of claims 1 to 3, characterized in that: The virtual image has a breathing state, and displaying the virtual image in a first form on the first interface includes: If the first interface is a screen-off interface, the first form is a breathing form including a first feature body and a second feature body, and the first feature body is displayed in a main form.

6. The method according to claim 5, characterized in that When the first interface is a screen-off interface, the breathing virtual image moves back and forth on the screen according to a preset rule.

7. The method according to claim 5, characterized in that When the first interface is a screen-off interface and the first state is a breathing state, the method further includes: A second user input is detected, and a third feature is displayed on the first feature, wherein the third feature is movable on the first feature and indicates the user's position.

8. The method according to any one of claims 1 to 4, characterized in that: The virtual image has a waiting state and a listening state, and the first interface is a lock screen interface or any screen interface after unlocking; The step of displaying the virtual image in a first form on the first interface includes: The first interface is displayed in a waiting-to-be-awakened state as a first form, the waiting-to-be-awakened state includes a first feature body, and the first feature body is displayed in a long strip as a navigation prompt line, and the navigation prompt line is used as a reference line for screen gesture navigation; wherein, when no input for waking up the digital assistant is received, the waiting-to-be-awakened state is permanently displayed on the first interface; or, A third user input is detected and the listening state is displayed on the first interface as the first form. The third user input is used to wake up the digital assistant. The listening state includes a first feature body and a second feature body, and the first feature body is displayed in the main form.

9. The method according to claim 8, characterized in that The first interface is a lock screen interface or any screen interface after unlocking, and the first state is a listening state; The step of displaying the virtual image in a first form on the first interface specifically includes: The first feature is first displayed in a geometric array; The first feature body is then changed from a geometric array to a main form, and the second feature body is displayed.

10. The method according to claim 9, characterized in that The first feature is first displayed in a geometric array, specifically: The geometrical figures of the partial array are displayed at the initial display position, and then the geometrical figures are displayed on the entire screen.

11. The method according to claim 10, characterized in that The initial display position is determined based on user input; And when no user input is detected, the initial position is the area where the virtual image is displayed.

12. The method according to any one of claims 1 to 11, characterized in that: The displaying of the virtual image in the first form on the first interface is specifically: Displaying the virtual image at a preset position on the first interface; or, The virtual image is overlaid and displayed on a target content element in the first interface, wherein the target content element is a content element of a preset type.

13. The method according to claim 12, characterized in that When the virtual image is displayed overlaid on the target content element in the first interface, the gesture event interface of the target content element is copied to the virtual image, so that performing a gesture on the virtual image can realize the gesture corresponding function of the target content element.

14. The method according to claim 13, characterized in that The first interface is a camera interface, and the target content element is a shutter control in the camera interface.

15. The method according to claim 8, characterized in that The first interface is a lock screen interface or any screen interface after unlocking, the first form is a wake-up state, and the second form is a listening state; the signal instruction is generated after the user performs a preset gesture on the digital assistant in the first form.

16. The method according to claim 15, characterized in that The plurality of features further includes a second feature; In the second form, the first feature body is in the main form of a ring, the second feature body is a sphere, and the ring is located on the equator of the sphere in a top-down perspective.

17. The method according to claim 8, characterized in that The virtual image also has a task execution state, and the task execution state includes the first feature body; In at least one process interface during the execution of an automated task by the digital assistant, the first feature body in the task execution state corresponds to at least one of a shape change and a position change.

18. The method according to claim 17, characterized in that The first feature body is a geometric array, and the display position of the geometric array matches the target control in the process interface, which is used to simulate a click event of the target control to enter the next process interface.

19. The method according to claim 17, characterized in that The first state is a listening state, and the second state is a task execution state. Both the listening state and the task execution state include the first feature body, and the first feature bodies in the listening state and the task execution state have different states respectively. In the task execution state, the shape of the first feature body includes: a geometric array within the interface, an outer frame of a dialog card, an outer frame of an input box when calling a digital assistant input, or an outer frame of a task result page.

20. The method according to claim 19, characterized in that Under different task progress of the automation task, the first feature body has different morphological changes.

21. The method according to claim 19, characterized in that When the automation task is an intelligent recognition task, the first feature body changes from a geometric array covering the entire screen to a geometric array displayed only on the recognized executable object.

22. The method according to claim 19 or 20, characterized in that In the task execution state, the morphological changes of the first feature body include: The first feature body is first displayed in the main form of a ring on the periphery of the second feature body; The first feature body is then separated from the second feature body by a ring and is enlarged and displayed to serve as an outer frame of the conversation card; The first feature body changes from the outer frame of the dialogue card to a geometric array within the dialogue bubble; After the automated task is completed, the virtual image is displayed in a listening state.

23. The method according to claims 1-4, characterized in that The method further comprises: Scene information is acquired, and a first feature body among the plurality of feature bodies is displayed in a target form based on the scene information.

24. The method according to claims 1-4, characterized in that The virtual image has a waiting state, and the waiting state includes at least a first characteristic body, and the first characteristic body is displayed in a long strip; In response to detecting that the focus of the human eye falls on the area where the screen is located, a second feature body is displayed.

25. The method according to claim 24, characterized in that The method further comprises: In response to detecting a human voice, the second feature body moves relative to the first feature body toward a direction approaching a source of the human voice.

26. The method according to claims 1-4, characterized in that The electronic device is a smart car, and the method further includes: The vehicle function or warning information is indicated by the deformation or movement of the first feature body among the multiple feature bodies.

27. A display device for a digital assistant virtual image, applied to an electronic device, characterized in that: The virtual image includes a plurality of features; the device includes: A display module, configured to display the virtual image in a first form on a first interface; The display module is further configured to detect a signal instruction generated by the electronic device and display the virtual image in a second form; Among them, the signal instruction is generated based on the first user input, the first form and the second form display at least one of the multiple feature bodies, and the change from the first form to the second form means that at least one feature body undergoes at least one of a shape change, a position change, a motion law change, and a display area change.

28. The device according to claim 27, characterized in that The plurality of features include a first feature, and the shape change of the first feature includes at least one of the following changes: enlargement, reduction, stretching, compression, rotation, and switching between a main form, a geometric array, or a strip.

29. The device according to claim 28, characterized in that The plurality of feature bodies further include a second feature body, wherein the main shape of the first feature body is a ring, the second feature body is a sphere, and the center of the ring and the center of the sphere are co-pointed; The position change of the first feature body includes at least one of a change in orbital position after flipping around the sphere and a position change caused by overall movement or rotation of the virtual image; The movement law of the first characteristic body includes that the ring rotates around the sphere according to a preset trajectory with the center of the circle as the fulcrum; The motion law of the second characteristic body includes that the sphere reciprocates along a straight line passing through the center of the circle and perpendicular to the plane where the circle is located according to a preset law.

30. The device according to claim 29, characterized in that The display module is further configured to detect a second user input and display a third feature body on the first feature body, wherein the third feature body can move on the first feature body and indicate the user's position.

31. The device according to any one of claims 27 to 29, characterized in that: The virtual image has a breathing state, and displaying the virtual image in a first form on the first interface includes: If the first interface is a screen-off interface, the first form is a breathing form including a first feature body and a second feature body, and the first feature body is displayed in a main form.

32. The device according to claim 31, characterized in that When the first interface is a screen-off interface, the breathing virtual image moves back and forth on the screen according to a preset rule.

33. The device according to claim 31, characterized in that When the first interface is a screen-off interface and the first state is a breathing state, the display module is further used to: A second user input is detected, and a third feature is displayed on the first feature, wherein the third feature is movable on the first feature and indicates the user's position.

34. The device according to any one of claims 27 to 30, characterized in that: The virtual image has a waiting state and a listening state, and the first interface is a lock screen interface or any screen interface after unlocking; The step of displaying the virtual image in a first form on the first interface includes: The first interface is displayed in a waiting-to-be-awakened state as a first form, the waiting-to-be-awakened state includes a first feature body, and the first feature body is displayed in a long strip as a navigation prompt line, and the navigation prompt line is used as a reference line for screen gesture navigation; wherein, when no input for waking up the digital assistant is received, the waiting-to-be-awakened state is permanently displayed on the first interface; or, A third user input is detected and the listening state is displayed on the first interface as the first form. The third user input is used to wake up the digital assistant. The listening state includes a first feature body and a second feature body, and the first feature body is displayed in the main form.

35. The device according to claim 34, characterized in that The first interface is a lock screen interface or any screen interface after unlocking, and the first state is a listening state; The step of displaying the virtual image in a first form on the first interface specifically includes: The first feature is first displayed in a geometric array; The first feature body is then changed from a geometric array to a main form, and the second feature body is displayed.

36. The device according to claim 35, characterized in that The first feature is first displayed in a geometric array, specifically: The geometrical figures of the partial array are displayed at the initial display position, and then the geometrical figures are displayed on the entire screen.

37. The device according to claim 36, characterized in that The initial display position is determined based on user input; And when no user input is detected, the initial position is the area where the virtual image is displayed.

38. The device according to any one of claims 27 to 37, characterized in that: The displaying of the virtual image in the first form on the first interface is specifically: Displaying the virtual image at a preset position on the first interface; or, The virtual image is overlaid and displayed on a target content element in the first interface, wherein the target content element is a content element of a preset type.

39. The device according to claim 38, characterized in that When the virtual image is displayed overlaid on the target content element in the first interface, the gesture event interface of the target content element is copied to the virtual image, so that performing a gesture on the virtual image can realize the gesture corresponding function of the target content element.

40. The device according to claim 39, characterized in that The first interface is a camera interface, and the target content element is a shutter control in the camera interface.

41. The device according to claim 34, characterized in that The first interface is a lock screen interface or any screen interface after unlocking, the first form is a wake-up state, and the second form is a listening state; the signal instruction is generated after the user performs a preset gesture on the digital assistant in the first form.

42. The device according to claim 39, characterized in that The plurality of features further includes a second feature; In the second form, the first feature body is in the main form of a ring, the second feature body is a sphere, and the ring is located on the equator of the sphere in a top-down perspective.

43. The device according to claim 34, characterized in that The virtual image also has a task execution state, and the task execution state includes the first feature body; In at least one process interface during the execution of an automated task by the digital assistant, the first feature body in the task execution state corresponds to at least one of a shape change and a position change.

44. The device according to claim 43, characterized in that The first feature body is a geometric array, and the display position of the geometric array matches the target control in the process interface, which is used to simulate a click event of the target control to enter the next process interface.

45. The device according to claim 43, characterized in that The first state is a listening state, and the second state is a task execution state. Both the listening state and the task execution state include the first feature body, and the first feature bodies in the listening state and the task execution state have different states respectively. In the task execution state, the shape of the first feature body includes: a geometric array within the interface, an outer frame of a dialog card, an outer frame of an input box when calling a digital assistant input, or an outer frame of a task result page.

46. ​​The device according to claim 45, characterized in that Under different task progress of the automation task, the first feature body has different morphological changes.

47. The device according to claim 46, characterized in that When the automation task is an intelligent recognition task, the first feature body changes from a geometric array covering the entire screen to a geometric array displayed only on the recognized executable object.

48. The device according to claim 45 or 46, characterized in that In the task execution state, the morphological changes of the first feature body include: The first feature body is first displayed in the main form of a ring on the periphery of the second feature body; The first feature body is then separated from the second feature body by a ring and is enlarged and displayed to serve as an outer frame of the conversation card; The first feature body changes from the outer frame of the dialogue card to a geometric array within the dialogue bubble; After the automated task is completed, the virtual image is displayed in a listening state.

49. The device according to claims 27-30, characterized in that The device also includes: The acquisition module is used to acquire scene information and display a first feature body among the multiple feature bodies in a target form based on the scene information.

50. The device according to claims 27-30, characterized in that The virtual image has a waiting state, and the waiting state includes at least a first characteristic body, and the first characteristic body is displayed in a long strip; The display module is further configured to display a second feature body in response to detecting that the focus of a person's sight falls on the area where the screen is located.

51. The device according to claim 50, characterized in that The display module is further configured to cause the second feature body to move toward a direction close to a sound source of the human voice relative to the first feature body in response to detecting the human voice.

52. The device according to claims 27-30, characterized in that The electronic device is a smart car, and the display module is also used for: The vehicle function or warning information is indicated by the deformation or movement of the first feature body among the multiple feature bodies.

53. A display device for a digital assistant virtual image, characterized in that: The device comprises a memory and a processor; the memory stores codes, the processor is configured to execute the codes, and when the codes are executed, the device executes the method according to any one of claims 1 to 26.

54. A computer storage medium, characterized in that The computer storage medium stores instructions, which, when executed by a computer, cause the computer to implement the method of any one of claims 1 to 26.

55. A computer program product, characterized in that The computer program product stores instructions, which, when executed by a computer, cause the computer to implement the method according to any one of claims 1 to 26.

Images