Method for displaying avatar of digital assistant and related device

Abstract

A method for displaying an avatar of a digital assistant, which can improve the naturalness of interaction between users and digital assistants. In the method, an avatar of a digital assistant is configured to comprise a plurality of features, and when the electronic device obtains user input, the electronic device generates a corresponding signal instruction, so as to trigger a change in the form of the avatar displayed on the interface. Thus, different working states of a digital assistant are represented in different forms, so that a user can determine the current state of interaction with the digital assistant by means of a change in the form of the digital assistant, thereby improving the naturalness of interaction between users and digital assistants.

Description

A method for displaying a digital assistant virtual image and related devices

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on June 29, 2023, with application number 202310786258.8 and invention name “A method for displaying a digital assistant and related devices”, and the Chinese patent application filed with the State Intellectual Property Office on February 8, 2024, with application number 202410178141.6 and invention name “A method for displaying a virtual image of a digital assistant and related devices”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of human-computer interaction technology, and in particular to a method for displaying a digital assistant virtual image and related devices. Background Art

[0003] With the rapid development of smart devices, configuring digital assistants on smart devices has gradually become mainstream. By replacing manual operations with digital assistants, a series of smart device operations can be automatically completed, freeing users' hands and improving the user experience of smart devices.

[0004] Currently, when using digital assistants, users usually need to wake up the digital assistant through voice commands or manual operations to enter the digital assistant user interface. The digital assistant user interface displays an assistant image, which is usually a static or continuously moving two-dimensional pattern.

[0005] In the user interface of the digital assistant, after the user further inputs a voice command or a gesture command, it is usually necessary to wait for the digital assistant to output the recognition result before the digital assistant can confirm whether the voice command or the gesture command has been received. After that, the digital assistant's form does not change during the process of recognizing or processing the command. Therefore, after the user inputs a voice command or a gesture command, it is usually necessary to wait for the digital assistant to output the recognition result before the digital assistant can confirm whether the voice command or the gesture command has been received. That is, when the digital assistant recognizes or processes the command input by the user, the user cannot get feedback from the digital assistant, and it is difficult to identify the current working status of the digital assistant. It is easy to mistakenly believe that the digital assistant has not received the command, which causes the user to repeat the command.

[0006] Summary of the Invention

[0007] The present application provides a method for displaying a digital assistant virtual image, which can improve the naturalness of the interaction between the user and the digital assistant.

[0008] The first aspect of the present application provides a method for displaying a digital assistant virtual image, which is applied to an electronic device, and the virtual image of the digital assistant includes multiple features. In this method, the virtual image is first displayed in a first form on a first interface; then, a signal instruction generated by the electronic device is detected, and the virtual image is displayed in a second form. The signal instruction is generated by the electronic device based on the received first user input, and the user input can be before or after the first form is displayed. That is, after the electronic device receives the first user input, it generates a signal instruction, thereby triggering the virtual image to switch from the first form to the second form, so that the form of the virtual image displayed on the interface of the electronic device changes.

[0009] In addition, the first form and the second form both display at least one of a plurality of characteristic bodies, and the change from the first form to the second form displays that at least one characteristic body undergoes at least one of a change in shape, position, movement pattern, and display area.

[0010] The different forms of the avatar represent the different working states of the digital assistant. That is, the digital assistant uses the form of the avatar to indicate its current working state. In this way, when the form of the avatar displayed on the interface changes, the user can intuitively and clearly understand the changes in the working state of the digital assistant and understand the current response of the digital assistant to the user's instructions.

[0011] In this solution, the virtual image of the digital assistant is set to include multiple features, and after the electronic device obtains user input, the electronic device generates a corresponding signal instruction, thereby triggering the change in the form of the virtual image displayed on the interface, thereby realizing the use of different forms to represent the different working states of the digital assistant, so that the user can confirm the current state of interaction with the digital assistant through the changes in the form of the digital assistant, thereby improving the naturalness of the interaction between the user and the digital assistant.

[0012] In one possible implementation, the plurality of features include a first feature, and the plurality of features include the first feature. The shape change of the first feature includes at least one of the following: enlargement, reduction, stretching, compression, rotation, and switching between a primary form, a geometric array, or a strip. That is, the shape of the first feature includes: a primary form, a geometric array, and a strip. Furthermore, the shape change of the first feature includes switching between multiple shapes, as well as enlargement, reduction, stretching, compression, or rotation of each shape. That is, in addition to switching between different shapes, the shape change of the first feature may also include enlargement, reduction, stretching, compression, or rotation within the same shape.

[0013] The main form refers to the form of the first feature body under normal circumstances, while the geometric array and the long strip are the forms of the first feature body under certain specific circumstances.

[0014] In one possible implementation, the plurality of feature bodies further includes a second feature body. The main form of the first feature body is a ring, and the second feature body is a sphere, with the center of the ring and the center of the sphere converging. The position change of the first feature body includes at least one of a change in orbital position after flipping around the sphere and a position change caused by the overall movement or rotation of the virtual image. The motion law of the first feature body includes the ring rotating around the sphere with its center as a fulcrum according to a preset trajectory. The motion law of the second feature body includes the sphere moving back and forth according to a preset law along a straight line passing through the center of the ring and perpendicular to the plane of the ring.

[0015] In a possible implementation, when the second user input is detected, a third feature is displayed on the first feature, and the third feature can move on the first feature and indicate the user's position.

[0016] In one possible implementation, the virtual image has a breathing state, and the virtual image is displayed in a first form on the first interface, including: if the first interface is an off-screen interface, the first form is a breathing state including a first feature body and a second feature body, and the first feature body is displayed in the main form.

[0017] In one possible implementation, the first interface is a screen-off interface, and the digital assistant in a breathing state moves back and forth on the screen according to a preset pattern.

[0018] In one possible implementation, when the first interface is the off-screen interface and the first state is the breathing state, in response to detecting a second user input, a third feature is displayed on the first feature. The third feature can move on the first feature and indicate the user's position. The first feature is a ring, and the third feature can be a light effect area on the ring.

[0019] In one possible implementation, the avatar has a waiting state and a listening state, and the first interface is the lock screen or any unlocked screen. Displaying the avatar in a first form on the first interface specifically includes: displaying the avatar in the waiting state as the first form on the first interface, the waiting state including a first feature body, and the first feature body being displayed as a long strip as a navigation prompt line, the navigation prompt line being used as a reference line for screen gesture navigation; wherein, when no input to wake up the digital assistant is received, the waiting state is permanently displayed on the first interface.

[0020] Alternatively, a third user input is detected and displayed on the first interface in a listening state as the first form. The third user input is used to wake up the digital assistant. The listening state includes a first feature body and a second feature body, and the first feature body is displayed in the main form.

[0021] In one possible implementation, the first interface is the lock screen or any unlocked screen, and the first state is the listening state. Displaying the avatar in the first state on the first interface specifically includes: first displaying the first feature in a geometric array; then, the first feature changes from the geometric array to the primary state, and the second feature appears. Specifically, initially, only the first feature is displayed, and the second feature is not displayed. The first feature is initially displayed in a geometric array, filling the entire screen; then, the first feature changes from the geometric array to the primary state, and the second feature appears.

[0022] That is to say, when waking up the digital assistant, the digital assistant first appears in the form of a geometric array (such as a dot matrix) and gradually fills the screen; then, the digital assistant gradually transforms from the geometric array to the main form of the first feature body (i.e., a ring) and displays the second feature body (i.e., a sphere), thereby realizing the awakening process of the digital assistant in the form of animation, increasing the intuitiveness of the interaction process, and helping users to identify the current working status of the digital assistant.

[0023] In one possible implementation, the first feature is first displayed as a geometric array. Specifically, a portion of the array is displayed at an initial display position, and then the geometric array is displayed across the entire screen. Specifically, the initial display position of the geometric array is first determined. Then, a portion of the array is displayed at the initial display position, and then the geometric array is gradually spread across the entire screen. For example, a portion of the array is displayed starting at an initial display position, and then the array is gradually spread across the entire screen. User input refers to the input used to wake up the digital assistant.

[0024] In one possible implementation, the initial display position may be near the user input that triggers the digital assistant to wake up. For example, when the user input is a long press of the power button on a smartphone, the initial display position of the geometric array may be on the screen near the power button.

[0025] Furthermore, when no user input is detected, the initial position is a default position, which is an area where the virtual image is displayed, such as the bottom of the screen.

[0026] In this solution, the initial display position of the geometric array is determined by user input, which can make the display process of the digital assistant more consistent with the user input, enhance the intuitiveness of the user's interaction with the digital assistant, and improve the user experience.

[0027] In one possible implementation, the first interface is the lock screen interface or any screen interface after unlocking, the first form is the wake-up state, and the second form is the listening state; the signal instruction is generated based on the user performing a preset gesture on the digital assistant in the first form.

[0028] In a possible implementation, the plurality of feature bodies further includes a second feature body; in the second form, the first feature body is in the main form of a ring, the second feature body is a sphere, and the ring is located on the equator of the sphere in a top-down perspective.

[0029] In one possible implementation, the virtual image is displayed in a first form on the first interface, specifically: the virtual image is displayed at a preset position on the first interface; or the virtual image is displayed overlaid on a target content element in the first interface, wherein the target content element is a content element of a preset type.

[0030] That is, the virtual image is covered on the target content element, so that the target content element is no longer displayed on the first interface.

[0031] In one possible implementation, when the virtual image is displayed overlaid on the target content element in the first interface, the gesture event interface of the target content element is copied to the virtual image so that performing a gesture on the virtual image can realize the gesture corresponding function of the target content element.

[0032] That is to say, even if the target content element is covered by the virtual image, the virtual image inherits the original function of the target content element, so that when a specific gesture (such as single-click) is performed on the virtual image, the gesture corresponding function of the target content element can still be executed.

[0033] In a possible implementation, the first interface is a camera interface, and the target content element is a shutter control in the camera interface.

[0034] In one possible implementation, the virtual image also has a task execution state, which includes a first feature body; in at least one process interface during the digital assistant's execution of an automated task, the first feature body in the task execution state corresponds to at least one of a shape change and a position change.

[0035] Specifically, in the process interface of the digital assistant performing automated tasks, the digital assistant is actually always in the task execution state, and the first feature body of the digital assistant in the task execution state can undergo various changes to show the user the automated process of the digital assistant performing tasks, making the automated process of the digital assistant performing tasks more intuitive and clear, and making it easier for users to observe whether there are any errors in the process of the digital assistant performing automated tasks.

[0036] In one possible implementation, the first feature is a geometric array, and the display position of the geometric array matches the target control within the process interface. By simulating a click event on the target control, the user enters the next process interface. In other words, during the execution of the automated task, the digital assistant can use the first feature, which is the geometric array, to simulate the user clicking on the control on the interface, thereby visualizing the entire process of the digital assistant simulating the user executing the automated task.

[0037] In one possible implementation, the first form is the listening state, and the second form is the task execution state. Both the listening state and the task execution state include a first feature body, and the first feature body in the listening state and the task execution state has different forms respectively; in the task execution state, the form of the first feature body includes: a geometric array within the interface, the outer frame of the digital assistant dialogue card, the outer frame of the input box when calling the digital assistant input, or the outer frame of the task result page.

[0038] That is, in different scenarios, the first feature body may be represented in different forms so as to be more consistent with the content displayed on the current interface.

[0039] In one possible implementation, the first feature body may have different morphological changes at different progress points in the automation task. That is, as the digital assistant executes the automation task, the first feature body may change to different morphologies at different task progress points to ensure that the morphology of the first feature body always closely matches the current interface.

[0040] In a possible implementation, when the automation task is an intelligent recognition task, the first feature body is changed from a geometric array covering the entire screen to a geometric array displayed only on the recognized executable object.

[0041] In one possible implementation, corresponding to the execution process of the automated task, in the task execution state, the morphological changes of the first feature body include: initially, the first feature body is displayed in the main form of a ring at an initial position relative to the second feature body (for example, the periphery of the second feature body), and the initial position is the same as or different from the position of the first feature body in the listening state; the first feature body is separated from the second feature body by the ring and is enlarged and displayed as the outer frame of the digital assistant dialogue card; the first feature body changes from the outer frame of the dialogue card to a geometric array within the digital assistant dialogue bubble; after the execution of the automated task is completed, the virtual image is displayed in the listening state.

[0042] In one possible implementation, the method may further include: the electronic device acquiring scene information, determining a target form for the first feature based on the scene information, and displaying the first feature in the target form. That is, the target form is related to the scene information. When the electronic device acquires specific scene information, the electronic device may display the first feature in the specific form.

[0043] In one possible implementation, the virtual image has a state to be awakened, which includes at least a first feature body, and the first feature body is displayed in a long strip; in response to detecting that the focus of the human eye falls on the area where the screen is located, the electronic device displays a second feature body.

[0044] That is to say, in some scenarios, when the digital assistant is in the wake-up state, only a long bar is displayed; when the digital assistant detects that the user's eyes fall on the screen, it can be considered that the user has an interactive tendency, so the second feature body is displayed, thereby indicating that the digital assistant detects that the user has an interactive tendency and is waiting for interaction, so as to improve the naturalness of human-computer interaction.

[0045] In a possible implementation, the method further includes: in response to detecting a human voice, the second feature body moves relative to the first feature body in a direction close to a source of the human voice.

[0046] In a possible implementation, the electronic device is a smart car, and the method further includes: indicating an in-vehicle function or warning information through deformation or movement of a first feature body among the multiple feature bodies.

[0047] A second aspect of the present application provides a display device for a digital assistant virtual image, which is applied to an electronic device. The virtual image includes multiple features. The device includes:

[0048] A display module, configured to display a virtual image in a first form on a first interface;

[0049] The display module is further configured to detect a signal instruction generated by the electronic device and display the virtual image in a second form;

[0050] Among them, the signal instruction is generated based on the first user input, the first form and the second form display at least one of multiple feature bodies, and the change from the first form to the second form means that at least one feature body undergoes at least one change in shape, position, movement pattern, and display area.

[0051] In one possible implementation, the plurality of features include a first feature, and the shape change of the first feature includes at least one of the following changes: enlargement, reduction, stretching, compression, rotation, and switching between a main form, a geometric array, or a strip.

[0052] In a possible implementation, the plurality of feature bodies further includes a second feature body, wherein the main shape of the first feature body is a ring, the second feature body is a sphere, and the center of the ring and the center of the sphere are concentric;

[0053] The position change of the first feature body includes at least one of a change in orbital position after flipping around the sphere and a position change caused by overall movement or rotation of the virtual image;

[0054] The motion law of the first characteristic body includes that the ring rotates around the sphere according to a preset trajectory with the center of the ring as the fulcrum;

[0055] The motion law of the second characteristic body includes that the sphere moves back and forth according to a preset law along a straight line passing through the center of the ring and perpendicular to the plane where the ring is located.

[0056] In a possible implementation, the display module is further configured to detect a second user input and display a third feature body on the first feature body, where the third feature body can move on the first feature body and indicate the user's position.

[0057] In a possible implementation, the avatar has a breathing state, and displaying the avatar in a first form on the first interface includes:

[0058] If the first interface is a screen-off interface, the first form is a breathing form including a first feature body and a second feature body, and the first feature body is displayed in the main form.

[0059] In a possible implementation, when the first interface is a screen-off interface, the virtual image in a breathing state moves back and forth on the screen according to a preset pattern.

[0060] In a possible implementation, when the first interface is the screen-off interface and the first state is the breathing state, the display module is further configured to:

[0061] A second user input is detected, and a third feature is displayed on the first feature. The third feature is movable on the first feature and indicates the user's position.

[0062] In one possible implementation, the avatar has a waiting state and a listening state, and the first interface is a lock screen interface or any screen interface after unlocking;

[0063] Displaying the virtual image in a first form on the first interface includes:

[0064] The first state is displayed on the first interface in a waiting-to-wake-up state as a first form, the waiting-to-wake-up state includes a first feature body, and the first feature body is displayed in a long strip as a navigation prompt line, and the navigation prompt line is used as a reference line for screen gesture navigation; wherein, when no input for waking up the digital assistant is received, the waiting-to-wake-up state is permanently displayed on the first interface;

[0065] or,

[0066] A third user input is detected and displayed on the first interface in a listening state as the first form. The third user input is used to wake up the digital assistant. The listening state includes a first feature body and a second feature body, and the first feature body is displayed in the main form.

[0067] In one possible implementation, the first interface is a lock screen interface or any screen interface after unlocking, and the first state is a listening state;

[0068] Displaying the virtual image in the first form on the first interface specifically includes:

[0069] The first feature is first displayed as a geometric array;

[0070] The first feature body then changes from the geometric array to the main form, and the second feature body is displayed.

[0071] In one possible implementation, the first feature is first displayed as a geometric array, specifically:

[0072] Displays a portion of the array's geometry at the initial display position, and then displays the geometry on the entire screen.

[0073] In one possible implementation, the initial display position is determined based on user input;

[0074] And when no user input is detected, the initial position is the area where the virtual image is displayed.

[0075] In a possible implementation, the virtual image is displayed in the first form on the first interface, specifically:

[0076] Displaying a virtual image at a preset position on the first interface; or,

[0077] The virtual image is overlaid and displayed on a target content element in the first interface, wherein the target content element is a content element of a preset type.

[0078] In one possible implementation, when the virtual image is displayed overlaid on the target content element in the first interface, the gesture event interface of the target content element is copied to the virtual image so that performing a gesture on the virtual image can realize the gesture corresponding function of the target content element.

[0079] In a possible implementation, the first interface is a camera interface, and the target content element is a shutter control in the camera interface.

[0080] In one possible implementation, the first interface is the lock screen interface or any screen interface after unlocking, the first form is the wake-up state, and the second form is the listening state; the signal instruction is generated based on the user performing a preset gesture on the digital assistant in the first form.

[0081] In a possible implementation, the plurality of features further includes a second feature;

[0082] In the second form, the first feature body is in the main form of a ring, the second feature body is a sphere, and the ring is located on the equator of the sphere in a top-down perspective.

[0083] In a possible implementation, the virtual image further has a task execution state, and the task execution state includes a first feature body;

[0084] In at least one process interface during the digital assistant's execution of an automated task, the first feature body in the task execution state produces at least one of a shape change and a position change.

[0085] In a possible implementation, the first feature body is a geometric array, and the display position of the geometric array matches the target control in the process interface, so as to simulate a click event of the target control to enter the next process interface.

[0086] In a possible implementation, the first state is a listening state, and the second state is a task execution state. Both the listening state and the task execution state include a first feature body, and the first feature bodies in the listening state and the task execution state have different forms.

[0087] In the task execution state, the form of the first feature body includes: a geometric array within the interface, the outer frame of a dialogue card, the outer frame of an input box when calling a digital assistant input, or the outer frame of a task result page.

[0088] In a possible implementation, the first feature body has different morphological changes at different task progresses of the automation task.

[0089] In a possible implementation, when the automation task is an intelligent recognition task, the first feature body is changed from a geometric array covering the entire screen to a geometric array displayed only on the recognized executable object.

[0090] In a possible implementation, in the task execution state, the morphological change of the first feature body includes:

[0091] The first feature body is first displayed in the main form of a ring on the periphery of the second feature body;

[0092] The first feature body is then separated from the second feature body by the ring and is enlarged and displayed to serve as the outer frame of the conversation card;

[0093] The first feature body changes from the outer frame of the dialogue card to the geometric array inside the dialogue bubble;

[0094] After the automated task is completed, the avatar is displayed in a listening state.

[0095] In a possible implementation, the apparatus further includes:

[0096] The acquisition module is used to acquire scene information and display a first feature body among the multiple feature bodies in a target form based on the scene information.

[0097] In a possible implementation, the virtual image has a waiting-to-awaken state, the waiting-to-awaken state includes at least a first characteristic body, and the first characteristic body is displayed in a long strip;

[0098] The display module is further configured to display a second feature body in response to detecting that the focus of a person's sight falls on the area where the screen is located.

[0099] In a possible implementation, the display module is further configured to, in response to detecting a human voice, cause the second feature body to move relative to the first feature body toward a direction closer to a source of the human voice.

[0100] In one possible implementation, the electronic device is a smart car, and the display module is further used to:

[0101] The vehicle function or warning information is indicated by the deformation or movement of the first feature body among the multiple feature bodies.

[0102] In a third aspect, the present application provides a display device for a digital assistant virtual image, which may include a processor coupled to a memory, wherein the memory stores program instructions, and when the program instructions stored in the memory are executed by the processor, the method of the first aspect or any implementation of the first aspect is implemented. For the steps in each possible implementation of the first aspect executed by the processor, please refer to the first aspect for details, and will not be repeated here.

[0103] In a fourth aspect, the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer-readable storage medium is run on a computer, the computer executes the method of any implementation of the first aspect.

[0104] A fifth aspect of the present application provides a circuit system, the circuit system including a processing circuit, and the processing circuit is configured to execute a method of any implementation manner of the above-mentioned first aspect.

[0105] In a sixth aspect, the present application provides a computer program product, which, when executed on a computer, enables the computer to execute the method of any implementation manner of the first aspect.

[0106] In a seventh aspect, the present application provides a chip system, which includes a processor for supporting a server or a threshold value acquisition device to implement the functions involved in any implementation of the first aspect, for example, sending or processing the data and / or information involved in the above method. In one possible design, the chip system also includes a memory for storing program instructions and data necessary for the server or communication device. The chip system can be composed of a chip or can include a chip and other discrete devices.

[0107] The beneficial effects of the second to seventh aspects mentioned above can be referred to the introduction of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0108] FIG1 is a schematic diagram of an architecture of an application scenario provided by an embodiment of the present application;

[0109] FIG2 is a schematic structural diagram of an electronic device 101 provided in an embodiment of the present application;

[0110] FIG3 is a flow chart of a method for displaying a digital assistant virtual image according to an embodiment of the present application;

[0111] FIG4 is a schematic diagram of a main image of a digital assistant provided in an embodiment of the present application;

[0112] FIG5 is a schematic diagram of a movement pattern of a digital assistant provided by an embodiment of the present application when the digital assistant is in a breathing state;

[0113] FIG6 is a schematic diagram of a digital assistant provided by an embodiment of the present application switching from a breathing state to a waiting-to-wake-up state;

[0114] FIG7 is a schematic diagram of a digital assistant provided by an embodiment of the present application in an awake state;

[0115] FIG8 is a schematic diagram of a digital assistant provided by an embodiment of the present application in a listening state;

[0116] FIG9 is a schematic diagram of a digital assistant provided by an embodiment of the present application in a thinking state;

[0117] FIG10 is a schematic diagram of a digital assistant provided by an embodiment of the present application in a task execution state;

[0118] FIG11 is a schematic diagram of various forms of a digital assistant provided in an embodiment of the present application;

[0119] FIG12 is a schematic diagram of a digital assistant provided by an embodiment of the present application displayed in a waiting-to-wake-up state;

[0120] FIG13 is a schematic diagram of a digital assistant switching from a waiting state to a listening state according to an embodiment of the present application;

[0121] FIG14 is a schematic diagram of a virtual image overlaid on a camera shutter control according to an embodiment of the present application;

[0122] FIG15 is a schematic diagram of a virtual image overlaid on an assistant suggestion component provided by an embodiment of the present application;

[0123] FIG16 is a schematic diagram of a process of a digital assistant performing an automated task provided by an embodiment of the present application;

[0124] FIG17 is a schematic diagram of a digital assistant provided by an embodiment of the present application, wherein a circular ring is converted into a dot matrix and laid out on a task execution interface;

[0125] FIG18 is a schematic diagram of using a digital assistant to input text on a desktop according to an embodiment of the present application;

[0126] FIG19 is a schematic diagram of a ring of a digital assistant provided by an embodiment of the present application deforming and then returning to its original shape;

[0127] FIG20 is a schematic diagram of the morphological changes of a digital assistant in a smart recognition scenario provided by an embodiment of the present application;

[0128] FIG21 is a schematic diagram showing the appearance of a digital assistant in a smart recognition scenario provided by an embodiment of the present application;

[0129] FIG22 is a schematic diagram of a digital assistant deforming based on scene information provided by an embodiment of the present application;

[0130] FIG23 is a schematic diagram showing changes in a digital assistant on a vehicle-mounted terminal provided by an embodiment of the present application;

[0131] FIG24 is a flow chart of a display method of a digital assistant provided in an embodiment of the present application;

[0132] FIG25 is a schematic diagram of an application scenario of a display method for a digital assistant provided in an embodiment of the present application;

[0133] FIG26 is a schematic diagram of a digital assistant in a dormant state provided by an embodiment of the present application;

[0134] FIG27 is a schematic diagram of a region division provided in an embodiment of the present application;

[0135] FIG28 is a schematic diagram of switching the display form of a digital assistant provided by an embodiment of the present application;

[0136] FIG29 is a schematic diagram showing changes in a digital assistant in an awake state provided by an embodiment of the present application;

[0137] FIG30 is a schematic diagram showing changes in the wake-up state of another digital assistant provided by an embodiment of the present application;

[0138] FIG31 is a schematic diagram of the structure of a display device for a digital assistant virtual image provided by an embodiment of the present application;

[0139] FIG32 is a schematic diagram of a structure of an execution device provided in an embodiment of the present application;

[0140] Figure 33 is a structural diagram of a computer-readable storage medium provided in an embodiment of the present application. DETAILED DESCRIPTION

[0141] In order to make the purpose, technical solutions and advantages of this application more clear, the embodiments of this application are described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only embodiments of a part of this application, rather than all embodiments. It is known to those skilled in the art that with the emergence of new application scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0142] The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the descriptions used in this way can be interchangeable where appropriate so that the embodiments can be implemented in a sequence other than that illustrated or described in this application. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or modules is not necessarily limited to those steps or modules clearly listed, but may include other steps or modules that are not clearly listed or that are inherent to these processes, methods, products or devices. The naming or numbering of steps in this application does not mean that the steps in the method flow must be executed in the time / logical sequence indicated by the naming or numbering. The named or numbered process steps can change the execution order according to the technical purpose to be achieved, as long as the same or similar technical effects can be achieved.

[0143] The division of units in this application is a logical division. In actual application, there may be other division methods. For example, multiple units can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between each other shown or discussed can be through some interfaces, and the indirect coupling or communication connection between units can be electrical or other similar forms, which are not limited in this application. Moreover, the units or sub-units described as separate components may or may not be physically separated, may or may not be physical units, or may be distributed into multiple circuit units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this application.

[0144] To facilitate understanding, some technical terms involved in the embodiments of this application are first introduced below.

[0145] (1) Digital Assistant

[0146] A digital assistant is a virtual assistant that receives user commands, replaces manual operations, and helps users complete a series of operations on smart devices. Generally speaking, most digital assistants are voice assistants, which complete smart device operations by receiving user voice commands. For example, if the smart device is a smartphone or smart TV, the digital assistant can receive user voice commands to open an application or play a song.

[0147] (2) Distance sensor

[0148] A distance sensor, also known as a displacement sensor, is a type of sensor that is used to sense the distance between it and an object to complete a preset function and has been widely used.

[0149] Distance sensors can be categorized into optical, infrared, and ultrasonic distance sensors, depending on their operating principles. Most distance sensors used in mobile phones are infrared sensors, which consist of an infrared transmitter and an infrared receiver. When infrared light from the transmitter is received by the receiver, it indicates a close distance and requires the screen to be turned off to prevent malfunctions. However, when the receiver fails to receive infrared light, it indicates a long distance and requires no screen turn-off. Other types of distance sensors operate on similar principles, also determining distance through the emission and reception of a substance. These emitters can include ultrasonic waves, light pulses, and so on.

[0150] Please refer to Figure 1, which is a schematic diagram of the architecture of an application scenario provided in an embodiment of the present application. As shown in Figure 1, the application scenario includes a user and an electronic device with a digital assistant. Among them, the user can issue instructions to the digital assistant in the electronic device through voice or gestures, thereby enabling the digital assistant to assist the user in automatically completing a series of operations of the electronic device, such as automatically executing an application in the electronic device.

[0151] The electronic device is equipped with a digital assistant, and the electronic device may be equipped with a command receiving device such as a microphone or a camera, which can receive the user's voice commands or gesture commands, and then automatically complete a series of operations by processing the user's input commands through the digital assistant, and feedback the processing results to the user. The electronic device also has a display screen, which can display the form of the digital assistant on the display screen. When the electronic device obtains user input, the electronic device generates a signal instruction to trigger the change of the digital assistant's virtual image, thereby making the process of interaction between the digital assistant and the user more intuitive. For example, when the electronic device is at a certain distance from the user, the electronic device recognizes the user's spatial interaction behavior by sensing the user's position change or posture change, and displays the form change of the digital assistant in response to the user's spatial interaction behavior on the display screen.

[0152] The above describes how electronic devices adjust the form of digital assistants by obtaining spatial interaction behaviors of users in the real physical world. In addition, the electronic device may also be a virtual reality (VR) device or an augmented reality (AR) device, and the electronic device adjusts the form of the digital assistant by obtaining spatial interaction behaviors of users in virtual worlds such as the VR world or the AR world. In the virtual world, the user who interacts with the electronic device may be a virtual image representing the real user in the virtual world. The real user performs spatial interaction behaviors by controlling the virtual image, thereby changing the form of the digital assistant in the virtual world.

[0153] Exemplarily, electronic devices with digital assistants may be, for example, smartphones, personal computers (PCs), laptops, tablet computers, smart TVs, mobile internet devices (MIDs), display devices in autonomous vehicles, smart speakers, wearable devices, VR devices, AR devices, display devices in industrial control, display devices in remote medical surgery, display devices in smart grids, wireless communication devices in smart cities, display devices in smart homes, etc.

[0154] Please refer to Figure 2, which is a schematic diagram of the structure of an electronic device 101 provided in an embodiment of the present application. As shown in Figure 2, electronic device 101 includes a processor 103, which is coupled to a system bus 105. Processor 103 can be one or more processors, each of which can include one or more processor cores. A display adapter (video adapter) 107 can drive a display 109, which is coupled to system bus 105. System bus 105 is coupled to an input / output (I / O) bus via a bus bridge 111. An I / O interface 115 is coupled to the I / O bus. The I / O interface 115 communicates with various I / O devices, such as an input device 117 (e.g., a touch screen), an external memory 121 (e.g., a hard disk, floppy disk, optical disk, or USB flash drive), a multimedia interface, etc., a transceiver 123 (capable of sending and / or receiving radio communication signals), a camera 155 (capable of capturing still and dynamic digital video images), and an external USB port 125. Optionally, the interface connected to the I / O interface 115 may be a USB interface.

[0155] The processor 103 may be any conventional processor, including a reduced instruction set computing (RISC) processor, a complex instruction set computing (CISC) processor, or a combination thereof. Alternatively, the processor may be a dedicated device such as an ASIC.

[0156] Electronic device 101 can communicate with software deployment server 149 via network interface 129. Exemplarily, network interface 129 is a hardware network interface, such as a network card. Network 127 can be an external network, such as the Internet, or an internal network, such as Ethernet or a virtual private network (VPN). Alternatively, network 127 can be a wireless network, such as a WiFi network or a cellular network.

[0157] The hard drive interface 131 is coupled to the system bus 105. The hard drive interface is connected to the hard drive 133. The internal memory 135 is coupled to the system bus 105. The data running in the internal memory 135 may include the operating system (OS) 137 of the electronic device 101, the application 143, and the scheduler.

[0158] The operating system consists of a shell 139 and a kernel 141. Shell 139 is an interface between the user and the operating system's kernel. The shell is the outermost layer of the operating system. The shell manages the interaction between the user and the operating system: it waits for user input, interprets user input to the operating system, and processes various operating system output.

[0159] The kernel 141 consists of the parts of the operating system that manage memory, files, peripherals, and system resources. The kernel 141 directly interacts with the hardware. The operating system kernel typically runs processes and provides inter-process communication, CPU time slice management, interrupts, memory management, and I / O management.

[0160] Please refer to Figure 3, which is a flowchart of a method for displaying a digital assistant virtual image provided in an embodiment of the present application. As shown in Figure 3, the method for displaying a digital assistant virtual image includes the following steps 301-302, and the method for displaying a digital assistant virtual image is applied to an electronic device equipped with a digital assistant.

[0161] Step 301: Display the virtual image of the digital assistant in a first form on a first interface.

[0162] In this embodiment, the digital assistant's avatar may include multiple features. When the avatar is displayed in different forms, the multiple features included in the avatar change. The first form is one of the multiple forms of the digital assistant's avatar, displayed on the first interface of the electronic device.

[0163] Step 302: Display the virtual image in a second form based on the signal instruction generated by the electronic device.

[0164] The signal instruction is generated by the electronic device based on the received first user input. After receiving the first user input, the electronic device generates the signal instruction, thereby triggering the avatar to switch from the first form to the second form, so that the form of the avatar displayed on the interface of the electronic device changes.

[0165] The virtual image in the first form and the second form each include at least one of the plurality of features. Specifically, the virtual image changing from the first form to the second form may be that at least one of the features undergoes at least one of a change in shape, position, motion pattern, or display area.

[0166] It should be noted that the first user input received by the electronic device may be before the electronic device displays the first form of the virtual image, or after the first form of the virtual image is displayed. If the first user input received may be before the electronic device displays the first form of the virtual image, it means that the first user input will prompt the electronic device to display the first form of the virtual image, and change from the first form to the second form, that is, cause at least two form changes. If the first user input received may be after the electronic device displays the first form of the virtual image, it means that the first user input will prompt the digital assistant's virtual image to change from the first form to the first form, that is, cause at least one form change.

[0167] In this embodiment, the different forms of the avatar represent the different working states of the digital assistant. That is, the digital assistant uses the form of the avatar to indicate its current working state. In this way, when the form of the avatar displayed on the interface changes, the user can intuitively and clearly understand the change in the working state of the digital assistant, and thus understand the current response of the digital assistant to the user's instructions.

[0168] Optionally, the plurality of feature bodies include a first feature body, and the shapes of the first feature body include: a main form, a geometric array, and a long strip. Furthermore, the shape change of the first feature body includes switching between multiple shapes, as well as enlarging, shrinking, stretching, compressing, or rotating each shape. That is, in addition to switching between different shapes, the shape change of the first feature body may also include enlarging, shrinking, stretching, compressing, or rotating within the same shape. The main form refers to the form of the first feature body under normal circumstances, while the geometric array and the long strip are the forms exhibited by the first feature body under certain specific circumstances.

[0169] Optionally, the plurality of feature bodies further include a second feature body. Among the plurality of feature bodies, the main form of the first feature body is a ring, and the second feature body is a sphere, with the center of the ring and the center of the sphere being co-located. The position change of the first feature body includes at least one of a change in orbital position after flipping around the sphere and a position change caused by the overall movement or rotation of the virtual image; the motion law of the first feature body includes the ring rotating around the sphere with its center as the fulcrum according to a preset trajectory; the motion law of the second feature body includes the sphere reciprocating along a straight line passing through the center of the ring and perpendicular to the plane of the ring according to a preset law.

[0170] For example, corresponding to different processing stages of the task, the digital assistant can have the following six different forms.

[0171] Breathing state: corresponds to the state of the digital assistant when it is not processing tasks, that is, it is used to indicate that the digital assistant is in a non-task processing state. Among them, the breathing state is the state of the digital assistant in the off-screen interface.

[0172] Waiting state: corresponds to the state of the digital assistant waiting to be awakened (essentially, the state of the digital assistant when it is not processing any tasks), that is, it is used to indicate that the digital assistant is in the state of waiting for wake-up instructions. Among them, the waiting state is the state of the digital assistant in the bright screen interface, where the bright screen interface includes the lock screen interface or the interface after the screen is unlocked.

[0173] Awakening state: corresponds to the state when the digital assistant is awakened and ready to receive commands, that is, it is used to indicate that the digital assistant is in the state of waiting to receive commands.

[0174] Listening state: corresponds to the state of the digital assistant during the process of receiving commands, that is, it is used to indicate that the digital assistant is in the command receiving state.

[0175] Thinking state: The state of the digital assistant when receiving and parsing the command, which indicates that the digital assistant is in the command parsing state.

[0176] Task execution state: The state of the task obtained by the digital assistant when executing the parsed instruction, which is used to indicate that the digital assistant is in the task execution state.

[0177] From the perspective of the degree of interaction intention, the above-mentioned multiple forms are sorted from low to high in order according to the degree of interaction intention: breathing state, waiting to be awakened state, awakened state, listening state, thinking state, and task execution state. In addition, in the off-screen interface, the breathing state can be converted into the awakened state. In the bright-screen interface, the waiting to be awakened state can be converted into the listening state, but not into the awakened state. When applied to different electronic devices, the forms of the digital assistant may also be different. For example, when applied to smartphones, the forms of the digital assistant include the above-mentioned breathing state, waiting to be awakened state, listening state, thinking state and task execution state, and the forms can be switched back and forth in sequence; when applied to smart TVs, the forms of the digital assistant include the above-mentioned breathing state, awakened state, listening state, thinking state and task execution state, and the forms can be switched back and forth in sequence.

[0178] The above introduces the different forms of digital assistants. The following will introduce the main image of digital assistants.

[0179] The main image of the digital assistant includes at least two features, and the different forms of the digital assistant are distinguished by the changes in the shape, position, and movement law of the features. Specifically for the main image of a digital assistant in a certain form, each feature can be dynamically presented to represent the concept of a living organism of the digital assistant's activities. Different forms of digital assistants are distinguished by changes in the shape, position and / or movement law of different feature bodies. Among them, the change in the shape of the feature body can mean that the feature body maintains its current position, while adjusting the position of the feature body presented on the display screen (for example, adjusting the front of the feature body to the back of the feature body), switching between a plurality of pre-set different shapes (including enlarging, reducing, stretching, compressing or rotating each shape), and adjusting the position or movement of certain parts on the feature body (for example, adjusting the arm on the feature body from static to waving). The change in the position of the feature body can mean that the position where the feature body is displayed changes, for example, from being displayed at the bottom of the screen to being displayed at the top of the screen or in the center of the screen. The change in the motion law of the feature body may refer to a change in the motion of the entire feature body on the display screen, for example, the feature body changes from being stationary to moving along a preset trajectory, or from moving along a sinusoidal trajectory to rotating around a certain point.

[0180] Please refer to Figure 4, which is a schematic diagram of the main image of a digital assistant provided in an embodiment of the present application. As shown in Figure 4, the main image of the digital assistant includes three characteristic bodies: a ring, a sphere, and an image body. Among them, the image body is wrapped in the sphere; the ring surrounds the sphere, similar to the rings of a planet. Digital assistants in different forms, the ring, the sphere, and the image body can have different postures and different motion laws. Moreover, between digital assistants in different forms, the posture and / or motion law of at least one characteristic body has changed.

[0181] Furthermore, to facilitate the description of the motion changes of each feature, a reference frame is defined for the digital assistant's main image: in the breathing state, the plane where the ring lies is the xoy plane, with the x-axis pointing toward the electronic device's screen and the vertical direction being the z-axis. Simply put, the width of the electronic device is the y-axis, and the height of the device is the z-axis.

[0182] The following will introduce in detail the posture and movement rules of the characteristic bodies in each form of the digital assistant with reference to the accompanying drawings.

[0183] 1. Breathing state

[0184] When the digital assistant is in a breathing state, the ring, sphere, and image body all move along a preset trajectory and regularly according to a preset period. During the movement, the relative positions of the ring, sphere, and image body remain unchanged, that is, the center points of the ring, sphere, and image body always coincide. In addition, the image body and the sphere can rotate around an axis or a point, for example, around the center of the sphere. That is, the image body and the sphere rotate while moving along the preset trajectory (similar to the earth rotating around the sun while also rotating).

[0185] For example, when the avatar has a breathing state, the avatar is displayed in the first form on the first interface. Specifically, if the first interface is an off-screen interface, the first form is the breathing state. The breathing state includes a first feature body and a second feature body, and the first feature body is displayed in the main form.

[0186] Optionally, when the first interface is a screen-off interface, the digital assistant in a breathing state moves back and forth on the screen according to a preset pattern.

[0187] Please refer to Figure 5, which is a schematic diagram of the movement law of a digital assistant provided by an embodiment of the present application when the digital assistant is in a breathing state. As shown in Figure 5, Figure 5 shows the breathing state image of the digital assistant at time t, and the ring moves back and forth along the z-axis direction, and the function of the moving distance and time conforms to a sine curve. During the circular motion, the sphere and the image rotate around the center of the sphere. In addition, in order to reflect the rotation effect of the sphere, corresponding light and shadow effects can be added to the sphere to indicate the rotation of the sphere.

[0188] 2. Waiting to wake up

[0189] When the digital assistant is in the wake-up state, the ring of the digital assistant (i.e. the first characteristic body mentioned above) is closed inward and displayed in the form of a gesture prompt line, which can be specifically a long strip.

[0190] The gesture prompt line is designed to support full-screen gestures. The gesture prompt line is a global control that can be displayed at the bottom of the screen to provide a reference for full-screen gestures. For example, swiping up from the bottom usually starts from the gesture prompt line.

[0191] Specifically, the digital assistant only displays the first feature body, and the shape of the first feature body is a long strip. Other feature bodies of the digital assistant (such as the sphere and image body of the digital assistant) are hidden. Generally, the gesture prompt line is usually displayed at the bottom of the screen and has a specific function. Correspondingly, the digital assistant in the wake-up state also has the function of the gesture prompt line, such as swiping left and right to switch applications.

[0192] Please refer to Figure 6, which is a schematic diagram of a digital assistant provided in an embodiment of the present application switching from a breathing state to a to-be-awakened state. As shown in (a) of Figure 6, in the off-screen interface, when the digital assistant is in a breathing state, the virtual image of the digital assistant includes a ring, a sphere, and an image body. As shown in (b) of Figure 6, when the user operation triggers the electronic device to switch from the off-screen interface to the bright-screen interface, the digital assistant switches from the breathing state to the to-be-awakened state. In the to-be-awakened state, the ring of the digital assistant is transformed into a long strip at the bottom of the screen, and the sphere and image body of the digital assistant are hidden.

[0193] 3. Awakening state

[0194] When the digital assistant is in the awake state, the ring is fixed to the equatorial plane of the sphere and rotates around the z-axis as the user moves. In addition, in order to indicate the rotation relationship of the ring, part of the area on the ring is set as a light effect area. The brightness of the light effect area on the ring is higher than the non-light effect area on the ring. In addition, the light effect area on the ring points to the user, and when the ring rotates, the light effect area moves and points to the user. Alternatively, when the user moves, the ring can also rotate with the user, so that the light effect area always points to the user's location. Alternatively, when the user approaches or moves away from the electronic device, the ring can change its pitch angle (that is, rotate around the plane where the display screen of the electronic device is located), so that the ring can simulate the "head-up" or "head-down" action when seeing the user approaching.

[0195] For example, when the first interface is the off-screen interface and the first form displayed on the first interface is the breathing form, if a second user input is detected, a third feature is displayed on the first feature. The third feature indicates the user's position and can move on the first feature. The first feature is the aforementioned ring, and the third feature can be a light effect area on the ring.

[0196] Please refer to Figure 7, which is a schematic diagram of a digital assistant provided by an embodiment of the present application in the awake state. As shown in Figure 7, the partial area on the ring close to the side of the screen is the light effect area, which indicates the user's orientation relative to the electronic device. When the user's orientation changes, the ring rotates around the z-axis so that the light effect area always faces the user. In addition, when the distance between the user and the electronic device is less than a preset value, the user continues to approach the TV, and in addition to rotating around the z-axis, the ring will also produce a change in pitch angle around the y-axis. For example, after the ring rotates upward around the y-axis for a certain angle, it gradually stops due to inertia, so that the ring can simulate the "head-up" action when seeing the user approaching.

[0197] 4. Listening attitude

[0198] When the digital assistant is in listening mode, the ring rotates around the y-axis on the xoy plane. After this rotation, the ring is located on the yoz plane, no longer obstructing the sphere and the main figure. Furthermore, when the ring flips, it can also cause the sphere and the main figure to flip together, so that the bottom or top of the sphere and the main figure are facing the screen.

[0199] Please refer to Figure 8, which is a schematic diagram of a digital assistant provided by an embodiment of the present application in a listening state. As shown in Figure 8, the ring rotates to the yoz plane, and the sphere and the image body also rotate.

[0200] 5. Thinking attitude

[0201] When the digital assistant is in the thinking state, the ring and the image body are always in a rotating state, and the fulcrum around which the ring and the image body rotate can be the center of the sphere. In addition, the rotation trajectory of the ring and the image body can be random or preset. In addition, in the thinking state, in order to represent the rotation relationship of the ring, part of the area on the ring can also be set as a light effect area, and the light effect area moves with the movement of the ring, or the light effect area continuously rotates on the ring. Among them, the method of setting the light effect area on the ring can refer to the description of the awakening state above, and will not be repeated here.

[0202] Please refer to Figure 9, which is a schematic diagram of a digital assistant provided by an embodiment of the present application in a thinking state. Figure 9 shows two states of the digital assistant during rotation, and in these two states, the ring and the image body are always rotating.

[0203] 6. Task execution state

[0204] When the digital assistant is in the task execution state, the ring of the digital assistant may be transformed into other shapes (such as a geometric array) and separated from the sphere and the image body, and displayed in other locations on the interface. For example, please refer to Figure 10, which is a schematic diagram of a digital assistant provided in an embodiment of the present application when the form is in the task execution state. As shown in Figure 10, the ring of the digital assistant is transformed into a geometric array and separated from the sphere and the image body of the digital assistant, and is displayed in other locations.

[0205] In general, please refer to Figure 11, which is a schematic diagram of multiple forms of a digital assistant provided in an embodiment of the present application. Figure 11 shows the multiple forms included in the digital assistant.

[0206] In combination with the various forms introduced above, the first form and the second form of the virtual image mentioned in this embodiment in various situations will be introduced below.

[0207] Optionally, when the virtual image has a waiting state and a listening state, the first interface showing the first form of the virtual image may be a lock screen interface or any screen interface after unlocking.

[0208] Then, displaying the virtual image in the first form on the first interface may specifically include: displaying the virtual image in the waiting state as the first form on the first interface. The waiting state includes a first feature body, and the first feature body is displayed in a long strip as a navigation prompt line. The navigation prompt line is used as a reference line for screen gesture navigation. In addition, when no input to wake up the digital assistant is received, the waiting state is permanently displayed on the display interface.

[0209] Specifically, after detecting user input, the electronic device determines a response strategy based on the recognition of the user input and displays the response result on the screen. The response result displayed on the screen by the electronic device can be that the interface remains unchanged or the interface has changed.

[0210] For example, when the electronic device is a smart phone, the user's probe is close to the smart phone, and the face recognition hardware on the smart phone recognizes the face, then it can be considered that the smart phone has detected the user input. Alternatively, the smart phone detects the approach of a human hand through an infrared sensor, and it can also be considered that the smart phone has detected the user input. According to the user input detected by the smart phone, the screen is switched from the off-screen interface to the lock screen interface (the lock screen interface is essentially the bright screen interface). Furthermore, after the face recognition verification on the lock screen interface is passed, it is unlocked to enter the desktop or the running application interface. For another example, the user directly double-clicks the off-screen interface, and the electronic device switches to the lock screen interface.

[0211] When the electronic device switches from the off-screen interface to the bright-screen interface and the digital assistant is not awakened, the digital assistant is displayed in the waiting-to-wake-up state. In the waiting-to-wake-up state, the ring of the digital assistant closes inward and is displayed in the form of a gesture prompt line, which can be specifically a long strip. The gesture prompt line is a global control that is displayed at the bottom of the screen in any interface and is used to provide a reference for full-screen gestures. For example, swiping up from the bottom usually swipes up from the position of the gesture prompt line.

[0212] That is to say, from turning the screen off to turning it on, and then to the user actively waking up the digital assistant and interacting with the digital assistant, the digital assistant is always displayed in a location that the user can reach and in different forms.

[0213] In this embodiment, the digital assistant can be displayed in a waiting state on the lock screen after the screen is unlocked, as well as on the desktop or in-app interface when the user has not activated the digital assistant through a preset operation. That is, the ring of the digital assistant closes to form a long strip, which serves as a system gesture prompt line (usually displayed at the bottom of the screen). As the ring closes, the image and sphere within the ring disappear (which may be accompanied by a corresponding disappearance animation).

[0214] For example, please refer to Figure 12, which is a schematic diagram of a digital assistant provided by an embodiment of the present application displayed in a waiting state. As shown in Figure 12 (a), in the off-screen interface, the digital assistant is displayed in a breathing state and can move within the off-screen interface according to a preset pattern. As shown in Figure 12 (b), when the user's input causes the electronic device to enter the lock screen interface from the off-screen interface, the digital assistant moves to the bottom, the ring shrinks inward, and the image body and the sphere are squeezed until they are completely hidden (this process is reflected in an animation, and the animation can also be hidden). At this time, the digital assistant becomes a bar and is displayed at the bottom of the screen as a system gesture prompt line. As shown in Figure 12 (c) and (d), after the screen is unlocked, the user clicks the gallery application icon (or any other application icon) to enter the gallery interface. During the above series of operations, the user has not woken up the digital assistant, so the digital assistant is always displayed in the waiting state and displayed on the interface as a gesture prompt line.

[0215] When the first form is the waiting state, the digital assistant switches from the first form to the second form, specifically, the waiting state can be switched from the waiting state to the listening state.

[0216] For example, when the first interface is the lock screen or any unlocked screen, the first form is the waiting state, and the second form displayed based on the signal instruction generated by the electronic device is the listening state. The signal instruction that triggers the display of the virtual image in the second form can be generated based on the user performing a preset gesture on the digital assistant in the first form. For example, the user performs a long press gesture or a double-click gesture on the digital assistant in the first form. This embodiment does not specifically limit the preset gesture.

[0217] Optionally, the plurality of features include a first feature and a second feature. In the second form, the first feature is in the main form of a ring, the second feature is a sphere, and the ring is located on the equator of the sphere in a top-down perspective.

[0218] The above describes an embodiment in which the first state is the waiting state. The following describes an embodiment in which the first state is the listening state.

[0219] Optionally, when a third user input is detected, the virtual image of the digital assistant is displayed on the first interface in a listening state as a first form, wherein the third user input is used to wake up the digital assistant. Exemplarily, the third user input can be, for example, a user's voice input, input from an external input device (such as a remote control or a mouse), touch input generated by a user directly interacting with the electronic device screen, input from a user interacting with physical buttons, shortcut operations such as back panel gestures, etc.

[0220] The listening state of the avatar includes a first characteristic body (such as the aforementioned ring) and a second characteristic body (such as the aforementioned sphere), and the first characteristic body is displayed in the main state.

[0221] Specifically, on the bright screen interface, click the gesture prompt line to wake up the digital assistant, so that the digital assistant enters the listening state. Please refer to Figure 13, which is a schematic diagram of a digital assistant switching from a waiting state to a listening state provided by an embodiment of the present application. As shown in Figure 13 (a), the user interacts with the gesture prompt line where the digital assistant is located with gestures such as single-click, double-click, or long press to trigger the awakening of the digital assistant. As shown in Figure 13 (b), after the digital assistant is awakened, the digital assistant is displayed in the current interface in a listening state.

[0222] In some scenarios, specific content may be displayed on the interface of the electronic device, and there may be little effective space on the interface to display the digital assistant. At this time, the virtual image of the digital assistant can be integrated with the specific content on the interface.

[0223] Exemplarily, the above-mentioned display of the virtual image in the first form on the first interface may specifically include: displaying the virtual image at a preset position on the first interface; or, displaying the virtual image as an overlay on the target content element in the first interface, wherein the target content element is a content element of a preset type. That is, the virtual image is overlaid on the target content element, so that the target content element is no longer displayed on the first interface. In this way, by overlaying the virtual image on the target content element, it is possible to avoid adding an additional interface for displaying the virtual image to the original interface and blocking too much content on the original interface (as shown on the right side of Figure 13, the floating window for displaying the virtual image of the digital assistant blocks part of the content at the bottom of the screen), thereby improving the integration of the digital assistant and the interface.

[0224] The target content element is, for example, a control or icon of a preset type predefined by the system. For example, the control or icon of the preset type may be a space or icon having a circular shape.

[0225] Optionally, when the avatar is displayed overlaying the target content element in the first interface, the gesture event interface of the target content element can be copied to the avatar, so that performing a gesture on the avatar can realize the gesture-related function of the target content element. In other words, even if the target content element is covered by the avatar, the avatar inherits the original function of the target content element, so that performing a specific gesture (such as a single click) on the avatar can still execute the gesture-related function of the target content element.

[0226] For example, when the first interface is a camera interface, the target content element is a shutter control in the camera interface. The virtual image is displayed overlaid on the shutter control of the camera interface, and clicking the virtual image can still achieve photo or video shooting.

[0227] That is to say, after the digital assistant's virtual image is overlaid on the target content element, when the display area of ​​the digital assistant's virtual image receives a gesture input for the original target content element, the gesture can be used to implement the function of the original control. Among them, the function of the original content element is implemented when the display area of ​​the virtual image receives the gesture input. The mapping relationship between the gesture and function of the target content element can be associated with the digital assistant, so that the digital assistant triggers the corresponding function after receiving the gesture input. Taking the target content element as an example, when the control is interacted, it usually uses whether the coordinates corresponding to the display area are touched as the trigger of the control and generates the corresponding instruction. Therefore, when the digital assistant integrates the original function of the control, it only needs to synchronize the trigger interface of the touch instruction to the digital assistant, so that when the area where the digital assistant is located is touched, the corresponding touch instruction is generated to realize the function of the original control.

[0228] Alternatively, as long as the display area corresponding to a control is touched by a preset gesture, the function corresponding to the control is executed according to the gesture type, regardless of whether there are other layers displayed above the control. In addition, implementing the digital assistant function itself only requires following the gesture recognition corresponding to the digital assistant and then triggering the function corresponding to the gesture.

[0229] Please refer to Figure 14, which is a schematic diagram of a virtual image overlaid on a camera shutter control provided in an embodiment of the present application. As shown in Figure 14, in one of the initial states, the digital assistant is not awakened, and is displayed on the interface in a waiting state as a navigation prompt line. When the user interacts with the digital assistant in the waiting state (i.e., the navigation prompt line), such as long pressing the navigation prompt line or long pressing the power button to wake up the digital assistant, the digital assistant is now in a listening state. At this time, when the system displays the digital assistant in the listening state, it first checks the type of control in the current interface. Since the camera shutter control in the current interface is a preset type of control, the digital assistant is displayed in a fusion with the camera shutter control. After the fusion display, the digital assistant is displayed in a listening state overlaid on the position of the original camera shutter control.

[0230] In addition, in another initial state, the digital assistant is not displayed in any state on the current interface. After long pressing a control (such as the power button) or calling out the digital assistant by voice, the digital assistant can also be displayed over a preset type of control or icon on the current interface.

[0231] After the digital assistant's avatar is overlaid on the original camera shutter control, tapping the digital assistant's avatar's display area (i.e., the original camera shutter control display area) can also trigger a photo. This can be achieved by the digital assistant passing instructions to the camera app to control the photo, or the camera app can directly recognize the gesture to achieve this.

[0232] In some cases, the navigation prompt line is permanently displayed in the system globally. Then, when the digital assistant is in the wake-up state, it can also be understood that the virtual image hides the sphere, changes the circle, and merges it with the navigation prompt line, so that the navigation prompt line not only has the function of a navigation prompt line, but also serves as the permanent state of the digital assistant virtual image.

[0233] In addition, when there are multiple controls or icons that can be overlaid on the interface, the digital assistant can switch between different controls or icons. Moreover, the digital assistant can switch from the controls of the first displayed interface to the controls or icons of the latest interface in the order of display time.

[0234] Please refer to Figure 15, which is a schematic diagram of a virtual image overlaid on an assistant suggestion component provided by an embodiment of the present application. As shown in Figure 15 (a), the digital assistant is displayed in a state to be awakened. In this state, the digital assistant integrates the navigation prompt line, and the deformed ring overlaps the navigation prompt line. As shown in Figure 15 (b), when switching to another home screen page, the home screen page contains an assistant suggestion widget, and an icon of a virtual image is displayed in the assistant suggestion widget, then the digital assistant can flow from the navigation prompt line to the virtual icon in the assistant suggestion widget. In addition, when the navigation prompt line is globally resident, it is the scheme shown in Figure 15 (b); when the navigation prompt line is only a deformed ring, then after the digital assistant flows from the navigation prompt line to the virtual icon in the assistant suggestion widget, the navigation prompt line is no longer displayed.

[0235] The above details the waiting state and listening state of the virtual image. The following will continue to introduce the task execution state of the virtual image.

[0236] Optionally, the digital assistant's avatar further has a task execution state, the task execution state including a first feature body. Corresponding to at least one process interface during the digital assistant's execution of an automated task, the first feature body in the task execution state undergoes at least one of a shape change and a position change.

[0237] Specifically, when the digital assistant performs an automated task, a series of changes will occur on the display interface of the electronic device. For example, when the digital assistant performs an automated task of calling a contact, the electronic device will first open the contact interface, then select a specific contact in the contact interface, and then enter the phone interface for calling the specific contact. Then, in the process interface of the digital assistant performing an automated task, the digital assistant is actually always in the task execution state, and the first feature body of the digital assistant in the task execution state can undergo various changes to show the user the automated process of the digital assistant performing the task, making the automated process of the digital assistant performing the task more intuitive and clear, and making it easier for the user to observe whether there are any errors in the process of the digital assistant performing the automated task.

[0238] In one possible example, the first feature is displayed as a geometric array, and the display position of the geometric array aligns with the target control within the process interface. The geometric array then simulates a click event on the target control to proceed to the next process interface. In other words, during the execution of an automated task, the digital assistant can use the first feature as the geometric array to simulate a user clicking on a control on the interface, thereby visualizing the entire process of the digital assistant simulating the user executing the automated task.

[0239] Simply put, if executing an automated task requires opening an application, and in the automated step of opening the application, you need to click the application icon on the desktop, then the application icon is the execution target of this step. When opening the application, the dot matrix that changes from a circle to a dot can be displayed on the application icon.

[0240] For example, please refer to Figure 16, which is a schematic diagram of the process of a digital assistant performing an automated task provided by an embodiment of the present application. As shown in Figure 16, first, the digital assistant receives the user's voice command "Share this photo to WeChat Moments". Then, the digital assistant displays the dot matrix on the "Send" control of the current interface to simulate the user's operation of clicking the "Send" control. Secondly, in response to the digital assistant simulating the user's operation of clicking the "Send" control, the interface changes. The current interface changes to display applications that can send photos, and these applications include the "WeChat" application; in the changed interface, the dot matrix is ​​displayed on the icon of the "WeChat" application to simulate the user's operation of clicking the "WeChat" application icon. Finally, the interface changes again, changing to an interface for editing a picture to be published in the Moments of the "WeChat" application. At this time, the dot matrix is ​​displayed on the "Publish" control to simulate the user's operation of clicking the "Publish" control.

[0241] Optionally, when the first state is the listening state and the second state is the task execution state, both the listening state and the task execution state include a first feature body, and the first feature bodies in the listening state and the task execution state have different forms.

[0242] Specifically, during task execution, the first feature body can take on various forms, including a geometric array within the interface, the frame of a digital assistant dialog card, the frame of the input box when invoking the digital assistant, or the frame of the task results page. In other words, the first feature body can be represented in different forms in different scenarios to better suit the content displayed on the current interface.

[0243] Optionally, the first feature body may have different morphological changes corresponding to the progress of the automated task. That is, as the digital assistant executes the automated task, the first feature body may change to different morphologies at different task progress levels, so that the morphology of the first feature body always closely matches the current interface.

[0244] Exemplarily, corresponding to the execution process of the automated task, in the task execution state, the morphological change of the first feature body includes the following process: initially, the first feature body is displayed in the main form of a ring at an initial position relative to the second feature body, and the initial position is the same as or different from the position of the first feature body in the listening state; then, the first feature body is separated from the second feature body, and after being enlarged and deformed, it serves as the outer frame of the digital assistant dialogue card; secondly, the first feature body is changed from the outer frame of the dialogue card to a geometric array within the dialogue bubble of the dialogue card; finally, after the execution of the automated task is completed, the virtual image is displayed in the listening state.

[0245] In general, in the task execution state, for the first feature body whose main form is a ring, the ring separates from the sphere, undergoes a shape change, and is displayed in the task execution interface in its changed feature state. The task execution interface refers to the interface when the digital assistant receives the task instruction and transmits the instruction to the local application for execution based on the analysis of the task instruction, or the execution result interface. Therefore, the task execution interface can be part of the interface of the execution application or the entire interface.

[0246] Among them, the morphological changes of the ring include: the ring concentric with the image body and the sphere disappears and transforms into a geometric array (such as a dot matrix) of the task execution interface on the screen, or an outer frame surrounding the task execution interface. In addition, the process of transforming from a ring to a geometric array or outer frame can be accompanied by animation. For example, the ring is decomposed into several points and then laid out on the task execution interface to form a dot matrix; the ring is deformed and stretched, transformed into a frame, and enlarged to become the outer frame of the task execution interface.

[0247] The following will use several application scenarios of digital assistants as examples to introduce the process of digital assistant morphology changes during task execution.

[0248] Please refer to Figure 17, which illustrates a digital assistant's circular form transformed into a dot matrix and displayed on a task execution interface, according to an embodiment of the present application. As shown in Figure 17, Figure 17 (a) corresponds to the digital assistant's listening state, where the user inputs the verbal instruction "Order me a cup of Xinba coffee..."

[0249] Figure 17(b) corresponds to the digital assistant's thinking state (where the thinking state is a dynamic image, with the ring rotating according to a certain pattern, and Figure 17(b) is a specific instantaneous state). Specifically, the thinking state corresponds to the digital assistant performing task analysis. At this time, the interface for the user's interaction with the digital assistant displays the digital assistant's task analysis process, "Combining Xinba's historical orders to make recommendations for you."

[0250] After completing the task parsing process, the task execution phase begins, and the digital assistant enters the task execution state, corresponding to Figure 17(c). In Figure 17(c), the circular ring of the digital assistant icon at the bottom disappears, and the task card (i.e., a conversation bubble, but presented in the form of a card) resulting from executing the task is displayed above. At the same time, a number of dots are displayed on the task card, which are formed by transforming the original circular ring of the digital assistant.

[0251] It is understandable that in order to avoid interference with the task execution interface, the dot matrix can disappear after being displayed for a period of time. In the two task cards in (c) of Figure 17, the dot matrix of the upper task card has disappeared, and the task card below shows an instantaneous state in the process of the dot matrix disappearing (the dot matrix and interface content are displayed at the same time, and both are displayed translucently). When the card interface is fully displayed, the dot matrix disappears. That is, for the task execution interface in Figure 17, the display process is: only the dot matrix is ​​displayed → the dot matrix gradually disappears, and the interface content gradually displays (the dot matrix and the interface exist at the same time) - only the interface content is displayed.

[0252] In another application scenario, when calling a digital assistant for voice-to-text input, the task execution interface can be an input box or a pop-up card interface. These cards can be components or subroutines provided by locally installed applications.

[0253] Please refer to Figure 18, which is a schematic diagram of using a digital assistant to input text on the desktop provided by an embodiment of the present application. As shown in Figure 18 (a), the user clicks on the search widget on the desktop, and the search widget enters edit mode. As shown in Figure 18 (b), the digital assistant is enabled by long pressing the power button. At this time, the digital assistant is in listening mode and uses the microphone to obtain user voice input. As shown in Figure 18 (c), the digital assistant receives the voice input and converts the voice into text and fills it into the input box; at this time, the digital assistant is in the task execution state, the ring disappears from the outside of the sphere, and is enlarged, deformed, and transferred to the outside of the search box, forming an outer frame surrounding the search box. As shown in Figure 18 (d), after the input is completed, the search widget expands into a search card, and the outer frame converted by the ring can be expanded and surround the search card; of course, the outer frame converted by the ring can also disappear after the search card is presented, or disappear after a preset time.

[0254] When the deformed ring disappears, or within a preset time after disappearing, the ring in its initial state (undeformed) may be displayed outside the sphere.

[0255] For example, please refer to Figure 19, which is a schematic diagram of a ring of a digital assistant provided by an embodiment of the present application deforming and then returning to its original shape. As shown in Figure 19 (a), after the digital assistant is initially called up, the digital assistant is displayed in a listening state, and the ring on the digital assistant is displayed outside the sphere.

[0256] As shown in Figure 19(b), after the user completes input, the digital assistant enters the task execution state after passing through the thinking state (not shown in Figure 19). In the initial stage of the task execution state, the digital assistant's ring deforms outward, overlapping with the border of the digital assistant's conversation card, and is displayed on or outside the border, or on top of the border. Visually, the digital assistant's ring becomes the outer frame of the conversation card.

[0257] As shown in FIG19( c ), when the card is no longer expanded, the ring disappears from the outer frame of the card and is re-displayed as a ring of the initial size outside the sphere.

[0258] As shown in Figure 19(d), when the result is presented in a speech bubble, the ring on the digital assistant transforms into a dot matrix again and is displayed on the speech bubble. It is understandable that the intermediate state shown in Figure 19(c) can be omitted, and the state change of the ring can be directly transformed from the card frame shown in Figure 19(b) to the dot matrix shown in Figure 19(d).

[0259] As shown in FIG19( e ), the result of the conversation card is fully displayed, the dot matrix disappears, and the ring is displayed outside the sphere again as a ring of the original size.

[0260] In another application scenario, when calling a digital assistant for intelligent recognition or other automatic tasks, the task execution interface is the application interface currently being operated. For intelligent recognition, a smart recognition result is usually generated, which is one or more executable objects obtained by recognition (such as smart removal of people / smart removal of objects in album editing), then the task execution interface can be the area where the executable object is located. In the process of executing intelligent recognition, the entire interface is recognized in real time, and finally the executable object is recognized. Corresponding to this task execution process, the dot matrix corresponding to the circle can also be adapted to the task execution process, that is, in the recognition process stage, the dot matrix is ​​spread all over the interface, and after the executable object is recognized, the dot matrix is ​​displayed only in the area of ​​the executable object.

[0261] For example, please refer to Figure 20, which is a schematic diagram of the morphological changes of a digital assistant in a smart recognition scenario provided by an embodiment of the present application. In Figure 20, the user calls up the digital assistant and uses the smart recognition function. As shown in Figure 20 (a), an image is displayed on the smartphone interface, and the image is displayed full screen. Then, the user long presses anywhere on the interface to trigger the smart recognition function. In addition, the digital assistant icon is displayed on the current interface. When the digital assistant first appears, the digital assistant is in a listening state. As shown in Figure 20 (b), the digital assistant performs smart recognition on the image. At this time, the digital assistant's ring disappears from the outside of the sphere and is transformed into a dot matrix displayed on the interface, with the dot matrix covering the entire screen. As shown in Figure 20 (c), after the image recognition is completed, if at least one executable object is identified, the dot matrix is ​​only displayed in the display area where the executable object is located. At this time, the points of the dot matrix may not be a regular dot matrix, but can move dynamically in the display area where the executable object is located, so that the dot matrix becomes a plurality of irregular discrete points.

[0262] It should be noted that in the above-mentioned Figures 19 and 20, in order to fully present the switching between the digital assistant forms, the established steps are used for illustration. When actually operating an electronic device, the electronic device supports calling out the digital assistant through quick operations to perform preset functions, such as the smart recognition function in Figure 20. At this time, the digital assistant can directly display the corresponding state, such as directly displaying the task execution state, or displaying the listening state at the moment of calling out, and then immediately switching to the task execution state.

[0263] Taking the smart recognition function as an example, assuming that the system defines long pressing the power button as a shortcut operation for smart recognition, when the user long presses the power button, the operating system displays the digital assistant icon on the interface and directly displays the task execution status, that is, after long pressing the power button, it directly enters the display interface (b) of Figure 20.

[0264] In some designs, in the task execution state, the image body and sphere of the digital assistant may not be displayed on the interface, but only the changed shape of the ring body may be displayed. For example, in Figure 20 (c), only the dot matrix may be displayed without displaying the sphere and image body below.

[0265] The above introduces the changing process of the digital assistant in the task execution state. In actual applications, when the digital assistant initially appears on the interface, it may also appear in a certain pattern.

[0266] Optionally, when the first interface is a lock screen or any unlocked screen, and the first state displayed on the first interface is a listening state, displaying the avatar in the first state on the first interface specifically includes: initially displaying only the first feature body without displaying the second feature body, and initially displaying the first feature body in a geometric array that fills the screen; then, the first feature body undergoes a shape change from the geometric array to be displayed in the main state, and the second feature body is displayed.

[0267] That is to say, when waking up the digital assistant, the digital assistant first appears in the form of a geometric array (such as a dot matrix) and gradually fills the screen; then, the digital assistant gradually transforms from the geometric array to the main form of the first feature body (i.e., a ring) and displays the second feature body (i.e., a sphere), thereby realizing the awakening process of the digital assistant in the form of animation, increasing the intuitiveness of the interaction process, and helping users to identify the current working status of the digital assistant.

[0268] The geometric array may be, for example, a dot matrix, i.e., an array formed by a plurality of dots arranged in a regular pattern. The geometric array may also be an array formed by other geometric shapes (such as polygons such as triangles and rectangles) arranged in a regular pattern.

[0269] Optionally, the first feature body is initially displayed as a geometric array, which may specifically include: first determining an initial display position of the geometric array; secondly, displaying a portion of the geometric pattern in the array at the initial display position, and then gradually spreading the geometric pattern to fill the entire screen. For example, a portion of the dot matrix may be displayed starting from an initial display position, and then gradually spreading the dot matrix to fill the entire screen.

[0270] Optionally, the initial display position of the geometric array is determined based on user input. The user input refers to an input used to wake up the digital assistant. Specifically, the initial display position may be near the user input that triggered the awakening of the digital assistant. For example, when the user input is a long press of the power button of a smartphone, the initial display position of the geometric array may be on the screen near the power button.

[0271] Furthermore, when no user input is detected, the initial position is a default position, which is an area where the virtual image is displayed, such as a preset area at the bottom of the screen.

[0272] In this solution, the initial display position of the geometric array is determined by user input, which can make the display process of the digital assistant more consistent with the user input, enhance the intuitiveness of the user's interaction with the digital assistant, and improve the user experience.

[0273] In general, in the process of calling up the digital assistant, the appearance of the dot matrix can have a preset pattern. When the user's operation is detected (touching or pressing a physical button), the initial display position of the digital assistant is determined based on the user's operation, and the dot matrix is ​​expanded from the initial display position to the full image display. When no user operation is detected, the display position of the digital assistant's virtual image is used as the initial display position, and the dot matrix is ​​expanded from the initial position to the full image display.

[0274] For example, please refer to Figure 21, which is a schematic diagram of the appearance of a digital assistant in a smart recognition scenario provided by an embodiment of the present application. As shown in Figure 21, when the user calls up the digital assistant by long pressing the power button on the right side of the smartphone, the digital assistant is initially displayed in the form of a dot matrix on the screen near the power button, and then gradually spreads to the entire screen. When the user calls up the digital assistant by long pressing the digital assistant at the bottom of the screen, the digital assistant is initially displayed in the form of a dot matrix near the area where the virtual image of the digital assistant is located at the bottom, and then gradually spreads to the entire screen.

[0275] In some scenarios, the morphological changes of the digital assistant may also be related to the scene information, so that the digital assistant's virtual image can accurately represent some information in the current usage scenario.

[0276] Optionally, in some embodiments, the electronic device may obtain scene information and, based on the scene information, determine a target form for the first feature of the digital assistant, and display the first feature in the target form. That is, the target form is related to the scene information. When the electronic device obtains specific scene information, the electronic device may display the first feature in a specific form.

[0277] Scenario information can include external environment information and system event information. External environment information is a signal obtained by sensors to indicate the external environment. Specifically, it can be obtained through sensors on the electronic device or the network, such as temperature, humidity, weather, and other external environment information. System event information is an event obtained from the internal system of the electronic device that occurs within a preset time, such as music playing, an alarm notification within a preset time, or a file being searched.

[0278] When the first feature body is a ring, the ring can be deformed based on the external environment information. For example, on a rainy day, the ring becomes an umbrella shape. Please refer to Figure 22, which is a schematic diagram of a digital assistant provided in an embodiment of the present application that is deformed based on scene information. As shown in Figure 22, after calling out the digital assistant, if you maintain the current interface, the digital assistant is always displayed in a listening state. After maintaining a certain state (such as the listening state in Figure 22) for a preset time, the ring of the digital assistant is deformed based on the scene information. In Figure 2, the current weather is light rain, so the ring becomes an umbrella shape.

[0279] The ring can also change shape based on system events. For example, when the screen is off and music is playing, the ring changes to the shape of headphones. If an alarm is set within a preset time, the ring changes to the shape of a physical alarm clock. Another example is when the user calls the digital assistant to search for a file, the ring changes to a magnifying glass.

[0280] Of course, in addition to the first feature body that can be deformed, other feature bodies of the digital assistant can also be deformed based on scene information, which will not be repeated here.

[0281] Optionally, when the avatar is in a waiting state, the waiting state includes at least a first feature, and the first feature is currently displayed in a long strip. In response to the electronic device detecting that the focus of the human eye falls on the area where the screen is located, the electronic device displays a second feature.

[0282] That is to say, in some scenarios, when the digital assistant is in the wake-up state, only a long bar is displayed; when the digital assistant detects that the user's eyes fall on the screen, it can be considered that the user has an interactive tendency, so the second feature body is displayed, thereby indicating that the digital assistant detects that the user has an interactive tendency and is waiting for interaction, so as to improve the naturalness of human-computer interaction.

[0283] Optionally, in response to detecting a human voice, the second feature body may move relative to the first feature body in a direction closer to the source of the human voice. Specifically, when the second feature body first appears, its position is relatively fixed, for example, located directly above or below the first feature body. When the electronic device detects a human voice, the electronic device may move the second feature body according to the direction of the human voice, so that the second feature body moves with the human voice, thereby improving the naturalness of the interaction.

[0284] Specifically, in another common terminal - the in-vehicle terminal, the digital assistant, as the main interaction method, can also be resident on the car's display screen (such as the central control screen) and interact with the user visually through at least one deformation in a ring or sphere, thereby maintaining the user's attention on the digital assistant and enhancing the sense of dialogue.

[0285] In the vehicle terminal environment, scene information mainly includes human information, system information and external environment information.

[0286] The human information can come from at least one of the driver and the passenger, including head orientation, eyes, gestures, posture, and voice, and can be distinguished as the main driver or the front passenger (it is usually difficult for the back seat to interact with the central control screen).

[0287] System information can refer to changes in manual or adaptive adjustments to vehicle control functions, such as driving mode, air conditioning settings, window defogger, and seat belt status, as well as changes in vehicle conditions, such as changes in vehicle speed, direction of travel, vehicle jolts, and unusual traffic conditions. System information can also include details such as the types of running applications.

[0288] External environmental information usually includes changes in weather, light, temperature, etc.

[0289] Based on this scene information, virtual images corresponding to different states can trigger different deformations of the virtual form, such as the deformation of the above-mentioned ring or the deformation of the sphere.

[0290] In addition, unlike handheld devices, the central control screen on an in-vehicle terminal rarely has an off-screen state. Therefore, the digital assistant on the in-vehicle terminal is displayed in a standby state upon startup. In this standby state, the digital assistant can change to different states depending on the detected scene conditions.

[0291] For example, please refer to Figure 23, which is a schematic diagram of the changes in the digital assistant on an in-vehicle terminal provided by an embodiment of the present application. In Figure 23, the digital assistant's initial wake-up state displays only a deformed circular ring. When the driver or passenger's gaze is detected toward the central control screen, a partial sphere is displayed. If a human voice is further detected, more of the sphere area is displayed, such as a hemisphere.

[0292] Furthermore, to correspond to different voices in the driver's seat and the front passenger's seat, the ring can be elongated, with the sphere positioned toward the side of the ring that corresponds to the voice's source. In Figure 23, when the driver's voice is detected, the hemisphere is positioned on the driver's side of the elongated ring, moving toward the driver's seat. When the front passenger's voice is detected, the hemisphere is positioned on the front passenger side of the elongated ring, moving toward the front passenger.

[0293] Optionally, in the case where the electronic device is a smart car, a digital assistant can be displayed on the screen of the electronic device based on the activation or alarm of the vehicle function, and the corresponding function or alarm information can be indicated by the deformation or movement of the first feature body.

[0294] In other words, a digital assistant can also be used on the car computer to provide vehicle status information. For example, when it is detected that the vehicle is in a bumpy road environment, the awakened digital assistant is displayed, and the sphere vibrates up and down along the screen where the ring is located. Alternatively, the sphere and the ring vibrate up and down together, and the vibration frequency of the two can be consistent or inconsistent. For another example, when it is detected that the vehicle has activated the window defogger function, the awakened digital assistant is displayed, and the sphere is rendered white. Then, the ring is fixed with the x-axis as the axis and generates an up and down swing in the z-axis direction. As it swings, the transparency of the white rendered on the sphere decreases until it is completely transparent and the ring stops swinging. For another example, when it is detected that the seat belt valve is not locked or the locking progress is not enough (i.e., the seat belt is not worn), the awakened digital assistant is displayed, and the ring is tilted at a preset angle. A part of the ring is broken in the middle, becoming an arc, and the arc gradually closes to form a ring.

[0295] The above mainly describes the process of digital assistant form changes when the electronic device is a smartphone or in-vehicle terminal. The following will describe the process of digital assistant form changes when the electronic device is a smart TV.

[0296] Please refer to Figure 24, which is a flow chart of a method for displaying a digital assistant according to an embodiment of the present application. As shown in Figure 24, the method for displaying a digital assistant includes the following steps 2401-2403, and the method for displaying a digital assistant is applied to an electronic device equipped with a digital assistant.

[0297] Step 2401, obtain the user's location and display the first form of the digital assistant based on the user's location.

[0298] In this embodiment, the electronic device configured with the digital assistant may be equipped with a sensor, such as an infrared sensor, an ultrasonic sensor, or an optical distance sensor, so that the electronic device can detect the user's location based on the deployed sensor. For example, when the electronic device is equipped with an infrared sensor, the electronic device emits infrared light through the infrared sensor and receives infrared light returned by the user's body, thereby calculating the direction and distance of the user relative to the electronic device, and thus determining the user's location.

[0299] Alternatively, the electronic device may obtain the user's location through other means, such as obtaining the user's location from other devices. For example, if the electronic device is a smart TV or smart speaker in a smart home, the electronic device may connect to other smart devices (such as home cameras, robot vacuums, or smart routers) through a network. After the other smart devices determine the user's location through their own sensors or positioning modules, the electronic device can obtain the user's location from the other smart devices.

[0300] When the electronic device obtains the user's location, it can display the first form of the digital assistant on the display screen of the electronic device based on the user's location, that is, the first form of the digital assistant displayed by the electronic device is related to the user's location. Simply put, the farther the user's location is from the electronic device, the lower the user's tendency to interact with the electronic device, so the electronic device can display a digital assistant form with a lower degree of interaction intention; the closer the user's location is to the electronic device, the higher the user's tendency to interact with the electronic device, so the electronic device can display a digital assistant form with a higher degree of interaction intention.

[0301] Step 2402: Acquire the user's spatial interaction behavior, where the spatial interaction behavior is used to indicate the user's activity in the space.

[0302] After determining the user's location and displaying the first form of the digital assistant, the electronic device can further capture the user's spatial interaction behavior through sensors, that is, the user's activities in the space, and then determine the user's further interaction tendencies.

[0303] Exemplarily, the user's spatial interaction behavior may include, but is not limited to, any of the following behaviors: approaching the electronic device, moving away from the electronic device, moving to the left, moving to the right, pointing a finger at the electronic device, pointing an arm at the electronic device, facing the electronic device, and preset gestures. In general, the user's spatial interaction behavior can be used to indicate the interaction tendency between the user and the digital assistant of the electronic device. For example, when the user's spatial interaction behavior is approaching the electronic device, it means that the user expects to conduct further interaction with the digital assistant of the electronic device; when the user's spatial interaction behavior is moving away from the electronic device, it means that the user does not expect to conduct further interaction with the digital assistant of the electronic device.

[0304] Step 2403: Display the second form of the digital assistant in response to the spatial interaction behavior.

[0305] In this embodiment, after acquiring the user's spatial interaction behavior, the electronic device switches from the first form of the digital assistant to displaying the second form of the digital assistant. The first form and the second form of the digital assistant are used to represent different states of the digital assistant's interaction with the user, thereby making it easier for the user to identify the digital assistant's current interaction state and issue corresponding interaction instructions.

[0306] Optionally, the electronic device may display the second form of the digital assistant only in relation to the spatial interaction behavior. That is, after acquiring the user's spatial interaction behavior, the electronic device determines to display the corresponding form of the digital assistant (i.e., the aforementioned second form) according to a pre-established mapping relationship between the spatial interaction behavior and the form of the digital assistant, regardless of the specific form of the first form of the digital assistant currently displayed.

[0307] In other embodiments, the second form of the digital assistant displayed by the electronic device may also be related to the first form of the digital assistant and the spatial interaction behavior that has been displayed by the electronic device. That is, the electronic device actually determines the second form of the digital assistant that needs to be displayed based on the currently displayed first form and the user's spatial interaction behavior. For example, the digital assistant can have multiple forms, and different forms correspond to different degrees of interaction intention between the user and the electronic device as judged by the electronic device. When the electronic device judges that the user's expected degree of interaction with the electronic device is increased based on the acquired spatial interaction behavior, the electronic device can determine to display a second form with a higher degree of interaction intention than the first form based on the currently displayed first form of the digital assistant, and display the second form; conversely, when the electronic device judges that the user's degree of interaction with the electronic device is reduced based on the acquired spatial interaction behavior, the electronic device can determine to display another second form with a lower degree of interaction intention than the first form based on the currently displayed first form of the digital assistant, and display the other second form.

[0308] Optionally, in some embodiments, the electronic device may divide the area near the electronic device into multiple areas based on the location of the electronic device, and different areas are not in the same position relative to the electronic device. For example, the electronic device can be divided into three areas, namely the first area, the second area and the third area. Among them, the electronic device is located in the first area, the second area is located outside the first area relative to the electronic device and surrounds the first area, and the third area is located outside the second area relative to the electronic device and surrounds the second area. For example, the first area, the second area and the third area can be areas divided with the electronic device as the center, and the plane shape of the first area is a circle or an ellipse, the plane shape of the second area is a ring surrounding the first area, and the plane shape of the third area is a ring surrounding the second area. In this way, when the electronic device recognizes that the user's spatial interaction behavior is moving between different areas, it can accurately determine the user's interaction tendency with the electronic device, and then adjust the form of the displayed digital assistant.

[0309] For example, assuming that the user's position obtained by the electronic device in step 2401 is in the first target area, and the spatial interaction behavior obtained by the electronic device in step 2402 is that the user moves from the first target area to the second target area, the electronic device switches the first form of the digital assistant to the second form. The first target area belongs to any one of the first area, the second area, and the third area, and the second target area can be an area adjacent to the first target area.

[0310] Optionally, in response to the user moving from the first target area to the second target area, the electronic device may display a second form of the digital assistant; wherein the second form corresponds to the second target area. In other words, each area divided around the electronic device corresponds to a form of the digital assistant. When the user moves from one area to another, the form of the digital assistant is switched so that the currently displayed form corresponds to the area where the user moved.

[0311] Specifically, when the first form is used to indicate that the digital assistant is in a non-task processing state and the first target area is outside the second target area, the second form is used to indicate that the digital assistant is in a state of waiting to receive instructions. Specifically, when the first target area is outside the second target area, the user moves from the first area to the second area, which means that the user moves from an area farther away from the electronic device to an area closer to the electronic device, that is, the user is approaching the electronic device, so the electronic device can recognize that the user has a further tendency to interact, thereby switching the form of the digital assistant to show the user that the digital assistant has switched from a non-task processing state to a state of waiting to receive instructions. Among them, the non-task processing state may be a state indicating that the digital assistant has no tasks to process and is in a dormant state; the state of waiting to receive instructions may refer to a state in which the digital assistant has been awakened and is ready to receive instructions at any time. The state of waiting to receive instructions of the digital assistant is a state with a higher degree of interaction intention than the non-task processing state.

[0312] Alternatively, when the first form is used to indicate that the digital assistant is in a state of waiting to receive instructions, and the first target area is outside the second target area, the second form is used to indicate that the digital assistant is in a state of receiving instructions. Similarly, when the first form is used to indicate that the digital assistant is in a state of waiting to receive instructions, the user moves from the first target area to the second target area, which means that the user has a further tendency to interact, thereby switching the form of the digital assistant to show the user that the digital assistant has switched from a state of waiting to receive instructions to a state of receiving instructions. Among them, the instruction receiving state may refer to a state in which the digital assistant has received the instructions issued by the user through the corresponding hardware on the electronic device, that is, the hardware for receiving instructions is already in working condition, for example, the digital assistant receives the voice instructions issued by the user through the microphone on the electronic device. The instruction receiving state of the digital assistant is a state with a higher degree of interaction intention than the state of waiting to receive instructions.

[0313] Alternatively, when the first form is used to indicate that the digital assistant is in a state of waiting to receive instructions and the second target area is outside the first target area, the second form is used to indicate that the digital assistant is in a state of no task processing. It is understandable that when the second target area is outside the first target area, the user moves from the first target area to the second target area, which means that the user is moving away from the electronic device. Therefore, the electronic device can recognize that the user has no further interaction intention, and thus switches from the state of waiting to receive instructions to the state of no task processing with a lower degree of interaction intention.

[0314] In this solution, the user's interaction tendency with the digital assistant is identified by obtaining the user's movement between areas with different distances from the electronic device, and then the morphology of the digital assistant is adjusted. This can well fit the relationship between social status and distance between humans, and improve the naturalness of the morphology change of the digital assistant.

[0315] Optionally, before switching the digital assistant's first form to the second form, the electronic device may also obtain the user's spatial activity behavior, which indicates the user's activities in space. The spatial activity behavior is actions performed by the user without the intention to interact with the digital assistant. That is, the spatial interaction behavior can be used to indicate that the user does not intend to interact with the digital assistant.

[0316] Then, in response to the spatial activity, the electronic device displays the adjusted first form of the digital assistant; wherein the adjusted first form is obtained by adjusting at least one characteristic of the first form, and the adjusted first form and the first form are used to represent the same state during the interaction between the digital assistant and the user. In other words, if the user does not have an intention to interact, the spatial activity performed by the user will change the characteristics of the digital assistant's form, but will not change the digital assistant's form, that is, will not cause the interactive state of the digital assistant to change.

[0317] The features adjusted in the first form may include, but are not limited to, the following features of the feature body used to represent the digital assistant: the posture of the feature body, the movement pattern of the feature body, the light effect of the feature body, the material of the feature body, the color of the feature body, the brightness of the feature body, and other features. For example, when the user walks in a certain divided area instead of moving to another area, the feature body of the digital assistant (such as a circular body) may rotate with the user; for another example, when the user points his finger in different directions, the feature body of the digital assistant may rotate or rotate according to the position pointed by the user's finger; for another example, when the user gradually approaches the display screen of the electronic device, the circular body of the digital assistant may rotate and change the pitch angle.

[0318] The above describes the process of switching the form of the digital assistant based on the user's spatial interaction behavior. The following will introduce the specific form of the digital assistant in detail.

[0319] Exemplarily, the first and second forms of the digital assistant each include multiple features, and the posture and / or motion pattern of at least one feature in the second form is altered compared to the first form. That is, the different forms of the digital assistant are all composed of the same multiple features, and the difference between the different forms is that the posture and / or motion pattern of at least one feature has changed.

[0320] In general, by using multiple feature bodies to construct the image of a digital assistant, different feature bodies can independently adjust the posture and movement rules. In this way, by adjusting the changes in the feature bodies, the digital assistant in different states can be represented, forming a unified style and a feedback method that responds to the input of spatial interactive behavior, which helps users understand the feedback of the digital assistant.

[0321] In one possible implementation, the multiple characteristic bodies used to represent the image of the digital assistant include a torus, a sphere, and an image body. The image body is enclosed in the sphere, and the torus surrounds the sphere. That is, in the image of the digital assistant, from the inside to the outside, they are the image body, the sphere, and the torus. In addition, the sphere can be transparent or transparent so that the user can see the image body enclosed in the sphere through the sphere. The image body can be an object of any shape designed for the digital assistant, such as an object similar to the shape of a human body, an object of irregular shape, etc.

[0322] In this solution, by constructing the image of the digital assistant based on the characteristic bodies of a torus, a sphere and an image body, it is possible to achieve a seamless and smooth transformation when switching the form of the digital assistant by adjusting the posture or movement pattern of the characteristic body, making the form change process of the digital assistant more natural and in line with human cognition, making it easier for users to understand the current interaction status with the digital assistant.

[0323] Exemplarily, when the first form or the second form is used to indicate that the digital assistant is in a non-task processing state, the first form or the second form is that multiple feature bodies move along a preset trajectory, and the sphere and the image body rotate during the movement.

[0324] Simply put, when the digital assistant is in a non-task processing state, it means that the user has no tendency to interact with the digital assistant. Therefore, the form of the digital assistant can be manifested as moving according to predetermined rules without any intention of interaction, that is, multiple feature bodies move along preset trajectories and the sphere and the image body rotate during the movement.

[0325] Exemplarily, when the first form or the second form is used to indicate that the digital assistant is in a state of waiting to receive instructions, the first form or the second form is that a portion of the area in the ring body has a light effect, and the portion of the area is facing the user's position.

[0326] When the digital assistant is in the state of waiting to receive instructions, it means that the user has the tendency to interact with the digital assistant, and the digital assistant is ready to receive the user's instructions at any time. Therefore, the form of the digital assistant can be manifested as always paying attention to the user (that is, the area with light effects is facing the user's position) and waiting for the user to issue instructions.

[0327] For example, when the second form is used to indicate that the digital assistant is in a command-receiving state, the second form is when the top or bottom of the multiple features are facing the user. That is, when the digital assistant is in a command-receiving state, it represents that the digital assistant is receiving the user's command. Therefore, the form of the digital assistant can be expressed as the digital assistant being fully focused on receiving the command (i.e., the top or bottom of the multiple features are facing the user, indicating that it is listening to the user's command).

[0328] In another possible implementation, the first form and the second form are both forms of a virtual human body, and the different forms of the virtual human body differ in any one or more of the following features: the expression, movement, and size of the virtual human body. In simple terms, the form of the digital assistant is also composed of multiple feature bodies, and the multiple feature bodies can be divided according to human body parts, that is, different feature bodies correspond to different parts of the virtual human body. For example, the multiple feature bodies can be divided into head feature bodies, eye feature bodies, mouth feature bodies, left hand feature bodies, right hand feature bodies, etc.

[0329] In this solution, by setting the form of the digital assistant to that of a virtual human body, it is possible to achieve human-like form changes when switching the form of the digital assistant in response to the user's spatial interaction input, making the form changes of the digital assistant more natural and easier for users to understand the current interaction status with the digital assistant.

[0330] For ease of understanding, the execution process of the digital assistant display method provided in the embodiment of the present application will be described in detail below with reference to specific examples.

[0331] Please refer to Figure 25, which is a schematic diagram of an application scenario of the display method of the digital assistant provided in an embodiment of the present application. The application scenario shown in Figure 25 includes a user 401 and an electronic device 402. Among them, electronic device 402 can be, for example, a smart TV in the user's home, and the digital assistant is deployed on the smart TV.

[0332] During the startup of the electronic device 402 , the user 401 can control the electronic device 402 through voice commands, so that the electronic device 402 automatically executes certain applications, such as automatically opening an application for playing movies and TV shows.

[0333] Electronic device 402 is equipped with hardware such as a distance sensor and a microphone. The distance sensor is used to detect the user's location, and the microphone is used to receive the user's voice commands. Electronic device 402 is also equipped with a digital assistant in software to process the user's voice commands and automatically execute certain applications based on the voice commands.

[0334] For the sake of convenience, the following will take a smart TV as an example of an electronic device that executes the digital assistant display method, and will introduce in detail the process of the smart TV executing the digital assistant display method.

[0335] Please refer to Figure 26, which is a schematic diagram of a digital assistant in a dormant state provided by an embodiment of the present application. As shown in Figure 26, when the smart TV is unused for an extended period of time and there are no users around the smart TV, the smart TV enters a lock screen standby state and displays the lock screen content on the screen. At this time, when the smart TV enters the lock screen standby state, the digital assistant is not displayed on the smart TV screen; the digital assistant runs in the background, waiting to be awakened.

[0336] Optionally, the lock screen content displayed on the display screen may be content unrelated to the digital assistant. Alternatively, the lock screen content displayed on the display screen may be a certain form of the digital assistant, such as the breathing form of the digital assistant.

[0337] During the startup of the smart TV, when the smart TV detects a user through a sensor, it can first determine the user's location based on the sensor, and then determine the area where the user is located based on the user's location, and determine the initial display form of the digital assistant (that is, the first form mentioned above) according to the area where the user is located.

[0338] In practical applications, the smart TV can be configured with preset logic to divide the space around the smart TV into multiple zones. When the user is in different zones, different digital assistant forms are displayed accordingly. Please refer to Figure 27, which is a schematic diagram of a zone division provided in an embodiment of the present application. As shown in Figure 27, in this embodiment, with the smart TV as the center, the space around the smart TV is divided into four zones according to different radii. From the outside to the inside, they are: unaware zone A1, presence-aware zone A2, natural behavior zone A3, and typical task zone A4. Specifically, with the smart TV as the center, three radii are defined: R1, R2, and R3, where R1>R2>R3. Therefore, the distance between unaware zone A1 and the smart TV is greater than R1, the distance between presence-aware zone A2 and the smart TV is between R1 and R2, the distance between natural behavior zone A3 and the smart TV is between R2 and R3, and the distance between typical task zone A4 and the smart TV is less than R3. The unaware zone corresponds to the breathing state, the presence-aware zone corresponds to the arousal state, and the natural behavior zone and typical task zone correspond to the listening state.

[0339] When the digital assistant of a smart TV is activated, for example, by a user using a wake-up word or remote control, the smart TV will determine the initial form of the digital assistant from the breathing state, wake-up state, and thinking state based on the user's location and display it on the screen.

[0340] For example, if a user activates the smart TV's digital assistant using a voice command on the remote control in a non-sensing zone, the initial state corresponding to the non-sensing zone is the breathing state. The smart TV will activate the digital assistant main program and display the digital assistant's breathing state. If a user activates the smart TV's digital assistant using a voice command on the remote control in a presence sensing zone, the digital assistant's awakening state will be displayed.

[0341] After the smart TV displays the initial form of the digital assistant based on the user's location, it will also capture the user's activities in the space through the sensors deployed on it, and parse the sensor data according to preset logic to identify whether there is spatial interaction behavior and the type of spatial interaction behavior from the captured user spatial activities.

[0342] For example, a smart TV uses a distance sensor to sense the user's position and distance relative to the TV. Based on changes in position and distance over time, it determines whether the user is walking toward the TV. If the distance between the user and the TV is detected to be decreasing over time and the actual distance between the user and the TV is less than a preset value, a walking input is detected. Similarly, if the user moves left or right relative to the TV, a lateral movement input to the left or right can be detected.

[0343] After detecting the user's spatial interaction behavior, the smart TV can adjust at least one feature of the digital assistant's initial form according to the type of spatial interaction behavior, thereby achieving form switching of the digital assistant.

[0344] Please refer to Figure 28, which is a schematic diagram of a switching display of the form of a digital assistant provided by an embodiment of the present application. As shown in (a) in Figure 28, when the digital assistant in the smart TV is activated, the user is in the non-perception area A1. At this time, the breathing state of the digital assistant is displayed on the smart TV, that is, the ring body moves regularly up and down along the z-axis. When the user is always walking in the non-perception area A1, the smart TV will not perceive the spatial input of the non-perception area A1. Therefore, the user's movement will not affect the form of the digital assistant, that is, the smart TV always displays the breathing state of the digital assistant.

[0345] As shown in Figure 28 (b), as the user moves from the non-sensing zone A1 to the presence sensing zone A2, the smart TV detects the user's spatial interaction behavior and approaches the smart TV. Furthermore, the user's location is within the presence sensing zone A2, so the smart TV switches the digital assistant to the awake state. When the digital assistant is awake, the torus is fixed to the equatorial plane of the sphere and rotates around the z-axis as the user moves. Furthermore, the light effect area on the torus points toward the user.

[0346] In addition, when the user moves within the presence sensing area A2 and adjusts the orientation relative to the TV, the annular body in the digital assistant rotates with the rotation of the human body and always points the light effect area towards the user. Please refer to Figure 29, which is a schematic diagram of the changes in the awake state of a digital assistant provided by an embodiment of the present application. As shown in (a) of Figure 29, when the user is within the presence sensing area A2, the digital assistant is in the awake state, and the light effect area on the annular body in the digital assistant points to the user; as shown in (b) of Figure 29, when the user walks within the presence sensing area A2 and moves from left to right, the digital assistant is still in the awake state, and the annular body in the digital assistant follows the user's movement and always points the light effect area on the annular body towards the user, that is, the light effect area on the annular body also moves from left to right. Moreover, when the user continuously approaches the smart TV, the smart TV detects the user's direction input, and the annular body can also rotate around the y-axis, resulting in a change in pitch direction. When the user always moves within the presence sensing area A2, the movement pattern of the sphere and the image body can remain unchanged.

[0347] When the user enters the natural behavior area A3 (including the natural behavior area A3 and the typical task area A4) from the presence perception area A2, the digital assistant switches from the awake state to the listening state, and the position and movement pattern of the ring body change. First, the ring body rotates 90 degrees around the y-axis from the equatorial plane of the sphere to the yoz plane. At the same time, the light effect can change, and the entire ring body is filled with light effects. The movement pattern of the sphere and the image body changes. For example, the sphere and the image body rotate 90 degrees around the y-axis once, and then rotate continuously around the z-axis. As shown in Figure 30, after the user enters the natural behavior area A3 from the presence perception area A2, the digital assistant displayed on the smart TV switches from the awake state to the listening state, indicating that the digital assistant is receiving the voice command issued by the user.

[0348] When the digital assistant is in the listening state, when the user enters a voice command, such as "What's the weather like today?", the digital assistant will parse the voice command and perform the corresponding operation. While the digital assistant is parsing the voice command and performing the corresponding operation, the smart TV may display the digital assistant's thinking state. The duration of the thinking state display is positively correlated with the length of time the digital assistant takes to parse the voice command and perform the corresponding operation.

[0349] Furthermore, when the user is in the natural behavior area A3, the digital assistant is in a listening state. When the user inputs voice commands to the digital assistant, they often need to first input a voice command as a wake-up command, such as "Hello, Xiaoyi" or "Xiaoyi Xiaoyi," so that the digital assistant can receive and interpret the voice command after the wake-up command. However, when the user enters the typical task area A4, the digital assistant remains in a listening state, but the user can directly input voice commands to the digital assistant without having to input a voice command as a wake-up command.

[0350] It should be noted that after the digital assistant receives and parses the user's voice command, the digital assistant can change its own form display according to the content of the user's voice command. For example, when the digital assistant parses the voice command to play music, the form of the digital assistant can be a dynamic change in color or a dynamic change in rotation rhythm following the rhythm of the music, that is, the digital assistant will also adjust its own form display accordingly when performing the music playback operation. For another example, when the digital assistant parses the voice command to turn on the lights, turn off the lights, or report the weather, the form of the digital assistant can change in brightness, color, or material following the action of turning on the lights, turning off the lights, or the type of weather reported.

[0351] In addition, when the digital assistant parses voice commands and performs corresponding tasks based on the content of the voice commands, the digital assistant can change its own display form according to the progress of its task execution. For example, the digital assistant can gradually change the material of the ring body, change the area of ​​the ring body filled with specific materials, etc. according to the progress of the task.

[0352] The above describes the changes in the form of the digital assistant displayed on the smart TV when the user moves from an area farther away from the smart TV to an area closer to the smart TV. In fact, the user may also move from an area closer to the smart TV to an area farther away from the smart TV. In this case, the smart TV can decide the form of the digital assistant based on the user's spatial interaction behavior and the area where the user is located after moving, thereby achieving the form switching of the digital assistant. For example, when the user moves from the natural behavior area A3 to the presence perception area A2, the digital assistant displayed on the smart TV switches from the listening state to the awake state; when the user moves from the presence perception area A2 to the non-perception area A1, the digital assistant displayed on the smart TV switches from the awake state to the breathing state.

[0353] The above describes the morphological changes of the digital assistant when its main image is composed of a torus, a sphere, and an image body. The following describes the morphological changes of the digital assistant when its main image is composed of a virtual human body.

[0354] Specifically, the digital assistant's main image can be a virtual human body, and the digital assistant's form is also composed of multiple features. The multiple features can be divided according to human body parts, that is, different features correspond to different parts of the virtual human body. For example, the multiple features can be divided into head features, eye features, mouth features, left hand features, right hand features, etc.

[0355] If the digital assistant's main form is a virtual human, the smart TV can also divide the space around the smart TV into multiple zones, with the smart TV's location as the center. This way, when the digital assistant is activated, the smart TV can display the corresponding form of the digital assistant based on the user's location in the space and the user's spatial interaction behavior.

[0356] Specifically, in this embodiment, the space near the smart TV can be divided into three areas: area 1 is 1.2-3.7 meters away from the smart TV, area 2 is 0.5-1.2 meters away from the smart TV, and area 3 is less than 0.5 meters away from the smart TV.

[0357] When the smart TV recognizes that the user is in area 1, the smart TV can consider the distance between the user and the digital assistant as an identification distance, and the spatial interaction behavior of the user obtained by the smart TV includes: the user faces the digital assistant, the user's gaze is on the digital assistant, and the user gazes at the digital assistant for more than 3 seconds, then the smart TV can display the form of the digital assistant as a form with expression and action. Specifically, the form of the digital assistant can be a virtual human body representing the digital assistant looking at the user, winking and smiling at the user, and the body and head slightly swaying, indicating that the digital assistant is willing to interact with the user. In addition, if there are multiple users in area 1, and multiple users have the above spatial interaction behaviors, 1.2 meters can be used as the dividing distance to control the virtual human body representing the digital assistant to look in the direction of different users in turn.

[0358] When the smart TV identifies the user as being in zone 2, it can assume the distance between the user and the digital assistant is a greeting distance. If the smart TV detects the user's spatial interaction behavior, including facing the digital assistant, maintaining gaze on the digital assistant, and gazing at the digital assistant for more than three seconds, the smart TV can display the digital assistant in an expression with facial expressions and gestures, and output a corresponding voice message. Specifically, the digital assistant can be represented by a virtual person waving and greeting the user. Furthermore, the voice message output by the smart TV can be inquiring about the user's needs, such as "How can I help you?"

[0359] When the smart TV recognizes that the user is in area 3, the smart TV can consider the distance between the user and the digital assistant to be a manual touch screen operation distance, and the spatial interaction behavior of the user obtained by the smart TV includes: the user facing the digital assistant, the user's gaze point is on the digital assistant, and the user gazes at the digital assistant for more than 3 seconds. Then, the smart TV can display task information that requires touch screen operation on the display screen and proportionally reduce the main image of the digital assistant. Specifically, the projected height of the main image of the digital assistant on the display screen can be H = the actual height of the main image * 0.5 / 1.2. Where H is the projected height of the main image of the digital assistant on the screen, and the actual height of the main image of the digital assistant is the real height of the virtual human body representing the digital assistant. The real height of the male virtual human body can be, for example, 174 cm, and the real height of the female virtual human body can be, for example, 161 cm. The specific value can be adjusted according to actual conditions, and this embodiment does not specifically limit this.

[0360] The above describes in detail the method provided in the embodiment of the present application. Next, the device provided in the embodiment of the present application for executing the above method will be introduced.

[0361] Please refer to Figure 31, which is a schematic diagram of the structure of a display device for a digital assistant virtual image provided by an embodiment of the present application. As shown in Figure 31, the display device for the digital assistant virtual image is applied to an electronic device, and the virtual image includes multiple features. The device includes:

[0362] Display module 3101, configured to display a virtual image in a first form on a first interface;

[0363] The display module 3101 is further configured to detect a signal instruction generated by the electronic device and display the virtual image in the second form;

[0364] Among them, the signal instruction is generated based on the first user input, the first form and the second form display at least one of multiple feature bodies, and the change from the first form to the second form means that at least one feature body undergoes at least one change in shape, position, movement pattern, and display area.

[0365] In one possible implementation, the plurality of features include a first feature, and the shape change of the first feature includes at least one of the following changes: enlargement, reduction, stretching, compression, rotation, and switching between a main form, a geometric array, or a strip.

[0366] In a possible implementation, the plurality of feature bodies further includes a second feature body, wherein the main shape of the first feature body is a ring, the second feature body is a sphere, and the center of the ring and the center of the sphere are concentric;

[0367] The position change of the first feature body includes at least one of a change in orbital position after flipping around the sphere and a position change caused by overall movement or rotation of the virtual image;

[0368] The motion law of the first characteristic body includes that the ring rotates around the sphere according to a preset trajectory with the center of the ring as the fulcrum;

[0369] The motion law of the second characteristic body includes that the sphere moves back and forth according to a preset law along a straight line passing through the center of the ring and perpendicular to the plane where the ring is located.

[0370] In a possible implementation, the display module 3101 is further configured to detect a second user input and display a third feature body on the first feature body, where the third feature body can move on the first feature body and indicate the user's position.

[0371] In a possible implementation, the avatar has a breathing state, and displaying the avatar in a first form on the first interface includes:

[0372] If the first interface is a screen-off interface, the first form is a breathing form including a first feature body and a second feature body, and the first feature body is displayed in the main form.

[0373] In a possible implementation, when the first interface is a screen-off interface, the virtual image in a breathing state moves back and forth on the screen according to a preset pattern.

[0374] In a possible implementation, when the first interface is the screen-off interface and the first state is the breathing state, the display module 3101 is further configured to:

[0375] A second user input is detected, and a third feature is displayed on the first feature. The third feature is movable on the first feature and indicates the user's position.

[0376] In one possible implementation, the avatar has a waiting state and a listening state, and the first interface is a lock screen interface or any screen interface after unlocking;

[0377] Displaying the virtual image in a first form on the first interface includes:

[0378] The first state is displayed on the first interface in a waiting-to-wake-up state as a first form, the waiting-to-wake-up state includes a first feature body, and the first feature body is displayed in a long strip as a navigation prompt line, and the navigation prompt line is used as a reference line for screen gesture navigation; wherein, when no input for waking up the digital assistant is received, the waiting-to-wake-up state is permanently displayed on the first interface;

[0379] or,

[0380] A third user input is detected and displayed on the first interface in a listening state as the first form. The third user input is used to wake up the digital assistant. The listening state includes a first feature body and a second feature body, and the first feature body is displayed in the main form.

[0381] In one possible implementation, the first interface is a lock screen interface or any screen interface after unlocking, and the first state is a listening state;

[0382] Displaying the virtual image in the first form on the first interface specifically includes:

[0383] The first feature is first displayed as a geometric array;

[0384] The first feature body then changes from the geometric array to the main form, and the second feature body is displayed.

[0385] In one possible implementation, the first feature is first displayed as a geometric array, specifically:

[0386] Displays a portion of the array's geometry at the initial display position, and then displays the geometry on the entire screen.

[0387] In one possible implementation, the initial display position is determined based on user input;

[0388] And when no user input is detected, the initial position is the area where the virtual image is displayed.

[0389] In a possible implementation, the virtual image is displayed in the first form on the first interface, specifically:

[0390] Displaying a virtual image at a preset position on the first interface; or,

[0391] The virtual image is overlaid and displayed on a target content element in the first interface, wherein the target content element is a content element of a preset type.

[0392] In one possible implementation, when the virtual image is displayed overlaid on the target content element in the first interface, the gesture event interface of the target content element is copied to the virtual image so that performing a gesture on the virtual image can realize the gesture corresponding function of the target content element.

[0393] In a possible implementation, the first interface is a camera interface, and the target content element is a shutter control in the camera interface.

[0394] In one possible implementation, the first interface is the lock screen interface or any screen interface after unlocking, the first form is the wake-up state, and the second form is the listening state; the signal instruction is generated based on the user performing a preset gesture on the digital assistant in the first form.

[0395] In a possible implementation, the plurality of features further includes a second feature;

[0396] In the second form, the first feature body is in the main form of a ring, the second feature body is a sphere, and the ring is located on the equator of the sphere in a top-down perspective.

[0397] In a possible implementation, the virtual image further has a task execution state, and the task execution state includes a first feature body;

[0398] In at least one process interface during the digital assistant's execution of an automated task, the first feature body in the task execution state produces at least one of a shape change and a position change.

[0399] In a possible implementation, the first feature body is a geometric array, and the display position of the geometric array matches the target control in the process interface, so as to simulate a click event of the target control to enter the next process interface.

[0400] In a possible implementation, the first state is a listening state, and the second state is a task execution state. Both the listening state and the task execution state include a first feature body, and the first feature bodies in the listening state and the task execution state have different forms.

[0401] In the task execution state, the form of the first feature body includes: a geometric array within the interface, the outer frame of a dialogue card, the outer frame of an input box when calling a digital assistant input, or the outer frame of a task result page.

[0402] In a possible implementation, the first feature body has different morphological changes at different task progresses of the automation task.

[0403] In a possible implementation, when the automation task is an intelligent recognition task, the first feature body is changed from a geometric array covering the entire screen to a geometric array displayed only on the recognized executable object.

[0404] In a possible implementation, in the task execution state, the morphological change of the first feature body includes:

[0405] The first feature body is first displayed in the main form of a ring on the periphery of the second feature body;

[0406] The first feature body is then separated from the second feature body by the ring and is enlarged and displayed to serve as the outer frame of the conversation card;

[0407] The first feature body changes from the outer frame of the dialogue card to the geometric array inside the dialogue bubble;

[0408] After the automated task is completed, the avatar is displayed in a listening state.

[0409] In a possible implementation, the apparatus further includes:

[0410] The acquisition module 3102 is configured to acquire scene information and display a first feature body among the plurality of feature bodies in a target form based on the scene information.

[0411] In a possible implementation, the virtual image has a waiting-to-awaken state, the waiting-to-awaken state includes at least a first characteristic body, and the first characteristic body is displayed in a long strip;

[0412] The display module 3101 is further configured to display a second feature body in response to detecting that the focus of the human eye falls on the area where the screen is located.

[0413] In a possible implementation, the display module 3101 is further configured to, in response to detecting a human voice, cause the second feature body to move relative to the first feature body toward a direction closer to a source of the human voice.

[0414] In one possible implementation, the electronic device is a smart car, and the display module 3101 is further configured to:

[0415] The vehicle function or warning information is indicated by the deformation or movement of the first feature body among the multiple feature bodies.

[0416] Please refer to Figure 32, which is a structural diagram of an execution device provided in an embodiment of the present application. The execution device 3200 can be specifically manifested as a mobile phone, a tablet, a laptop computer, a smart wearable device, a server, etc., which is not limited here. Specifically, the execution device 3200 includes: a receiver 3201, a transmitter 3202, a processor 3203 and a memory 3204 (wherein the number of processors 3203 in the execution device 3200 can be one or more, and Figure 32 takes one processor as an example), wherein the processor 3203 may include an application processor 32031 and a communication processor 32032. In some embodiments of the present application, the receiver 3201, the transmitter 3202, the processor 3203 and the memory 3204 may be connected via a bus or other means.

[0417] Memory 3204 may include read-only memory and random access memory, and provides instructions and data to processor 3203. A portion of memory 3204 may also include non-volatile random access memory (NVRAM). Memory 3204 stores processor and operation instructions, executable modules, or data structures, or subsets or extended sets thereof. The operation instructions may include various operation instructions for implementing various operations.

[0418] Processor 3203 controls the operation of the execution device. In specific applications, the various components of the execution device are coupled together via a bus system. In addition to a data bus, the bus system may also include a power bus, a control bus, and a status signal bus. However, for clarity, all bus systems are referred to as a bus system in the figure.

[0419] The method disclosed in the above embodiment of the present application can be applied to the processor 3203, or implemented by the processor 3203. The processor 3203 can be an integrated circuit chip with signal processing capabilities. During the implementation process, each step of the above method can be completed by the hardware integrated logic circuit in the processor 3203 or the instructions in the form of software. The above-mentioned processor 3203 can be a general-purpose processor, a digital signal processor (digital signal processing, DSP), a microprocessor or a microcontroller, and can further include an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.

[0420] The processor 3203 can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as being executed by a hardware decoding processor, or can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium mature in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 3204, and the processor 3203 reads the information in the memory 3204 and completes the steps of the above method in combination with its hardware.

[0421] Receiver 3201 can be used to receive input digital or character information and generate signal input related to executing device-related settings and function control. Transmitter 3202 can be used to output digital or character information through the first interface. Transmitter 3202 can also be used to send instructions to the disk pack through the first interface to modify data in the disk pack. Transmitter 3202 can also include a display device such as a display screen.

[0422] The electronic device provided in the embodiments of the present application may specifically be a chip, and the chip includes: a processing unit and a communication unit. The processing unit may be, for example, a processor, and the communication unit may be, for example, an input / output interface, a pin, or a circuit. The processing unit may execute computer-executable instructions stored in the storage unit so that the chip in the execution device executes the method described in the above embodiment. Optionally, the storage unit is a storage unit in the chip, such as a register, a cache, etc. The storage unit may also be a storage unit located outside the chip in the wireless access device, such as a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM), etc.

[0423] Please refer to Figure 33, which is a schematic diagram of the structure of a computer-readable storage medium provided in an embodiment of the present application. The present application also provides a computer-readable storage medium. In some embodiments, the method disclosed in Figure 3 above can be implemented as computer program instructions encoded in a machine-readable format on a computer-readable storage medium or on other non-transitory media or products.

[0424] 33 schematically illustrates a conceptual partial view of an example computer-readable storage medium including a computer program for executing a computer process on a computing device, arranged in accordance with at least some embodiments presented herein.

[0425] In one embodiment, computer readable storage medium 3300 is provided using signal bearing medium 3301. Signal bearing medium 3301 may include one or more program instructions 3302 that, when executed by one or more processors, may provide the functionality or portions of the functionality described above with respect to FIG.

[0426] In some examples, the signal bearing medium 3301 may include a computer readable medium 3303 such as, but not limited to, a hard drive, a compact disk (CD), a digital video disk (DVD), a digital tape, a memory, a ROM or RAM, or the like.

[0427] In some embodiments, signal-bearing medium 3301 may include computer-recordable media 3304, such as, but not limited to, memory, read / write (R / W) CDs, R / W DVDs, and the like. In some embodiments, signal-bearing medium 3301 may include communication media 3305, such as, but not limited to, digital and / or analog communication media (e.g., fiber optic cables, waveguides, wired communication links, wireless communication links, and the like). Thus, for example, signal-bearing medium 3301 may be communicated via a wireless form of communication media 3305 (e.g., a wireless communication medium conforming to IEEE 802.X standards or other transmission protocols).

[0428] The one or more program instructions 3302 may be, for example, computer-executable instructions or logic-implemented instructions. In some examples, the computing device may be configured to provide various operations, functions, or actions in response to the program instructions 3302 communicated to the computing device via one or more of computer-readable media 3303, computer-recordable media 3304, and / or communication media 3305.

[0429] It should also be noted that the device embodiments described above are merely illustrative, in which the units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. In addition, in the drawings of the device embodiments provided in this application, the connection relationship between the modules indicates that there is a communication connection between them, which can be specifically implemented as one or more communication buses or signal lines.

[0430] Through the description of the above embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course can also be implemented by special hardware including application-specific integrated circuits, special CPUs, special memories, special components, etc. In general, all functions performed by computer programs can be easily implemented with corresponding hardware, and the specific hardware structures used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits, etc. However, for the present application, software program implementation is a better implementation method in most cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer's floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk or optical disk, etc., and includes a number of instructions to enable a computer device (which can be a personal computer, training equipment, or network equipment, etc.) to execute the methods of each embodiment of the present application.

[0431] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware, or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0432] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, computer, training equipment or data center to another website, computer, training equipment or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as a training equipment, data center, etc. that includes one or more available media integrations. Available media can be magnetic media, (e.g., floppy disk, hard disk, tape), optical media (e.g., DVD), or semiconductor media (e.g., solid-state drive (SSD)), etc.

Claims

1. A method for displaying a digital assistant virtual image, applied to an electronic device, characterized in that: The virtual image includes a plurality of features, and the method includes: Displaying the virtual image in a first form on a first interface; Detecting a signal instruction generated by the electronic device, and displaying the virtual image in a second form; Among them, the signal instruction is generated based on the first user input, the first form and the second form display at least one of the multiple feature bodies, and the change from the first form to the second form means that at least one feature body undergoes at least one of a shape change, a position change, a motion law change, and a display area change.

2. The method according to claim 1, characterized in that The plurality of features include a first feature, and the shape change of the first feature includes at least one of the following changes: enlargement, reduction, stretching, compression, rotation, and switching between a main form, a geometric array, or a strip.

3. The method according to claim 2, characterized in that The plurality of feature bodies further include a second feature body, wherein the main shape of the first feature body is a ring, the second feature body is a sphere, and the center of the ring and the center of the sphere are co-pointed; The position change of the first feature body includes at least one of a change in orbital position after flipping around the sphere and a position change caused by overall movement or rotation of the virtual image; The movement law of the first characteristic body includes that the ring rotates around the sphere according to a preset trajectory with the center of the circle as the fulcrum; The motion law of the second characteristic body includes that the sphere reciprocates along a straight line passing through the center of the circle and perpendicular to the plane where the circle is located according to a preset law.

4. The method according to claim 3, characterized in that The method further comprises: A second user input is detected, and a third feature is displayed on the first feature, wherein the third feature is movable on the first feature and indicates the user's position.

5. The method according to any one of claims 1 to 3, characterized in that: The virtual image has a breathing state, and displaying the virtual image in a first form on the first interface includes: If the first interface is a screen-off interface, the first form is a breathing form including a first feature body and a second feature body, and the first feature body is displayed in a main form.

6. The method according to claim 5, characterized in that When the first interface is a screen-off interface, the breathing virtual image moves back and forth on the screen according to a preset rule.

7. The method according to claim 5, characterized in that When the first interface is a screen-off interface and the first state is a breathing state, the method further includes: A second user input is detected, and a third feature is displayed on the first feature, wherein the third feature is movable on the first feature and indicates the user's position.

8. The method according to any one of claims 1 to 4, characterized in that: The virtual image has a waiting state and a listening state, and the first interface is a lock screen interface or any screen interface after unlocking; The step of displaying the virtual image in a first form on the first interface includes: The first interface is displayed in a waiting-to-be-awakened state as a first form, the waiting-to-be-awakened state includes a first feature body, and the first feature body is displayed in a long strip as a navigation prompt line, and the navigation prompt line is used as a reference line for screen gesture navigation; wherein, when no input for waking up the digital assistant is received, the waiting-to-be-awakened state is permanently displayed on the first interface; or, A third user input is detected and the listening state is displayed on the first interface as the first form. The third user input is used to wake up the digital assistant. The listening state includes a first feature body and a second feature body, and the first feature body is displayed in the main form.

9. The method according to claim 8, characterized in that The first interface is a lock screen interface or any screen interface after unlocking, and the first state is a listening state; The step of displaying the virtual image in a first form on the first interface specifically includes: The first feature is first displayed in a geometric array; The first feature body is then changed from a geometric array to a main form, and the second feature body is displayed.

10. The method according to claim 9, characterized in that The first feature is first displayed in a geometric array, specifically: The geometrical figures of the partial array are displayed at the initial display position, and then the geometrical figures are displayed on the entire screen.

11. The method according to claim 10, characterized in that The initial display position is determined based on user input; And when no user input is detected, the initial position is the area where the virtual image is displayed.

12. The method according to any one of claims 1 to 11, characterized in that: The displaying of the virtual image in the first form on the first interface is specifically: Displaying the virtual image at a preset position on the first interface; or, The virtual image is overlaid and displayed on a target content element in the first interface, wherein the target content element is a content element of a preset type.

13. The method according to claim 12, characterized in that When the virtual image is displayed overlaid on the target content element in the first interface, the gesture event interface of the target content element is copied to the virtual image, so that performing a gesture on the virtual image can realize the gesture corresponding function of the target content element.

14. The method according to claim 13, characterized in that The first interface is a camera interface, and the target content element is a shutter control in the camera interface.

15. The method according to claim 8, characterized in that The first interface is a lock screen interface or any screen interface after unlocking, the first form is a wake-up state, and the second form is a listening state; the signal instruction is generated after the user performs a preset gesture on the digital assistant in the first form.

16. The method according to claim 15, characterized in that The plurality of features further includes a second feature; In the second form, the first feature body is in the main form of a ring, the second feature body is a sphere, and the ring is located on the equator of the sphere in a top-down perspective.

17. The method according to claim 8, characterized in that The virtual image also has a task execution state, and the task execution state includes the first feature body; In at least one process interface during the execution of an automated task by the digital assistant, the first feature body in the task execution state corresponds to at least one of a shape change and a position change.

18. The method according to claim 17, characterized in that The first feature body is a geometric array, and the display position of the geometric array matches the target control in the process interface, which is used to simulate a click event of the target control to enter the next process interface.

19. The method according to claim 17, characterized in that The first state is a listening state, and the second state is a task execution state. Both the listening state and the task execution state include the first feature body, and the first feature bodies in the listening state and the task execution state have different states respectively. In the task execution state, the shape of the first feature body includes: a geometric array within the interface, an outer frame of a dialog card, an outer frame of an input box when calling a digital assistant input, or an outer frame of a task result page.

20. The method according to claim 19, characterized in that Under different task progress of the automation task, the first feature body has different morphological changes.

21. The method according to claim 19, characterized in that When the automation task is an intelligent recognition task, the first feature body changes from a geometric array covering the entire screen to a geometric array displayed only on the recognized executable object.

22. The method according to claim 19 or 20, characterized in that In the task execution state, the morphological changes of the first feature body include: The first feature body is first displayed in the main form of a ring on the periphery of the second feature body; The first feature body is then separated from the second feature body by a ring and is enlarged and displayed to serve as an outer frame of the conversation card; The first feature body changes from the outer frame of the dialogue card to a geometric array within the dialogue bubble; After the automated task is completed, the virtual image is displayed in a listening state.

23. The method according to claims 1-4, characterized in that The method further comprises: Scene information is acquired, and a first feature body among the plurality of feature bodies is displayed in a target form based on the scene information.

24. The method according to claims 1-4, characterized in that The virtual image has a waiting state, and the waiting state includes at least a first characteristic body, and the first characteristic body is displayed in a long strip; In response to detecting that the focus of the human eye falls on the area where the screen is located, a second feature body is displayed.

25. The method according to claim 24, characterized in that The method further comprises: In response to detecting a human voice, the second feature body moves relative to the first feature body toward a direction approaching a source of the human voice.

26. The method according to claims 1-4, characterized in that The electronic device is a smart car, and the method further includes: The vehicle function or warning information is indicated by the deformation or movement of the first feature body among the multiple feature bodies.

27. A display device for a digital assistant virtual image, applied to an electronic device, characterized in that: The virtual image includes a plurality of features; the device includes: A display module, configured to display the virtual image in a first form on a first interface; The display module is further configured to detect a signal instruction generated by the electronic device and display the virtual image in a second form; Among them, the signal instruction is generated based on the first user input, the first form and the second form display at least one of the multiple feature bodies, and the change from the first form to the second form means that at least one feature body undergoes at least one of a shape change, a position change, a motion law change, and a display area change.

28. The device according to claim 27, characterized in that The plurality of features include a first feature, and the shape change of the first feature includes at least one of the following changes: enlargement, reduction, stretching, compression, rotation, and switching between a main form, a geometric array, or a strip.

29. The device according to claim 28, characterized in that The plurality of feature bodies further include a second feature body, wherein the main shape of the first feature body is a ring, the second feature body is a sphere, and the center of the ring and the center of the sphere are co-pointed; The position change of the first feature body includes at least one of a change in orbital position after flipping around the sphere and a position change caused by overall movement or rotation of the virtual image; The movement law of the first characteristic body includes that the ring rotates around the sphere according to a preset trajectory with the center of the circle as the fulcrum; The motion law of the second characteristic body includes that the sphere reciprocates along a straight line passing through the center of the circle and perpendicular to the plane where the circle is located according to a preset law.

30. The device according to claim 29, characterized in that The display module is further configured to detect a second user input and display a third feature body on the first feature body, wherein the third feature body can move on the first feature body and indicate the user's position.

31. The device according to any one of claims 27 to 29, characterized in that: The virtual image has a breathing state, and displaying the virtual image in a first form on the first interface includes: If the first interface is a screen-off interface, the first form is a breathing form including a first feature body and a second feature body, and the first feature body is displayed in a main form.

32. The device according to claim 31, characterized in that When the first interface is a screen-off interface, the breathing virtual image moves back and forth on the screen according to a preset rule.

33. The device according to claim 31, characterized in that When the first interface is a screen-off interface and the first state is a breathing state, the display module is further used to: A second user input is detected, and a third feature is displayed on the first feature, wherein the third feature is movable on the first feature and indicates the user's position.

34. The device according to any one of claims 27 to 30, characterized in that: The virtual image has a waiting state and a listening state, and the first interface is a lock screen interface or any screen interface after unlocking; The step of displaying the virtual image in a first form on the first interface includes: The first interface is displayed in a waiting-to-be-awakened state as a first form, the waiting-to-be-awakened state includes a first feature body, and the first feature body is displayed in a long strip as a navigation prompt line, and the navigation prompt line is used as a reference line for screen gesture navigation; wherein, when no input for waking up the digital assistant is received, the waiting-to-be-awakened state is permanently displayed on the first interface; or, A third user input is detected and the listening state is displayed on the first interface as the first form. The third user input is used to wake up the digital assistant. The listening state includes a first feature body and a second feature body, and the first feature body is displayed in the main form.

35. The device according to claim 34, characterized in that The first interface is a lock screen interface or any screen interface after unlocking, and the first state is a listening state; The step of displaying the virtual image in a first form on the first interface specifically includes: The first feature is first displayed in a geometric array; The first feature body is then changed from a geometric array to a main form, and the second feature body is displayed.

36. The device according to claim 35, characterized in that The first feature is first displayed in a geometric array, specifically: The geometrical figures of the partial array are displayed at the initial display position, and then the geometrical figures are displayed on the entire screen.

37. The device according to claim 36, characterized in that The initial display position is determined based on user input; And when no user input is detected, the initial position is the area where the virtual image is displayed.

38. The device according to any one of claims 27 to 37, characterized in that: The displaying of the virtual image in the first form on the first interface is specifically: Displaying the virtual image at a preset position on the first interface; or, The virtual image is overlaid and displayed on a target content element in the first interface, wherein the target content element is a content element of a preset type.

39. The device according to claim 38, characterized in that When the virtual image is displayed overlaid on the target content element in the first interface, the gesture event interface of the target content element is copied to the virtual image, so that performing a gesture on the virtual image can realize the gesture corresponding function of the target content element.

40. The device according to claim 39, characterized in that The first interface is a camera interface, and the target content element is a shutter control in the camera interface.

41. The device according to claim 34, characterized in that The first interface is a lock screen interface or any screen interface after unlocking, the first form is a wake-up state, and the second form is a listening state; the signal instruction is generated after the user performs a preset gesture on the digital assistant in the first form.

42. The device according to claim 39, characterized in that The plurality of features further includes a second feature; In the second form, the first feature body is in the main form of a ring, the second feature body is a sphere, and the ring is located on the equator of the sphere in a top-down perspective.

43. The device according to claim 34, characterized in that The virtual image also has a task execution state, and the task execution state includes the first feature body; In at least one process interface during the execution of an automated task by the digital assistant, the first feature body in the task execution state corresponds to at least one of a shape change and a position change.

44. The device according to claim 43, characterized in that The first feature body is a geometric array, and the display position of the geometric array matches the target control in the process interface, which is used to simulate a click event of the target control to enter the next process interface.

45. The device according to claim 43, characterized in that The first state is a listening state, and the second state is a task execution state. Both the listening state and the task execution state include the first feature body, and the first feature bodies in the listening state and the task execution state have different states respectively. In the task execution state, the shape of the first feature body includes: a geometric array within the interface, an outer frame of a dialog card, an outer frame of an input box when calling a digital assistant input, or an outer frame of a task result page.

46. ​​The device according to claim 45, characterized in that Under different task progress of the automation task, the first feature body has different morphological changes.

47. The device according to claim 46, characterized in that When the automation task is an intelligent recognition task, the first feature body changes from a geometric array covering the entire screen to a geometric array displayed only on the recognized executable object.

48. The device according to claim 45 or 46, characterized in that In the task execution state, the morphological changes of the first feature body include: The first feature body is first displayed in the main form of a ring on the periphery of the second feature body; The first feature body is then separated from the second feature body by a ring and is enlarged and displayed to serve as an outer frame of the conversation card; The first feature body changes from the outer frame of the dialogue card to a geometric array within the dialogue bubble; After the automated task is completed, the virtual image is displayed in a listening state.

49. The device according to claims 27-30, characterized in that The device also includes: The acquisition module is used to acquire scene information and display a first feature body among the multiple feature bodies in a target form based on the scene information.

50. The device according to claims 27-30, characterized in that The virtual image has a waiting state, and the waiting state includes at least a first characteristic body, and the first characteristic body is displayed in a long strip; The display module is further configured to display a second feature body in response to detecting that the focus of a person's sight falls on the area where the screen is located.

51. The device according to claim 50, characterized in that The display module is further configured to cause the second feature body to move toward a direction close to a sound source of the human voice relative to the first feature body in response to detecting the human voice.

52. The device according to claims 27-30, characterized in that The electronic device is a smart car, and the display module is also used for: The vehicle function or warning information is indicated by the deformation or movement of the first feature body among the multiple feature bodies.

53. A display device for a digital assistant virtual image, characterized in that: The device comprises a memory and a processor; the memory stores codes, the processor is configured to execute the codes, and when the codes are executed, the device executes the method according to any one of claims 1 to 26.

54. A computer storage medium, characterized in that The computer storage medium stores instructions, which, when executed by a computer, cause the computer to implement the method of any one of claims 1 to 26.

55. A computer program product, characterized in that The computer program product stores instructions, which, when executed by a computer, cause the computer to implement the method according to any one of claims 1 to 26.

Images