Terminal voice interaction method, system and corresponding terminal device

Abstract

A terminal interaction method, system and device are proposed. The terminal interaction method comprises: presenting an interaction scene, wherein the interaction scene comprises one or more interaction objects; receiving a user's operation on the interaction object; and presenting the operation result of the operation in the interaction scene. The present invention provides a specific interaction scene and enables the user to operate the interaction object through voice interaction in the interaction scene, such as obtaining the operation effect on the interaction object, thereby improving participation, and can more clearly display the various functions of the interaction object, thereby promoting the promotion of the interaction object.

Description

Technical Field

[0001] The present disclosure relates to an interaction field, and in particular to a terminal voice interaction method, system and corresponding terminal equipment. Background Art

[0002] In the Internet era, all kinds of content information are delivered to users in various forms. In order to carry out commercial promotion, various types of APPs have various operation resource positions. Different operation resource positions undertake various types of information that merchants want to deliver to users. The content of these promotional information is fixed. Even if different products are delivered to different users, the promotional information of these products itself is unchanged and usually hinders the normal APP operation of users.

[0003] To this end, an improved way of promoting information is needed. Summary of the invention

[0004] A technical problem to be solved by the present disclosure is to provide a terminal voice interaction solution, which provides a specific interaction scenario and enables users to operate interaction objects through voice interaction in the interaction scenario, such as obtaining the operation effects on the interaction objects, thereby improving participation and being able to more clearly display the various functions of the interaction objects.

[0005] According to a first aspect of the present disclosure, a terminal voice interaction method is proposed, comprising: presenting an interaction scene, wherein the interaction scene includes one or more interaction objects; receiving a user's operation on the interaction object; and presenting an operation result of the operation in the interaction scene.

[0006] According to the second aspect of the present disclosure, a terminal voice interaction system is proposed, including multiple terminals and a server, wherein the multiple terminals communicate with the server, and the server is used to: send an interaction scene, wherein the interaction scene includes one or more interaction objects; obtain the uploaded user's operation on the interaction object; generate and send an operation response instruction, and the terminal is used to: receive the sent interaction scene and present it; receive and upload the user's operation on the interaction object; receive the sent operation response instruction; and based on the operation response instruction, present the operation result of the operation in the interaction scene.

[0007] According to a third aspect of the present disclosure, a terminal display method is proposed, comprising: displaying a virtual interaction scene page, the virtual scene page including one or more virtual interaction objects; and in response to a user's operation on the virtual interaction object, displaying the operation result of the operation in the virtual scene page.

[0008] According to the fourth aspect of the present disclosure, a terminal device is proposed, comprising: an output device for presenting an interactive scene, the interactive scene including one or more interactive objects; an input device for receiving a user's operation on the interactive object; and a processing device for presenting the operation result of the operation in the interactive scene using the presentation device based on the operation.

[0009] According to the fifth aspect of the present disclosure, a terminal voice interaction method is proposed, including: presenting commercial promotion content based on predetermined conditions in a current interaction scenario; receiving a user's voice command operation on the commercial promotion content; and presenting the operation result of the operation on the commercial promotion content in the interaction scenario.

[0010] According to the sixth aspect of the present disclosure, a voice interaction method is proposed, including: playing a film or TV series, wherein interactive objects are presented in the playback scene of the film or TV series; receiving a user's voice command operation on the interactive object; and presenting the operation result of the operation in the playback scene of the film or TV series.

[0011] According to the seventh aspect of the present disclosure, a commercial promotion screen voice interaction method is proposed, including: prompting a user to perform voice interaction; obtaining the user's voice interaction input; presenting commercial promotion content; receiving the user's voice command operation on the commercial promotion content; and presenting the operation result of the voice command operation on the commercial promotion content.

[0012] The present invention provides interactive scenes for users, receives user operations and displays operation results, so that users can participate in the promotion process itself and can accurately know the various functions of the operation object. The present invention is particularly suitable for being implemented as an intelligent promotion solution for voice interaction within an APP, that is, users within the APP can perform voice interaction in the promotion scene, and the promotion scene presents different content information according to the user's voice instructions, thereby displaying the services or products to be promoted, improving user participation and the fun of information promotion, as well as the accuracy of delivery. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The above and other objects, features and advantages of the present disclosure will become more apparent through a more detailed description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, wherein like reference numerals generally represent like components in the exemplary embodiments of the present disclosure.

[0014] Figure 1 A schematic flow chart of a terminal interaction method according to an embodiment of the present invention is shown.

[0015] Figure 2 An example of a terminal interaction system capable of implementing the present invention is shown.

[0016] Figure 3 An example of a server participating in the terminal interaction of the present invention is shown.

[0017] Figure 4 A schematic flow chart of a terminal display method according to an embodiment of the present invention is shown.

[0018] Figure 5 A schematic diagram of the structure of a terminal device that can be used to implement the above interaction method according to an embodiment of the present invention is shown.

[0019] Figure 6A-6B An example of a terminal interaction scenario according to the present invention is shown. DETAILED DESCRIPTION

[0020] The preferred embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the preferred embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to make the present disclosure more thorough and complete, and to fully convey the scope of the present disclosure to those skilled in the art.

[0021] In the Internet era, all kinds of content information are delivered to users in various forms. In order to carry out commercial promotion, various types of APPs have various operation resource positions. Different operation resource positions undertake various types of information that merchants want to deliver to users. The content of these promotional information is fixed. Even if different products are delivered to different users, the promotional information of these products itself is unchanged and usually hinders the normal APP operation of users.

[0022] To this end, the present invention proposes a terminal interaction solution, which improves participation by providing a specific interaction scenario and enabling users to learn the operation effects on the interaction object through active operations in the interaction scenario, and can more clearly display the various functions of the interaction object.

[0023] Figure 1 A flow chart of a terminal interaction method according to an embodiment of the present invention is shown. Here, terminal interaction may refer to the interaction between a terminal device and its user, so the terminal interaction method is a method in which the terminal performs corresponding operations in response to the user's input. In different embodiments, the terminal device may be implemented as various types of mobile terminals, such as smartphones and tablet computers with interactive APPs installed, special AR helmets, and the like.

[0024] In step S110, the interactive scene is presented. The presented interactive scene includes one or more interactive objects. Here, "presentation" refers to letting the user know through various perception methods. In one embodiment, presenting the interactive scene can be displaying an interactive scene page on a display screen, for example, displaying it in an APP installed on a mobile phone. As an alternative or supplement, the above presentation can also include sound presentation. At this time, the speaker or headphones can play corresponding scene sounds, such as music, voice prompts or descriptions, or sounds that simulate a real scene (for example, rain, wind, etc.). In the other two above, the interactive scene can also be presented in other ways such as vibration.

[0025] Here, "interactive scene" refers to a situation in which a user can operate on one or more objects. For example, in order to facilitate users to operate one or more Internet of Things (IoT) devices, an interactive scene that simulates a home environment can be provided to users. In order to facilitate users to operate outdoor equipment such as barbecue grills, an interactive scene that simulates an outdoor environment can be provided to users. The interactive scene may include one or more interactive objects. In some embodiments, the interactive scene and / or interactive object may be a virtual scene and / or object, such as an animated scene and object. In other embodiments, the interactive scene may be a real scene, and the interactive object may be a virtual AR object displayed on the real scene.

[0026] In step S120, a user's voice command operation on an interactive object is received. In the presented interactive scene, the user can perform voice operations on the interactive objects included therein, and the terminal device receives the above operations. In other embodiments, the user can also perform touch input operations through a touch screen, use a mouse to perform point operations, or even use a game controller to perform similar operations.

[0027] In step S130, the operation result of the operation is presented in the interactive scene. In some embodiments, the terminal device can directly generate a corresponding operation response based on the acquired user operation and present it in the interactive scene. In other embodiments, the terminal device can be connected to the server, upload the acquired user operation (or partially processed user operation), obtain the operation response sent by the server, or intermediate data, and present it in the interactive scene based on the operation response obtained.

[0028] Here, receiving a user's operation on an interactive object may include: receiving a user's operation to trigger a function contained in the interactive object. Accordingly, presenting the operation result of the operation in the interactive scene may include: presenting the triggering result of the function in the interactive scene. For example, a user may perform an operation to turn on a certain virtual interactive object, and then the effect of the virtual object after being turned on may be presented in the interactive scene. Subsequently, the user may turn off the virtual interactive object, and present the corresponding turning-off effect in the interactive scene. Thus, by providing a specific interactive scene and enabling the user to obtain the corresponding operation effect on the interactive object through active operation in the interactive scene, the interactive object can be displayed in an interactive manner.

[0029] In the present invention, various methods can be used to make the user's operation on the trigger object simulate the operation on the real object. For example, the user can simulate the scene of clicking the real button of the physical object in the real scene by clicking the virtual button of the virtual object in the virtual scene. However, there is always a gap between the above-mentioned simulation operation and the operation in the real scene (for example, the touch of clicking the touch screen is definitely different from the touch of turning on the switch button of a real device). For this reason, the present invention is particularly suitable for receiving the user's voice command operation on the interactive object. In other words, in the case of receiving the voice command, the voice operation of the real scene can be completely simulated. More specifically, step S120 may include: receiving the user's voice command operation on the voice interaction object, wherein the voice command operation enables the voice interaction object to control other interactive objects to enable corresponding functions. At this time, the interactive scene includes a voice interaction object and other interactive objects controlled by the voice of the voice interaction object. Thus, the user can control other objects via the voice interaction object. For example, a smart speaker and a smart TV connected to the smart speaker are displayed in the virtual scene. Users can input "XXX (for example, the smart speaker wake-up word), turn on the TV" to turn on the TV that was originally turned off in the virtual scene, simulating the scene of controlling the smart TV through the smart speaker in a real home scene. In this way, users can know how to use the smart speaker to control smart home appliances and feel its effect.

[0030] In actual operation, the generation of the above-mentioned response instructions can be completed locally on the terminal device. For example, when the user enters a simple sentence that can be recognized locally. However, in general, the generation of the response instruction requires the participation of the server. At this time, the terminal interaction method may also include: uploading the voice instruction operation to the server; obtaining the generated voice instruction response result, and step S130 may include: presenting the operation result of the operation based on the response result.

[0031] In addition, in order to facilitate user participation, the interactive method of the present invention may also include presenting operation prompts for the interactive object. Similarly, the above presentation can be perceived by the user in other ways such as sound, image or vibration. For example, when the user enters the virtual scene page, a voice prompt is given: "Try to say 'XXX (for example, the wake-up word of the smart speaker), turn on the light'".

[0032] Furthermore, in the case of voice operation, the interactive method of the present invention may also include giving voice feedback of the voice command operation to the user. The above-mentioned voice feedback can be multi-round dialogue feedback when additional information is needed, or it can be a description after the operation is completed. For example, when the user says "XXX (for example, the smart speaker wake-up word), turn on the lights", if the virtual scene includes multiple smart lights (for example, one in the living room and one in the dining room), all the lights can be turned on in response to the user's instructions, or the user can be further asked "which light to turn on", and the user can answer "turn on the lights in the restaurant". At this time, the lights in the restaurant can be turned on in the virtual scene, and a voice description of "the lights in the restaurant are on" can be given.

[0033] Since the interactive method of the present invention can well present the functions of the interactive object and provide the user with a sense of participation, it is particularly suitable for subsequent promotion of the interactive object. At this time, the terminal interaction method may also include: jumping to the promotion page of the interactive object being operated. In different embodiments, the above jump can be based on the user's active operation, or it can be triggered by the user exiting the interactive scene, or it can be triggered in other ways. For example, when a user uses a smart speaker to control a smart TV and a smart light in a virtual scene, when the user exits the virtual scene, product links including smart speakers, smart TVs, and smart lights can be displayed to the user. As an alternative or supplement, the user can also directly jump to the shopping link of the real product corresponding to the object in the virtual scene, for example, by clicking or double-clicking the corresponding interactive object, or directly add the product to the shopping cart.

[0034] In order to further enhance user participation and flexibility, in certain implementations, the presented interactive scenes and the interactive objects contained in the scenes may also be changed or updated with various settings.

[0035] An interactive scene for presentation may be generated based on user information, wherein the user information includes at least one of the following: user portrait information; user shopping information; user historical operation information; user selection information for interactive scenes; and interactive scene information input by the user.

[0036] Specifically, the user's preferred interaction scenes or interaction objects, as well as subsequent promotional information, can be presented through the user's previous information and preference information. For example, if the user portrait information shows that the user is a post-90s who loves basketball, a pair of new basketball shoes can be placed in the interactive scene to facilitate the user's subsequent operations. If the shopping information shows that the user has purchased a smart speaker, the subsequent promotion link can only promote smart devices that other users have not yet purchased. In addition, if the server supports it, for example, if the server can provide interactive scenes for various home room types, the user can enter the room type information of interest by himself, thereby generating, for example, a virtual scene that is consistent with the room type (even color matching) of the user's own home, so that subsequent operations are more immersive.

[0037] After the interactive scene is determined and presented, the interactive objects in the interactive scene can also be updated based on the trigger conditions. The above trigger can be user-triggered, terminal / server setting trigger, or time trigger. For example, if the user feels that the brightness of the smart light in the virtual scene is not enough after it is turned on, or the style is not liked, the interactive object displayed in the virtual scene can be replaced by, for example, voice "change to a brighter light". In addition, an interactive object library can be provided to the user, so that after the initial display of the interactive scene, the user can add or delete objects to the interactive scene by operating in the object library. The server can also actively push special offers, for example, at the beginning of a special offer.

[0038] As previously shown, presenting an interactive scene may include, for example, displaying a virtual scene page in a mobile phone APP, wherein the virtual scene page includes one or more virtual interactive objects. The virtual interactive object may be a virtual object corresponding to an actual commodity, and presenting the operation result of the operation in the interactive scene may include: demonstrating the function of the actual commodity in the interactive scene. In other embodiments, the virtual interactive object may also be a virtual object corresponding to an actual service, and the function of the actual service may then be demonstrated. For example, a virtual tablet in a virtual scene may display an "online classroom" service, and a user may experience it by clicking on the "online classroom" service.

[0039] In other embodiments, presenting an interactive scene may include, for example, shooting an actual scene, such as an actual scene in one's own home, via an AR helmet or a mobile phone, and adding a virtual interactive object thereto according to the actual scene shot and displayed. At this time, the user can see the use effect of the virtual object in the real scene by operating the virtual object, thereby further increasing the sense of immersion and fun. For example, the user is shopping for a bedroom lighting. The user can open the corresponding APP on the mobile phone and click the AR shopping function, so the mobile phone camera turns on and shoots the bedroom picture. The APP recognizes the bedroom scene and finds that the bedroom is equipped with a smart speaker. At this time, the smart speaker model registered by the user can be obtained through the server, and when the user says "I want to buy a lamp", the smart lamp is pushed to the user. The user can select the installation location of the virtual smart lamp in the AR scene, and view the virtual lighting effect of the lamp in the real scene through the light-on instruction, thereby determining whether it needs to be purchased.

[0040] Although the terminal interaction solution of the present invention can be implemented locally in some cases, it usually requires the participation of the server. For this reason, the present invention can also be implemented as a terminal interaction system. Figure 2 An example of a terminal interaction system capable of implementing the present invention is shown. As shown in the figure, the server 210 can connect to multiple terminals 220 and provide services therefor, and is used to: send down an interaction scene, which includes one or more interaction objects; obtain uploaded user operations on the interaction objects; and generate and send down operation feedback instructions. Multiple terminals 220 communicate with the server, and are used to: receive the interaction scene sent down and present it; receive and upload the user's operations on the interaction objects; receive the operation response instructions sent down; and based on the operation response instructions, present the operation results of the operation in the interaction scene.

[0041] Specifically, when the terminal has voice interaction capability, the terminal 220 can be used to: receive a user's voice command operation on an interactive object, the voice command operation is used to trigger the interactive object to contain a function; and based on the operation feedback instruction, demonstrate the triggering result of the function in the interactive scene. Correspondingly, the server 210 can be used to: identify the voice content of the voice command operation; and send the voice content text to the terminal. Further, the server 210 can be used to: generate a response action of the interactive object as the operation response instruction based on the recognized voice content.

[0042] The one or more interactive objects may correspond to one or more IoT devices, and the server 210 may be used to push promotion information of the operated interactive objects to the terminal 220. Figure 3 An example of a server participating in the terminal interaction of the present invention is described.

[0043] To better describe the process, let's take a promotion (advertising) scenario as an example. For example, a smart home promotion page is set up in the IoT device management app to introduce different models of smart speakers, IoT devices, and how to use them.

[0044] Users can pre-install the APP or get an update with smart advertising functions. Then, the user opens the APP and enters the promotion page ("Smart Advertising" in the figure), where the user will see a virtual smart home scene. For example, the living room is equipped with smart speakers, smart lights and smart air conditioners. Users can see some tips on how to interact with the smart speakers. Then, the user can make voice input, such as saying to the APP: "XXX, turn on the light." After the APP receives the voice, it uploads it to the voice interaction smart cloud through the voice interaction SDK. The smart cloud recognizes the voice command and sends it to the APP to present the voice text. At the same time, after the voice input reaches the cloud, the server sends the response action to the APP through the cloud intelligent algorithm. After the APP receives the sent command, it refreshes the local information of the advertisement through the end intelligent algorithm and lights up the "light in the living room" in the above scene. While the virtual light is lit, the APP can realize smart advertising interaction through voice playback, such as saying "OK, the light is on." Through the above scenes and operations, users can clearly know how to control IoT products through smart speakers, and it is interesting. It not only achieves the disclosure of promotional information, but also realizes real-time user participation. Furthermore, users can interact with smart advertisements through voice interaction to obtain the content information they need, for example, by voice inputting "where to buy" or "change to a brighter lamp".

[0045] Furthermore, the present invention may also be implemented as a terminal display method. Figure 4 A schematic flow chart of a terminal display method according to an embodiment of the present invention is shown. The method is particularly suitable for implementation on a mobile intelligent terminal such as a smart phone or a tablet computer.

[0046] In step S410, a virtual interaction scene page is displayed, and the virtual scene page includes one or more virtual interaction objects. Specifically, the user can display the virtual interaction scene page by opening the corresponding APP and clicking or selecting the corresponding scene. Subsequently, in step S420, in response to the user's operation on the virtual interaction object, the operation result of the operation is displayed in the virtual scene page.

[0047] In one embodiment, step S420 may include: in response to a user's voice command operation that triggers a function contained in an interactive object, displaying the interactive object that triggers the function in the interactive scene. For example, a TV screen of a turned-on TV may be displayed in the virtual scene. Accordingly, a page portion of the virtual interactive scene page that contains the corresponding virtual interactive object may be refreshed.

[0048] Further, step S420 may include: displaying the effect after the interactive object triggers the function in the interactive scene. The above effect may affect other parts other than the interactive object, for example, the lighting effect after turning on the light may illuminate the entire virtual scene. Accordingly, the effect after triggering the function may be rendered in the virtual scene page.

[0049] The present invention may also be implemented as a terminal device. The terminal device may be used to execute the above-mentioned interaction method and may be a part of the above-mentioned interaction system. Figure 5 A schematic diagram of the structure of a terminal device that can be used to implement the above interaction method according to an embodiment of the present invention is shown.

[0050] See also Figure 5 The terminal device 500 includes an output device 510, an input device 520 and a processing device 530.

[0051] The output device 510 is used to present an interactive scene, which includes one or more interactive objects. The input device 520 is used to receive a user's operation on the interactive object. The processing device 530 is used to present the operation result of the operation in the interactive scene using the presentation device based on the operation.

[0052] The output device 510 may include: a display for displaying the interaction scene. In one embodiment, the display may be a touch screen and is used as an input device for receiving a user's touch input operation on an interaction object.

[0053] As an alternative or supplement, the input device 520 may include: a microphone for receiving a user's voice command operation on the interactive object. Accordingly, the output device 510 may also include: a voice input device for giving the user voice feedback of the voice command operation and / or giving a voice operation prompt for the interactive object.

[0054] Furthermore, the terminal device 500 may include a networking device for communicating with the server. Specifically, the networking device may be used to: obtain the interaction scene; report the user's operation on the interaction object; and obtain an operation response instruction of the operation, and the response instruction is used by the processing device to present the operation result of the operation in the interaction scene.

[0055] Specifically, the processing device 530 may be a multi-core processor or may include multiple processors. In some embodiments, the processor 530 may include a general-purpose main processor and one or more special coprocessors, such as a graphics processing unit (GPU), a digital signal processor (DSP), etc. In some embodiments, the processor 530 may be implemented using a customized circuit, such as an application-specific integrated circuit (ASIC) or a field programmable gate array (FPGA).

[0056] The terminal device 500 may also include a memory to store the above device management APP and other data content. The memory may include various types of storage units, such as system memory, read-only memory (ROM), and permanent storage. Among them, ROM can store static data or instructions required by the processor 820 or other modules of the computer. The permanent storage device may be a readable and writable storage device. The permanent storage device may be a non-volatile storage device that does not lose the stored instructions and data even after the computer is powered off. In some embodiments, the permanent storage device uses a large-capacity storage device (such as a magnetic or optical disk, flash memory) as a permanent storage device. In some other embodiments, the permanent storage device may be a removable storage device (such as a floppy disk, an optical drive). The system memory may be a readable and writable storage device or a volatile readable and writable storage device, such as a dynamic random access memory. The system memory may store some or all instructions and data required by the processor at run time. In addition, the memory may include a combination of any computer-readable storage media, including various types of semiconductor memory chips (DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), and disks and / or optical disks may also be used. In some embodiments, the memory may include a readable and / or writable removable storage device, such as a laser disc (CD), a read-only digital versatile disc (e.g., DVD-ROM, double-layer DVD-ROM), a read-only Blu-ray disc, an ultra-density optical disc, a flash memory card (e.g., SD card, mini SD card, Micro-SD card, etc.), a magnetic floppy disk, etc. The computer-readable storage medium does not include carrier waves and transient electronic signals transmitted wirelessly or wired.

[0057] The memory stores executable codes, and when the executable codes are processed by the processor 530 , the processor 530 can execute the interactive access method mentioned above.

[0058] The terminal interaction scheme according to the present invention has been described in detail above with reference to the accompanying drawings. The scheme provides interactive scenes for users, receives user operations and displays operation results, so that users can participate in the promotion process itself and can accurately know the various functions of the operation object. The present invention is particularly suitable for being implemented as an intelligent promotion scheme for voice interaction within an APP, that is, users within the APP can perform voice interaction in the promotion scene, and the promotion scene presents different content information according to the user's voice instructions, thereby displaying the services or products to be promoted, improving user participation and the fun of information promotion, as well as the accuracy of delivery.

[0059] As mentioned above, the present invention can examine the corresponding function of an interactive object (e.g., a product or a service) by performing voice operation on the interactive object in an interactive scene. However, in a broader embodiment, the interactive object can be the commercial promotion content itself (e.g., a commercial advertisement, not the product promoted in the advertisement).

[0060] To this end, the present invention can also be implemented as a terminal voice interaction method, including: presenting commercial promotion content based on predetermined conditions in the current interaction scene; receiving a user's voice command operation on the commercial promotion content; and presenting the operation result of the operation on the commercial promotion content in the interaction scene.

[0061] The current interaction scene may be, for example, a video playback scene, and the predetermined conditions for triggering the presentation of the commercial promotion content may include a start operation of video playback and a pause operation of video playback, etc. In other embodiments, it may also include a voice command of a terminal user.

[0062] In addition, the current interaction scene may also be the APP startup scene, thereby implementing the APP startup advertisement as the voice interaction advertisement of the present invention.

[0063] In some embodiments, predetermined commercial promotion content can be generated. For example, all users of a certain APP will be presented with the same commercial advertisement when opening the APP, and the optional operation mode of the advertisement is the same. In a preferred embodiment, the promotion object and voice operation mode of the commercial promotion content can be generated or selected in real time for a specific user. Specifically, the promotion object and / or voice operation mode of the commercial promotion content can be generated based on at least one of the terminal user portrait information; terminal user shopping information; terminal user historical operation information; terminal user selection information for interactive scenes; and interactive scene information input by the terminal user.

[0064] For example, new interactive games can be recommended to users based on their gaming preferences, and interactive methods of different difficulty levels can be presented based on the user's game level. For another example, if a user clicks on a cooking video, cooking tools can be recommended to the user, and different voice cooking methods can be given based on the user's previous online shopping list.

[0065] Furthermore, the above-mentioned voice command operation can be used to operate the presentation of the commercial promotion content itself, for example, to control the play, pause, and fast forward of the commercial advertisement. Preferably, the above-mentioned voice command operation can also be used to operate the promotion object contained in the commercial promotion content. For example, the smart speaker presented in the smart speaker advertisement is operated as described above.

[0066] The present invention can also be used for commercial promotion in film and television dramas. To this end, the present invention can also be implemented as a voice interaction method, including: playing a film and television drama, wherein an interactive object is presented in the playing scene of the film and television drama; receiving a user's voice command operation on the interactive object; and presenting the operation result in the playing scene of the film and television drama.

[0067] For example, in a certain indoor scene of a film or TV series, a new Bluetooth lighting system is displayed. At this time, a prompt can be given that there are interactive objects in the playback scene of the film or TV series, for example, a certain device in the scene is lit up, or a pop-up box or bullet screen prompt is given, so that the user knows that there are interactive objects in the current scene. At this time, the user can perform corresponding operations on the interactive objects, for example, change the lighting system in the current scene from evening mode to night mode. Therefore, the effect of the interactive object on the playback scene after the corresponding function is enabled based on the voice command operation can be presented in the playback scene of the film or TV series. For example, the film or TV series continues to develop the plot under the lighting of the night mode.

[0068] The present invention can also be used for commercial promotion screens. Commercial promotion screens can be advertising screens facing non-specific users, such as outdoor advertising screens, elevator advertising screens, and advertising screens on transportation vehicles (for example, interactive screens installed behind the front seats of taxis, high-speed trains, and airplanes).

[0069] To this end, a commercial promotion screen voice interaction method includes: prompting a user to perform voice interaction; obtaining the user's voice interaction input; presenting commercial promotion content; receiving the user's voice command operation on the commercial promotion content; and presenting the operation result of the voice command operation on the commercial promotion content.

[0070] The commercial promotion screen can keep displaying the prompt "You can interact with me by voice" when idle, so that users can interact with it on their own. In some embodiments, the commercial promotion screen can sense the approach of the user, for example, based on an image sensor or a pressure sensor. At this time, the commercial promotion screen can be lit up and give a prompt that it can interact with voice (for example, based on voice or text display).

[0071] The user can interact with the commercial promotion screen based on the prompt. Similarly, the promotional content in the commercial promotion screen can be generated by an unrelated user, for example, according to the bidding or time period. In other embodiments, the user information of the user can also be obtained; and the commercial promotion content is presented based on the user information. Here, the user information includes at least one of the following: the user's biometric information obtained on site; the user's voice interaction input obtained on site; and the user information obtained online based on the user's identity information.

[0072] For example, based on the identification of the user's gender and age (e.g., face or voice recognition), promotional content with corresponding age group labels can be selected from the database. For another example, based on the content information obtained from the user's voice interaction, for example, "I'm a little tired now", advertisements with caring content can be given. In addition, user identity information can be obtained through auxiliary means such as face recognition and scanning QR codes. For example, the user ID can be matched, and the user information stored based on the user ID can be associated to select promotional content and voice interaction forms based on preferences, historical operations, or shopping habits.

[0073] In order to deepen the understanding of the present invention, the following Figure 6A-6B An example of an interaction scenario based on the present invention is described. Figure 6A-6B An example of a terminal interaction scenario according to the present invention is shown.

[0074] For example, a smart speaker APP or other APP installed in a smartphone can push a link to "experience smart home" to the user in a prominent position, and the user can then click on the link to enter the virtual scene page of the smart home, such as Fig. 6A Here, the smart advertisement may be presented in a Native page, an H5 page, a mini program, a mini game, etc., and the present invention does not limit this.

[0075] Fig. 6A A virtual smart home scene is shown. On the coffee table in the living room, a smart speaker 1 is arranged, and the living room also includes IoT devices such as a smart TV 2, a sweeping robot 3, smart curtains 4, and a smart light 5. In a specific implementation, these devices can inform the user that they are operable by flashing.

[0076] Specifically, the smart speaker 1 can inform the user "Welcome to the virtual home, please try to wake up XXX (the name of the smart speaker and the wake-up word)" by flashing itself, or by a text prompt box or voice prompt next to it, or even by a virtual avatar that can speak. The user can then wake up the smart speaker 1 by saying "Hello, XXX".

[0077] At this time, the smart speaker 1 can answer "Good evening, master, what can I do for you", and then the user can answer "Open the curtains". At this time, the terminal can upload the above voice input to the server and obtain the response command of the server, and then display the animation of the curtains being opened, and make the smart speaker 1 answer "The curtains are opened".

[0078] Similarly, the user can further interact with the smart speaker 1, for example, turn on the light 5, turn on the TV 2, start the sweeping robot 3, and get the following information: Figure 6B The scene shown is triggered by the interactive object function. Afterwards, the user can also control the interactive object through closing instructions or other instructions. In this way, the user can clearly experience the voice operation effect of the smart home.

[0079] The user can jump to the purchase page of the interactive object or put it into the shopping cart by clicking on the object or voice input in the above virtual scene. After the user exits the page, a list of purchase links including the above device can be displayed to the user, thereby facilitating the user's subsequent purchase.

[0080] Through the present invention, users can interact with advertisements by voice in the APP. Through two-way communication with smart advertisements, advertisements can present different content information according to the user's voice instructions, making the advertisements smarter and more humane. In addition, the advertising resources on the APP are limited, and users can interact with advertisements by voice to make the content information presented by the advertisements the information that the users themselves care about to the greatest extent.

[0081] In addition, the method according to the present invention may also be implemented as a computer program or a computer program product, which includes computer program code instructions for executing the above steps defined in the above method of the present invention.

[0082] Alternatively, the present invention can also be implemented as a non-temporary machine-readable storage medium (or computer-readable storage medium, or machine-readable storage medium) on which executable code (or computer program, or computer instruction code) is stored. When the executable code (or computer program, or computer instruction code) is executed by a processor of an electronic device (or computing device, server, etc.), the processor executes the various steps of the above-mentioned method according to the present invention.

[0083] Those skilled in the art will further appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein may be implemented as electronic hardware, computer software, or a combination of both.

[0084] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system and method according to multiple embodiments of the present invention. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or code, and the part of the module, program segment or code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0085] The embodiments of the present invention have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terms used herein are selected to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A terminal voice interaction method, comprising: Displaying a virtual interaction scene page, wherein the virtual scene page includes a plurality of virtual interaction objects, wherein the plurality of virtual interaction objects include a voice virtual interaction object and other virtual interaction objects controlled by voice of the voice virtual interaction object; Receiving a user's voice command operation on the voice virtual interaction object to trigger functions contained in other virtual interaction objects; as well as In response to the user's voice command operation, the voice virtual interactive object controls other virtual interactive objects to enable corresponding functions, and displays the effects of other virtual interactive objects triggering the functions in the virtual scene page. The virtual interactive object corresponds to an actual commodity.

2. The method of claim 1, further comprising: Uploading the voice command operation to the server; Get the generated voice command response result, and The effect of displaying the virtual interactive object in the virtual scene page after triggering the function includes: Based on the response result, the effect is presented.

3. The method of claim 1, further comprising: Provide voice feedback of the voice command operation to the user.

4. The method of claim 1, further comprising: An operation prompt for the virtual interactive object is presented.

5. The method of claim 1, further comprising: Relevant information of the operated virtual interactive object is presented.

6. The method of claim 1, further comprising: Jump to the promotion page of the virtual interactive object being operated.

7. The method of claim 1, further comprising: Based on the trigger condition, the virtual interaction object in the virtual interaction scene is updated.

8. The method of claim 1, further comprising: The virtual interactive scene page for presentation is generated according to the user information.

9. The method of claim 8, wherein: The user information includes at least one of the following: User profile information; User shopping information; User historical operation information; User selection information for the interaction scenario; and Interaction scene information input by the user.

10. A terminal voice interaction system, comprising a plurality of terminals and a server, wherein the plurality of terminals communicate with the server, and The server is used to: A virtual interaction scene is issued, wherein the virtual interaction scene includes a plurality of virtual interaction objects, and the plurality of virtual interaction objects include a voice virtual interaction object and other virtual interaction objects controlled by voice of the voice virtual interaction object. Acquire the uploaded user's voice command operation on the voice virtual interactive object, where the voice command operation is used to trigger other virtual interactive objects to enable corresponding functions. Generate and issue operation response instructions; The terminal is used for: receiving the virtual interaction scene sent down and displaying the virtual interaction scene page, receiving and uploading the user's voice command operation on the voice virtual interaction object, receiving the issued operation response instruction, Based on the operation response instruction, the voice virtual interaction object controls other virtual interaction objects to enable corresponding functions, and displays the effect of the virtual interaction object triggering the function in the virtual scene page; in, The virtual interactive object corresponds to an actual commodity.

11. The system of claim 10, wherein: The server is used to: Recognizing the voice content of the voice command operation; and Send the voice content text to the terminal.

12. The system of claim 11, wherein: The server is used to: A response action of the interactive object is generated based on the recognized voice content as the operation response instruction.

13. The system of claim 10, wherein: The one or more virtual interactive objects correspond to one or more IoT devices.

14. The system of claim 10, wherein: The server is used to: Pushing promotion information of the operated interactive object to the terminal.

15. A terminal device, comprising: An output device, used for displaying a virtual interaction scene page, wherein the virtual scene page includes a plurality of virtual interaction objects, wherein the plurality of virtual interaction objects include a voice virtual interaction object and other virtual interaction objects voice-controlled by the voice virtual interaction object; An input device, used to receive a voice command operation by a user to trigger the voice virtual interactive object to control other virtual interactive objects to enable corresponding functions; a processing device, for responding to the voice command operation, the voice virtual interactive object controlling other virtual interactive objects to enable corresponding functions, and displaying the effect of the virtual interactive object triggering the function in the virtual scene page, The virtual interactive object corresponds to an actual commodity.

16. The terminal device according to claim 15, wherein: The input device comprises: Microphone, used to receive user's voice commands to the virtual interactive object, And the output device also includes: The voice input device is used to provide the user with voice feedback of the voice command operation and / or provide voice operation prompts for the virtual interactive object.

17. The terminal device according to claim 15, further comprising: Networking devices for: Acquire the virtual interaction scene; Reporting the user's voice command operation on the virtual interactive object; as well as An operation response instruction of the voice command operation is obtained, and the response instruction is used by the processing device to display the effect of the virtual interactive object after triggering the function in the virtual interactive scene page.

Images