Intelligent voice device control method, device, computer device and storage medium

Abstract

The present invention relates to a method, device, computer device and storage medium for controlling an intelligent voice device, the method comprising: acquiring an image captured within a video capture range; identifying a facial image in the image; determining a corresponding sight line direction according to the facial image; and when the determined sight line direction points to a virtual interactive character presented locally on the intelligent voice device, controlling the intelligent voice device to enter a wake-up state. The solution of the present application improves the accuracy of the wake-up process of the intelligent voice device.

Description

[0001] This application was submitted to the China Patent Office on January 3, 2018, with application number 2018100052080, and the invention name is "Intelligent voice device control method, device, computer equipment and storage medium". All its contents are incorporated into this application by reference. Technical Field

[0002] The present invention relates to the field of computer technology, and in particular to an intelligent voice device control method, device, computer equipment and storage medium. Background Art

[0003] With the rapid development of science and technology, artificial intelligence technology has become more and more popular due to the convenience it brings. Some artificial intelligence products have emerged. For example, smart speakers, users can speak to smart speakers to achieve a series of interactive controls.

[0004] Traditionally, when using smart speakers for interactive control, the smart speakers need to collect voice data and identify keywords from the collected voice data for activation. However, the voice data often contains a large amount of interference sound, such as the noise on the road outside the room. Due to these large amounts of interference, the accuracy of activating the smart speaker based on voice data will be low. Summary of the invention

[0005] Based on this, it is necessary to provide a smart voice device control method, device, computer device and storage medium to address the problem of low accuracy of traditional methods in activating smart speakers based on voice data.

[0006] A method for controlling an intelligent voice device, the method comprising:

[0007] The holographic intelligent voice imaging device holographically projects the bound virtual interactive character to present a three-dimensional holographic virtual interactive character on the holographic intelligent voice imaging device;

[0008] Acquire images captured within the video acquisition range;

[0009] When a user object is identified in the image, the three-dimensional holographic virtual interactive character is controlled to always face the position of the user represented by the user object in the real scene;

[0010] When the sight line direction corresponding to the user object in the image points to the three-dimensional holographic virtual interactive character, the holographic intelligent voice imaging device is controlled to enter a wake-up state.

[0011] An intelligent voice device control device, the device comprising:

[0012] A role presentation module is used for the holographic intelligent voice imaging device to perform holographic projection on the bound virtual interactive role, so as to present a three-dimensional holographic virtual interactive role on the holographic intelligent voice imaging device;

[0013] An acquisition module, used to acquire images captured within a video acquisition range;

[0014] The control module is used to control the three-dimensional holographic virtual interactive character to always remain oriented toward the position of the user represented by the user object in the real scene when a user object is recognized in the image; and to control the holographic intelligent voice imaging device to enter a wake-up state when the line of sight corresponding to the user object in the image points to the three-dimensional holographic virtual interactive character.

[0015] A computer device comprises a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor executes the following steps:

[0016] The holographic intelligent voice imaging device holographically projects the bound virtual interactive character to present a three-dimensional holographic virtual interactive character on the holographic intelligent voice imaging device;

[0017] Acquire images captured within the video acquisition range;

[0018] When a user object is identified in the image, the three-dimensional holographic virtual interactive character is controlled to always face the position of the user represented by the user object in the real scene;

[0019] When the sight line direction corresponding to the user object in the image points to the three-dimensional holographic virtual interactive character, the holographic intelligent voice imaging device is controlled to enter a wake-up state.

[0020] A storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor executes the following steps:

[0021] The holographic intelligent voice imaging device holographically projects the bound virtual interactive character to present a three-dimensional holographic virtual interactive character on the holographic intelligent voice imaging device;

[0022] Acquire images captured within the video acquisition range;

[0023] When a user object is identified in the image, the three-dimensional holographic virtual interactive character is controlled to always face the position of the user represented by the user object in the real scene;

[0024] When the sight line direction corresponding to the user object in the image points to the three-dimensional holographic virtual interactive character, the holographic intelligent voice imaging device is controlled to enter a wake-up state.

[0025] The above-mentioned intelligent voice device control method, device, computer device and storage medium, the holographic intelligent voice imaging device holographically projects the bound virtual interactive character to present a three-dimensional holographic virtual interactive character on the holographic intelligent voice imaging device, so that the three-dimensional holographic virtual interactive character can be used to interact with the user. Acquire an image captured within the video acquisition range; when a user object is identified in the image, the three-dimensional holographic virtual interactive character is controlled to always remain facing the position of the user represented by the user object in the real scene; when the line of sight corresponding to the user object in the image points to the three-dimensional holographic virtual interactive character, the holographic intelligent voice imaging device is controlled to enter the wake-up state. That is, the holographic intelligent voice imaging device is not only a receiver of instructions, but can holographically project a three-dimensional holographic virtual interactive character, and control the three-dimensional holographic virtual interactive character to remain facing the position of the user, combine the line of sight detection with the three-dimensional holographic virtual interactive character, and perform more accurate wake-up control on the holographic intelligent voice imaging device when the line of sight direction points to the three-dimensional holographic virtual interactive character. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 An application scenario diagram of an intelligent voice device control method in an embodiment;

[0027] Figure 2 This is an application scenario diagram of the intelligent voice device control method in another embodiment;

[0028] Figure 3 A flowchart of a method for controlling an intelligent voice device in one embodiment;

[0029] Figure 4 A schematic diagram of the architecture of an intelligent voice device in an embodiment;

[0030] Figure 5 is a flowchart of a wake-up control step in an embodiment;

[0031] Figure 6 A schematic diagram of a scenario of multiple intelligent voice devices in one embodiment;

[0032] Figure 7 A schematic diagram of a method for controlling an intelligent voice device in multiple intelligent voice device scenarios in one embodiment;

[0033] Figure 8 A schematic diagram of a scenario in which an intelligent voice device responds to voice control in one embodiment;

[0034] Fig. 9 A schematic diagram of continuous power supply in one embodiment;

[0035] Fig.10 A flowchart of a method for controlling an intelligent voice device in another embodiment;

[0036] Fig.11 A block diagram of an intelligent voice device control device in one embodiment;

[0037] Fig.12 A block diagram of an intelligent voice device control device in another embodiment;

[0038] Fig.13 A block diagram of an intelligent voice device control device in yet another embodiment;

[0039] Fig.14 Schematic diagram of the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0041] Figure 1 FIG. 1 is an application scenario diagram of an intelligent voice device control method in an embodiment. Figure 1 , the application scenario includes an intelligent voice device 110 and a server 120 connected via a network. Among them, the intelligent voice device 110 is a device with voice collection and output functions, as well as intelligent computing functions. It can be understood that the intelligent voice device 110 can also have other functions, which are not limited here. The intelligent voice device 110 can include a video acquisition device and a display device. The video acquisition device is used to capture images, and the video acquisition device can be a camera. The display device can be used to display virtual interactive characters.

[0042] In one embodiment, the intelligent voice device 110 may be a holographic intelligent voice imaging device, wherein the holographic intelligent voice imaging device is an intelligent voice device with a holographic imaging function. In this embodiment, the display device of the intelligent voice device 110 may include a holographic display device, such as a liquid crystal projection device and a holographic film.

[0043] In one embodiment, the intelligent voice device 110 may also be a terminal with a display function and a voice collection and output function. In this embodiment, the display device may be an ordinary display screen. The terminal may be an intelligent speaker, an intelligent television, a desktop computer or a mobile terminal with a display function, and the mobile terminal may include at least one of a mobile phone, a tablet computer, a laptop computer, a personal digital assistant and a wearable device. The server 120 may be implemented as an independent server or a server cluster consisting of multiple physical servers.

[0044] The intelligent voice device 110 can obtain images captured within the video capture range; and recognize facial images in the images. The intelligent voice device 110 can determine the corresponding sight direction based on the facial image; when the determined sight direction points to the virtual interactive character presented locally on the intelligent voice device, the intelligent voice device 110 is controlled to enter the awake state. It can be understood that after the intelligent voice device 110 enters the awake state, the user can use imperative statements to issue operation instructions to the intelligent voice device 110, and the intelligent voice device 110 can collect corresponding voice data and send the voice data to the server 120. The server 120 can perform semantic analysis on the voice data to determine the operation that the user wants to perform.

[0045] Figure 2 FIG. 1 is an application scenario diagram of an intelligent voice device control method in another embodiment. Figure 2 There are multiple intelligent voice devices 110, and the multiple intelligent voice devices 110 are connected to the same network. The multiple intelligent voice devices 110 connected to the same network can send and receive broadcasts to each other. Multiple intelligent voice devices 110 communicate with the server 120 through the network respectively. It can be understood that multiple intelligent voice devices 110 can respond to voice interaction control in parallel (that is, each intelligent voice device can respond to voice interaction control independently from each other), or one of the intelligent voice devices 110 can be awakened and send a role exclusive status broadcast to other intelligent voice devices that have not yet been awakened, so that other intelligent voice devices 110 do not respond to voice interaction control before the role exclusive status ends. It can be understood that the intelligent voice device control method in each embodiment of the present application is applicable to each intelligent voice device 110.

[0046] Figure 3 This is a flow chart of a method for controlling an intelligent voice device in an embodiment. Figure 1 or Figure 2 The intelligent voice device 110 in FIG. Figure 3 , the method specifically comprises the following steps:

[0047] S302, acquiring images captured within a video capture range.

[0048] It should be noted that when the smart voice device is not in the awakening state, it is in a semi-dormant state.

[0049] The semi-dormant state is a state before entering the awakening state, and can perform logical processing of events such as image acquisition, face recognition, and voice acquisition. That is, when the intelligent voice device is in a semi-dormant state, it can perform logical processing of events such as image acquisition, face recognition, and voice acquisition.

[0050] The awake state is a state in which commands corresponding to voice data can be executed. That is, after the intelligent voice device enters the awake state, it can execute commands corresponding to voice data.

[0051] The video acquisition range is the area in which images can be acquired, that is, the intelligent voice device can acquire images within the video acquisition range.

[0052] In one embodiment, the intelligent voice device can directly obtain the image collected within the video collection range. In one embodiment, the intelligent voice device can collect images within the video collection range through a video collection device.

[0053] It should be noted that the image collection, voice collection, and user line of sight analysis and other processing for the user in this application are all carried out with the authorization of the user or after full authorization by all parties.

[0054] S304, identifying a facial image in the image.

[0055] In one embodiment, the intelligent voice device can perform face recognition processing on the acquired image locally to identify the facial image from the image. It can be understood that the intelligent voice device can identify the facial image from the image based on the composition features of the face. The intelligent voice device can also locate the facial image from the image based on the position of the face in the body structure and combined with the skin features.

[0056] In one embodiment, the intelligent voice device may also send the image to the face recognition platform, which recognizes the facial image from the image and obtains the facial image recognition result returned by the face recognition platform.

[0057] It is understandable that there may be situations where a facial image is recognized from an image or a situation where a facial image is not recognized. When a facial image is recognized from an image, the intelligent voice device may execute step S306. In one embodiment, when a facial image is not recognized from an image, the intelligent voice device may be awakened by voice to control the intelligent voice device to enter an awake state.

[0058] S306: Determine a corresponding sight line direction according to the facial image.

[0059] In one embodiment, step S306 includes: identifying a pupil position in the facial image; and determining a corresponding sight line direction according to the identified pupil position.

[0060] The pupil position is the position of the pupil in the facial image.

[0061] Specifically, the intelligent voice device can identify the pupil from the facial image according to the preset pupil features to determine the pupil position (i.e. the position of the pupil in the facial image). The intelligent voice device can determine the corresponding sight direction according to the identified pupil position.

[0062] In one embodiment, the intelligent voice device can determine the pupil center according to the pupil position, and determine the eyeball center from the facial image, and use the line connecting the pupil center and the eyeball center as the line of sight direction. It can be understood that the intelligent voice device can directly take the center point of the eyeball in the facial image to obtain the eyeball center. The intelligent voice device can also determine the eyeball center from the facial image based on the preset position difference between the eyeball center and the pupil center, and the determined pupil center.

[0063] In one embodiment, step S306 includes: determining the orientation of the face relative to the intelligent voice device based on the facial image; and determining the corresponding line of sight direction based on the orientation of the face relative to the intelligent voice device.

[0064] Specifically, the intelligent voice device can parse the facial image, determine the ratio between the areas of the two sides of the face in the facial image, and determine the orientation of the face relative to the intelligent voice device based on the ratio. In one embodiment, when the difference in the ratio between the areas of the two sides of the face is less than or equal to a first threshold, the orientation of the face relative to the intelligent voice device is determined to be front facing. When the difference in the ratio between the areas of the two sides of the face is greater than the first threshold and less than or equal to the second threshold, the orientation of the face relative to the intelligent voice device is determined to be side facing. When the difference in the ratio between the areas of the two sides of the face is greater than the second threshold, it is determined that the face is not facing the intelligent voice device.

[0065] For example, when the ratio between the left and right faces in a facial image is less than or equal to 1.05, that is, the areas occupied by the left and right faces in the facial image are approximately equal, it can be considered that the orientation of the face relative to the intelligent voice device is frontal.

[0066] It is understandable that there may be situations where a facial image is recognized from an image or a situation where a facial image is not recognized from an image. When a facial image is recognized from an image, the intelligent voice device may execute step S306. In one embodiment, when a facial image is not recognized from an image, the intelligent voice device may be awakened by voice to control the intelligent voice device to enter an awake state.

[0067] S308: When the determined sight line direction points to the virtual interactive character presented locally on the intelligent voice device, the intelligent voice device is controlled to enter a wake-up state.

[0068] Among them, the virtual interactive character is a virtual image used to interact with the user. It can be understood that the virtual interactive character is a virtual image without a hardware entity. The awakened state is a state in which commands corresponding to voice data can be executed. That is, after the intelligent voice device enters the awakened state, it can execute commands corresponding to voice data. It can be understood that there is a binding relationship between the virtual interactive character presented locally on the intelligent voice device and the intelligent voice device.

[0069] In one embodiment, if the intelligent voice device does not receive voice data within a preset time after entering the awake state, it can re-enter the semi-dormant state.

[0070] In one embodiment, the method also includes a role binding step, which specifically includes the following steps: scanning a role QR code representing the identity of a virtual interactive role; extracting a download link address of corresponding virtual interactive role information from the role QR code; downloading the virtual interactive role information according to the download link address, and binding the corresponding virtual interactive role to the local according to the downloaded virtual interactive role information.

[0071] The character QR code is used to represent the identity of the virtual interactive character, and includes a download link address for the corresponding virtual interactive character information.

[0072] Specifically, the intelligent voice device can scan the character QR code, extract the download link address of the corresponding virtual interactive character information from the character QR code, and download the virtual interactive character information according to the download link address. The intelligent voice device can bind the corresponding virtual interactive character to the intelligent voice device locally according to the downloaded virtual interactive character information.

[0073] It is understandable that before the intelligent voice device enters the awake state (i.e., in the semi-dormant state), the virtual interactive role can be presented locally. In one embodiment, the virtual interactive role can be a virtual image displayed in the display screen.

[0074] In one embodiment, the virtual interactive character may be a three-dimensional holographic virtual interactive character. A three-dimensional holographic virtual interactive character is a virtual interactive character projected by front-projected holographic display technology and presented in a three-dimensional form in a real environment.

[0075] In one embodiment, the intelligent voice device may be a holographic intelligent voice imaging device, wherein the holographic intelligent voice imaging device is an intelligent voice device with a holographic imaging function. In this embodiment, the method further comprises: projecting the virtual interactive role information three-dimensionally to a holographic film in the holographic intelligent voice imaging device through a liquid crystal projection device in the holographic intelligent voice imaging device; and presenting a three-dimensional holographic virtual interactive role on the holographic film of the holographic intelligent voice imaging device.

[0076] Among them, the holographic film, that is, the holographic projection film, is a film that displays the image of the holographic projection. While maintaining a clear image, it can allow the audience to see the real scene behind it through the projection film.

[0077] Figure 4 FIG. 1 is a schematic diagram of the architecture of an intelligent voice device in an embodiment. Figure 4 The gateway is used for external communication, such as communication with a smart TV or server. The audio receiving array is used to collect voice data. The intelligent computing center is used to perform computing logic processing. The liquid crystal projection device and the holographic film are used for three-dimensional holographic projection. The liquid crystal projection device can project the virtual interactive role information into the holographic film in three-dimensional holographic projection. The camera is used to collect images within the video acquisition range. The audio output array is used to output voice data.

[0078] In one embodiment, step S308 includes: obtaining voice data; when the line of sight is directed to the intelligent voice device and there is a preset wake-up word in the voice data, controlling the intelligent voice device to enter a wake-up state, and presenting the bound virtual interactive character locally on the intelligent voice device.

[0079] It can be understood that in this embodiment, when the line of sight is directed to the intelligent voice device and there is a preset wake-up word in the acquired voice data, after the intelligent voice device is controlled to enter the wake-up state, the intelligent voice device locally presents the bound virtual interactive role. In one embodiment, the intelligent voice device can be a terminal with display function and voice receiving and sending function. For example, the intelligent voice device can be a mobile phone, a tablet computer or a desktop computer. It can be understood that the intelligent voice device can be implemented with a general terminal such as a mobile phone without the need for a dedicated voice device, which saves costs and improves resource utilization.

[0080] In one embodiment, the method further includes: when the intelligent voice device is in a semi-dormant state, selecting a target picture from a gallery corresponding to the intelligent voice device; and displaying the selected target picture.

[0081] Among them, the semi-dormant state is a state before entering the awake state and capable of performing logical processing of events such as image acquisition, face recognition, and voice acquisition. A gallery is a library for storing pictures. The gallery includes a collection of pictures. The gallery corresponding to the intelligent voice device includes a gallery stored locally on the intelligent voice device and / or a gallery stored in the cloud corresponding to the intelligent voice device.

[0082] In one embodiment, the intelligent voice device can select the most recent picture or the preset cover picture from the corresponding gallery as the target picture. The intelligent voice device can display the selected target picture to achieve the effect of a smart photo frame.

[0083] It can be understood that the intelligent voice device can determine whether to control the intelligent voice device to enter the awake state based solely on the line of sight direction, and when the determined line of sight direction points to the virtual interactive character presented locally on the intelligent voice device, the intelligent voice device is controlled to enter the awake state. The intelligent voice device can also obtain voice data, and determine whether to control the intelligent voice device to enter the awake state based on the obtained voice data and line of sight direction.

[0084] In one embodiment, if the intelligent voice device does not receive voice data within a preset time after entering the awake state, it can re-enter the semi-dormant state and display the target image again.

[0085] In one embodiment, step S308 includes: obtaining voice data; when the determined line of sight direction points to a virtual interactive character presented locally on the intelligent voice device and there is a preset wake-up word in the voice data, controlling the intelligent voice device to enter a wake-up state.

[0086] In one embodiment, the step of determining whether a preset wake-up word exists in the voice data includes: extracting feature words in the voice data; matching the extracted feature words with the preset wake-up words; when the feature words match the preset wake-up words, determining that the preset wake-up word exists in the voice data.

[0087] Among them, the characteristic word is a word that can reflect the semantic characteristics. The preset wake-up word is a pre-set word used to put the intelligent voice device into the wake-up state.

[0088] Specifically, the intelligent voice device can perform semantic analysis on the voice data and extract the feature words therein. The preset wake-up words are pre-stored in the intelligent voice device. The intelligent voice device library matches the extracted feature words with the preset wake-up words. When the feature words match the preset wake-up words, it is determined that the preset wake-up words exist in the voice data. It can be understood that when the feature words match the preset wake-up words, it can be determined that the preset wake-up words do not exist in the voice data.

[0089] It should be noted that the step of determining whether there is a preset wake-up word in the voice data can be applied to other embodiments of the present application that require determining a preset wake-up word in the voice data.

[0090] The above-mentioned intelligent voice device control method obtains the image collected within the video acquisition range, and determines the corresponding sight direction according to the facial image recognized in the image. The facial image recognized within the video acquisition range and capable of identifying the sight direction is usually close to the intelligent voice device, and is more likely to be a user who wants to wake up the intelligent voice device, eliminating interference from other users who are far away. When the determined sight direction points to the virtual interactive character presented locally on the intelligent voice device, the intelligent voice device is controlled to enter the wake-up state, simulating the scene in which you generally need to look at the other party when you have a conversation in a real scene. Therefore, when the sight points to the virtual interactive character of the local program of the intelligent voice device, the intelligent voice device is awakened, making the awakening of the intelligent voice device more accurate.

[0091] In addition, the virtual interactive characters simulate the actual conversation scene, which enhances the anthropomorphism of the intelligent voice device. Compared with the interaction of physical robots, it reduces the hardware cost and is not limited by hardware limitations, which improves scalability.

[0092] In one embodiment, the method further includes: acquiring voice data; when the line of sight is not directed to the virtual interactive character presented locally on the intelligent voice device and there is a preset wake-up word in the voice data, controlling the intelligent voice device to enter a wake-up state.

[0093] Specifically, the intelligent voice device can directly obtain existing voice data to detect the preset wake-up word, or collect voice data in the real environment. The intelligent voice device can control the intelligent voice device to enter the wake-up state when the line of sight is not directed to the virtual interactive character presented locally on the intelligent voice device and the preset wake-up word exists in the voice data.

[0094] In one embodiment, when the corresponding sight direction cannot be determined according to the facial image and there is a preset wake-up word in the voice data, the intelligent voice device is controlled to enter the wake-up state. For example, if the corresponding sight direction cannot be determined because the facial image is blurry, the intelligent voice device can also be controlled to enter the wake-up state when there is a preset wake-up word in the voice data.

[0095] In the above embodiment, the recognition of the gaze direction in the facial image is not only relied on, but the recognition of the gaze direction in the facial image is combined with the wake-up word of the voice recognition. When the gaze direction is not directed to the virtual interactive character, the preset wake-up word in the voice data can be recognized to control the intelligent voice device to enter the wake-up state, thereby improving the flexibility of waking up the intelligent voice device. In addition, the efficiency of waking up is also improved.

[0096] In one embodiment, the method further includes: when a user object is recognized in the image, entering the near-field interaction mode and executing step S304; when no user object is recognized in the image and voice data is acquired, entering the far-field interaction mode, and when the acquired voice data meets the wake-up conditions in the far-field interaction mode, controlling the intelligent voice device to enter the wake-up state.

[0097] The user object is the image content that represents the user in the acquired image. The near-field interaction mode is a close-range interaction mode that represents the user interacting with the smart voice device within the video acquisition range. The far-field interaction mode is a long-range interaction mode that represents the user interacting with the smart voice device without appearing in the video acquisition range.

[0098] It can be understood that the near-field interaction mode and the far-field interaction mode correspond to different interaction processing logics. If the user object is recognized in the image, it can be determined that the user appears in the video acquisition range. If the user object is not recognized in the image, it can be determined that the user does not appear in the video acquisition range.

[0099] It should be noted that the situation where a user object is recognized in an image but a facial image is not recognized (for example, the user object is facing away from the intelligent voice device) can also be considered a situation in the near-field interaction mode.

[0100] Among them, the wake-up conditions in the far-field interaction mode are the conditions for waking up the intelligent voice device in the far-field interaction mode.

[0101] In one embodiment, the wake-up condition in the far-field interaction mode includes: the acquired voice data includes a preset wake-up word that appears a preset number of times within a preset period. For example, if the voice data includes a preset wake-up word that appears twice within 3 seconds, it can be determined that the voice data meets the wake-up condition in the far-field interaction mode.

[0102] In the above embodiments, when different interaction modes are entered according to whether a user object can be recognized in the image, the intelligent voice device is controlled to enter a wake-up state through the interaction logic of different interaction modes, thereby improving the accuracy of wake-up.

[0103] In one embodiment, the local and at least one non-local intelligent voice device bound to the virtual interactive character are connected to the same network.

[0104] like Figure 5 As shown, in this embodiment, when the acquired voice data meets the wake-up condition in the far-field interaction mode, the step of controlling the intelligent voice device to enter the wake-up state (referred to as the wake-up control step) specifically includes the following steps:

[0105] S502: When the acquired voice data meets the wake-up condition in the far-field interaction mode, the acquired voice data is sent to the server.

[0106] In this embodiment, multiple intelligent voice devices are bound to the same virtual interactive role and connected to the same network. Figure 6 FIG. 1 is a schematic diagram of a scenario in which multiple intelligent voice devices are used in an embodiment. Figure 6 , intelligent voice devices 601, 602, 603 and 604 are respectively set in different rooms, and these four intelligent voice devices are all bound to the same virtual interactive character and connected to the same local area network. These four intelligent voice devices all have their own corresponding video acquisition range.

[0107] Specifically, each intelligent voice device can analyze the interaction mode it is in. When each intelligent voice device is in the far-field interaction mode, each intelligent voice device can obtain and analyze the voice data it has collected, determine whether its own voice data meets the wake-up conditions in the far-field interaction mode, and send the voice data it has obtained to the server when it meets the wake-up conditions in the far-field interaction mode.

[0108] In one embodiment, when no user object is recognized in the image acquired by each intelligent voice device and each intelligent voice device acquires voice data, it can be determined that each intelligent voice device is in the far-field interaction mode.

[0109] S504, receiving a target device designated instruction returned by the server, wherein the target device designated instruction is received when the volume of the voice data is the largest compared with similar voice data, where the similar voice data is voice data similar to the voice data and sent by a non-local intelligent voice device.

[0110] Among them, the target device specifying instruction is used to specify the target intelligent voice device to enter the wake-up state.

[0111] Specifically, when the server receives the voice data sent by each intelligent voice device, it can perform semantic analysis on the received voice data to determine the similarity between the voice data. The server can select the voice data with the highest volume from the similar voice data, designate the intelligent voice device corresponding to the voice data with the highest volume as the target device, generate a target device designation instruction, and return the target device designation instruction to the intelligent voice device corresponding to the selected voice data with the highest volume.

[0112] It can be understood that for an intelligent voice device that receives a target device specified instruction, the target device specified instruction it receives is the voice data sent by the device when the volume is the highest compared with similar voice data. The similar voice data is voice data sent by a non-local intelligent voice device that is similar to the voice data sent locally by the intelligent voice device.

[0113] For example, smart voice device A sends voice data 1 to the server, smart voice device B sends voice data 2 to the server, and smart voice device C sends voice data 3 to the server. The server compares the similarities between the three voice data and determines that voice data 1 and 2 are similar voice data, where the volume of voice data 1 is greater than that of voice data 2. The server can then specify smart voice device A that sent voice data 1 as the target device and return the target device designation instruction to smart voice device A.

[0114] In other embodiments, the target device designation instruction in S404 may also be received when the voice data is sent earliest compared with similar voice data. That is, the server may select the voice data with the earliest sending time from the similar voice data received, designate the intelligent voice device corresponding to the voice data with the earliest sending time as the target device, generate the target device designation instruction, and return the target device designation instruction to the intelligent voice device corresponding to the voice data with the earliest sending time.

[0115] S506, in response to the target device specified instruction, control the intelligent voice device to enter the wake-up state locally.

[0116] Specifically, the intelligent voice device that receives the target device specified instruction can control the local of the intelligent voice device to enter the wake-up state in response to the target device specified instruction.

[0117] In the above embodiment, when there are multiple intelligent voice devices connected to the same network and bound to the same virtual interactive role, by performing voice data similarity analysis, the intelligent voice device with the loudest volume in the voice data is determined as the target device, and the local of the intelligent voice device is controlled to enter the wake-up state. Among them, the intelligent voice device with the loudest volume in similar voice data is closest to the user, and the user is most likely to issue voice control to it, so it is more accurate to control the intelligent voice device with the loudest volume in similar voice data to enter the wake-up state. In addition, due to the relatively large volume, the voice data will be clearer, making the subsequent execution of the voice control instructions issued by the user more accurate.

[0118] In one embodiment, when a local and at least one non-local intelligent voice device bound to a virtual interactive character are connected to the same network, the intelligent voice device can also send a role exclusive state broadcast to the non-local intelligent voice device when controlling the intelligent voice device to enter a wake-up state. The role exclusive state broadcast is used to instruct the intelligent voice device that receives the role exclusive state broadcast not to respond to voice interaction control before the role exclusive state ends.

[0119] Among them, the role exclusive state is the state of being in an exclusive virtual interactive role. The role exclusive state broadcast is a broadcast that transmits the information that the local role is in the exclusive state to non-local intelligent voice devices. The role exclusive state broadcast is used to instruct the intelligent voice device that receives the role exclusive state broadcast not to respond to voice interaction control before the role exclusive state ends.

[0120] It can be understood that the non-local intelligent voice device described in the embodiment of the present application refers to an intelligent voice device that is locally bound to the same virtual interactive role as the intelligent voice device and connected to the same network. For example, the intelligent voice devices B and C are locally bound to the same virtual interactive role as the intelligent voice device A and connected to the same network. Then, the intelligent voice devices B and C are non-local intelligent voice devices relative to the intelligent voice device A.

[0121] It should be noted that the intelligent voice device can send a role-exclusive status broadcast to all non-local intelligent voice devices. The intelligent voice device can also send a role-exclusive status broadcast to a non-local intelligent voice device that sends similar voice data. Similarly, in conjunction with the example given in step 404, intelligent voice device A can send a role-exclusive status broadcast to intelligent voice devices B and C, or it can only send a role-exclusive status broadcast to intelligent voice device B that sends similar voice data.

[0122] In the above embodiment, when the intelligent voice device controls the intelligent voice device to enter the wake-up state, it can also send a role exclusive state broadcast to the non-local intelligent voice device, so that the intelligent voice device receiving the role exclusive state broadcast does not respond to voice interaction control before the role exclusive state ends, thereby avoiding conflicts between multiple intelligent voice devices and saving resources.

[0123] Figure 7It is a schematic diagram of a method for controlling an intelligent voice device in a scenario of multiple intelligent voice devices in one embodiment. Each intelligent voice device is connected to the same wireless network. When each intelligent voice device detects a user object in the image acquired within the scope of its respective video acquisition (a user object is detected in the image, which means that the user appears within the scope of the video acquisition) and identifies the direction of sight, it can send a role exclusive status broadcast to other non-local intelligent voice devices, and the intelligent voice device in the role exclusive state (referred to as the exclusive device) enters the wake-up state and waits for voice commands. If each intelligent voice device does not recognize a user object in the image acquired within the scope of its respective video acquisition, but obtains voice data, it can send its own voice data to the server for voice similarity analysis, and specify the target device from the intelligent voice devices with similar voice data through volume determination or time determination. The intelligent voice device designated as the target device can send a role exclusive status broadcast to other non-local intelligent voice devices, and the intelligent voice device in the role exclusive state (referred to as the exclusive device) enters the wake-up state and waits for voice commands.

[0124] In one embodiment, the method further includes: identifying the user object position in the image; mapping the user object position to the user position in the real scene; and controlling the orientation of the virtual interactive character so that the orientation changes as the user position moves and always remains toward the direction where the user position is.

[0125] The user object is the image content representing the user in the acquired image. The user object position is the position of the user object in the image. The user position in the real scene is the position of the user corresponding to the user object in the real scene.

[0126] It is understandable that the user object position can be characterized according to the position of the entire image content of the user object in the acquired image, or according to the position of a portion of the image content of the user object in the acquired image, such as the position of the facial image of the user object in the image.

[0127] Specifically, the intelligent voice device can identify the user object in the image, determine the position of the identified user object in the image, and obtain the user object position. The intelligent voice device can also identify the facial image in the image, and obtain the user object position according to the position of the facial image in the image.

[0128] It can be understood that the image is obtained by collecting images of real scenes. The position distribution of each image content in the image matches the position distribution in the real scene, and there is a mapping relationship between the position of each image content in the image and the position of its entity in the real scene. The intelligent voice device can map the user object position to the user position in the real scene based on the mapping relationship.

[0129] The intelligent voice device can control the orientation of the virtual interactive character according to the mapped user position. In one embodiment, the intelligent voice device can control the orientation of the virtual interactive character so that the orientation changes with the movement of the user position and always remains toward the direction where the user position is located.

[0130] In one embodiment, the user object has multiple positions. Controlling the orientation of the virtual interactive character so that the orientation changes with the movement of the user position and always faces the direction of the user position includes: determining the nearest user position from the mapped user positions; and controlling the virtual interactive character to always face the direction of the nearest user position.

[0131] Specifically, the intelligent voice device can determine the distance between each mapped user position and the intelligent voice device, and determine the user position closest to the intelligent voice device from the mapped user positions. The intelligent voice device can control the virtual interactive character to always face the direction of the user position closest to the intelligent voice device.

[0132] In one embodiment, there are multiple user object positions. Controlling the orientation of the virtual interactive character so that the orientation changes with the movement of the user position and always remains oriented toward the position where the user position is located includes: determining a target user object from the user objects corresponding to each user object position; controlling the orientation of the virtual interactive character so that the orientation changes with the movement of the target user position obtained by mapping corresponding to the target user object and always remains oriented toward the position where the target user position is located.

[0133] The target user position is to map the target user object position to the user position in the real scene. The target user object position is the position of the target user object in the image.

[0134] In one embodiment, the intelligent voice device can perform authority authentication on the user objects corresponding to each user object position, and determine the user objects that have passed the authority authentication as the target user objects. In one embodiment, the intelligent voice device can match the user objects corresponding to each user object position with the pre-stored authority user images, and determine the user objects that match the authority user images as having passed the authority authentication and can be determined as the target user objects.

[0135] In the above embodiment, the orientation of the virtual interactive character is controlled so that the orientation changes with the movement of the user's position and always remains oriented toward the user's position, thereby better simulating the actual conversation scene, improving the anthropomorphism of the intelligent voice device, and improving the interaction efficiency and accuracy.

[0136] In one embodiment, the method further includes: acquiring voice data including command statements; after entering the wake-up state, performing semantic analysis on the command statements; and generating and executing operation instructions corresponding to the naming statements based on the results of the semantic analysis.

[0137] Among them, a command statement is a statement that describes the command to be issued. It can be understood that a command statement can be a direct command statement or an implicit command statement. A direct command statement is a statement directly expressed in an imperative tone. An implicit command statement is a statement that expresses the command to be issued in a non-imperative tone. For example, "turn on the TV" is a direct command statement, and "I want to watch TV" is an "implicit command statement."

[0138] It can be understood that the intelligent voice device can obtain voice data including command statements before entering the wake-up state, and can also obtain voice data including command statements after entering the wake-up state.

[0139] Specifically, the intelligent voice device can parse the voice data, extract the command statements in the voice data, perform semantic analysis on the named statements, and generate and execute operation instructions corresponding to the named statements based on the results of the semantic analysis.

[0140] In one embodiment, the step of performing semantic analysis on the naming statement includes: performing word segmentation processing on the command statement, performing context semantic analysis on each obtained word segment, and obtaining a result of the semantic analysis.

[0141] In one embodiment, after entering the wake-up state, performing semantic analysis on the command statement includes: after entering the wake-up state, extracting facial features from the facial image and extracting voiceprint features from the voice data; identifying operation permissions based on the facial features and voiceprint features; when the operation permission identification is passed, performing semantic analysis on the command statement.

[0142] Specifically, after entering the wake-up state, the intelligent voice device can extract facial features from the facial image and extract voiceprint features from the voice data.

[0143] In one embodiment, the intelligent voice device may match the extracted facial features with the pre-stored facial features, and the extracted voiceprint features with the pre-stored voiceprint features. When the extracted facial features match the pre-stored facial features, and the extracted voiceprint features match the pre-stored voiceprint features, it is determined that the operation authority recognition has passed. In one embodiment, when any one of the facial feature matching and the voiceprint feature matching fails, it can be determined that the operation authority recognition has failed.

[0144] It can be understood that the intelligent voice device can perform semantic analysis on the command statement when the operation permission recognition is passed.

[0145] In one embodiment, the method further includes: when the operation authority identification fails, outputting authorization prompt information; obtaining authorization information corresponding to the authorization prompt information; authorizing the user corresponding to the facial image according to the authorization information; and after the authorization is successful, performing semantic analysis on the command statement.

[0146] The authorization prompt information is information used to prompt authorization. In one embodiment, the authorization prompt information can be information in the form of text, picture, video, voice, etc.

[0147] Authorization information is information for granting operation permissions. In one embodiment, the authorization information may be information in the form of voice, text, picture, video, etc. For example, a user with authorization permissions may speak a voice message granting operation permissions, or send a video message granting operation permissions, etc. A user with authorization permissions is a user who has the function of granting operation permissions.

[0148] In one embodiment, the authorization information may also be the binding information between the device used by the user to be authorized and the virtual interactive character. Among them, the device used by the user to be authorized is a device that performs operation authority authentication through face recognition. It can be understood that, for example, if device 1 is a device that needs to perform operation authority authentication through face recognition, then the binding between device 1 and the virtual interactive character means that device 1 has control authority over the intelligent voice device bound to the virtual interactive character, and the user to be authorized can operate device 1 through face recognition, so the binding information between the device used by the user to be authorized and the virtual interactive character can be used to represent the operation authority granted to the user to be authorized to the intelligent voice device bound to the virtual interactive character.

[0149] Specifically, after obtaining the authorization information, the intelligent voice device can authorize the user corresponding to the facial image according to the authorization information; after the authorization is successful, the command statement is semantically analyzed.

[0150] In one embodiment, the intelligent voice device can record the facial features and voiceprint features to authorize the user corresponding to the facial image. In one embodiment, the intelligent voice device can also display an authorization information entry interface, through which the facial features of the user corresponding to the facial image are collected from multiple angles, and the collected facial features and previously extracted voiceprint features are recorded to authorize the user corresponding to the facial image.

[0151] In the above embodiment, after entering the wake-up state, the intelligent voice device performs semantic analysis on the command statements in the voice data; based on the results of the semantic analysis, an operation instruction corresponding to the named statement is generated and executed. Voice control is realized without the need for manual operation by the user, thereby improving operation efficiency. Secondly, the combination of realization and voice control reduces the probability of misoperation and improves accuracy compared to single voice control.

[0152] In addition, the semantic analysis of command statements is combined with the identification of operation permissions to improve security.

[0153] In one embodiment, based on the results of semantic analysis, an operation instruction corresponding to the naming statement is generated and executed, including: when a command statement including a visualization object display keyword is obtained through semantic analysis, the visualization object keyword targeted by the visualization object display keyword in the command statement is obtained; based on the visualization object keyword, the display information corresponding to the visualization object corresponding to the visualization keyword is queried; and the display information corresponding to the visualization object obtained by the query is output.

[0154] Among them, the visualization object is an object that can be used for display. The visualization object includes videos, pictures or texts. The visualization object display keyword is a word that indicates that the visualization object is to be displayed, such as the words play, watch, put or broadcast. The visualization object keyword is a word that describes the visualization object, which can be a word that directly describes the name of the visualization object, or a word that expresses the description of the visualization object. For example, "I want to watch the Big Bang Theory", in which "watch" is the visualization object display keyword, and "The Big Bang Theory" is the video keyword for "watch". It can be understood that "The Big Bang Theory" is a word that directly describes the name of the visualization object - the video "The Big Bang Theory". For another example, "I want to watch a movie with the Civil War as the background and the heroine named Scarlett" "with the Civil War as the background and the heroine named Scarlett" is a word that expresses the description of the visualization object - the video "Gone with the Wind".

[0155] The display information corresponding to the visualization object includes the substantial content and description information of the visualization object.

[0156] In one embodiment, the visualization object is a video; the visualization object display keyword is a video playback keyword; and the visualization object keyword is a video keyword. According to the visualization object keyword, querying the display information corresponding to the corresponding visualization object includes: matching the video keyword with the video name in a preset video library; and obtaining the video information corresponding to the matched video name from the video library.

[0157] The video play keyword is a word used to indicate the play of a video, such as "play" or "watch". The video keyword is a word describing a video. In one embodiment, the video keyword can be a video name or a keyword that indirectly describes the video.

[0158] Specifically, the intelligent voice device can perform semantic analysis on the command statement. When the command statement includes a video playback keyword through semantic analysis, the video keyword targeted by the video playback keyword in the command statement is matched with the video name in the preset video library; and the video information corresponding to the matched video name is searched from the video library.

[0159] The video library is a library for storing video collections. The video library includes video information and corresponding video names. The video information includes video content and video description information.

[0160] Specifically, a video library is pre-set in the intelligent voice device. The intelligent voice device can match the video keyword with the video name in the preset video library in response to the video play instruction. The intelligent voice device can search for corresponding video information from the video library according to the matched video name.

[0161] In one embodiment, the intelligent voice device can match the video keyword with the video name in the preset video library, obtain the corresponding matching probability, compare the matching probability with the preset matching threshold, and when the matching probability is greater than or equal to the preset matching threshold, it is determined that the video keyword matches the video name. It can be understood that when the matching probability is less than the preset matching threshold, it is determined that the video keyword does not match the video name.

[0162] It can be understood that the preset matching threshold can be a single matching threshold or a multi-stage matching threshold, for example, a first matching threshold and a second matching threshold, wherein the first matching threshold is greater than the second matching threshold.

[0163] In one embodiment, the intelligent voice device can match the video keyword with each video name in the video library word by word, obtain the matching probability of each word according to the matching result of each word, and obtain the corresponding matching probability of the video keyword according to the matching probability of each word in the video keyword. It can be understood that the intelligent voice device can sum or average the matching probabilities of each word in the video keyword to obtain the corresponding matching probability of the video keyword.

[0164] In one embodiment, based on the results of semantic analysis, generating and executing operation instructions corresponding to the naming statement also includes: when semantic analysis determines that the command statement is a search command statement, determining the target search object corresponding to the command statement; generating a search instruction corresponding to the target search object; and in response to the search instruction, obtaining and displaying the search results corresponding to the target search object.

[0165] The search command statement is a command statement used to instruct content search. The target search object is an object to be searched.

[0166] Specifically, the intelligent voice device can directly extract the target search object from the command sentence, or determine the target search object described and expressed according to the context semantics of the naming sentence. For example, "I want to know the age of Justin Bieber", the command sentence includes the target search object "Justin Bieber's age", and for another example, "I want to know the age of the singer who sang 'baby'", the target search object "Justin Bieber's age" can be determined according to the context semantics.

[0167] The intelligent voice device can generate a search instruction corresponding to the target search object; in response to the search instruction, obtain and display the search results corresponding to the target search object. In one embodiment, the intelligent voice device can obtain the search results corresponding to the target search object from the server and display the search results. The intelligent voice device can display the search results locally or through a smart TV.

[0168] It can be understood that the smart voice device can display the search results on the smart TV when the smart TV is turned on, or it can turn on the smart TV and control the smart TV to display the search results when the smart TV is turned off.

[0169] In the above embodiment, when the command statement includes a visualization object display keyword through semantic analysis, the visualization object keyword targeted by the visualization object display keyword in the command statement is obtained; according to the visualization object keyword, the display information corresponding to the corresponding visualization object is queried; and the queried display information corresponding to the visualization object is output. Compared with the traditional smart speaker that can only output voice information, it can realize the processing related to information display, which enhances the function of the smart voice device.

[0170] In one embodiment, outputting the queried display information corresponding to the visualization object includes: turning on a smart TV bound to the smart voice device; and controlling the smart TV to display video information corresponding to the matched video name.

[0171] Among them, a smart TV is a TV with computer processing function. Video information includes video content and video description information.

[0172] Specifically, the smart voice device can bind the smart voice device to the smart TV in advance. After the smart voice device obtains the video information corresponding to the matched video name from the video library, it can trigger the smart TV bound to the smart voice device to turn on; and control the smart TV to display the video information corresponding to the matched video name.

[0173] It can be understood that in other embodiments, when the smart TV itself is already in the turned-on state, the smart voice device can directly control the turned-on smart TV to display the video information corresponding to the matched video name.

[0174] It is understandable that one smart voice device can be bound to one or more smart TVs. When multiple smart TVs are bound, the smart voice device can turn on all bound smart TVs, or turn on the smart TV closest to the smart voice device or the default smart TV.

[0175] In one embodiment, the intelligent voice device can control the smart TV to run the video client corresponding to the video library, and display the video information corresponding to the matched video name through the opened video client. For example, if the video library of the Tencent video client (a video playback client developed by Tencent) is stored in the intelligent voice device, after matching the video keyword with the video name in the video library of the Tencent video client, the intelligent voice device can turn on the bound smart TV and control the smart TV to run the Tencent video client, and display the video information corresponding to the matched video name through the Tencent video client.

[0176] In one embodiment, controlling the smart TV to display video information corresponding to the matched video name includes: when the probability of matching the video keyword and the matched video name is greater than or equal to a first matching threshold, controlling the smart TV to play the video content corresponding to the matched video name. When the probability of matching the video keyword and the matched video name is greater than or equal to a second matching threshold and less than the first matching threshold, controlling the smart TV to display video description information corresponding to the matched video name.

[0177] The video description information is the basic information describing the video.

[0178] Figure 8 FIG. 1 is a schematic diagram of a scenario in which an intelligent voice device responds to voice control in an embodiment. Figure 8, a virtual interactive character "Xiao Q" 804 is presented on the intelligent voice device 802, the sight direction of the user 806 points to "Xiao Q" 804, and the intelligent voice device 802 enters the awakening state. The intelligent voice device 802 obtains the voice data including the command statement issued by the user 806 and performs semantic analysis. When the intelligent voice device 802 analyzes the command statement including the video playback keyword, the video keyword targeted by the video playback keyword in the command statement can be obtained, and the video information corresponding to the video keyword can be queried from the video library. The intelligent voice device 802 can turn on the bound smart TV 808 and control the smart TV 808 to play the video information.

[0179] In the above embodiment, by binding with a smart TV, the smart TV is controlled to display video information corresponding to the matched video name, thereby improving the quality of video information display.

[0180] In one embodiment, a continuous power supply device may also be integrated into the intelligent voice device. It is understood that the continuous power supply device may be integrated with the device where the card slot of the intelligent voice device is located, or may be integrated into the device where the power source is located. The intelligent voice device achieves continuous power supply through the continuous power supply device. In one embodiment, the intelligent voice device may be triggered to implement the intelligent voice device control method in each embodiment of the present application when placed on the continuous power supply device. Fig. 9 This is a schematic diagram of continuous power supply in one embodiment. Fig. 9 , the continuous power supply device is connected to the card slot of the intelligent voice device. In the above embodiment, by integrating the continuous power supply device, continuous power supply can be maintained, avoiding the situation where the power is insufficient and the device cannot work, thereby improving the practicality of the intelligent voice device.

[0181] like Fig.10 As shown, in one embodiment, a method for controlling an intelligent voice device is provided, and the method specifically includes the following steps:

[0182] S1002, scan the role QR code representing the identity of the virtual interactive role; extract the download link address of the corresponding virtual interactive role information from the role QR code; download the virtual interactive role information according to the download link address, and bind the corresponding virtual interactive role to the local intelligent voice device according to the downloaded virtual interactive role information.

[0183] In one embodiment, the intelligent voice device is a holographic intelligent voice imaging device; the virtual interactive role information is three-dimensional holographic virtual interactive role information. The method also includes: using a liquid crystal projection device in the holographic intelligent voice imaging device, three-dimensionally projecting the virtual interactive role information onto a holographic film in the holographic intelligent voice imaging device; and presenting a three-dimensional holographic virtual interactive role on the holographic film of the holographic intelligent voice imaging device.

[0184] S1004, connecting to the same network via a local and non-local intelligent voice device bound to the same virtual interactive role.

[0185] S1006, acquiring an image captured within the video capture range. When a user object is recognized in the image, the near-field interaction mode is entered, and step S1008 is executed. When no user object is recognized in the image and voice data is acquired, the far-field interaction mode is entered, and step S1014 is executed.

[0186] S1008, identifying a facial image in the image; and determining a corresponding sight line direction according to the facial image. When the determined sight line direction points to the virtual interactive character presented locally on the intelligent voice device, proceed to step S1010; when the sight line direction does not point to the virtual interactive character presented locally on the intelligent voice device, proceed to step S1012.

[0187] In one embodiment, determining the corresponding line of sight direction based on a facial image includes: identifying the pupil position in the facial image; determining the corresponding line of sight direction based on the identified pupil position; or determining the orientation of the face relative to the intelligent voice device based on the facial image; and determining the corresponding line of sight direction based on the orientation of the face relative to the intelligent voice device.

[0188] S1010, controlling the intelligent voice device to enter a wake-up state.

[0189] In one embodiment, the intelligent voice device can also obtain voice data, and when the line of sight points to the intelligent voice device and there is a preset wake-up word in the voice data, the intelligent voice device is controlled to enter the wake-up state. It can be understood that in this embodiment, the virtual interactive character can be presented before the intelligent voice device enters the wake-up state (that is, when it is in a semi-dormant state), or it can be presented locally on the intelligent voice device after the intelligent voice device is controlled to enter the wake-up state.

[0190] In one embodiment, the method further includes: when the intelligent voice device is in a semi-dormant state, selecting a target picture from a gallery corresponding to the intelligent voice device; and displaying the selected target picture.

[0191] S1012, obtaining voice data; when a preset wake-up word exists in the voice data, controlling the intelligent voice device to enter a wake-up state.

[0192] In one embodiment, the intelligent voice device can determine whether there is a preset wake-up word in the voice data through the following steps: extracting feature words in the voice data; matching the extracted feature words with the preset wake-up words; when the feature words match the preset wake-up words, determining that the preset wake-up words exist in the voice data.

[0193] S1014, when the wake-up conditions in the far-field interaction mode are met, the acquired voice data is sent to the server; the target device specified instruction returned by the server is received; and in response to the target device specified instruction, the intelligent voice device is controlled to enter the wake-up state locally.

[0194] The target device specifies the instruction when the voice data is received at the maximum volume compared to similar voice data, where the similar voice data is voice data similar to the voice data and sent by a non-local intelligent voice device;

[0195] S1016, sending a role exclusive state broadcast to a non-local intelligent voice device, where the role exclusive state broadcast is used to instruct the intelligent voice device that receives the role exclusive state broadcast not to respond to voice interaction control before the role exclusive state ends.

[0196] S1018, identifying the user object position in the image; mapping the user object position to the user position in the real scene; and controlling the orientation of the virtual interactive character so that the orientation changes with the movement of the user position and always remains toward the direction where the user position is located.

[0197] In one embodiment, there are multiple user object positions. Controlling the orientation of the virtual interactive character so that the orientation changes with the movement of the user position and always remains oriented toward the position where the user position is located includes: determining the user position closest to the user position obtained by mapping; controlling the virtual interactive character to always face the position where the user position closest to the user position is located; or determining a target user object from the user objects corresponding to the user object positions; controlling the orientation of the virtual interactive character so that the orientation changes with the movement of the target user position obtained by mapping corresponding to the target user object and always remains oriented toward the position where the target user position is located.

[0198] S1020, obtain voice data including command statements; after entering the awakening state, extract facial features from the facial image and extract voiceprint features from the voice data; perform operation authority identification based on the facial features and voiceprint features. If the operation authority identification fails, proceed to step S1022; if the operation authority identification passes, proceed to step S1024.

[0199] S1022, output authorization prompt information; obtain authorization information corresponding to the authorization prompt information; authorize the user corresponding to the facial image according to the authorization information. After the authorization is successful, enter step S1024.

[0200] S1024, performing semantic analysis on the command statement. When the semantic analysis shows that the command statement includes a video playback keyword, matching the video keyword targeted by the video playback keyword in the command statement with the video name in a preset video library; and obtaining video information corresponding to the matched video name from the video library.

[0201] S1026, turn on the smart TV bound to the smart voice device; when the matching probability between the video keyword and the matching video name is greater than or equal to the first matching threshold, control the smart TV to play the video content corresponding to the matched video name; when the matching probability between the video keyword and the matching video name is greater than or equal to the second matching threshold and less than the first matching threshold, control the smart TV to display the video description information corresponding to the matched video name.

[0202] It should be noted that the user information (including but not limited to user device information, user personal information, etc., such as collected user images, facial images, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc., such as analyzed user location or user line of sight, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0203] like Fig.11 As shown, in one embodiment, a smart voice device control device 1100 is provided, the device 1100 includes: an acquisition module 1102, an image recognition module 1104, a sight direction determination module 1106 and a control module 1108, wherein:

[0204] The acquisition module 1102 is used to acquire images captured within the video acquisition range.

[0205] The image recognition module 1104 is used to recognize facial images in the image.

[0206] The sight line direction determination module 1106 is used to determine the corresponding sight line direction according to the facial image.

[0207] The control module 1108 is used to control the intelligent voice device to enter a wake-up state when the determined sight line direction points to the virtual interactive character presented locally on the intelligent voice device.

[0208] In one embodiment, the gaze direction determination module 1106 is also used to identify the pupil position in the facial image; determine the corresponding gaze direction based on the identified pupil position; or determine the orientation of the face relative to the intelligent voice device based on the facial image; and determine the corresponding gaze direction based on the orientation of the face relative to the intelligent voice device.

[0209] In one embodiment, the control module 1108 is also used to obtain voice data; when the determined line of sight direction points to a virtual interactive character presented locally on the intelligent voice device and there is a preset wake-up word in the voice data, the intelligent voice device is controlled to enter a wake-up state.

[0210] In one embodiment, the control module 1108 is also used to extract feature words from the voice data; match the extracted feature words with the preset wake-up words; and when the feature words match the preset wake-up words, determine that the preset wake-up words exist in the voice data.

[0211] In one embodiment, the control module 1108 is also used to obtain voice data; when the line of sight is directed to the intelligent voice device and there is a preset wake-up word in the voice data, the intelligent voice device is controlled to enter the wake-up state and the bound virtual interactive character is presented locally on the intelligent voice device.

[0212] In one embodiment, the control module 1108 is also used to select a target picture from a gallery corresponding to the intelligent voice device when the intelligent voice device is in a semi-dormant state; and display the selected target picture.

[0213] In one embodiment, the control module 1108 is also used to obtain voice data; when the line of sight is not directed to the virtual interactive character presented locally on the intelligent voice device and there is a preset wake-up word in the voice data, the intelligent voice device is controlled to enter a wake-up state.

[0214] like Fig.12 As shown, in one embodiment, the apparatus 1100 further includes:

[0215] The interaction mode determination module 1103 is used to enter the near-field interaction mode when a user object is recognized in the image, and notify the image recognition module 1104 to execute the step of recognizing the facial image in the image; when the user object is not recognized in the image and voice data is obtained, it enters the far-field interaction mode, and notifies the control module 1108 to execute the step of controlling the intelligent voice device to enter the wake-up state when the acquired voice data meets the wake-up conditions in the far-field interaction mode.

[0216] In one embodiment, the wake-up condition in the far-field interaction mode includes: the acquired voice data includes a preset wake-up word that appears a preset number of times within a preset period.

[0217] In one embodiment, the local and at least one non-local intelligent voice device bound to the virtual interactive character are connected to the same network. The control module 1108 is also used to send the acquired voice data to the server when the wake-up condition in the far-field interaction mode is met; receive the target device specified instruction returned by the server, the target device specified instruction is received when the voice data has the largest volume compared with similar voice data, and the similar voice data is voice data similar to the voice data sent by the non-local intelligent voice device; in response to the target device specified instruction, control the local intelligent voice device to enter the wake-up state.

[0218] In one embodiment, the local and at least one non-local intelligent voice device bound to the virtual interactive character are connected to the same network. The control module 1108 is also used to send a role exclusive state broadcast to the non-local intelligent voice device, and the role exclusive state broadcast is used to instruct the intelligent voice device that receives the role exclusive state broadcast not to respond to voice interaction control before the role exclusive state ends.

[0219] In one embodiment, the control module 1108 is also used to identify the user object position in the image; map the user object position to the user position in the real scene; and control the orientation of the virtual interactive character so that the orientation changes as the user position moves and always remains toward the direction of the user position.

[0220] In one embodiment, there are multiple user object positions. The control module 1108 is further used to determine the nearest user position from the mapped user positions; control the virtual interactive character to always face the direction of the nearest user position; or determine the target user object from the user objects corresponding to the user object positions; control the direction of the virtual interactive character so that the direction changes with the movement of the target user position corresponding to the mapped target user object and always faces the direction of the target user position.

[0221] like Fig.13 As shown, in one embodiment, the apparatus 1100 further includes:

[0222] The command execution module 1110 is used to obtain voice data including command statements; after entering the awakening state, perform semantic analysis on the command statements; and generate and execute operation instructions corresponding to the named statements based on the results of the semantic analysis.

[0223] In one embodiment, the command execution module 1110 is also used to extract facial features from facial images and voiceprint features from voice data after entering the wake-up state; identify operation permissions based on facial features and voiceprint features; and perform semantic analysis on the command statement when the operation permission identification is passed.

[0224] In one embodiment, the command execution module 1110 is also used to output authorization prompt information when the operation permission identification fails; obtain authorization information corresponding to the authorization prompt information; authorize the user corresponding to the facial image according to the authorization information; and perform semantic analysis on the command statement after the authorization is successful.

[0225] In one embodiment, the command execution module 1110 is also used to obtain the visual object keyword targeted by the visual object display keyword in the command statement when a semantic analysis is performed to obtain a visual object keyword; query the display information corresponding to the corresponding visual object based on the visual object keyword; and output the queried display information corresponding to the visual object.

[0226] In one embodiment, the visualization object is a video; the visualization object display keyword is a video playback keyword; the visualization object keyword is a video keyword. The command execution module 1110 is also used to match the video keyword with the video name in the preset video library; and obtain video information corresponding to the matched video name from the video library.

[0227] In one embodiment, the command execution module 1110 is also used to turn on a smart TV bound to the smart voice device; and control the smart TV to display video information corresponding to the matched video name.

[0228] In one embodiment, the command execution module 1110 is also used to control the smart TV to play the video content corresponding to the matched video name when the matching probability between the video keyword and the matched video name is greater than or equal to the first matching threshold; when the matching probability between the video keyword and the matched video name is greater than or equal to the second matching threshold and less than the first matching threshold, control the smart TV to display the video description information corresponding to the matched video name.

[0229] In one embodiment, the command execution module 1110 is also used to determine the target search object corresponding to the command statement when a semantic analysis is performed to determine that the command statement is a search command statement; generate a search instruction corresponding to the target search object; and in response to the search instruction, obtain and display the search results corresponding to the target search object.

[0230] In one embodiment, the apparatus 1100 further includes:

[0231] A role binding module (not shown in the figure) is used to scan a role QR code representing the identity of a virtual interactive role; extract a download link address of corresponding virtual interactive role information from the role QR code; download the virtual interactive role information according to the download link address, and bind the corresponding virtual interactive role to the local according to the downloaded virtual interactive role information.

[0232] In one embodiment, the intelligent voice device is a holographic intelligent voice imaging device; the virtual interactive role information is three-dimensional holographic virtual interactive role information. The device 1100 also includes:

[0233] The role presentation module (not shown in the figure) is used to perform three-dimensional holographic projection of virtual interactive role information onto a holographic film in the holographic intelligent voice imaging device through a liquid crystal projection device in the holographic intelligent voice imaging device; and present a three-dimensional holographic virtual interactive role on the holographic film of the holographic intelligent voice imaging device.

[0234] Fig.14 FIG. 1 is a schematic diagram of the internal structure of a computer device in one embodiment. Fig.14 , the computer device may be Figure 1 The intelligent voice device shown in the figure, the computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium of the computer device can store an operating system and a computer program. When the computer program is executed, the processor can execute a method for controlling an intelligent voice device. The processor of the computer device is used to provide computing and control capabilities to support the operation of the entire computer device. The internal memory can store a computer program, and when the computer program is executed by the processor, the processor can execute a method for controlling an intelligent voice device. The network interface of the computer device is used for network communication. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, etc. The input device of the computer device can be a touch layer covered on the display screen, or a button, trackball or touchpad set on the terminal housing, or an external keyboard, touchpad or mouse, etc. The computer device can be a personal computer, a mobile terminal or a vehicle-mounted device, and the mobile terminal includes at least one of a mobile phone, a tablet computer, a personal digital assistant or a wearable device, etc.

[0235] Those skilled in the art will understand that Fig.14 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0236] In one embodiment, the intelligent voice device control device provided by the present application can be implemented in the form of a computer program. The computer program can be Fig.14 The non-volatile storage medium of the computer device can store various program modules constituting the intelligent voice device control device, for example, Fig.11The acquisition module 1102, image recognition module 1104, sight direction determination module 1106 and control module 1108 are shown. The computer program composed of each program module is used to enable the computer device to execute the steps of the intelligent voice device control method of each embodiment of the present application described in this specification. For example, the computer device can Fig.11 The acquisition module 1102 in the intelligent voice device control device 1100 shown acquires the image captured within the video acquisition range, and recognizes the facial image in the image through the image recognition module 1104. The intelligent voice device can determine the corresponding sight direction according to the facial image through the sight direction determination module 1106, and control the intelligent voice device to enter the awakening state through the control module 1108 when the determined sight direction points to the virtual interactive character presented locally on the intelligent voice device.

[0237] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor executes the following steps: acquiring an image captured within a video capture range; identifying a facial image in the image; determining a corresponding sight line direction based on the facial image; and when the determined sight line direction points to a virtual interactive character presented locally on the intelligent voice device, controlling the intelligent voice device to enter a wake-up state.

[0238] In one embodiment, determining the corresponding line of sight direction based on a facial image includes: identifying the pupil position in the facial image; determining the corresponding line of sight direction based on the identified pupil position; or determining the orientation of the face relative to the intelligent voice device based on the facial image; and determining the corresponding line of sight direction based on the orientation of the face relative to the intelligent voice device.

[0239] In one embodiment, when the determined line of sight direction points to a virtual interactive character presented locally on the smart voice device, controlling the smart voice device to enter a wake-up state includes: obtaining voice data; when the determined line of sight direction points to the virtual interactive character presented locally on the smart voice device and there is a preset wake-up word in the voice data, controlling the smart voice device to enter a wake-up state.

[0240] In one embodiment, the computer program also causes the processor to perform the following steps: extracting feature words from the voice data; matching the extracted feature words with preset wake-up words; and when the feature words match the preset wake-up words, determining that the preset wake-up words exist in the voice data.

[0241] In one embodiment, when the determined line of sight direction points to a virtual interactive character presented locally on the smart voice device, controlling the smart voice device to enter a wake-up state includes: obtaining voice data; when the line of sight direction points to the smart voice device and there is a preset wake-up word in the voice data, controlling the smart voice device to enter a wake-up state and presenting the bound virtual interactive character locally on the smart voice device.

[0242] In one embodiment, the computer program further causes the processor to perform the following steps: when the intelligent voice device is in a semi-dormant state, selecting a target picture from a gallery corresponding to the intelligent voice device; and displaying the selected target picture.

[0243] In one embodiment, the computer program also causes the processor to perform the following steps: obtaining voice data; when the line of sight is not directed to the virtual interactive character presented locally on the intelligent voice device and there is a preset wake-up word in the voice data, controlling the intelligent voice device to enter a wake-up state.

[0244] In one embodiment, the computer program also causes the processor to perform the following steps: when a user object is recognized in the image, it enters the near-field interaction mode and executes the step of recognizing the facial image in the image; when the user object is not recognized in the image and voice data is acquired, it enters the far-field interaction mode, and when the acquired voice data meets the wake-up conditions in the far-field interaction mode, the intelligent voice device is controlled to enter the wake-up state.

[0245] In one embodiment, the wake-up condition in the far-field interaction mode includes: the acquired voice data includes a preset wake-up word that appears a preset number of times within a preset period.

[0246] In one embodiment, the local and at least one non-local intelligent voice device bound to the virtual interactive role are connected to the same network. When the acquired voice data meets the wake-up condition in the far-field interaction mode, controlling the intelligent voice device to enter the wake-up state includes: sending the acquired voice data to the server when the wake-up condition in the far-field interaction mode is met; receiving the target device specified instruction returned by the server, the target device specified instruction is received when the voice data has the largest volume compared with similar voice data, and the similar voice data is voice data similar to the voice data sent by the non-local intelligent voice device; in response to the target device specified instruction, controlling the local intelligent voice device to enter the wake-up state.

[0247] In one embodiment, the local and at least one non-local intelligent voice device bound to the virtual interactive character are connected to the same network. The computer program further causes the processor to perform the following steps: sending a role exclusive state broadcast to the non-local intelligent voice device, the role exclusive state broadcast is used to instruct the intelligent voice device that receives the role exclusive state broadcast not to respond to voice interactive control before the role exclusive state ends.

[0248] In one embodiment, the computer program further causes the processor to perform the following steps: identifying the user object position in the image; mapping the user object position to the user position in the real scene; and controlling the orientation of the virtual interactive character so that the orientation changes as the user position moves and remains always oriented toward the direction where the user position is located.

[0249] In one embodiment, there are multiple user object positions. Controlling the orientation of the virtual interactive character so that the orientation changes with the movement of the user position and always remains oriented toward the position where the user position is located includes: determining the user position closest to the user position obtained by mapping; controlling the virtual interactive character to always face the position where the user position closest to the user position is located; or determining a target user object from the user objects corresponding to the user object positions; controlling the orientation of the virtual interactive character so that the orientation changes with the movement of the target user position obtained by mapping corresponding to the target user object and always remains oriented toward the position where the target user position is located.

[0250] In one embodiment, the computer program further causes the processor to execute the following steps: acquiring voice data including command statements; after entering the wake-up state, performing semantic analysis on the command statements; and generating and executing operation instructions corresponding to the naming statements based on the results of the semantic analysis.

[0251] In one embodiment, after entering the wake-up state, performing semantic analysis on the command statement includes: after entering the wake-up state, extracting facial features from the facial image and extracting voiceprint features from the voice data; identifying operation permissions based on the facial features and voiceprint features; when the operation permission identification is passed, performing semantic analysis on the command statement.

[0252] In one embodiment, the computer program also causes the processor to perform the following steps: when the operation authority identification fails, output authorization prompt information; obtain authorization information corresponding to the authorization prompt information; authorize the user corresponding to the facial image according to the authorization information; after the authorization is successful, perform semantic analysis on the command statement.

[0253] In one embodiment, based on the results of semantic analysis, an operation instruction corresponding to the naming statement is generated and executed, including: when a command statement including a visual object display keyword is obtained through semantic analysis, the visual object keyword targeted by the visual object display keyword in the command statement is obtained; based on the visual object keyword, the display information corresponding to the corresponding visual object is queried; and the display information corresponding to the visual object obtained is output.

[0254] In one embodiment, the visualization object is a video; the visualization object display keyword is a video playback keyword; and the visualization object keyword is a video keyword. According to the visualization object keyword, querying the display information corresponding to the corresponding visualization object includes: when the command statement includes the video keyword through semantic analysis, matching the video keyword with the video name in the preset video library; and obtaining the video information corresponding to the matched video name from the video library.

[0255] In one embodiment, outputting the queried display information corresponding to the visualization object includes: turning on a smart TV bound to the smart voice device; and controlling the smart TV to display video information corresponding to the matched video name.

[0256] In one embodiment, controlling a smart TV to display video information corresponding to a matched video name includes: when the probability of a match between a video keyword and a matched video name is greater than or equal to a first match threshold, controlling the smart TV to play video content corresponding to the matched video name; when the probability of a match between a video keyword and a matched video name is greater than or equal to a second match threshold and less than the first match threshold, controlling the smart TV to display video description information corresponding to the matched video name.

[0257] In one embodiment, based on the results of semantic analysis, generating and executing operation instructions corresponding to the naming statement also includes: when semantic analysis determines that the command statement is a search command statement, determining the target search object corresponding to the command statement; generating a search instruction corresponding to the target search object; and in response to the search instruction, obtaining and displaying the search results corresponding to the target search object.

[0258] In one embodiment, the computer program further causes the processor to perform the following steps: scanning a character QR code representing the identity of a virtual interactive character; extracting a download link address of corresponding virtual interactive character information from the character QR code; downloading the virtual interactive character information according to the download link address, and binding the corresponding virtual interactive character to the local according to the downloaded virtual interactive character information.

[0259] In one embodiment, the intelligent voice device is a holographic intelligent voice imaging device; the virtual interactive role information is three-dimensional holographic virtual interactive role information. The computer program further causes the processor to perform the following steps: three-dimensionally project the virtual interactive role information onto a holographic film in the holographic intelligent voice imaging device through a liquid crystal projection device in the holographic intelligent voice imaging device; and present a three-dimensional holographic virtual interactive role on the holographic film of the holographic intelligent voice imaging device.

[0260] In one embodiment, a storage medium storing a computer program is provided. When the computer program is executed by a processor, the processor executes the following steps: acquiring an image captured within a video capture range; identifying a facial image in the image; determining a corresponding sight line direction based on the facial image; and when the determined sight line direction points to a virtual interactive character presented locally on the intelligent voice device, controlling the intelligent voice device to enter a wake-up state.

[0261] In one embodiment, determining the corresponding line of sight direction based on a facial image includes: identifying the pupil position in the facial image; determining the corresponding line of sight direction based on the identified pupil position; or determining the orientation of the face relative to the intelligent voice device based on the facial image; and determining the corresponding line of sight direction based on the orientation of the face relative to the intelligent voice device.

[0262] In one embodiment, when the determined line of sight direction points to a virtual interactive character presented locally on the smart voice device, controlling the smart voice device to enter a wake-up state includes: obtaining voice data; when the determined line of sight direction points to the virtual interactive character presented locally on the smart voice device and there is a preset wake-up word in the voice data, controlling the smart voice device to enter a wake-up state.

[0263] In one embodiment, the computer program also causes the processor to perform the following steps: extracting feature words from the voice data; matching the extracted feature words with preset wake-up words; and when the feature words match the preset wake-up words, determining that the preset wake-up words exist in the voice data.

[0264] In one embodiment, when the determined line of sight direction points to a virtual interactive character presented locally on the smart voice device, controlling the smart voice device to enter a wake-up state includes: obtaining voice data; when the line of sight direction points to the smart voice device and there is a preset wake-up word in the voice data, controlling the smart voice device to enter a wake-up state and presenting the bound virtual interactive character locally on the smart voice device.

[0265] In one embodiment, the computer program further causes the processor to perform the following steps: when the intelligent voice device is in a semi-dormant state, selecting a target picture from a gallery corresponding to the intelligent voice device; and displaying the selected target picture.

[0266] In one embodiment, the computer program also causes the processor to perform the following steps: obtaining voice data; when the line of sight is not directed to the virtual interactive character presented locally on the intelligent voice device and there is a preset wake-up word in the voice data, controlling the intelligent voice device to enter a wake-up state.

[0267] In one embodiment, the computer program also causes the processor to perform the following steps: when a user object is recognized in the image, it enters the near-field interaction mode and executes the step of recognizing the facial image in the image; when the user object is not recognized in the image and voice data is acquired, it enters the far-field interaction mode, and when the acquired voice data meets the wake-up conditions in the far-field interaction mode, the intelligent voice device is controlled to enter the wake-up state.

[0268] In one embodiment, the wake-up condition in the far-field interaction mode includes: the acquired voice data includes a preset wake-up word that appears a preset number of times within a preset period.

[0269] In one embodiment, the local and at least one non-local intelligent voice device bound to the virtual interactive role are connected to the same network. When the acquired voice data meets the wake-up condition in the far-field interaction mode, controlling the intelligent voice device to enter the wake-up state includes: sending the acquired voice data to the server when the wake-up condition in the far-field interaction mode is met; receiving the target device specified instruction returned by the server, the target device specified instruction is received when the voice data has the largest volume compared with similar voice data, and the similar voice data is voice data similar to the voice data sent by the non-local intelligent voice device; in response to the target device specified instruction, controlling the local intelligent voice device to enter the wake-up state.

[0270] In one embodiment, the local and at least one non-local intelligent voice device bound to the virtual interactive character are connected to the same network. The computer program further causes the processor to perform the following steps: sending a role exclusive state broadcast to the non-local intelligent voice device, the role exclusive state broadcast is used to instruct the intelligent voice device that receives the role exclusive state broadcast not to respond to voice interactive control before the role exclusive state ends.

[0271] In one embodiment, the computer program further causes the processor to perform the following steps: identifying the user object position in the image; mapping the user object position to the user position in the real scene; and controlling the orientation of the virtual interactive character so that the orientation changes as the user position moves and remains always oriented toward the direction where the user position is located.

[0272] In one embodiment, there are multiple user object positions. Controlling the orientation of the virtual interactive character so that the orientation changes with the movement of the user position and always remains oriented toward the position where the user position is located includes: determining the user position closest to the user position obtained by mapping; controlling the virtual interactive character to always face the position where the user position closest to the user position is located; or determining a target user object from the user objects corresponding to the user object positions; controlling the orientation of the virtual interactive character so that the orientation changes with the movement of the target user position obtained by mapping corresponding to the target user object and always remains oriented toward the position where the target user position is located.

[0273] In one embodiment, the computer program further causes the processor to execute the following steps: acquiring voice data including command statements; after entering the wake-up state, performing semantic analysis on the command statements; and generating and executing operation instructions corresponding to the naming statements based on the results of the semantic analysis.

[0274] In one embodiment, after entering the wake-up state, performing semantic analysis on the command statement includes: after entering the wake-up state, extracting facial features from the facial image and extracting voiceprint features from the voice data; identifying operation permissions based on the facial features and voiceprint features; when the operation permission identification is passed, performing semantic analysis on the command statement.

[0275] In one embodiment, the computer program also causes the processor to perform the following steps: when the operation authority identification fails, output authorization prompt information; obtain authorization information corresponding to the authorization prompt information; authorize the user corresponding to the facial image according to the authorization information; after the authorization is successful, perform semantic analysis on the command statement.

[0276] In one embodiment, based on the results of semantic analysis, an operation instruction corresponding to the naming statement is generated and executed, including: when a command statement including a visual object display keyword is obtained through semantic analysis, the visual object keyword targeted by the visual object display keyword in the command statement is obtained; based on the visual object keyword, the display information corresponding to the corresponding visual object is queried; and the display information corresponding to the visual object obtained is output.

[0277] In one embodiment, the visualization object is a video; the visualization object display keyword is a video playback keyword; and the visualization object keyword is a video keyword. According to the visualization object keyword, querying the display information corresponding to the corresponding visualization object includes: matching the video keyword with the video name in a preset video library; and obtaining the video information corresponding to the matched video name from the video library.

[0278] In one embodiment, outputting the queried display information corresponding to the visualization object includes: turning on a smart TV bound to the smart voice device; and controlling the smart TV to display video information corresponding to the matched video name.

[0279] In one embodiment, controlling a smart TV to display video information corresponding to a matched video name includes: when the probability of a match between a video keyword and a matched video name is greater than or equal to a first match threshold, controlling the smart TV to play video content corresponding to the matched video name; when the probability of a match between a video keyword and a matched video name is greater than or equal to a second match threshold and less than the first match threshold, controlling the smart TV to display video description information corresponding to the matched video name.

[0280] In one embodiment, based on the results of semantic analysis, generating and executing operation instructions corresponding to the naming statement also includes: when semantic analysis determines that the command statement is a search command statement, determining the target search object corresponding to the command statement; generating a search instruction corresponding to the target search object; and in response to the search instruction, obtaining and displaying the search results corresponding to the target search object.

[0281] In one embodiment, the computer program further causes the processor to perform the following steps: scanning a character QR code representing the identity of a virtual interactive character; extracting a download link address of corresponding virtual interactive character information from the character QR code; downloading the virtual interactive character information according to the download link address, and binding the corresponding virtual interactive character to the local according to the downloaded virtual interactive character information.

[0282] In one embodiment, the intelligent voice device is a holographic intelligent voice imaging device; the virtual interactive role information is three-dimensional holographic virtual interactive role information. The computer program further causes the processor to perform the following steps: three-dimensionally project the virtual interactive role information onto a holographic film in the holographic intelligent voice imaging device through a liquid crystal projection device in the holographic intelligent voice imaging device; and present a three-dimensional holographic virtual interactive role on the holographic film of the holographic intelligent voice imaging device.

[0283] It should be understood that, although each step in each embodiment of the present application is not necessarily performed in sequence according to the order indicated by the step number. Unless there is clear explanation in this article, the execution of these steps does not have strict order restriction, and these steps can be performed in other order. And, in each embodiment, at least a part of steps can include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these sub-steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a part of other steps or sub-steps of other steps or stages.

[0284] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0285] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0286] The above embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for those of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A method for controlling an intelligent voice device, the method comprising: When the holographic intelligent voice imaging device is in a semi-dormant state, the bound virtual interactive character is holographically projected by the holographic intelligent voice imaging device to present a three-dimensional holographic virtual interactive character on the holographic intelligent voice imaging device; Acquire images captured within the video acquisition range; When a user object is identified in the image, a near-field interaction mode is entered, and the position of the user object in the image is identified according to the interaction processing logic corresponding to the near-field interaction mode, and the position of the user object is mapped to the user position in the real scene according to the mapping relationship between the position of each image content in the image and the position of the entity of the image content in the real scene, and the orientation of the three-dimensional holographic virtual interactive character is controlled according to the mapped user position, so that the three-dimensional holographic virtual interactive character always maintains the orientation of the user represented by the user object in the real scene; Parsing the facial image in the image, determining the ratio between the areas of the two sides of the facial image, and determining the orientation of the face relative to the holographic intelligent voice imaging device according to the ratio; Determine the corresponding sight direction according to the orientation of the face relative to the holographic intelligent voice imaging device; When the sight line direction corresponding to the user object in the image points to the three-dimensional holographic virtual interactive character, controlling the holographic intelligent voice imaging device to enter a wake-up state; When no user object is recognized in the image and voice data is acquired, the far-field interaction mode is entered. When the acquired voice data meets the wake-up conditions in the far-field interaction mode, the holographic intelligent voice imaging device is controlled to enter the wake-up state.

2. The method according to claim 1, characterized in that The method further comprises: Get voice data; When the sight line direction corresponding to the user object in the image points to the three-dimensional holographic virtual interactive character, controlling the holographic intelligent voice imaging device to enter the awakening state includes: When the sight line direction corresponding to the user object in the image points to the three-dimensional holographic virtual interactive character and there is a preset wake-up word in the voice data, the holographic intelligent voice imaging device is controlled to enter a wake-up state.

3. The method according to claim 1, characterized in that The local and at least one non-local intelligent voice device bound to the virtual interactive character are connected to the same network; When the acquired voice data meets the wake-up condition in the far-field interaction mode, controlling the holographic intelligent voice imaging device to enter the wake-up state includes: When the wake-up conditions in the far-field interaction mode are met, the acquired voice data is sent to the server; receiving a target device designation instruction returned by the server, wherein the target device designation instruction is received when the volume of the voice data is the largest compared with similar voice data, wherein the similar voice data is voice data similar to the voice data and sent by a non-local intelligent voice device; In response to the target device specified instruction, the holographic intelligent voice imaging device is controlled to enter a wake-up state.

4. The method according to claim 1, characterized in that: Also includes: Acquiring voice data including command sentences; After entering the awakening state, performing semantic analysis on the command statement; According to the result of the semantic analysis, an operation instruction corresponding to the command statement is generated and executed.

5. The method according to claim 4, characterized in that The step of generating and executing an operation instruction corresponding to the command statement according to the result of the semantic analysis includes: When performing semantic analysis to obtain that the command statement includes a visualization object display keyword, obtaining the visualization object keyword targeted by the visualization object display keyword in the command statement; According to the visualization object keyword, query the display information corresponding to the corresponding visualization object; Output the queried display information corresponding to the visualization object.

6. The method according to claim 5, characterized in that The visualization object is a video; the visualization object display keyword is a video playback keyword; The visualization object keyword is a video keyword; The querying of display information corresponding to the corresponding visualization object according to the visualization object keyword includes: Matching the video keywords with video names in a preset video library; The video information corresponding to the matched video name is obtained from the video library.

7. The method according to claim 6, characterized in that The output query corresponding to the visualization object includes: Turn on the smart TV bound to the holographic smart voice imaging device; The smart TV is controlled to display video information corresponding to the matched video name.

8. The method according to claim 7, characterized in that The controlling the smart TV to display video information corresponding to the matched video name includes: When the matching probability between the video keyword and the matching video name is greater than or equal to a first matching threshold, controlling the smart TV to play the video content corresponding to the matching video name; When the matching probability between the video keyword and the matching video name is greater than or equal to the second matching threshold and less than the first matching threshold, the smart TV is controlled to display the video description information corresponding to the matched video name.

9. The method according to any one of claims 1 to 2, characterized in that The holographic intelligent voice imaging device in a semi-dormant state performs holographic projection on the bound virtual interactive character to present a three-dimensional holographic virtual interactive character on the holographic intelligent voice imaging device, including: When the holographic intelligent voice imaging device is in a semi-dormant state, the virtual interactive role information is three-dimensionally projected onto a holographic film in the holographic intelligent voice imaging device through a liquid crystal projection device in the holographic intelligent voice imaging device; A three-dimensional holographic virtual interactive character is presented on the holographic film of the holographic intelligent voice imaging device.

10. An intelligent voice device control device, characterized in that: The device comprises: A role presentation module, used to perform holographic projection of a bound virtual interactive role through the holographic intelligent voice imaging device when the holographic intelligent voice imaging device is in a semi-dormant state, so as to present a three-dimensional holographic virtual interactive role on the holographic intelligent voice imaging device; An acquisition module, used to acquire images captured within a video acquisition range; The control module is used to enter the near-field interaction mode when a user object is recognized in the image, and to identify the position of the user object in the image according to the interaction processing logic corresponding to the near-field interaction mode, and to map the user object position to the user position in the real scene according to the mapping relationship between the position of each image content in the image and the position of the entity of the image content in the real scene, and to control the orientation of the three-dimensional holographic virtual interactive character according to the mapped user position, so that the three-dimensional holographic virtual interactive character always maintains the orientation of the user represented by the user object in the real scene; and to analyze the facial image in the image to determine The ratio between the areas of the two sides of the face in the facial image is used to determine the orientation of the face relative to the holographic intelligent voice imaging device according to the ratio; the corresponding line of sight direction is determined according to the orientation of the face relative to the holographic intelligent voice imaging device; when the line of sight direction corresponding to the user object in the image points to the three-dimensional holographic virtual interactive character, the holographic intelligent voice imaging device is controlled to enter a wake-up state; when the user object is not recognized in the image and voice data is obtained, the far-field interaction mode is entered, and when the obtained voice data meets the wake-up conditions in the far-field interaction mode, the holographic intelligent voice imaging device is controlled to enter a wake-up state.

11. The device according to claim 10, characterized in that The acquisition module is also used to acquire voice data; the control module is also used to control the holographic intelligent voice imaging device to enter a wake-up state when the line of sight corresponding to the user object in the image points to the three-dimensional holographic virtual interactive character and there is a preset wake-up word in the voice data.

12. The device according to claim 10, characterized in that The local and at least one non-local intelligent voice device bound to the virtual interactive role are connected to the same network; the control module is also used to send the acquired voice data to the server when the wake-up condition in the far-field interaction mode is met; Receive a target device designation instruction returned by the server, wherein the target device designation instruction is received when the volume of the voice data is the largest compared with similar voice data, and the similar voice data is voice data similar to the voice data sent by a non-local intelligent voice device; in response to the target device designation instruction, control the holographic intelligent voice imaging device to enter a wake-up state.

13. The device according to claim 10, characterized in that The control module is also used to obtain voice data including command statements; after entering the awakening state, perform semantic analysis on the command statements; and generate and execute operation instructions corresponding to the command statements based on the results of the semantic analysis.

14. The device according to claim 13, characterized in that The control module is further configured to obtain the visual object keyword targeted by the visual object display keyword in the command statement when the command statement includes the visual object display keyword through semantic analysis; According to the visualization object keyword, query the display information corresponding to the corresponding visualization object; Output the queried display information corresponding to the visualization object.

15. The device according to claim 14, characterized in that The visualization object is a video; the visualization object display keyword is a video playback keyword; the visualization object keyword is a video keyword; The control module is also used to match the video keyword with a video name in a preset video library; and obtain video information corresponding to the matched video name from the video library.

16. The device according to claim 15, characterized in that The control module is also used to turn on the smart TV bound to the holographic intelligent voice imaging device; and control the smart TV to display video information corresponding to the matched video name.

17. The device according to claim 16, characterized in that The control module is also used to control the smart TV to play the video content corresponding to the matched video name when the matching probability between the video keyword and the matched video name is greater than or equal to a first matching threshold; and to control the smart TV to display the video description information corresponding to the matched video name when the matching probability between the video keyword and the matched video name is greater than or equal to a second matching threshold and less than the first matching threshold.

18. The device according to claim 10, characterized in that The role presentation module is also used for, when the holographic intelligent voice imaging device is in a semi-dormant state, projecting the virtual interactive role information in three-dimensional holography onto a holographic film in the holographic intelligent voice imaging device through a liquid crystal projection device in the holographic intelligent voice imaging device; A three-dimensional holographic virtual interactive character is presented on the holographic film of the holographic intelligent voice imaging device.

19. A computer device comprising a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor executes the steps of the method according to any one of claims 1 to 9.

20. A storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor executes the steps of the method according to any one of claims 1 to 9.

Images