Voice control method and device, projection equipment and storage medium

Abstract

The embodiment of the invention discloses a voice control method, which comprises the following steps: when new page content is loaded on the basis of the current page content, starting a voice acquisition function to obtain the new page content, and after a voice instruction of a user is acquired, determining the intention of the user according to the voice instruction and the new page content, the newly loaded page content is obtained in time, so that the intention of the user can be accurately determined, and the voice instruction of the user is accurately responded. The embodiment of the invention further discloses a voice control device, projection equipment and a storage medium.

Description

Technical Field

[0001] This application relates to, but is not limited to, the field of voice control technology, and in particular to a voice control method, device, projection equipment, and storage medium. Background Technology

[0002] In the smart projector industry, far-field voice control can be used, that is, the projector can be woken up by a wake word and then controlled by voice.

[0003] However, unlike traditional smart speakers, smart projectors offer a vast amount of resources and a large number of functions for users to choose from. For smart projector users, a huge amount of resources and a large number of functions will appear in front of them. That is, the content presented to users is constantly changing, and sometimes the content will change in a minute, which is unpredictable.

[0004] Therefore, the voice control method used in related technologies is not suitable for smart projectors. As a result, smart projectors suffer from the technical problem of low efficiency in voice control. Summary of the Invention

[0005] In view of the above, embodiments of this application provide at least one voice control method, device, projection equipment, and storage medium.

[0006] The technical solution of this application embodiment is implemented as follows:

[0007] On one hand, embodiments of this application provide a voice control method, including:

[0008] With the second page content loaded on top of the first page content currently displayed on the screen, enable the voice capture function to obtain the second page content on the current display screen.

[0009] Upon receiving a user's voice command, the user's intent is determined based on the voice command and the content of the second page.

[0010] Perform the operation corresponding to the user's intent.

[0011] On the other hand, embodiments of this application provide a voice control device, including:

[0012] The acquisition module is used to capture audio to obtain the second page content on the current display screen when the first page content is displayed on the current display screen and the second page content is loaded.

[0013] The determination module is used to determine the user's intent based on the voice command and the content of the second page when the user's voice command is received;

[0014] The execution module is used to perform the operation corresponding to the user's intent.

[0015] In another aspect, embodiments of this application provide a projection device, including a memory and a processor. The memory stores a computer program that can run on the processor, and the processor executes the program to implement some or all of the steps in the above-described method.

[0016] This application provides a voice control method, device, projection equipment, and storage medium, comprising: when loading second page content on top of the first page content displayed on the current display interface, activating a voice acquisition function to obtain the second page content on the current display interface; when a user's voice command is obtained, determining the user's intent based on the voice command and the second page content, and executing the operation corresponding to the user's intent; that is, in this application embodiment, when loading new page content on top of the current page content, the voice acquisition function is activated to obtain the new page content, and after the user's voice command is acquired, the user's intent is determined based on the voice command and the new page content. Because the newly loaded page content is obtained in a timely manner, the user's intent can be accurately determined, and thus the user's voice command can be responded to accurately. Attached Figure Description

[0017] Figure 1 A flowchart illustrating an optional voice control method provided in an embodiment of this application;

[0018] Figure 2 A flowchart illustrating an example of an optional voice control method provided in this application.

[0019] Figure 3 A flowchart illustrating an optional voice control method provided in this application embodiment, specifically Example 2.

[0020] Figure 4a A schematic diagram of an optional first current display interface provided in an embodiment of this application. Figure 1 ;

[0021] Figure 4b A schematic diagram of an optional first current display interface provided in an embodiment of this application. Figure 2 ;

[0022] Figure 4c A schematic diagram of an optional first current display interface provided in an embodiment of this application. Figure 3 ;

[0023] Figure 5a A schematic diagram of an optional second current display interface provided in an embodiment of this application. Figure 1 ;

[0024] Figure 5b A schematic diagram of an optional second current display interface provided in an embodiment of this application. Figure 2 ;

[0025] Figure 5c A schematic diagram of an optional second current display interface provided in an embodiment of this application. Figure 3 ;

[0026] Figure 6 A schematic diagram of an optional third current display interface provided for an embodiment of this application;

[0027] Figure 7 Schematic diagram four showing an optional second current display interface provided for an embodiment of this application;

[0028] Figure 8 A schematic diagram of an optional voice control device provided in an embodiment of this application;

[0029] Figure 9 This is a schematic diagram of an optional projection device provided in an embodiment of this application. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application are further described in detail below with reference to the accompanying drawings and embodiments. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0031] In the following description, references to "some embodiments" refer to a subset of all possible embodiments. It is understood that "some embodiments" may be the same or different subsets of all possible embodiments and may be combined with each other without conflict. The terms "first / second / third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first / second / third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0032] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. The terminology used herein is for descriptive purposes only and is not intended to limit the scope of this application.

[0033] To address the low efficiency of voice control in projection devices, this application provides a voice control method. Figure 1A flowchart illustrating an optional voice control method provided in an embodiment of this application is shown below. Figure 1 As shown, the voice control method may include:

[0034] S101: When loading the second page content on top of the first page content currently displayed on the display interface, enable the voice capture function to obtain the second page content on the current display interface;

[0035] The voice control method provided in this application embodiment can be applied to projection devices, between projection devices and servers, and to other electronic devices. Here, this application embodiment does not specifically limit it.

[0036] To improve voice control efficiency, in this embodiment, a second page needs to be loaded on top of the first page content currently displayed on the display interface. Taking a projection device as an example, when the projection device is on the main interface of the operating system or running an application, and the main page is displayed on the current display interface, it receives a voice command and responds to the voice command. When it needs to load the page corresponding to a certain control on the main page, the page content of the main page is equivalent to the first page content, and the page content corresponding to the certain control is equivalent to the second page content. At this time, it is determined that the second page content is loaded on top of the first page content currently displayed on the display interface.

[0037] The content of the first page and the content of the second page can be different pages of the same application or different applications. The application is not limited to built-in applications of the operating system or third-party applications. The first page is not limited to the main interface of the operating system or the main interface of the application, but can be any display interface of the device. Here, this application embodiment does not make specific limitations in this regard.

[0038] Typically, for voice control, taking a projection device as an example, the voice capture function is activated only when the projection device confirms that it has detected the wake word. At this point, it needs to acquire the content of the currently displayed interface. After activating the voice capture function, it no longer acquires the page content of the currently displayed interface.

[0039] Therefore, in full-duplex mode, if the voice capture function is enabled but the content of the currently displayed page is no longer being acquired, and new page content is reloaded, the new page content cannot be acquired. Based on this, in step S101, when a second page is loaded on top of the first page content currently displayed, the voice capture function is enabled to acquire the second page content. In other words, when a second page is loaded on top of the first page content currently displayed, the voice capture function needs to be enabled to acquire the second page content. Thus, by controlling the activation of the voice capture function, the newly loaded page content can be acquired in a timely manner.

[0040] It should be noted that, regarding the aforementioned function of activating voice acquisition, the voice acquisition function may be in an on or off state before activation. This application embodiment does not specifically limit this.

[0041] Specifically, if the voice capture function is enabled before enabling it, you need to disable it first and then enable it. If the voice capture function is disabled before enabling it, you can enable it directly.

[0042] In this way, new page content can be obtained in a timely manner when new page content is loaded.

[0043] S102: Upon receiving the user's voice command, determine the user's intent based on the voice command and the content of the second page;

[0044] The page content on the current display interface can be obtained in a timely manner through the above S101. In S102, based on the page content on the current display interface, denoted as the second page content, the user's voice command is obtained. In order to respond to the voice command, the user's intention is determined according to the voice command and the second page content.

[0045] The aforementioned voice commands can be either voice signals or voice text; however, this embodiment does not specifically limit them.

[0046] Here, in determining the user's intent based on voice commands and the content of the second page, a Natural Language Processing (NLP) model can be used to determine the user's intent. The user's intent can be a visible and speakable intent, such as playing a video on the second page, or a general intent, such as checking the weather.

[0047] The aforementioned NLP model can include two models: a first NLP model and a second NLP model. The first NLP model is used to identify visible and speakable intents, while the second NLP model is used to identify general intents. The first NLP model and the second NLP model can be obtained by training different training data.

[0048] It should be noted that, in determining the user's intent as described above, one can choose either the first NLP model or the second NLP model, or one can select the first NLP model first and then the second NLP model. Here, the embodiments of this application do not specifically limit this.

[0049] S103: Execute the operation corresponding to the user's intent.

[0050] After determining the user's intent in S102, the operation corresponding to the user's intent can be executed in S103. For visible and speakable intents, for example, a click operation on the control corresponding to the intent can be executed to realize the user's intent. For general intents, for example, a search operation on the content corresponding to the intent can be executed to realize the user's intent.

[0051] The above operations can be click operations on controls of the second page content, or text input operations and search operations for a certain resource. Here, the embodiments of this application do not specifically limit them.

[0052] It should be noted that S101-S102 can be executed by a projection device or an electronic device. Alternatively, after executing S101, the projection device or electronic device can send the user's voice command and the content of the second page to the server. The server determines the user's intent and returns it to the projection device or electronic device, which then executes S103. Here, this application embodiment does not specifically limit this.

[0053] To determine the user's intent, in one alternative embodiment, determining the user's intent based on voice commands and second page content may include:

[0054] The voice command, the information of the controls in the second page content, and the labels of the controls are input into the first NLP model to obtain the user's intent; correspondingly, S103 may include: if the user's intent is the first information in the display information of the second page content, performing an operation on the control corresponding to the first information.

[0055] Understandably, voice commands, information about controls on the second page content, and the labels of those controls can be input into the first NLP model to obtain the user's intent. Here, the information about controls on the second page content and the labels of those controls can be obtained when acquiring the second page content, or the controls on the second page content can be extracted after acquiring the second page content and then labeled to obtain the information about controls on the second page content and the labels of those controls. Here, this application embodiment does not specifically limit this.

[0056] After obtaining the information and tags of the controls in the second page content, they are input into the first NLP model along with the user's voice commands.

[0057] The first NLP model is trained based on training data, which can include the text corresponding to the collected voice commands, information about controls on the page content, control labels, and visible-to-speak intent.

[0058] So, after obtaining the user's intent using the first NLP model, it's necessary to determine whether the user's intent corresponds to the first piece of information displayed on the second page. For example, is the user's intent the first marker among the tags of controls on the second page? Here, the first marker being the tag of a control on the second page indicates that the user's intent is information displayed on the second page—a visible, indicative intent. Therefore, the operation on the control corresponding to the first piece of information can be performed. If not, it means the user's intent is not among the displayed elements on the second page, and further processing using the second NLP model is required. The first piece of information displayed on the second page refers to information visible to the user on the display interface, including but not limited to control tags, control titles, video names, etc.

[0059] It should be noted that if the user's intent is not exactly the first piece of information displayed in the second page content, for example, it is very similar to the information displayed in the second page content. In this case, the information obtained by the first NLP model may be related to the user's intent, such as a part of the video name. The similarity of this part with the title name of the control in the second page content can be calculated by word vector. Based on the relationship between the obtained similarity and the preset similarity value, it can be determined whether the user's intent is the information displayed in the second page content.

[0060] Thus, by using the first NLP model to determine the first marker, the visible-to-speak intent can be realized, enabling users to control the page content on the current display interface via voice.

[0061] In an optional embodiment, for cases where the user's intent is not indicated by the tags in the second page content, the method may further include:

[0062] If the user's intent is not the first piece of information displayed on the second page, the user's intent is input into the second NLP model to obtain the user's intent again.

[0063] Understandably, if the user's intent is not the first piece of information displayed on the second page, the user's intent obtained at this time is the result of natural language processing, which may be information related to the determined final intent of the user. Therefore, the user's intent obtained through the first NLP model is input into the second NLP model.

[0064] The second NLP model is also trained based on training data, which can be the text corresponding to the voice command and the user's intent.

[0065] Here, the user's intent can be obtained again through the second NLP model mentioned above. The user's intent here is a general intent, such as checking the weather, checking traffic restrictions, watching a movie, playing music, etc.

[0066] Thus, if the first NLP model cannot obtain the tagging of the controls in the second page content, the user's intent can be obtained again through the second NLP model. This makes the voice control provided in this application embodiment applicable to the determination of visible and speakable intents, as well as the determination of general intents, thereby more comprehensively and accurately determining the user's true intent.

[0067] In an optional embodiment, to obtain the content of the second page when loading the second page content, S101 may include:

[0068] Load the second page content based on the first page content currently displayed on the screen, and with the voice capture function enabled, disable the voice capture function and then enable it again to obtain the second page content.

[0069] Understandably, when loading the second page content on top of the first page content currently displayed on the screen, it is necessary to check whether the voice capture function is enabled. If the voice capture function is enabled, in order to obtain the second page content, it is necessary to first disable the voice capture function and then enable it again. This way, even if the voice capture function is enabled, the second page content can be obtained.

[0070] Here, the triggering method for turning off and then on the voice recording function can be determined based on the actual situation.

[0071] In this way, by first turning off the voice capture function and then turning it back on, the voice capture function can be restarted. This action of turning it off and on allows the content of the second page on the currently displayed interface to be obtained, providing accurate data support for determining the user's intent and helping to determine the user's true intent.

[0072] In order to obtain all the content of the second page, in one optional embodiment, if the second page content is loaded on top of the first page content currently displayed on the screen and the voice capture function is enabled, disabling the voice capture function and then re-enabling it to obtain the second page content may include:

[0073] Load the second page content on top of the first page content currently displayed on the screen, and disable the voice capture function if the voice capture function is enabled.

[0074] After receiving the notification message, then enable the voice recording function.

[0075] Understandably, when loading the second page content on top of the first page content currently displayed on the screen, and if the voice capture function is enabled, it is best to first disable the voice capture function. It should be noted that the key to obtaining the full content of the second page is when to enable the voice capture function after disabling it.

[0076] Here, the function to start recording audio is triggered by a received notification message. This notification message indicates that the second page of content on the current display screen has finished loading. In other words, the audio recording function is activated after receiving a message indicating that the second page of content on the current display screen has finished loading.

[0077] For example, after the content of the second page is loaded, the application to which the content of the second page belongs will send a notification message, which will cause the client operating system of the projection device or electronic device to start the voice capture function after receiving the notification message. At this time, the current display interface displays the full content of the second page, so the full content of the second page can be captured.

[0078] Therefore, by sending a notification message to turn the voice capture function off and then on again, it helps to obtain the full content of the second page, thereby improving the accuracy of determining the user's intent.

[0079] In addition, in order to obtain all the content of the second page, in an optional embodiment, when the second page content is loaded on top of the first page content displayed on the current display interface and the voice capture function is enabled, turning off the voice capture function and then turning it on again to obtain the second page content may include:

[0080] Load the second page content based on the first page content currently displayed on the screen. If the voice capture function is enabled, disable the voice capture function, and then enable the voice capture function again after a preset time to obtain the second page content.

[0081] Understandably, if the second page content is loaded on top of the first page content currently displayed on the screen, and the voice capture function is enabled, the key to obtaining the full content of the second page is to first disable the voice capture function and then re-enable it after disabling it.

[0082] Here, a preset duration is set to disable the voice recording function, and then enable the voice recording function again after the preset duration. In other words, by setting a preset duration, the voice recording function is enabled again after a preset duration following the disabling of the voice recording function. This preset duration can be a fixed value or a value determined according to the actual situation. Here, this application embodiment does not specifically limit this.

[0083] Therefore, by setting a preset duration to turn off the voice recording function and then turn it on again, it helps to obtain all the content of the second page and improves the accuracy of determining the user's intent.

[0084] Regarding the aforementioned preset duration setting, in one optional embodiment, the method may further include:

[0085] When executing the user's intended action to load the second page content on top of the first page content displayed on the current screen, a preset duration is determined based on the user's intent.

[0086] Understandably, when executing the user's intention to load the second page content on top of the first page content displayed on the current screen, the preset duration can be determined based on the user's intention.

[0087] The user's intent here can be determined based on the content of the second page that is about to be loaded. In other words, the user's intent here is: the intent of the user determined before loading the content of the second page, that is, the intent of the user determined when the current display interface is the content of the first page. This user's intent is directed to the content of the second page that is about to be loaded. This is different from the user's intent determined in S103, which is the user's intent determined based on the second page content when the current display interface is the content of the second page, combined with the voice command collected again.

[0088] The user's intent mentioned above can also be derived from voice commands based on a machine learning model. User intent can include playback-related and operation-related intents. For example, playback-related intents could include playing video, playing music, or playback control. Operation-related intents could include page turning or settings adjustments. Furthermore, a fixed preset duration can be determined based on the user intent identified by the machine learning model. For instance, a playback-related intent might have a third preset duration, while an operation-related intent might have a fourth. It's understood that playback-related intents require more time to load video or music resources, while operation-related intents require less time to load the page; therefore, the third duration is set longer than the fourth.

[0089] The user's intent used to determine the preset duration can be either a visible-and-say intent or a general intent. Regardless of whether it is a visible-and-say intent or a general intent, it can know relevant information about the content of the second page loaded by the user, such as whether the content of the second page is a local resource, the resource size of the content of the second page, etc.

[0090] After knowing the user's intent, the preset duration can be determined based on the resource type and size of the second page content loaded in the user's intent. Of course, the preset duration can also be determined based on the specific resource content. Here, this application embodiment does not make specific limitations on this.

[0091] Thus, by determining the user's intent when loading the second page content, the preset duration can be set based on the approximate time required to load the second page content. This allows the voice capture function to be activated after the preset duration, and the second page content to be retrieved after it has been fully loaded, thus obtaining the full content of the second page.

[0092] To determine a reasonable preset duration, in one optional embodiment, the preset duration is determined based on the user's intent, including:

[0093] If the user's intent indicates that the content of the second page is not a local resource, and the resource size of the content of the second page is greater than the preset value, the preset duration is determined to be the first duration.

[0094] If the user's intent indicates that the content of the second page is a local resource, then the preset time period is determined as the second duration.

[0095] If the user's intent indicates that the content of the second page is not a local resource, and the size of the dynamic resource on the second page is less than the preset value, the preset duration is determined to be the second duration.

[0096] Understandably, setting the preset duration too long will result in excessively long waiting times for users, while setting it too short will result in the second page content being retrieved before it has been fully loaded, thus resulting in incomplete second page content. In order to set a reasonable preset duration, in this embodiment of the application, the user's intent is used to determine whether the second page content is a local resource and to determine the resource size of the second page content, and then the preset duration is set based on this.

[0097] Specifically, when the user's intent indicates that the content of the second page is a non-local resource, and the size of the dynamic resource on the second page is greater than a preset value, for example, when the content of the second page is playing a movie, the preset duration is determined to be the first duration.

[0098] If the user's intent indicates that the content of the second page is a local resource, for example, the content of the second page is a page in the settings function, the preset duration is determined to be the second duration.

[0099] If the user's intent indicates that the content of the second page is not a local resource, and the resource size of the content of the second page is less than a preset value, for example, when switching tabs (tab pages), the preset duration is determined to be the second duration.

[0100] The first duration is longer than the second duration. In other words, the preset duration for the second page content that takes longer to load is set to be longer, and the preset duration for the second page content that takes shorter to load is set to be shorter.

[0101] Thus, by setting the preset duration based on the resource type and size of the second page indicated by the user's intent, a reasonable preset duration can be determined for the content of the second page, so that the full content of the second page can be obtained without affecting the user's experience.

[0102] Regarding the content of the second page, in one optional embodiment, the above method may further include:

[0103] Identify the controls for the content of the second page; mark the controls to obtain their tags.

[0104] Understandably, after obtaining the content of the second page, the content of the second page can include the information and tags of the controls in the content of the second page. Then, even without the information and tags of the controls in the content of the second page, the controls can be extracted from the content of the second page, and the extracted controls can be marked to obtain the tags of the controls. At the same time as extracting the controls from the content of the second page, the information of the controls can also be obtained.

[0105] In this way, information about the controls and their tags on the second page can be obtained, providing effective data for understanding the user's intent and improving the accuracy of the obtained user intent.

[0106] The information of a control includes one or more of the following: the control's title, the control's content type, the control's link address, and the control's coordinates.

[0107] Here, the title of the control can be the title of a control on the second page, such as the name of a video. The content type of the control can be video, audio, or text, etc. The link address of the control can be its Uniform Resource Locator (URL). The coordinates of the control can be its coordinates on the currently displayed screen.

[0108] Thus, the information from the aforementioned controls can not only serve as input data for the first NLP model when the user's intent is obtained, but can also be used together with the output data to further redefine the user's intent. Furthermore, it can determine the preset duration when loading the content of the second page, thereby further improving the control efficiency of voice control.

[0109] The following examples illustrate the voice control method described in one or more of the above embodiments.

[0110] Figure 2 A flowchart illustrating an example of an optional voice control method provided in this application is shown below. Figure 2 As shown, the voice control method may include:

[0111] S201: The user sends a voice command to the smart projector;

[0112] Among them, the smart projector is equivalent to the aforementioned projection device.

[0113] S202: The smart projector uploads voice commands to the voice recognition cloud server;

[0114] S203: The voice recognition cloud server returns the voice command text to the smart projector;

[0115] S204: The smart projector sends the current page content and voice command text to the NLP cloud server;

[0116] S205: The NLP cloud server returns the visible and speakable intent or general intent to the smart projector;

[0117] S206: The smart projector executes the operation corresponding to the intent to provide feedback to the user.

[0118] Regarding S204-S206 above, Figure 3 The intention of the process interaction in Example 2 of an optional voice control method provided in the embodiments of this application is as follows: Figure 3 As shown, the voice control method may include:

[0119] S301: The smart projector sends the voice command text and the current page content to the NLP cloud server;

[0120] S302: The NLP cloud server inputs the voice command text and the current page content into the NLP visible and speakable model for named entity recognition and slot filling. If at least one slot (or a required slot) is filled, execute S303; otherwise, execute S306.

[0121] S303: Based on the slot filling result, identify the user's intent, and completely match the key information in the voice command text for slot filling with the information on the current page. If the match is successful, execute S305; otherwise, execute S304.

[0122] For example, key information includes: name, location, time, video name, numbers, etc.

[0123] S304: Perform fuzzy matching. If the match is successful, proceed to S305; if the fuzzy matching fails, proceed to S306.

[0124] S305: Return the visible and speakable intent and the corresponding page coordinates to the smart projector;

[0125] S306: Input the intermediate information from the visible-and-sayable model processing into the general NLP model processing to obtain the general intent; execute S307;

[0126] S307: Return the general intent to the smart projector.

[0127] Specifically, in this example, firstly, the smart projector obtains the user interface content and content tags based on the user's request, which may include:

[0128] When a user invokes far-field voice commands, the voice client in the smart projector needs to obtain the current page content and mark the current page content with a number, so that the user can interact in two ways.

[0129] Obtaining the content of the current page involves interaction between the voice client and the application on the current page to retrieve information about the current page content and its corresponding content. At the same time, each piece of content is marked with a serial number. This function is mainly implemented by the voice client, which obtains the information of the controls of the current page content through Android's native auxiliary services.

[0130] Figure 4a A schematic diagram of an optional first current display interface provided in an embodiment of this application. Figure 1 ,like Figure 4a As shown, the user can say "the second one" based on the marker number.

[0131] After fulfilling the user's request and redirecting to a new interface, the user interface content can be retrieved, which may include:

[0132] The "see-and-say" interaction typically uses a full-duplex interaction method, where the user only needs to say the wake word once, and subsequent interactions do not require saying the wake word again. Taking the wake word AAA as an example, the interaction without full-duplex would be: AAA, I want to watch a movie; AAA, the second one; AAA, fast forward to 20 minutes. The interaction with full-duplex would be: AAA, I want to watch a movie; the second one; fast forward to 20 minutes.

[0133] Because "visible and speakable" works best with full-duplex mode, it greatly enhances the user experience. However, a problem exists: in full-duplex mode, the voice dot (equivalent to the aforementioned voice acquisition function) does not disappear and is continuously acquiring voice information. This causes the voice client to only obtain the page content from the first time the voice is invoked, such as... Figure 4a However, it cannot retrieve new page content.

[0134] The conventional full-duplex voice interaction implementation method is as follows: after receiving the user's voice wake-up word, the second wake-up word is then spoken. The judgment of continuous dialogue is that there is no need to say another wake-up word, and the voice ball is used to continue to receive voice and interact with the user.

[0135] When the user enters a new page, the voice ball can be set to "disappear" for a certain period of time based on the user's intent. When new page content appears, the voice ball will reappear, thus achieving full-duplex communication while also allowing the user to access new page content.

[0136] As described above, a key aspect of the new interaction method is setting the "disappearance" time of the voice ball based on the user's intent. This is because different content takes different amounts of time to display. For example, loading a video can take a long time. If the voice ball reappears too quickly, the user won't be able to access the new page content. Similarly, if the display time is too slow when opening settings or tab pages, it can make the user feel lag or think that full-duplex mode is not working. Therefore, it is necessary to use NLP to recognize the intent and the time it takes for the new page to load to determine the "disappearance" time of the voice ball. For example, only after the new page has finished loading should the voice ball be activated to ensure that what is visible can be spoken and the user can obtain the complete information of the new page.

[0137] Based on the above Figure 4a , Figure 4b A schematic diagram of an optional first current display interface provided in an embodiment of this application. Figure 2 ,like Figure 4b As shown, in Figure 4a Based on the displayed page, the user issues the second command. Simultaneously, the voice client clicks the control corresponding to marker 2 and closes the voice ball. Figure 4b The page displays "New page content XX is loading", but the new page content information and tags cannot be obtained.

[0138] Based on the above Figure 4b , Figure 4c A schematic diagram of an optional first current display interface provided in an embodiment of this application. Figure 3 If the voice ball is turned off and then turned on again, the voice client will obtain... Figure 4b The page content is displayed, and the page content is marked up to obtain the marked page, as shown below. Figure 4c As shown, it can be seen that Figure 4b yes Figure 4a and Figure 4c The intermediate state.

[0139] This section defines certain intent ranges. For example, if the intent is related to playback, the disappearance time will be set to be longer. If the intent is related to settings, menus, or page turning, and a new interface can be opened quickly, the disappearance time will be set to be shorter.

[0140] Figure 5a A schematic diagram of an optional second current display interface provided in an embodiment of this application. Figure 1 ,like Figure 5a As shown, users can say BCDEF based on the page content.

[0141] Based on the above Figure 5a , Figure 5b A schematic diagram of an optional second current display interface provided in an embodiment of this application. Figure 2 ,like Figure 5b As shown, in Figure 5a Based on the displayed page, the user utters: BCDEF. The voice client executes the click operation of the control corresponding to BCDEF while simultaneously closing the voice ball. Figure 5b The page with new BCDEF content is loaded, but the new page content information and tags cannot be retrieved.

[0142] Based on the above Figure 5b , Figure 5c A schematic diagram of an optional second current display interface provided in an embodiment of this application. Figure 3 If the voice ball is turned off and then turned on again, the voice client will obtain... Figure 5b The page content is displayed, and the page content is marked up to obtain the marked page, as shown below. Figure 5c As shown, it can be seen that Figure 5b yes Figure 5a and Figure 5c The intermediate state.

[0143] NLP model processing based on user-defined data can include:

[0144] In the technical description above, based on user page data and user verbal instructions, the visible-to-speak intent is returned. Regarding this step:

[0145] Firstly, regarding the ambiguity of intent, intent can be divided into visible-and-sayable intent and general intent, for example... Figure 4a As shown, the user's dialogue is: "Second." At this point, should we return to the second intent, which is usually the second intent, to play the relevant location intent, or return to the "visible and speakable" page markup content? If multiple intents are returned, the playback intent is executed first, but it cannot be executed. Then the "visible and speakable" intent is executed, which will lead to delays. Similarly, if it is on the playback interface, the location intent should be executed, not the "visible and speakable" intent. Therefore, NLP needs to combine the real-time information of the user to determine the NLP intent.

[0146] For example, regarding page content and playback status, the current request carries visible-to-talk information and has already completed content tagging. If a relevant positioning intent is identified, it will be automatically converted into a visible-to-talk intent.

[0147] In this example, the visible-to-say information passed in based on the current page content is given the first priority. If the current NLP request includes the user's visible-to-say information, the visible-to-say related intent will be returned first. Otherwise, a general domain intent will be returned. Here, a location intent is used as an example to confirm this.

[0148] exist Figure 4aThe interface states that, for the second instance, since the visible-as-speakable information has been passed in and the control's sequence number has been marked, when the user says "second instance," it will be recognized as an intent within the visible-as-speakable domain.

[0149] Figure 6 A schematic diagram of an optional third current display interface provided for an embodiment of this application, such as... Figure 6 The image shown is the local page content of the voice client. Figure 6 On the interface, the user sends out: the second one. Since there is no information that can be seen and spoken, and no content is marked, the second one at this time is the positioning intent of the general control domain.

[0150] Secondly, regarding the incompleteness of the user's statement... Figure 7 A schematic diagram four shows an optional second current display interface provided for an embodiment of this application, as shown below. Figure 7 As shown, the user sends "BCDE", but the actual page is "BCDEF". This fails to accurately match the page content. Therefore, two improvements are needed in the model to meet the current requirements: 1. Recognize "BCDE" as a video name; 2. Calculate the similarity between "BCDE" and all names in the page information using the user's word vectors. When a certain threshold is reached, return the visible / sayable intent, i.e., return the tag "BCDEF". Similarly, user phrases such as "BCEF", "CDEF", and "DEF" can also be matched.

[0151] The goal here is to reduce false positives. After the model recognizes the video, the threshold for fuzzy matching can be lowered to expand the recall, while avoiding other information, such as a noisy message on the page that might lead to a match for the visible intent. A general named entity recognition model is used here, and the video information on the page is passed into the model as a feature to assist in the model's recognition.

[0152] As can be seen, this example demonstrates that the overall "See-It-Speak" solution addresses the pain point of requiring a remote control for users interacting with smart projectors via far-field voice, allowing for seamless voice interaction without a remote. By integrating with NLP and employing a brief disappearance of the voice ball, it resolves the issue of failing to acquire new page information in full-duplex mode, preventing subsequent voice interactions from using "See-It-Speak" in full-duplex scenarios. Furthermore, by combining with NLP, it addresses the problem of excessively long or short disappearance times of the voice ball in full-duplex mode, which can lead to page retrieval failures or poor user experience, providing a more intelligent handling of the voice ball disappearance time. When performing NLP processing with user data, it resolves ambiguities between the "See-It-Speak" intent and general intents, correctly returning the relevant "See-It-Speak" intent. Finally, when performing NLP processing with user data, it calculates similarity between the model's identified results and user data using word vectors, determining whether the NLP result should be a click-through page response within the "See-It-Speak" domain or a video search within the general NLP domain.

[0153] This application provides a voice control method, including: when loading second page content on top of the first page content currently displayed on the display interface, activating a voice acquisition function to obtain the second page content on the current display interface; when a user's voice command is obtained, determining the user's intent based on the voice command and the second page content, and executing the operation corresponding to the user's intent; that is, in this application embodiment, when loading second page content on top of the first page content, the voice acquisition function is activated to obtain the second page content, and after the user's voice command is acquired, the user's intent is determined based on the voice command and the second page content. Because the newly loaded page content is obtained in a timely manner, the user's intent can be accurately determined, and thus the user's voice command can be responded to accurately.

[0154] Based on the same inventive concept as the foregoing embodiments, this application provides a voice control device. Figure 8 A schematic diagram of an optional voice control device provided for an embodiment of this application is shown below. Figure 8 As shown, the voice control device includes: an acquisition module 81, a determination module 82, and an execution module 83, wherein:

[0155] The acquisition module 81 is used to enable the voice acquisition function to obtain the second page content on the current display interface when the second page content is loaded on the basis of the first page content displayed on the current display interface.

[0156] The determination module 82 is used to determine the user's intent based on the voice command and the content of the second page when the user's voice command is obtained;

[0157] Execution module 83 is used to execute the operation corresponding to the user's intent.

[0158] In one optional embodiment, the determining module 82 determines the user's intent based on the voice command and the second page content, including: inputting the voice command, the information of the controls in the second page content, and the labels of the controls into the first NLP model to obtain the user's intent; correspondingly, the execution module 83 is specifically used to: when the user's intent is the first information in the display information of the second page content, perform an operation on the control corresponding to the first information.

[0159] In an optional embodiment, the device is further configured to: input the user's intent into a second NLP model to obtain the user's intent again when the user's intent is not the first information in the display information of the second page content.

[0160] In one optional embodiment, when the acquisition module 81 loads the second page content on top of the first page content displayed on the current display interface, the function of activating the voice acquisition function to obtain the second page content on the current display interface includes: when the second page content is loaded on top of the first page content displayed on the current display interface and the voice acquisition function is activated, the voice acquisition function is deactivated and then activated again to obtain the second page content.

[0161] In one optional embodiment, when the acquisition module 81 loads the second page content based on the first page content displayed on the current display interface and the voice acquisition function is enabled, the process of disabling and then re-enabling the voice acquisition function to acquire the second page content includes: disabling the voice acquisition function when loading the second page content based on the first page content displayed on the current display interface and the voice acquisition function is enabled; and re-enabling the voice acquisition function after receiving a notification message; wherein the notification message is used to indicate that the second page content has been loaded on the current display interface.

[0162] In one optional embodiment, when the acquisition module 81 loads the second page content on top of the first page content displayed on the current display interface and the voice acquisition function is enabled, it disables the voice acquisition function and then enables the voice acquisition function to obtain the second page content. This includes: when the second page content is loaded on top of the first page content displayed on the current display interface and the voice acquisition function is enabled, disabling the voice acquisition function and then enabling the voice acquisition function again after a preset time to obtain the second page content.

[0163] In an optional embodiment, the device is further configured to: determine a preset duration based on the user's intent when loading the second page content on the basis of performing an operation that executes the user's intent to display the first page content on the current display interface.

[0164] In one optional embodiment, the device determines the preset duration based on the user's intent, including: determining a first duration when the user's intent indicates that the content of the second page is a non-local resource and the resource size of the second page content is greater than a preset value; determining a second duration when the user's intent indicates that the content of the second page is a local resource; and determining a second duration when the user's intent indicates that the content of the second page is a non-local resource and the resource size of the second page content is less than a preset value; wherein the first duration is greater than the second duration.

[0165] In an alternative embodiment, the device is further configured to: determine controls for the content of the second page; and mark the controls to obtain a mark of the controls.

[0166] In one alternative embodiment, the information of the control includes one or more of the following: the title name of the control, the content type of the control, the link address of the control, and the coordinates of the control.

[0167] In practical applications, the acquisition module 81, determination module 82 and execution module 83 mentioned above can be implemented by a processor located on the voice control device 800, which can be a central processing unit (CPU), microprocessor unit (MPU), digital signal processor (DSP) or field programmable gate array (FPGA), etc.

[0168] Figure 9 The following is a schematic diagram of an optional projection device provided in an embodiment of this application, such as... Figure 9 As shown, this application embodiment provides a projection device 900, including:

[0169] The processor 91 and the storage medium 92 storing the processor-executable instructions; the storage medium 92 performs operations dependent on the processor 91 via a communication bus 93, and when the instructions are executed by the processor, the abnormal detection method executed on the processor side in one or more of the above embodiments is performed.

[0170] It should be noted that in practical applications, the various components in the electronic device are coupled together via the communication bus 93. It can be understood that the communication bus 93 is used to achieve communication between these components. In addition to the data bus, the communication bus 93 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 9 The general labeled all buses as communication bus 93.

[0171] This application provides a computer storage medium storing executable instructions. When the executable instructions are executed by one or more processors, the processors execute the voice control method described in one or more of the above embodiments.

[0172] The computer-readable storage medium can be a magnetic random access memory (FRAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM), etc.

[0173] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0174] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0175] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0176] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0177] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application.

Claims

1. A voice control method, characterized in that, include: With the second page content loaded on top of the first page content currently displayed on the screen, enable the voice capture function to obtain the second page content on the current display screen. Upon receiving a user's voice command, the user's intent is determined based on the voice command and the content of the second page. Perform the operation corresponding to the user's intent.

2. The method according to claim 1, wherein determining the user's intent based on the voice command and the content of the second page includes: The voice command, the information of the controls in the second page content, and the tags of the controls are input into the first NLP model to obtain the user's intent; Correspondingly, the operation corresponding to the user intent includes: If the user's intent is the first information in the display information of the second page content, then the operation on the control corresponding to the first information is performed.

3. The method according to claim 2, characterized in that, The method further includes: If the user's intent is not the first information displayed in the second page content, the user's intent is input into the second NLP model to obtain the user's intent again.

4. The method according to any one of claims 1 to 3, characterized in that, When loading a second page of content on top of the first page of content currently displayed on the screen, the function of activating voice capture to obtain the second page of content on the current screen includes: If the second page content is loaded on top of the first page content displayed on the current display interface, and the voice capture function is enabled, then the voice capture function is disabled and then enabled again to obtain the second page content.

5. The method according to claim 4, characterized in that, When loading second page content based on the first page content displayed on the current display interface and the voice capture function is enabled, disabling the voice capture function and then re-enabling it to obtain the second page content includes: If the content of the second page is loaded on the basis of the content of the first page displayed on the current display interface, and the function of collecting audio is enabled, then the function of collecting audio is disabled. After receiving the notification message, the function of collecting voice is then enabled; wherein, the notification message is used to indicate that the content of the second page has been loaded on the currently displayed interface.

6. The method according to claim 4, characterized in that, When loading the second page content based on the first page content displayed on the current display interface and the voice capture function is enabled, disabling the voice capture function and then enabling the voice capture function to obtain the second page content includes: If the second page content is loaded based on the first page content displayed on the current display interface, and the voice capture function is enabled, then the voice capture function is disabled, and the voice capture function is enabled again after a preset time to obtain the second page content.

7. The method according to claim 6, characterized in that, The method further includes: When the user's intention is to perform an operation that causes the current display interface to display the content of the first page and then load the content of the second page, the preset duration is determined based on the user's intention.

8. The method according to claim 7, characterized in that, Determining the preset duration based on the user's intent includes: If the user's intent indicates that the content of the second page is a non-local resource, and the resource size of the content of the second page is greater than a preset value, then the preset duration is determined to be the first duration. If the user's intent indicates that the content of the second page is a local resource, the preset time period is determined to be the second duration. If the user's intent indicates that the content of the second page is a non-local resource, and the size of the dynamic resource in the second page is less than a preset value, then the preset duration is determined to be the second duration. Wherein, the first duration is longer than the second duration.

9. The method according to any one of claims 1 to 3, characterized in that, The method further includes: A control that determines the content of the second page; The control is marked to obtain the control's label.

10. The method according to claim 9, characterized in that, The information of the control includes one or more of the following: The title name of the control, the content type of the control, the link address of the control, and the coordinates of the control.

11. A voice control device, characterized in that, include: The acquisition module is used to capture audio to obtain the second page content on the current display screen when the first page content is displayed on the current display screen and the second page content is loaded. The determination module is used to determine the user's intent based on the voice command and the content of the second page when the user's voice command is received; The execution module is used to perform the operation corresponding to the user's intent.

12. A projection device, comprising a memory and a processor, wherein the memory stores a computer program executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method according to any one of claims 1 to 10.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the method according to any one of claims 1 to 10.

Images