A voice interaction method, device and electronic equipment

Abstract

The application provides a voice interaction method and device and electronic equipment, acquires voice instruction text of a user; determines text content of an interactive element in a current interaction interface; processes the text content of the interactive element in combination with attribute information of an interface region to which the interactive element belongs in the interaction interface, generates a hot word library of the current interaction interface; the hot word library includes at least one hot word of each interactive element in the current interaction interface; determines a target hot word in the hot word library that matches the voice instruction text, and triggers an interactive element to which the target hot word belongs in the current interaction interface. The application generates a hot word library of the current interaction interface in combination with attribute information of an interface region to which an interactive element belongs in the current interaction interface, alleviates semantic differences between user colloquial input and structured page element text, realizes quick and accurate matching of a user natural colloquial voice instruction and a corresponding interactive element on a current interface of the electronic equipment, and improves user voice interaction experience.

Description

Technical Field

[0001] This application relates to the field of intelligent voice interaction technology, specifically to a voice interaction method, device, and electronic device. Background Technology

[0002] In the field of intelligent voice interaction, especially in application scenarios such as smart cockpits and smart homes, the "see and speak" function has become a core interaction method to enhance user experience. This function allows users to directly control the interactive elements displayed on the current electronic device interface through natural language voice commands, thereby realizing voice interaction with electronic devices.

[0003] With users' growing demand for natural and fluent voice interaction experiences, providing a voice interaction method that can quickly and accurately match users' natural spoken voice commands with the corresponding interactive elements on the current interface of electronic devices to trigger the correct operation plays a crucial role in improving the voice interaction experience. Summary of the Invention

[0004] Based on the defects and shortcomings of the existing technology, this application proposes a voice interaction method, device and electronic device to achieve rapid and accurate matching between the user's natural spoken voice commands and the corresponding interactive elements on the current interface of the electronic device, thereby improving the user's voice interaction experience.

[0005] The first aspect of this application provides a voice interaction method, including: Obtain the text of the user's voice commands; Determine the text content of interactive elements in the current interactive interface; The text content of the interactive element is processed by combining the attribute information of the interface area to which the interactive element belongs in the interactive interface to generate a hot word library for the current interactive interface; the hot word library includes at least one hot word for each interactive element in the current interactive interface. Identify the target hot word in the hot word library that matches the voice command text, and trigger the interactive element to which the target hot word belongs in the current interactive interface.

[0006] In one possible implementation, obtaining the user's voice command text includes: Receive the user's original voice command text; Remove non-semantic text from the original voice command text to obtain the voice command text.

[0007] In one possible implementation, the step of processing the text content of the interactive element by combining the attribute information of the interface area to which the interactive element belongs in the interactive interface to generate a hot word library for the current interactive interface includes: The text content of the interactive element is segmented to obtain at least one segment of the interactive element; Determine the attribute information of the interface area to which the interactive element belongs in the interactive interface, and the attribute information of the interface area is determined based on the difference information between different interface areas in the current interactive interface. The word segmentation and attribute information are associated to generate hot words corresponding to the word segmentation; the hot words corresponding to the word segmentation of each interactive element in the current interactive interface constitute the hot word library of the current interactive interface.

[0008] In one possible implementation, the attribute information of the interface area includes the area type and / or area position of the interface area, wherein the area type is based on the functional definition of the interface area, and the area position is based on the position information of the interface area in the current interactive interface.

[0009] In one possible implementation, determining the attribute information of the interface area to which the interactive element belongs in the interactive interface includes: Identify at least one logical partition in the current interactive interface, where each logical partition corresponds to a UI area; Obtain the title text of each of the at least one logical partitions; The attribute information of each logical partition is determined based on the difference information between the title texts of each logical partition in the at least one logical partition.

[0010] In one possible implementation, determining the attribute information of the interface area to which the interactive element belongs in the interactive interface includes: Identify at least one logical partition in the current interactive interface, where each logical partition corresponds to a UI area; Obtain the position information of each of the at least one logical partitions in the current interactive interface; The attribute information of each logical partition is determined based on the relative position information between the logical partitions in the at least one logical partition.

[0011] In one possible implementation, determining the target hot word in the hot word library that matches the voice command text includes: Determine the target scene information of the current interactive interface, wherein the target scene information represents the application scene of the current interactive interface; Based on the pre-trained text feature extraction model, the first text feature of the voice command text and the second text feature of each hot word in the hot word library are extracted respectively. Based on the first text features of the voice command text and the second text features of the hot words, the similarity of each hot word in the hot word library is calculated respectively; the similarity of the hot words represents the semantic similarity between the hot words and the voice command text; Determine whether the highest similarity of each of the hot words in the hot word library exceeds the target similarity threshold corresponding to the target scene information; If the highest similarity of each of the hot words in the hot word library exceeds the target similarity threshold corresponding to the target scene information, the hot word to which the highest similarity belongs is determined as the target hot word.

[0012] In one possible implementation, the text feature extraction model is obtained in the following way: Distilling the first model yields the second model, where the first model is a pre-set large model and the second model is used for text feature extraction. The sample pair is input into the second model to obtain the first text feature of the first sample and the second text feature of the second sample in the sample pair; Calculate the first semantic similarity between the first sample and the second sample based on the first text features of the first sample and the second text features of the second sample; Based on the first model, a second semantic similarity is determined between the first sample and the second sample in the sample pair; Based on the difference between the first semantic similarity and the second semantic similarity, the second model is optimized and trained to obtain a text feature extraction model.

[0013] A second aspect of this application provides a voice interaction device, comprising: The voice command text acquisition unit is used to acquire the user's voice command text; The text content determination unit is used to determine the text content of interactive elements in the current interactive interface. A hot word library generation unit is used to process the text content of the interactive element by combining the attribute information of the interface area to which the interactive element belongs in the interactive interface, and generate a hot word library for the current interactive interface; the hot word library includes at least one hot word for each interactive element in the current interactive interface. The voice interaction unit is used to determine the target hot word in the hot word library that matches the voice command text, and to trigger the interactive element to which the target hot word belongs in the current interactive interface.

[0014] A third aspect of this application provides an electronic device, including a memory and a processor; The memory is connected to the processor and is used to store programs; The processor is used to implement the voice interaction method as described in the first aspect of this application or any possible implementation thereof by running a program in the memory.

[0015] The fourth aspect of this application provides a computer program product, including a computer program that, when executed by a processor, implements the voice interaction method as described in the first aspect of this application or any possible implementation thereof.

[0016] The fifth aspect of this application provides a chip including a processor and a data interface, wherein the processor reads and runs a program stored in a memory through the data interface to perform a voice interaction method as described in the first aspect of this application or any possible implementation thereof.

[0017] The sixth aspect of this application provides a storage medium storing a computer program, which, when executed by a processor, implements the voice interaction method as described in the first aspect of this application or any possible implementation thereof.

[0018] This application provides a voice interaction method, device, and electronic device. The method involves acquiring the user's voice command text; determining the text content of interactive elements in the current interactive interface; processing the text content of the interactive elements based on the attribute information of the interface area to which the interactive elements belong in the interactive interface to generate a hot word library for the current interactive interface; the hot word library includes at least one hot word for each interactive element in the current interactive interface; determining a target hot word in the hot word library that matches the voice command text, and triggering the interactive element to which the target hot word belongs in the current interactive interface. This application generates a hot word library for the current interactive interface by combining the attribute information of the interface area to which the interactive elements belong in the current interactive interface. This alleviates the semantic difference between the user's spoken input and the structured page element text, reduces matching failures caused by semantic differences, and achieves fast and accurate matching between the user's natural spoken voice commands and the corresponding interactive elements on the current interface of the electronic device, thus improving the user's voice interaction experience. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0020] Figure 1 A flowchart of a voice interaction method provided in an embodiment of this application.

[0021] Figure 2 The flowchart illustrates a method for processing the text content of interactive elements by combining the attribute information of the interface area to which the interactive element belongs in the interactive interface, and generating a hot word library for the current interactive interface, as provided in this application embodiment.

[0022] Figure 3 A flowchart illustrating a method for determining target hot words in a hot word library that match voice command text, provided in an embodiment of this application.

[0023] Figure 4 This is a schematic diagram of the structure of a voice interaction device provided in an embodiment of this application.

[0024] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0025] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0026] Exemplary methods This embodiment provides a voice interaction method, which can be applied to, but is not limited to, electronic devices with voice interaction functions and display interfaces, such as in-vehicle infotainment systems, smartphones, smart speakers, and tablet computers. This method aims to solve the problem in related technologies where voice interaction matching is based solely on the text content of the interactive elements themselves, leading to mismatches or matching failures when interactive elements with similar text content exist in different interface areas.

[0027] Figure 1 The diagram shown is a flowchart of a voice interaction method provided in an embodiment of this application. This method is executed by a processor in the aforementioned electronic device, and its specific interaction steps are detailed below.

[0028] In step S110, the user's voice command text is obtained.

[0029] In this embodiment, obtaining the user's voice command text includes: receiving the user's original voice command text; removing non-semantic text from the original voice command text to obtain the voice command text.

[0030] This step opens the data input port for the entire voice interaction process. Specifically, the user issues an operation intention via voice, and the electronic device's voice acquisition module (such as a microphone array) receives the voice command. This voice command is processed by the automatic speech recognition module running on the electronic device's processor and converted into a computer-readable text sequence. This text sequence is the original voice command text. The original voice command text carries the specific intention that the user wants the electronic device to perform. For example, if the user's voice command is "I want to listen to Shan Tianfang's 'White-Browed Hero' for free," then the corresponding original voice command text is "I want to listen to Shan Tianfang's 'White-Browed Hero' for free."

[0031] To further improve the accuracy of voice interaction and reduce the computational load of subsequent voice interaction matching processes, this embodiment also discloses a technical solution for optimizing the user's original voice command text.

[0032] In this application, "non-semantic text" refers to morphemes or words in the user's original voice command text that do not directly contribute to expressing the core matching intent or target content and can be removed.

[0033] In one specific implementation, the speech recognition module first receives all the recognition results of the user's spoken voice command, which constitutes the original speech command text. Based on the inventor's in-depth research and observation of such human-computer voice interaction scenarios, users often unconsciously include some expressions unrelated to the core operational intent when issuing voice commands. For example, a user might say, "I want to listen to Guo Degang's crosstalk," where "I want to listen" is a guiding expression and not the target of the user's intent; similarly, a user might say, "Help me open that car window," where words like "help me," "open," and "that" are all non-target semantic content. This application's embodiments collectively refer to such text as "non-semantic text."

[0034] To address the issue of non-semantic text interfering with matching calculations, embodiments of this application pre-define rules or models to identify and remove them. Specific means of achieving this functionality may include: at the software level, establishing a non-semantic text lexicon containing common semantic guide words, function words, or modal particles such as "I want to listen," "Play it for me," "Click," "Please help me," and "that." Embodiments of this application perform word segmentation and part-of-speech analysis on the received original voice command text. By comparing it with the pre-defined non-semantic text lexicon, non-semantic text in the original voice command text is located and deleted to obtain the semantic command text.

[0035] As another implementation method, a matching and extraction scheme based on regular expressions can be adopted. Specifically, a series of regular expression rules are pre-compiled to match and locate common command sentence structures. For example, a regular expression rule can be designed to match and extract the content following "I want to listen" or "Play for me". When non-semantic text preset in the regular expression rules is detected in the original voice command text, the original voice command text is processed based on the regular expression rules to locate the non-semantic text in the original voice command text and extract the text at the corresponding position in the original voice command text (for example, the text following the non-semantic text in the original voice command text), so as to identify the extracted text as the voice command text. It can be understood that this process can be seen not only as the extraction of voice command text from the original voice command text, but also as the removal of non-semantic text from the original voice command text.

[0036] After the above processing, the core part retained in the original voice command text is the purified voice command text. For example, if the original voice command text is "I want to listen to Shan Tianfang's 'White-Browed Hero' for free," then the voice command text will be "Shan Tianfang's 'White-Browed Hero' is free." This method effectively filters out redundant information carried by the user in their natural language expression that is irrelevant to the core matching intent, resulting in extracted voice command text with a purer target semantics. This not only effectively reduces the computational load of subsequent feature extraction and similarity calculation but also significantly improves the accuracy of core semantic matching.

[0037] In step S120, the text content of the interactive elements in the current interactive interface is determined.

[0038] The current interactive interface referred to herein is the graphical user interface that is displayed on the screen of the electronic device when the user issues a voice command. This interface contains multiple controls or views that can respond to user operations, i.e., the interactive elements referred to in the embodiments of this application.

[0039] In this application, "interactive element" refers to any interface component in an interactive interface that can respond to user operations or trigger specific functions and content displays. For example, it may include, but is not limited to, function buttons, content list items, program cards, icon controls, hyperlink text, etc. in a graphical user interface.

[0040] Each interactive element has a text description that represents its content or function, i.e., text content.

[0041] For example, the text content of an interactive element is the element name of the interactive element.

[0042] The above are merely preferred contents of the text content of the interactive elements provided in the embodiments of this application. The specific content of the text content of the interactive elements can be set by those skilled in the art according to their own needs, and is not limited here.

[0043] As an example implementation, in response to a user's voice command, the system obtains the text of the user's voice command and determines the text content of each interactive element in the current electronic device's interface.

[0044] In step S130, the text content of the interactive element is processed by combining the attribute information of the interface area to which the interactive element belongs in the interactive interface, and a hot word library of the current interactive interface is generated.

[0045] In this embodiment of the application, the hot word library includes at least one hot word for each interactive element in the current interactive interface.

[0046] In this application, "hot words" generally refers to words generated based on word segmentation of the text content of interactive elements in the current interactive interface and combined with its interface area attribute information, used to match the voice command text corresponding to the user's voice command.

[0047] In this application, "hot word library" refers to a collection of one or more hot words that is constructed in real time or dynamically in the current interactive interface. This collection stores words used to match voice command text, specifically including at least one hot word for each interactive element in the current interactive interface.

[0048] This step is the core of the voice interaction method provided in this application embodiment. In order to overcome the mismatch problem caused by interactive elements with similar text content in different areas of the interactive interface in related technologies, this application embodiment, when generating hot words for voice command text matching for interactive elements, not only uses the text content of the interactive elements themselves, but also creatively introduces the contextual environment information of the interactive elements in the interface layout, that is, the attribute information of the interface area to which the interactive element belongs in the current interactive interface.

[0049] It should be noted that, in this application, "interface area" refers to a spatial unit within the current interactive interface that can be divided according to function or layout logic. For example, in an audiobook application, "Popular Crosstalk and Storytelling Chart," "New Crosstalk and Storytelling Chart," and "Free Crosstalk and Storytelling Chart" are different interface areas. "Attribute information of the interface area" refers to the characteristic labels used to describe and distinguish these areas, such as the area's functional type or location information.

[0050] In this application, "attribute information" refers to characteristic descriptive information used to characterize the interface area to which an interactive element belongs in the interactive interface, and which can distinguish that area from other areas on the interface. For example, it may include, but is not limited to: area type attributes determined based on the functional definition of the interface area (such as "popular list", "free list", "new product list"), or area position attributes determined based on the position information of the interface area in the interactive interface (such as coordinates or layout order) (such as "top left area", "bottom third column").

[0051] In specific implementation, for each interactive element identified in step S120, its corresponding interface area in the interactive interface is identified, and the attribute information of that area is extracted. For example, for an interactive element with the text "White-browed Hero | Shan Tianfang's Storytelling", if it is located in the "Popular Crosstalk and Storytelling List" interface area, its attribute information is determined to be "Popular List". Simultaneously, for an interactive element with the same text content located under the "Free Crosstalk and Storytelling List" interface, its attribute information is determined to be "Free List". Subsequently, the text content of the interactive element is processed by combining the attribute information of the interface area to which it belongs in the interactive interface to obtain at least one hot word corresponding to the interactive element.

[0052] A typical approach is to use the attribute information of the interface area to which the interactive element belongs in the interactive interface as a context marker, and combine, concatenate or establish an association mapping with each word in the text content of the interactive element to generate one or more hot words carrying the regional context features.

[0053] Thus, in the generated hot word library, each interactive element corresponds to at least one hot word. These hot words are no longer isolated text fragments, but rather composite semantic entities endowed with spatial or functional dimensions. For example, an interactive element named "White-Browed Hero | Shan Tianfang's Storytelling" located under the "Popular Crosstalk and Storytelling List" will generate at least one hot word including "White-Browed Hero Popular List" and "Shan Tianfang's Storytelling Popular List"; while an interactive element with the same text content located under the "Free Crosstalk and Storytelling List" will generate at least one hot word such as "White-Browed Hero Free List" and "Shan Tianfang's Storytelling Free List". In this way, the hot word library has the ability to distinguish interactive elements located in different interface areas but with similar text content.

[0054] In step S140, a target hot word that matches the voice command text in the hot word library is determined, and the interactive element to which the target hot word belongs in the current interactive interface is triggered.

[0055] In the embodiment of the present application, after the hot word library is constructed, matching calculation is performed between the user's voice instruction text and all hot words in the hot word library. This matching process aims to find the hot word that is semantically closest to the user intent represented by the voice instruction text from the hot word library (this hot word is referred to as the target hot word). Specifically, the matching process can be completed based on semantic similarity calculation. First, the voice instruction text and each hot word in the hot word library are respectively converted into text features through a text feature extraction model, for example, converting the voice instruction text "Shan Tianfang's White Eyebrow Hero is free" into a high-dimensional text feature; and converting each hot word in the hot word library into the corresponding text feature respectively; then, for each hot word in the hot word library, the semantic similarity between the text feature of the hot word and the text feature of the voice instruction text is calculated as the similarity of the hot word.

[0056] As an example, the text feature is a text feature vector, and the semantic similarity may be cosine similarity. The above is only the preferred content of the text feature and semantic similarity provided by the embodiments of the present application, and the specific content related to the text feature and semantic similarity can be set by those skilled in the art according to their own needs, which is not limited herein.

[0057] In the embodiment of the present application, the hot word with the highest similarity is determined from the hot word library. If the similarity of the hot word exceeds the target similarity threshold corresponding to the target scene information of the current interaction interface, the hot word is determined as the target hot word matching the voice instruction text.

[0058] Since regional attribute information has been injected in the hot word generation stage, even if the voice instruction text is "Shan Tianfang's White Eyebrow Hero is free", its text feature will be more inclined to be closer to the text feature of the hot word "White Eyebrow Hero Free List" rather than "White Eyebrow Hero Popular List", so that the correct target hot word can be "hit" to trigger the correct interactive element "White Eyebrow Hero | Shan Tianfang's Ping Shu" under "Cross-talk and Ping Shu Free List" in the current interactive interface.

[0059] In the embodiment of the present application, the interactive element to which the target hot word belongs is triggered, that is, the functional operation corresponding to the interactive element is executed. For example, the audio book program "White Eyebrow Hero | Shan Tianfang's Ping Shu" under "Cross-talk and Ping Shu Free List" is triggered to play. This triggering action may simulate a click operation on the interactive element on the screen, or directly call the interface or callback function bound to the interactive element in the background, which is not limited in the embodiments of the present application.

[0060] It should be understood that the acquisition of the text content of the interactive element in step S120 described above provides basic semantic materials for generating the hot word library with regional context in step S130. The manner of introducing the interface region attribute information in step S130 further provides key discrimination information for accurately locating the target hot word from multiple hot words in the hot word library in step S140. The organic collaboration of these three steps enables the understanding of the implicit region restriction intent (such as "free") in the user's natural language, thereby effectively solving the false matching problem caused by the similar text content of interface elements, and achieving accurate targeting of the user's intent.

[0061] In this embodiment, the technical details of how to combine the interface region attribute information to specifically generate discriminative hot words so as to construct a hot word library are further disclosed. This preferred solution is devoted to solving the technical problem of systematically generating composite hot words with regional context for each interactive element, so as to support accurate matching of the user's composite intent.

[0062] Figure 2 is a flowchart of a method for processing the text content of an interactive element in combination with the attribute information of the interface region to which the interactive element belongs in the interactive interface and generating a hot word library of the current interactive interface provided by the embodiments of the present application, and the specific interaction steps are described in detail as follows.

[0063] In step S210, word segmentation is performed on the text content of the interactive element to obtain at least one segmented word of the interactive element.

[0064] In the embodiment of the present application, after each interactive element in the current interactive interface is determined, word segmentation processing is performed on the text content of each interactive element to obtain at least one segmented word of each interactive element. For example, for an interactive element whose text content is "Bai Mei Da Xia | Shan Tianfang's Pingshu", it will be segmented into a plurality of independent segmented words such as "Bai Mei Da Xia" and "Shan Tianfang's Pingshu".

[0065] In step S220, the attribute information of the interface region to which the interactive element belongs in the interactive interface is determined, and the attribute information of the interface region is determined based on the difference information between different interface regions in the current interactive interface.

[0066] In the embodiment of the present application, the attribute information of the interface region includes the region type and / or region position of the interface region, the region type is defined based on the function of the interface region, and the region position is defined based on the position information of the interface region in the current interactive interface. That is, the region type is determined based on the function definition undertaken by the interface region, and the region position is defined based on the position information of the interface region in the current interactive interface.

[0067] The key takeaway is that the essential differences between different areas of an interactive interface lie in two aspects: what they do (function) and where they are located (location).

[0068] For example, in a content aggregation application, different interface areas serve different functions. Some are used to display popular recommendations, some to display new product launches, and some to provide free content. Accordingly, tags such as "Popular List," "New Product List," and "Free List" can be used as their area type attributes. These tags reflect the content organization and display logic of that interface area and are derived from its functional definition.

[0069] On the other hand, even if the function definition is vague, the location information of the interface area on the interactive interface (physical location, such as being located at the top, middle or bottom of the interface, or being defined as the first column, second column, third column) is its inherent and quantifiable spatial attribute.

[0070] By specifying attribute information into one or both of these technical dimensions, a clear and actionable technical path is provided for automatically identifying and extracting the contextual features of the interface area where interactive elements are located. Whether qualitatively determining the functional type by recognizing the title text of the area, or quantitatively determining its spatial location by parsing the coordinate information of the interface area in the interactive interface, this definition can be used to generate distinctive hot words for interactive elements in different interface areas. This ensures the universality of the entire hot word library generation logic and its feasibility in different applications and interface designs.

[0071] To further improve the automation and accuracy of the technical solution, enabling it to adaptively understand the interface structure of various content aggregation applications without requiring manual pre-configuration of rules for each interface, this embodiment provides an optimal solution for automatically determining the attribute information of interface areas, particularly their area type attributes. It is understood that if the attribute information of an interface area includes its area type, the attribute information of the interface area can be referred to as the area type attribute.

[0072] In a preferred embodiment for automatically identifying region type attributes, determining the attribute information of the interface region to which an interactive element belongs in the interactive interface includes: identifying at least one logical partition in the current interactive interface, each logical partition corresponding to an interface region; obtaining the title text of each logical partition in the at least one logical partition; and determining the attribute information of each logical partition based on the differences between the title texts of each logical partition in the at least one logical partition.

[0073] In this application, "logical partition" generally refers to a regional unit on an interactive interface that is divided according to visual, layout, or functional criteria to organize and accommodate a group of related interactive elements.

[0074] In this embodiment of the application, the DOM tree or view tree of the current interactive interface is subjected to structured analysis. By searching for specific container controls (such as list containers, card containers) or visual elements such as dividing lines, the interactive interface is automatically identified and divided into multiple independent logical partitions.

[0075] For example, on the "Crosstalk and Storytelling" page of an audiobook app, three parallel list areas (logical partitions) can be automatically identified. These three parallel list areas are the popular crosstalk and storytelling list area, the new crosstalk and storytelling list area, and the free crosstalk and storytelling list area.

[0076] Each identified logical partition can be considered a UI area, and the system further searches for text elements with title characteristics within it. Typically, these title texts appear at the top of the partition, with fonts and styles distinct from the rest of the content. In this example, three title texts can be obtained: "Popular Crosstalk and Storytelling List," "New Crosstalk and Storytelling List," and "Free Crosstalk and Storytelling List." These three title texts are then compared and analyzed, and unique attribute information is assigned to each logical partition based on their textual differences (i.e., "Popular List," "New List," and "Free List" are different from each other). For example, "Popular List" from the extracted title text of the "Popular Crosstalk and Storytelling List" area can be used as the area type attribute for this logical partition. In this way, the solution autonomously maps the UI visual structure to logical functional attributes, automatically distinguishing "White-Eyebrowed Hero" under "Popular Crosstalk and Storytelling List" from "Free Crosstalk and Storytelling List" from a functional perspective, significantly improving the automation and robustness of the solution when handling dynamic and complex interfaces.

[0077] This embodiment provides another scheme for automatically determining the attribute information of an interface region, focusing on defining the attribute information from the spatial location dimension to enhance the adaptability of this scheme in weak text or non-standard interfaces. It is understood that if the attribute information of an interface region includes the regional location of the interface region, the attribute information of the interface region can be called the regional location attribute.

[0078] In another preferred embodiment for automatically identifying the location attributes of a region, determining the attribute information of the interface region to which the interactive element belongs in the interactive interface includes: identifying at least one logical partition in the current interactive interface, each logical partition corresponding to an interface region; obtaining the location information of each logical partition in the current interactive interface; and determining the attribute information of each logical partition based on the relative location information between the logical partitions in the at least one logical partition.

[0079] In practice, the process of identifying logical partitions is similar to that described above. Following this, the embodiments of this application will obtain the position information of these logical partitions in the interface coordinate system.

[0080] For example, the position information of the logical partition in the interface coordinate system of the interactive interface includes the coordinate range of the logical partition on the X-axis and Y-axis respectively.

[0081] After obtaining the position information of each logical partition in the current interactive interface, the relative positional relationships between the logical partitions are analyzed. For example, the vertical arrangement order of each logical partition can be determined by comparing the Y-axis coordinates of its top-left corner or center point. Assuming an interactive interface has three logical partitions arranged vertically, the partition with the smallest Y-coordinate is located at the top, labeled as the "first area" or "top area," the partition with the next smallest Y-coordinate is designated as the "second area" or "middle area," and the partition with the largest Y-coordinate is designated as the "third area" or "bottom area." This information representing the relative order or orientation is the regional position attribute of the logical partitions determined based on their relative position information.

[0082] This solution provides spatial dimension attribute information independent of specific textual descriptions. Therefore, even if the interface area lacks explicit functional title text, or the title text is semantically unclear, it can still create distinctive hot words for interactive elements in different logical areas based on the spatial layout between the various logical partitions in the interactive interface. For example, associating the word segmentation of the interactive element text content with regional position attributes such as "top area" and "middle area" effectively enhances the robustness and versatility of this solution in non-standard interfaces.

[0083] In step S230, the word segmentation and attribute information are associated to generate hot words corresponding to the word segmentation; the hot words corresponding to the word segmentation of each interactive element in the current interactive interface constitute the hot word library of the current interactive interface.

[0084] In the present application, at least one participle of each interactive element in the interactive interface is determined respectively; for each participle of the interactive element, the participle is associated with the attribute information of the interactive element to obtain a hot word corresponding to the participle. As a result, a hot word corresponding to the participle can be obtained for each participle of each interactive element in the interactive interface, and then the hot words corresponding to each participle of each interactive element in the interactive interface form a hot word library corresponding to the interactive interface.

[0085] There may be a variety of specific ways to implement the above association to generate hot words. A relatively direct way is to perform text-level processing based on preset splicing rules. For example, the extracted participle of the text content of the interactive element may be spliced and combined with the attribute information of the interactive element, so as to generate a hot word.

[0086] For the above example, the participle "Bai Mei Da Xia" located in the area of "Crosstalk and Pingshu Popular Ranking" will be spliced to generate the hot word "Bai Mei Da Xia Popular Ranking"; while the same participle located in the area of "Crosstalk and Pingshu Free Ranking" will be spliced to generate the hot word "Bai Mei Da Xia Free Ranking". In this way, each participle of each interactive element can generate a composite hot word clearly bound with its regional attribute identifier, which is then stored in the hot word library. In this manner, the hot word library can systematically and completely express the combined semantics of "content" and "function / location", providing an accurate matching basis for subsequent accurate identification of composite intents such as the voice command text "Shan Tianfang's Bai Mei Da Xia that is free", and avoiding ambiguity during matching.

[0087] In order to improve the precision and recall rate of hot word matching, so that it can dynamically adapt to the semantic distribution characteristics in different application scenarios, this embodiment proposes a scenario-adaptive dynamic similarity threshold matching scheme. This scheme aims to solve the problem of reduced matching accuracy or matching failure caused by using the same fixed similarity threshold for all application scenarios.

[0088] In this preferred solution, Figure 3 is a flowchart of a method for determining a target hot word in a hot word library that matches a voice command text provided by the embodiment of the present application, which is specifically defined to include the following steps.

[0089] In step S310, target scenario information of the current interactive interface is determined, and the target scenario information characterizes the application scenario of the current interactive interface.

[0090] In this application, "target scene information" refers to any descriptive identifier that can characterize the application scene to which the current interactive interface belongs. This identifier is used to associate with or invoke the corresponding matching strategy parameters (target similarity threshold). For example, it may include, but is not limited to: scene identifiers that characterize the type of audio content provided by the interface (such as "radio scene", "audiobook scene", "music scene"), or functional environment descriptions inferred based on the application name and the current interactive interface information (such as "vehicle control scene", "navigation settings scene"), etc.

[0091] That is, the target scene information in this application embodiment is used to characterize the application scene to which the current interactive interface belongs. For example, in an in-vehicle system, when a user opens a radio application, the target scene information is determined to be a "radio scene"; when a user opens an audiobook application, the target scene information is an "audiobook scene"; similarly, it can also include "music scene", "vehicle control scene", etc.

[0092] In step S320, the first text features of the voice command text and the second text features of each hot word in the hot word library are extracted based on the pre-trained text feature extraction model.

[0093] In this application, "text feature extraction model" refers to any model that can convert input text into a vectorized representation (i.e., text features) that carries its semantic information. The text features can be text feature vectors, and this is not limited thereto.

[0094] In this embodiment of the application, the text feature extraction model is a lightweight neural network model (such as the BGE model) that has been trained by knowledge distillation and contrastive learning and then deployed on the vehicle terminal.

[0095] Based on a pre-trained text feature extraction model, the first text features of the voice command text obtained in the aforementioned steps, and the second text features of each hot word in the hot word library are extracted respectively.

[0096] In step S330, the similarity of each hot word in the hot word library is calculated based on the first text feature of the voice command text and the second text feature of the hot words.

[0097] In this application, "similarity" refers to a scalar measure used to quantify the degree of association or proximity between two texts in a semantic space. For example, it may include, but is not limited to, semantic similarity calculated based on the text feature vectors of two texts.

[0098] In this embodiment, the similarity of each hot word is calculated based on the first text feature of the voice command text and the second text feature of each hot word. The similarity of the hot word is used to characterize the semantic similarity (i.e., the degree of semantic similarity) between the hot word and the voice command text.

[0099] In step S340, it is determined whether the highest similarity of each hot word in the hot word library exceeds the target similarity threshold corresponding to the target scene information; if the highest similarity of each hot word in the hot word library exceeds the target similarity threshold corresponding to the target scene information, then step S350 is executed.

[0100] Based on steps S310-S330 above, this scheme performs dynamic similarity threshold determination. Specifically, it determines whether the highest similarity calculated between all hot words in the hot word library and the voice command text exceeds the target similarity threshold corresponding to the target scene information; if the highest similarity is determined to exceed the target similarity threshold corresponding to the target scene information, then the hot word to which the highest similarity belongs is determined as the target hot word matching the voice command text.

[0101] In this application, "target similarity threshold" refers to a similarity threshold dynamically determined based on the target scenario information of the current interactive interface, used to determine whether hot words and voice command text constitute a match. For example, it may include, but is not limited to: a lenient threshold (such as 0.6) that can be lowered in a radio scenario to improve generalization recall, or a strict threshold (such as 0.8) required to maintain accurate matching in a music scenario.

[0102] Different target scenarios have different pre-set target similarity thresholds. For example, in a radio scenario, because the radio station name may differ significantly from the user's colloquial expression (e.g., the user says "financial radio station," while the trending topic on the interface is "Beijing Economic Radio"), a relatively low target similarity threshold, such as 0.6, can be set to improve the recall rate. In contrast, in a music scenario, where matching song titles or artist names focuses more on accuracy, a relatively high target similarity threshold, such as 0.8, can be set.

[0103] In step S350, the hot word with the highest similarity is determined as the target hot word.

[0104] In this embodiment of the application, if the highest similarity is determined to exceed the target similarity threshold corresponding to the target scene information, then the hot word to which the highest similarity belongs is determined as the target hot word that matches the voice command text.

[0105] Furthermore, if the highest similarity score does not exceed the target similarity threshold corresponding to the target scene information, the matching is deemed to have failed. In this case, a prompt message can be sent to the user. The prompt message can be broadcast in voice format. The prompt message includes guidance information, which guides the user to input accurate voice commands. For example, the guidance information may include distinguishing information between multiple interactive elements in the interactive interface related to the voice command that failed to match. The voice command that the user inputs again based on the prompt message contains the distinguishing information. This solution can determine the interactive element that needs to be triggered by combining the user's re-input voice command and the previously failed voice command.

[0106] Through such Figure 3 The matching mechanism shown allows the hot word matching criteria to be flexible and adaptable based on the application scenario of the current interface. In scenarios with high generalization requirements, such as radio, a more lenient similarity threshold is applied to improve the hit rate; while in precise matching scenarios, such as music, a strict similarity threshold is maintained to ensure accuracy. This effectively solves the problem of inconsistent application of a fixed similarity threshold across different scenarios, achieving accurate matching in various contexts.

[0107] In order to ensure semantic matching accuracy while meeting the need for local real-time computing on resource-constrained edge devices such as vehicles, this embodiment also discloses a method for obtaining the aforementioned text feature extraction model. The method for obtaining the text feature extraction model is specifically a model training scheme based on knowledge distillation and contrastive learning.

[0108] In this embodiment, the text feature extraction model is obtained as follows: a second model is obtained by distilling a first model, where the first model is a pre-set large model and the second model is used for text feature extraction; a sample pair is input into the second model to obtain the first text feature of the first sample and the second text feature of the second sample in the sample pair; a first semantic similarity between the first sample and the second sample is calculated based on the first text feature of the first sample and the second text feature of the second sample; a second semantic similarity between the first sample and the second sample in the sample pair is determined based on the first model; and the second model is optimized and trained based on the difference between the first semantic similarity and the second semantic similarity to obtain the text feature extraction model.

[0109] First, a first model is distilled to obtain a second model.

[0110] In this application, the "first model" is a pre-set large and more powerful model that includes text feature extraction capabilities and plays the role of a "teacher model" during training. For example, it could be the Qwen3 model.

[0111] In this application, the "second model" is a lightweight model with a small number of parameters, playing the role of a "student model". As a specific implementation, the second model can be a BGE (BAAI General Embedding) model (small model) with less than 50M parameters. By distilling the knowledge of the first model, the text feature extraction function of the first model can be transferred to the second model.

[0112] Next comes the core loop of training and optimization. A batch of sample pairs is obtained, including semantically similar positive sample pairs and semantically unrelated negative sample pairs. The first and second samples from each sample pair are input into the current second model to obtain their respective first and second text features. The semantic similarity between the first and second samples is calculated based on the first text features of the first sample and the second text features of the second sample, and denoted as the first semantic similarity.

[0113] Meanwhile, using the first model as the teacher model, the semantic similarity between the first and second samples in the sample pair is also calculated, and this is used as a supervision signal, denoted as the second semantic similarity. Because the first model is more powerful, its second semantic similarity more accurately reflects the true semantic relationship between the two samples in the sample pair.

[0114] Finally, the first semantic similarity calculated by the student model is compared with the second semantic similarity given by the teacher model to obtain a difference value. This difference value is used as a core component of the loss function to perform backpropagation and iterative optimization training on the parameters of the second model. For example, for a positive sample pair, the second semantic similarity given by the teacher model is 0.95, while the first semantic similarity calculated by the student model is only 0.7. This difference will drive the student model to adjust its internal weight parameters, so that the text features of the positive sample pair are closer to each other in the semantic space. Conversely, for negative sample pairs, the difference will drive the student model to push their text features further apart. Through iterative training with a large number of sample pairs, the optimized second model eventually becomes a mature text feature extraction model. It can raise the semantic similarity of similar texts to above 0.9 and reduce the semantic similarity of dissimilar texts to below 0.5 with a very small model size, significantly widening the decision boundary for matching and ensuring the accuracy and real-time performance of local NPU inference on automotive-grade chips such as the Qualcomm 8295p.

[0115] After completing the training of the knowledge distillation and contrastive learning models, in order to achieve efficient local semantic computation on resource-constrained in-vehicle devices, operators such as LayerNorm and GELU in the text feature extraction model can be reconstructed using fixed-point quantization and deployed to the NPU using INT8 quantization for exclusive NPU operation. This achieves inference performance with a single-processing latency of ≤30ms in batch processing scenarios with a batch size of 50, meeting the real-time interaction requirements of automotive-grade standards. The specific process is as follows: Step a: Fixed-point reconstruction using operators such as LayerNorm and GELU For the characteristics of the target platform (e.g., the Qualcomm 8295P platform Hexagon NPU architecture), key operators such as LayerNorm and GELU in the text feature extraction model are reconstructed using fixed-point methods and their memory layout is optimized to reduce floating-point computation overhead. This lays the foundation for quantization deployment, supporting INT8 quantization deployment and efficient inference. Step a mainly involves the following two processes: 1. The LayerNorm operator transforms division operations into fixed-point shift operations by converting the mean-variance calculation into integer operations [e.g., approximating 1 / sqrt(var+ε) as a multiplication shift of 2^n / round(sqrt(var+ε))].

[0116] 2. The GELU activation function uses piecewise linear approximation (e.g., tanh approximation or least squares fitting) or lookup table (LUT) to replace the exponential and error function operations, reducing computational complexity and controlling accuracy loss to within 0.5%. At the same time, the tensor memory layout is optimized to NHWC format to match the Hexagon NPU SIMD data stream, and per-channel symmetric quantization (scale is zero-point symmetric) and 128-byte alignment are performed on the weights. The activation uses per-tensor dynamic quantization to eliminate the data transfer overhead between the CPU and NPU.

[0117] Step b: INT8 quantization deployment, NPU dedicated inference The text feature extraction model is quantized using INT8 and deployed to the NPU of the Qualcomm 8295p platform, running in dedicated inference mode to ensure the stability of computing resources and inference efficiency.

[0118] Step c: Batch processing delay determination Evaluate the inference latency of the text feature extraction model in batch processing scenarios, and determine whether the single processing latency meets the threshold requirement when the batch size is 50 (for example, the threshold requirement can be a single processing latency ≤ 30ms).

[0119] Step d: Meet automotive-grade real-time interaction requirements If the single-processing latency meets the threshold requirement (i.e., the single-processing latency is ≤30ms), it indicates that the text feature extraction model performance meets the response time requirements of automotive-grade real-time voice interaction scenarios.

[0120] Step e: Performance not up to standard (does not meet automotive-grade real-time interaction requirements) If the single-processing latency does not meet the threshold requirement (i.e., single-processing latency > 30ms), the text feature extraction model structure or operator implementation needs to be further optimized until the performance of the text feature extraction model meets the response time requirements of automotive-grade real-time voice interaction scenarios.

[0121] By adopting the above-mentioned model training scheme based on knowledge distillation and contrastive learning, and combining operator fixed-point reconstruction and INT8 quantization deployment, not only can the small text feature extraction model significantly widen the decision boundary of semantic matching, ensuring the accuracy of local semantic computation on resource-constrained edge devices such as in-vehicle devices; but it can also ensure the real-time performance of local semantic computation on resource-constrained edge devices such as in-vehicle devices with extremely low inference latency.

[0122] This application provides a voice interaction method that acquires the user's voice command text; determines the text content of interactive elements in the current interactive interface; processes the text content of interactive elements based on the attribute information of the interface area to which the interactive elements belong in the interactive interface to generate a hot word library for the current interactive interface; the hot word library includes at least one hot word for each interactive element in the current interactive interface; determines a target hot word in the hot word library that matches the voice command text, and triggers the interactive element to which the target hot word belongs in the current interactive interface. This application generates a hot word library for the current interactive interface by combining the attribute information of the interface area to which the interactive elements belong in the current interactive interface, alleviating the semantic difference between the user's spoken input and the structured page element text, reducing matching failures caused by semantic differences, and achieving fast and accurate matching between the user's natural spoken voice commands and the corresponding interactive elements on the current interface of the electronic device, thus improving the user's voice interaction experience. That is, by introducing the attribute information of the interface area to which the interactive element belongs when generating the hot word library, the hot words carry the context features of the interface area, so that interface elements (interactive elements) located in different interface areas but with similar text can be effectively distinguished during matching, reducing false matching and achieving accurate matching of the user's voice command intent.

[0123] Exemplary device Accordingly, this application also provides a voice interaction device.

[0124] Please see Figure 4 In one exemplary embodiment, a voice interaction device 400 is provided, the voice interaction device 400 including: Voice command text acquisition unit 410 is used to acquire the user's voice command text; The text content determination unit 420 is used to determine the text content of interactive elements in the current interactive interface; The hot word library generation unit 430 is used to process the text content of the interactive element by combining the attribute information of the interface area to which the interactive element belongs in the interactive interface, and generate the hot word library of the current interactive interface; the hot word library includes at least one hot word for each interactive element in the current interactive interface. The voice interaction unit 440 is used to determine the target hot word in the hot word library that matches the voice command text and to trigger the interactive element to which the target hot word belongs in the current interactive interface.

[0125] In one possible implementation, the voice command text acquisition unit includes: The raw voice command text receiving unit is used to receive the user's raw voice command text. The voice command text acquisition subunit is used to remove non-semantic text from the original voice command text to obtain the voice command text.

[0126] In one possible implementation, the hot word library generation unit includes: A word segmentation unit is used to segment the text content of interactive elements to obtain at least one word of the interactive element. The attribute information determination unit is used to determine the attribute information of the interface area to which the interactive element belongs in the interactive interface. The attribute information of the interface area is determined based on the difference information between different interface areas in the current interactive interface. The hot word generation unit is used to associate word segmentation and attribute information to generate hot words corresponding to the word segmentation; the hot words corresponding to the word segmentation of each interactive element in the current interactive interface constitute the hot word library of the current interactive interface.

[0127] In one possible implementation, the attribute information of the interface area includes the area type and / or area location of the interface area, where the area type is based on the functional definition of the interface area and the area location is based on the position information of the interface area in the current interactive interface.

[0128] In one possible implementation, the attribute information determination unit includes: The first identification unit is used to identify at least one logical partition in the current interactive interface, and each logical partition corresponds to an interface area. The title text acquisition unit is used to acquire the title text of each logical partition in at least one logical partition; The first determining unit is used to determine the attribute information of each logical partition based on the difference information between the title texts of each logical partition in at least one logical partition.

[0129] In one possible implementation, the attribute information determination unit includes: The second identification unit is used to identify at least one logical partition in the current interactive interface, and each logical partition corresponds to an interface area. The location information acquisition unit is used to acquire the location information of each logical partition in at least one logical partition in the current interactive interface; The second determining unit is used to determine the attribute information of each logical partition based on the relative position information between each logical partition in at least one logical partition.

[0130] In one possible implementation, the voice interaction unit includes: The target scene information determination unit is used to determine the target scene information of the current interactive interface. The target scene information represents the application scene of the current interactive interface. The first feature extraction unit is used to extract the first text features of the voice instruction text and the second text features of each hot word in the hot word library based on the pre-trained text feature extraction model. The first similarity calculation unit is used to calculate the similarity of each hot word in the hot word library based on the first text features of the voice command text and the second text features of the hot words; the similarity of the hot words represents the semantic similarity between the hot words and the voice command text. The similarity determination unit is used to determine whether the highest similarity of each hot word in the hot word library exceeds the target similarity threshold corresponding to the target scene information; The target hot word determination unit is used to determine the hot word with the highest similarity as the target hot word if the highest similarity of each hot word in the hot word library exceeds the target similarity threshold corresponding to the target scene information.

[0131] In one possible implementation, the voice interaction device further includes a model generation unit, which includes: The distillation unit is used to distill the first model to obtain the second model. The first model is a pre-set large model, and the second model is used for text feature extraction. The second feature extraction unit is used to input the sample pair into the second model to obtain the first text feature of the first sample and the second text feature of the second sample in the sample pair. The second similarity calculation unit is used to calculate the first semantic similarity between the first sample and the second sample based on the first text features of the first sample and the second text features of the second sample. The third similarity calculation unit is used to determine the second semantic similarity between the first sample and the second sample in the sample pair based on the first model; The model generation subunit is used to optimize and train the second model based on the difference between the first semantic similarity and the second semantic similarity to obtain the text feature extraction model.

[0132] The voice interaction device provided in this embodiment belongs to the same concept as the voice interaction method provided in the above embodiments of this application. It can execute the voice interaction method provided in any of the above embodiments of this application and has the corresponding functional modules and beneficial effects for executing the voice interaction method. Technical details not described in detail in this embodiment can be found in the corresponding method embodiments of this application, and will not be repeated here.

[0133] The functions implemented by each unit in the above device can be implemented by the same or different processors, and this application embodiment does not limit this.

[0134] It should be understood that the units in the above device can be implemented by a processor calling software. For example, the device includes a processor connected to a memory containing instructions. The processor calls the instructions stored in the memory to implement any of the above methods or to implement the functions of each unit in the device. The processor can be a general-purpose processor, such as a CPU or microprocessor, and the memory can be internal or external to the device. Alternatively, the units in the device can be implemented as hardware circuits. By designing the hardware circuits, some or all of the unit functions can be implemented. The hardware circuits can be understood as one or more processors. For example, in one implementation, the hardware circuit is an ASIC, and the functions of some or all of the above units are implemented by designing the logical relationships between the components within the circuit. In another implementation, the hardware circuit can be implemented using a PLD, such as an FPGA, which can include a large number of logic gates. The connection relationships between the logic gates are configured through configuration files to implement the functions of some or all of the above units. All units in the above device can be implemented entirely by a processor calling software, entirely by hardware circuits, or partially by a processor calling software with the remaining parts implemented by hardware circuits.

[0135] In this application embodiment, a processor is a circuit with signal processing capabilities. In one implementation, the processor can be a circuit with instruction reading and execution capabilities, such as a CPU, microprocessor, GPU, or DSP. In another implementation, the processor can implement certain functions through the logical relationships of hardware circuits. These logical relationships are fixed or reconfigurable. For example, the processor may be a hardware circuit implemented as an ASIC or PLD, such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document and configuring the hardware circuit can be understood as the processor loading instructions to implement the functions of some or all of the above units. Furthermore, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as an NPU, TPU, or DPU.

[0136] As can be seen, each unit in the above device can be one or more processors (or processing circuits) configured to implement the above methods, such as: CPU, GPU, NPU, TPU, DPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor forms.

[0137] Furthermore, the units in the above devices can be integrated in whole or in part, or they can be implemented independently. In one implementation, these units are integrated together and implemented in the form of a System-on-Chip (SoC). The SoC may include at least one processor for implementing any of the above methods or implementing the functions of the units in the device. The at least one processor may be of different types, such as CPU and FPGA, CPU and artificial intelligence processor, CPU and GPU, etc.

[0138] Exemplary electronic devices Another embodiment of this application also proposes an electronic device. See [link to relevant documentation]. Figure 5 As shown, the electronic device may include: a memory 500 and a processor 510; wherein the memory 500 is connected to the processor 510 and is used to store programs; the processor 510 is used to implement the voice interaction method disclosed in any of the above embodiments by running the programs stored in the memory 500.

[0139] Specifically, the aforementioned electronic device may also include: a bus, a communication interface 520, an input device 530, and an output device 540.

[0140] The processor 510, memory 500, communication interface 520, input device 530, and output device 540 are interconnected via a bus. Among them: A bus can include a pathway for transmitting information between various components of a computer system.

[0141] The processor 510 can be a general-purpose processor, such as a general-purpose central processing unit (CPU), a microprocessor, etc., or an application-specific integrated circuit (ASIC), or one or more integrated circuits used to control the execution of the program of the present application. It can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0142] The processor 510 may include a main processor, as well as a baseband chip, modem, etc.

[0143] The memory 500 stores a program for executing the technical solution of this application, and may also store an operating system and other critical business functions. Specifically, the program may include program code, which includes computer operation instructions. More specifically, the memory 500 may include read-only memory (ROM), other types of static storage devices capable of storing static information and instructions, random access memory (RAM), other types of dynamic storage devices capable of storing information and instructions, disk storage, flash memory, etc.

[0144] Input device 530 may include a device for receiving data and information input by a user, such as a keyboard, mouse, camera, scanner, light pen, voice input device, touch screen, pedometer, or gravity sensor.

[0145] Output device 540 may include devices that allow information to be output to a user, such as a display screen, printer, speaker, etc.

[0146] The communication interface 520 may include a device that uses any transceiver to communicate with other devices or communication networks, such as Ethernet, Radio Access Network (RAN), Wireless Local Area Network (WLAN), etc.

[0147] The processor 510 executes the program stored in the memory 500 and calls other devices, which can be used to implement the various steps of any of the voice interaction methods provided in the above embodiments of this application.

[0148] This application also proposes a chip, which includes a processor and a data interface. The processor reads and runs a program stored in the memory through the data interface to execute any of the voice interaction methods provided in the above embodiments. For the specific processing procedure and its beneficial effects, please refer to the above-described embodiments of the voice interaction method.

[0149] Exemplary computer program products and storage media In addition to the methods and devices described above, embodiments of this application may also be computer program products, which include computer program instructions that, when executed by a processor, cause the processor to perform the steps in the voice interaction methods according to various embodiments of this application as described in any of the above embodiments of this specification.

[0150] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this application. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0151] Furthermore, embodiments of this application may also be storage media storing a computer program, which is executed by a processor to perform the steps of the voice interaction method according to various embodiments of this application as described in any of the above embodiments of this specification.

[0152] For the foregoing method embodiments, in order to simplify the description, they are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, because according to this application, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0153] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For apparatus embodiments, since they are basically similar to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0154] The steps in the methods of the various embodiments of this application can be adjusted, merged, or deleted in order according to actual needs, and the technical features described in each embodiment can be replaced or combined.

[0155] The modules and sub-modules in the apparatus and terminal in the various embodiments of this application can be merged, divided, and deleted according to actual needs.

[0156] It should be understood that the disclosed terminals, devices, and methods can be implemented in other ways, given the several embodiments provided in this application. For example, the terminal embodiments described above are merely illustrative. For instance, the division of modules or sub-modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple sub-modules or modules may be combined or integrated into another module, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or modules, and may be electrical, mechanical, or other forms.

[0157] The modules or submodules described as separate components may or may not be physically separate. The components that constitute a module or submodule may or may not be physical modules or submodules; that is, they may be located in one place or distributed across multiple network modules or submodules. Some or all of the modules or submodules can be selected to achieve the purpose of this embodiment's solution, depending on actual needs.

[0158] Furthermore, the functional modules or sub-modules in the various embodiments of this application can be integrated into one processing module, or each module or sub-module can exist physically separately, or two or more modules or sub-modules can be integrated into one module. The integrated modules or sub-modules described above can be implemented in hardware or in the form of software functional modules or sub-modules.

[0159] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0160] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software unit executed by a processor, or a combination of both. The software unit can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0161] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0162] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A voice interaction method, characterized in that, include: Obtain the text of the user's voice commands; Determine the text content of interactive elements in the current interactive interface; The text content of the interactive element is processed by combining the attribute information of the interface area to which the interactive element belongs in the interactive interface to generate a hot word library for the current interactive interface. The hot word library includes at least one hot word for each interactive element in the current interactive interface; Identify the target hot word in the hot word library that matches the voice command text, and trigger the interactive element to which the target hot word belongs in the current interactive interface.

2. The method according to claim 1, characterized in that, The acquisition of the user's voice command text includes: Receive the user's original voice command text; Remove non-semantic text from the original voice command text to obtain the voice command text.

3. The method according to claim 1, characterized in that, The step of processing the text content of the interactive element by combining the attribute information of the interface area to which the interactive element belongs in the interactive interface to generate a hot word library for the current interactive interface includes: The text content of the interactive element is segmented to obtain at least one segment of the interactive element; Determine the attribute information of the interface area to which the interactive element belongs in the interactive interface, and the attribute information of the interface area is determined based on the difference information between different interface areas in the current interactive interface. The word segmentation and attribute information are associated to generate hot words corresponding to the word segmentation; the hot words corresponding to the word segmentation of each interactive element in the current interactive interface constitute the hot word library of the current interactive interface.

4. The method according to claim 1, characterized in that, The attribute information of the interface area includes the area type and / or area location of the interface area. The area type is based on the functional definition of the interface area, and the area location is based on the position information of the interface area in the current interactive interface.

5. The method according to claim 4, characterized in that, Determining the attribute information of the interface area to which the interactive element belongs in the interactive interface includes: Identify at least one logical partition in the current interactive interface, where each logical partition corresponds to a UI area; Obtain the title text of each of the at least one logical partitions; The attribute information of each logical partition is determined based on the difference information between the title texts of each logical partition in the at least one logical partition.

6. The method according to claim 4, characterized in that, Determining the attribute information of the interface area to which the interactive element belongs in the interactive interface includes: Identify at least one logical partition in the current interactive interface, where each logical partition corresponds to a UI area; Obtain the position information of each of the at least one logical partitions in the current interactive interface; The attribute information of each logical partition is determined based on the relative position information between the logical partitions in the at least one logical partition.

7. The method according to claim 1, characterized in that, The step of determining the target hot words in the hot word library that match the voice command text includes: Determine the target scene information of the current interactive interface, wherein the target scene information represents the application scene of the current interactive interface; Based on the pre-trained text feature extraction model, the first text feature of the voice command text and the second text feature of each hot word in the hot word library are extracted respectively. Based on the first text features of the voice command text and the second text features of the hot words, the similarity of each hot word in the hot word library is calculated respectively; the similarity of the hot words represents the semantic similarity between the hot words and the voice command text; Determine whether the highest similarity of each of the hot words in the hot word library exceeds the target similarity threshold corresponding to the target scene information; If the highest similarity of each of the hot words in the hot word library exceeds the target similarity threshold corresponding to the target scene information, the hot word to which the highest similarity belongs is determined as the target hot word.

8. The method according to claim 7, characterized in that, The text feature extraction model is obtained in the following way: Distilling the first model yields the second model, where the first model is a pre-set large model and the second model is used for text feature extraction. The sample pair is input into the second model to obtain the first text feature of the first sample and the second text feature of the second sample in the sample pair; Calculate the first semantic similarity between the first sample and the second sample based on the first text features of the first sample and the second text features of the second sample; Based on the first model, a second semantic similarity is determined between the first sample and the second sample in the sample pair; Based on the difference between the first semantic similarity and the second semantic similarity, the second model is optimized and trained to obtain a text feature extraction model.

9. A voice interaction device, characterized in that, include: The voice command text acquisition unit is used to acquire the user's voice command text; The text content determination unit is used to determine the text content of interactive elements in the current interactive interface. The hot word library generation unit is used to process the text content of the interactive element by combining the attribute information of the interface area to which the interactive element belongs in the interactive interface, and generate the hot word library of the current interactive interface. The hot word library includes at least one hot word for each interactive element in the current interactive interface; The voice interaction unit is used to determine the target hot word in the hot word library that matches the voice command text, and to trigger the interactive element to which the target hot word belongs in the current interactive interface.

10. An electronic device, characterized in that, Including memory and processor; The memory is connected to the processor and is used to store programs; The processor is used to implement the voice interaction method as described in any one of claims 1 to 8 by running a program in the memory.

Images