Voice interaction method and related equipment

Abstract

The invention discloses a voice interaction method and related equipment, and relates to the field of vehicle-mounted technology, and the method comprises the following steps: when a hot word containing an embedded external component identifier exists in a visible-to-speak hot word set uploaded by a vehicle-mounted terminal application program, carrying out voice interaction on the embedded external component identifier; based on a barrier-free service, scanning an embedded external component of an interface corresponding to the hot word in the vehicle-mounted terminal application program to obtain a text set; determining a target text based on the text set, and binding the target text with the embedded external component identifier; and based on the target text, the embedded external component identifier and the barrier-free service, interaction between a target voice and an interface of the vehicle-mounted terminal application program is carried out.

Description

Technical Field

[0001] This application relates to the field of automotive in-vehicle technology, and in particular to a voice interaction method and related equipment. Background Technology

[0002] In the field of in-vehicle voice interaction, related technologies mainly rely on two independent interface control response schemes: registration-based visibility and accessibility-based visibility. Registration-based visibility, or the speaking mechanism, uses an integrated Software Development Kit (SDK) to scan native controls (such as buttons and text labels) within the vehicle's infotainment system, generating visible hotwords and uploading them to the voice service. While this solution achieves accurate control responses without requiring system permissions, it cannot recognize embedded external components, such as widget controls (desktop widgets) or embedded web content (Web content) like HTML5 pages, resulting in incomplete functionality. Summary of the Invention

[0003] The embodiments of this application provide a voice interaction method and related devices, which can at least to some extent enable the response to embedded external components in the vehicle system.

[0004] The summary section introduces a series of simplified concepts, which will be further explained in detail in the detailed description section. This summary section is not intended to limit the key and essential technical features of the claimed technical solution, nor is it intended to determine the scope of protection of the claimed technical solution.

[0005] This application specifically includes the following aspects: Firstly, this application proposes a voice interaction method, the method comprising: If a hot word containing an embedded external component identifier exists in the set of visible and speakable hot words uploaded by the vehicle infotainment application, the embedded external components of the interface corresponding to the hot word in the vehicle infotainment application are scanned based on the accessibility service to obtain a text set. The target text is determined based on the text set, and the target text is bound to the embedded external component identifier; The interaction between the target voice and the interface of the vehicle-mounted application is based on the target text, the embedded external component identifier, and the accessibility service.

[0006] In one feasible implementation, the interaction between the target voice and the interface of the vehicle-mounted application based on the target text, the embedded external component identifier, and the accessibility service includes: When the speech recognition result of the target speech includes the target text and the embedded external component identifier, the control corresponding to the target text in the interface of the vehicle system application is responded to based on the accessibility service.

[0007] In one feasible implementation, after scanning the embedded external components of the interface corresponding to the hot words in the vehicle-mounted application based on accessibility services to obtain the text set, the method further includes: The set of visible and talkable hot words is merged with the set of text to obtain the target set; The interaction between the target voice and the interface of the vehicle-mounted application is performed based on the target set.

[0008] In one feasible implementation, the method further includes: No generalization processing is performed for the control type corresponding to the embedded external component identifier.

[0009] In one feasible implementation, the method further includes: If the speech recognition result of the target speech includes hot words from the set of visible and speakable hot words, the controls corresponding to the hot words in the set of visible and speakable hot words are controlled to respond based on the registration-based visible and speakable software development kit integrated into the vehicle application.

[0010] In one feasible implementation, the voice interaction method further includes: Identify the control type corresponding to the hot words in the visible and talkable hot word set; Based on a preset control type mapping table, determine the application internal component identifier corresponding to the control type; The internal component identifiers of the application are bound to the corresponding hot words in the set of visible and talkable hot words.

[0011] In one feasible implementation, the method further includes: For each control type corresponding to the internal component identifier of the application, different generalization processing is performed based on the preset control type mapping table.

[0012] Secondly, this application proposes a voice interaction system, applied to the voice interaction method described in any of the above embodiments, comprising: The interface scanning module is used to scan the embedded external components of the interface corresponding to the hot words in the vehicle system application based on accessibility services when there are hot words containing embedded external component identifiers in the set of visible and speakable hot words uploaded by the vehicle system application. An identifier binding module is used to determine target text based on the text set and bind the target text to the embedded external component identifier; The response execution module is used to perform interaction between the target voice and the interface of the vehicle-mounted application based on the target text, the embedded external component identifier, and the accessibility service.

[0013] Thirdly, an electronic device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program stored in the memory to implement the steps of the voice interaction method as described in any of the first aspects above.

[0014] Fourthly, this application also proposes a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the voice interaction method of any of the first aspects.

[0015] This application proposes a voice interaction method that enables an application that has already integrated a registration-based "see-and-talk" software development kit to achieve seamless and complete "see-and-talk" functionality when encountering embedded external components such as Widget controls or H5 pages, while maintaining the excellent experience of the registration-based solution on ordinary controls as much as possible.

[0016] This application proposes a voice interaction method and related device. Other advantages, objectives and features of this application will be partly apparent from the following description, and partly understood by those skilled in the art through study and practice of this application. Attached Figure Description

[0017] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit this specification. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 A flowchart illustrating a voice interaction method provided in an embodiment of this application; Figure 2 A flowchart illustrating a registration-based, visible-and-speakable software development kit (SDK) process is provided for embodiments of this application. Figure 3 This application provides a response routing decision graph based on control type. Figure 4 A schematic diagram of the functional modules of a voice interaction system provided in an embodiment of this application; Figure 5This is a schematic diagram of the structure of a voice interaction control device provided in an embodiment of this application. Detailed Implementation

[0018] To better understand the technical solutions provided in the embodiments of this specification, the technical solutions of the embodiments of this specification will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments of this specification and the specific features in the embodiments are detailed descriptions of the technical solutions of the embodiments of this specification, rather than limitations on the technical solutions of this specification. In the absence of conflict, the embodiments of this specification and the technical features in the embodiments can be combined with each other.

[0019] In this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, without necessarily requiring or implying any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. The term "two or more" includes two or more cases.

[0020] Please see Figure 1 This is a flowchart illustrating a voice interaction method provided in an embodiment of this application, which may specifically include: S110. If there are hot words containing embedded external component identifiers in the set of visible and speakable hot words uploaded by the vehicle infotainment application, scan the embedded external components of the interface corresponding to the hot words in the vehicle infotainment application based on the accessibility service to obtain the text set.

[0021] For example, when the in-vehicle infotainment application uploads a set of "visible and speakable" hot words, the system first checks whether the set contains an embedded external component identifier (such as specialView: "H5" or specialView: "Widget"). If the identifier exists, it means that the current interface contains special controls (such as buttons or widget components in an H5 page) that cannot be scanned by the registered "visible and speakable" software development kit (SDK). At this time, the system scans the interface corresponding to the hot words for embedded external components through accessibility services, obtains the text information of all embedded external components, and generates a text set.

[0022] For example, a user opens an "Online Music" application integrated with the registered "Visible and Speakable" SDK in the car's infotainment system. The application embeds the `android:contentDescription="SpectialView:H5"` field into the corresponding H5 interface controls on this page. The application's uploaded "Visible and Speakable" hot word set contains the embedded external component identifier `specialView: "H5"`. Upon detecting this identifier, the voice service immediately invokes the accessibility service to scan the current interface, obtaining text such as "Play All" and "Favorite Playlist" from the H5 module, generating a text set `[{text="Play All", prompt="H5"}, {text="Favorite Playlist", prompt="H5"}, {text="Share", prompt="H5"}]`. Through the triggering mechanism of the embedded external component identifier, "on-demand scanning" is achieved, activating the accessibility service only when special controls are present, avoiding invalid scanning of ordinary interfaces. This ensures coverage of special scenarios while reducing system resource consumption.

[0023] For example, the accessibility service iterates through the Widgets / H5 controls of the current interface to obtain the text information of each control (such as the control display text, content description, etc.), and then the obtained text information is deduplicated, invalid text (such as empty strings) is filtered, and then summarized into a structured text collection.

[0024] For example, when the accessibility service scans an H5 page, it comes across a button control whose displayed text is "Next Song" and whose content description is "Switch to the next song". The system extracts the core text "Next Song" and adds it to the text collection; for icon controls with no displayed text but whose content description is "Refresh", "Refresh" is extracted and added to the collection, and finally the text collection ["Next Song", "Refresh", "Pause"] is generated.

[0025] In summary, by clearly defining the text extraction rules for accessibility scanning, it is ensured that even if the control does not display text (only through content description), it can still be identified as a valid hot word, thus expanding the coverage of "what you see is what you can say," which is especially suitable for interfaces with many icon-based controls in in-vehicle scenarios.

[0026] S120. Determine the target text based on the text set, and bind the target text to the embedded external component identifier.

[0027] In some examples, after scanning the embedded external components of the interface corresponding to hot words in the in-vehicle infotainment application based on accessibility services to obtain the text set, it also includes: The target set is obtained by fusing the set of visible and talkable hot words with the text set; The interaction between the target voice and the interface of the vehicle-mounted application is based on the target set.

[0028] For example, the system merges the set of visible and speakable hot words uploaded by the application (the text of ordinary controls scanned by the registration-based visible and speakable SDK) with the text set obtained from accessibility scanning (the text of special controls). When the voice service receives the formatted string of visible hot words uploaded by the application, it first parses whether the string contains the specialView field. If a special identifier is detected, it does not upload directly, but first uses the accessibility service to scan and traverse the current page. During the traversal, it first reads the mContentDescription field of the node (AccessibilityNodeInfo). If the value of the accessibility description field in the node is equal to SpecialView:H5 or SpecialView:Widget, it means that the node is a Widget control or H5 interface in the application. It is necessary to traverse the content of special nodes. During the traversal of the identified special nodes, if the leaf nodes under the node have visible special features, such as UI text with click events or mapped text (such as icons mapped to text), they need to be appended to the formatted set of visible hot words mentioned above.

[0029] In some examples, the control type corresponding to the embedded external component identifier is not generalized.

[0030] For example, before adding visible text to the set of visible hot words, it is necessary to first identify a control type for the text based on the special annotation of the scanning node, such as prompt="Widget" or prompt="H5". The algorithm will perform generalization or non-generalization processing for these two control types.

[0031] In some examples, for the control type corresponding to the internal component identifier of the application, different generalization processing is performed for each control type based on the preset control type mapping table.

[0032] For example, control type is a way to generalize the visibility of UI controls in a registration-based visibility scheme. For instance, if there's a switch control called "Bluetooth" on the UI, the uploaded data would be `{prompt="Switch", text="Bluetooth"}`. `prompt="Switch"` means the text "Bluetooth" is a switch. On the voice side, the text "Bluetooth" will be generalized, resulting in various visibility expressions such as "{on|on|off|off|turned off|turned off}Bluetooth", etc. Different prompts have different generalizations.

[0033] In some examples, when the speech recognition result of the target speech includes hot words from the Visible and Speakable Hot Words set and the identifiers of internal application components, the controls corresponding to the hot words in the Visible and Speakable Hot Words set are controlled based on the registration-based Visible and Speakable Software Development Kit integrated into the vehicle application.

[0034] For example, each hot word in the "Visible and Speakable" hot word set is pre-bound to an internal application component identifier (such as a control ID, resource path, or custom attribute identifier). This identifier is used to uniquely locate the corresponding visual control in the application interface. When the user issues a target voice command, if the voice recognition result contains both the hot word text and the corresponding internal application component identifier, the system will directly call the registration-based "Visible and Speakable" software development kit (SDK) integrated into the vehicle's infotainment application. The registration-based "Visible and Speakable" SDK will then locate the control based on the identifier and execute the response operation (such as clicking, switching, etc.). By establishing a direct mapping between hot words and controls through the internal application component identifier, cross-process communication or external service intervention is avoided, ensuring that the response process is completed in a closed loop within the application.

[0035] For example, when the "Driving Preferences" interface loads, the registered visible and speakable software development kit (SDK) automatically scans the interface controls, binds the hot word "economy mode" to the application's internal component identifier "btn_economy_mode" (corresponding to the resource ID of the "economy mode" button in the interface), and uploads it to the voice service. When the user says "Switch to economy mode" while driving, the voice recognition result includes the hot word "economy mode" and the component identifier "btn_economy_mode". The voice service sends the result to the application, and the SDK directly locates the "economy mode" button through the identifier "btn_economy_mode", calls the application's internal interface to execute the click operation, and the interface immediately switches to economy mode, displaying a "Switched to economy mode" feedback animation.

[0036] By directly locating controls using internal component identifiers, the cross-process scanning and gesture simulation steps of the accessibility service are eliminated, significantly reducing response latency. Simultaneously, the identifiers are bound one-to-one with controls, avoiding false responses caused by repetitive interface text (such as multiple "OK" buttons), thus significantly reducing the error rate. The response process is executed by the application's internal SDK, supporting application-customized control animations (such as button click feedback and state transition animations), providing a user experience completely consistent with manual operation.

[0037] In some examples, it also includes: Identify the control type corresponding to the hot words in the visible and speakable hot word set; Based on the preset control type mapping table, determine the application internal component identifier corresponding to the control type; Bind the application's internal component identifiers to the corresponding hot words in the visible and speakable hot word set.

[0038] S130: Interaction between target voice and vehicle infotainment application interface based on target text, embedded external component identification, and accessibility services.

[0039] In some examples, the interaction between the target voice and the in-vehicle application interface is based on the target text, embedded external component identifiers, and accessibility services, including: When the speech recognition result of the target speech includes the target text and the identifier of the embedded external component, the control corresponding to the target text in the interface of the vehicle-mounted application based on the accessibility service responds.

[0040] For example, for embedded external components (such as Widgets / H5 controls) that cannot be covered by the registration-based visible and speakable software development kit, the system binds embedded external component identifiers (such as "prompt=H5 or Widget") to their hotwords to distinguish between internal and external components. When the target speech recognition result contains both the external component hotword text and the embedded external component identifier, the system invokes the accessibility service mechanism to respond by scanning interface controls and simulating user gestures. The response path is distinguished by the identifier: internal components (registration-based visible and speakable software development kit) and embedded external components (accessibility service) each perform their respective functions, retaining the efficiency of the registration-based scheme while supplementing the interactive capabilities of embedded external components through the accessibility service.

[0041] Taking the H5 music application integrated into the vehicle's infotainment system as an example (H5 pages are embedded external components, which cannot be directly scanned by the registration-based SDK): Embedded external component identifier binding: When the H5 page loads, the voice service scans the interface through the accessibility service, identifies the "Next Song" button (a hot word), and binds it to the embedded external component identifier "external_h5_next_song" (identifying it as an H5 external component). When the user says "Next Song," the voice recognition result includes the hot word "Next Song" and the identifier "external_h5_next_song." The system determines that the identifier is an external component type, calls the accessibility service to scan the current H5 interface, locates the screen coordinates of the "Next Song" button through the identifier, simulates the user's finger tap gesture (coordinate accuracy up to ±2 pixels), and triggers the song switching operation.

[0042] By filling the gap in support for embedded external components such as Widgets / H5 in the registration-based solution, the "what you see is what you speak" functionality achieves 100% interface coverage in the vehicle's infotainment system. Simultaneously, through an automatic switching response mechanism (registration-based / accessibility-based) using embedded external component identifiers, users no longer need to distinguish between control types and can interact uniformly via voice commands, reducing operational complexity. Furthermore, this application only invokes the accessibility service when an external component identifier is recognized, avoiding invalid scanning of internal application components and reducing system resource consumption.

[0043] The technical solution of this application will be further described in detail below through specific embodiments.

[0044] If a page in the vehicle infotainment application contains special interfaces such as widgets or H5 web pages, the page needs to be specially marked in advance so that the application includes the marked information in the uploaded visible hot words. This information is used to notify the voice service interface whether it contains special pages or controls. If there are special controls, the accessibility service needs to be used to scan the page to obtain the UI text of these special controls or interfaces.

[0045] like Figure 2 As shown, when a set of visible hot words for registration-based visible upload is received, if a special identifier is detected, the upload will not be performed directly. Instead, the accessibility service will be used to scan the current page to obtain a set of UI text, which will include the visible text scanned from the Widget / H5 interface.

[0046] The text set is appended to the visible hot word set of the registration-based visible upload before uploading, generating a new set to continue the upload process.

[0047] The voice ASR / NLU service does not perform any generalization processing on visible hot words with control types of Widget or H5; the control type is still returned as is when uploaded. Here, control type is a way of generalizing UI controls. For example, if there is a switch control called Bluetooth on the UI, the uploaded data would be {prompt="Switch", text="Bluetooth"}. Here, prompt="Switch" means that the text "Bluetooth" is a switch. On the voice side, the text "Bluetooth" will be generalized to various visible expressions such as "{on|on|off|off|turn off|turned off}Bluetooth".

[0048] like Figure 3 As shown, when the voice service determines that the control type in the visible result is a Widget or H5, it sends the visible result to the accessibility service of the voice service for a response. The accessibility service can then simulate a gesture click on the screen to respond. For control types that are not Widgets or H5, the visible result is sent to the application, which then handles the response in the application service.

[0049] Furthermore, this application also proposes a voice interaction system, applied to an embodiment of any of the above-mentioned voice interaction methods, specifically as follows: Figure 4 The diagram shown is a functional module diagram of a voice interaction system proposed in this application, including: The interface scanning module 21 is used to scan the embedded external components of the interface corresponding to the hot words in the vehicle system application based on accessibility services when there are hot words containing embedded external component identifiers in the set of visible and speakable hot words uploaded by the vehicle system application. The identifier binding module 22 is used to determine the target text based on the text set and bind the target text to the embedded external component identifier; The response execution module 23 is used to enable interaction between the target voice and the interface of the vehicle-mounted application based on the target text, embedded external component identifiers, and accessibility services.

[0050] It should be noted that the above embodiments are merely best examples and are not intended to limit the implementation of this application.

[0051] Furthermore, such as Figure 5 As shown, this application embodiment also provides an electronic device 300, including a processor 310, a memory 320, and a computer program 321 stored in the memory 320 and executable on the processor. When the processor 310 executes the computer program 321, it implements the steps of any of the above-mentioned voice interaction methods.

[0052] Since the electronic device described in this embodiment is a device used to implement a voice interaction method in the embodiments of this application, those skilled in the art can understand the specific implementation method and various variations of the electronic device in this embodiment based on the method described in the embodiments of this application. Therefore, how the electronic device implements the method in the embodiments of this application will not be described in detail here. Any device used by those skilled in the art to implement the method in the embodiments of this application falls within the scope of protection of this application.

[0053] In practical implementation, when the computer program 321 is executed by the processor, it can achieve the following: Figure 1 Any of the corresponding implementation methods in the embodiments.

[0054] It should be noted that the descriptions of each embodiment in the above embodiments have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0055] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-readable program code.

[0056] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0057] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0058] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0059] This application also provides a computer program product, which includes computer software instructions that, when executed on a processing device, cause the processing device to execute a voice interaction method.

[0060] A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0061] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0062] In the several embodiments provided in this application, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between devices or units, and may be electrical, mechanical, or other forms.

[0063] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0064] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0065] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0066] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

[0067] Although preferred embodiments have been described in this specification, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this specification.

[0068] Obviously, those skilled in the art can make various modifications and variations to this specification without departing from its spirit and scope. Therefore, if such modifications and variations fall within the scope of the claims and their equivalents, this specification is also intended to include such modifications and variations.

Claims

1. A voice interaction method, characterized in that, The method includes: If a hot word containing an embedded external component identifier exists in the set of visible and speakable hot words uploaded by the vehicle infotainment application, the embedded external components of the interface corresponding to the hot word in the vehicle infotainment application are scanned based on the accessibility service to obtain a text set. The target text is determined based on the text set, and the target text is bound to the embedded external component identifier; The interaction between the target voice and the interface of the vehicle-mounted application is based on the target text, the embedded external component identifier, and the accessibility service.

2. The voice interaction method according to claim 1, characterized in that, The interaction between the target voice and the interface of the vehicle-mounted application based on the target text, the embedded external component identifier, and the accessibility service includes: When the speech recognition result of the target speech includes the target text and the embedded external component identifier, the control corresponding to the target text in the interface of the vehicle system application is responded to based on the accessibility service.

3. The voice interaction method according to claim 1, characterized in that, After scanning the embedded external components of the interface corresponding to the hot words in the vehicle-mounted application based on accessibility services to obtain the text set, the process further includes: The set of visible and talkable hot words is merged with the set of text to obtain the target set; The interaction between the target voice and the interface of the vehicle-mounted application is performed based on the target set.

4. The voice interaction method according to claim 1, characterized in that, The method further includes: No generalization processing is performed for the control type corresponding to the embedded external component identifier.

5. The voice interaction method according to claim 1, characterized in that, The method further includes: If the speech recognition result of the target speech includes hot words from the set of visible and speakable hot words, the controls corresponding to the hot words in the set of visible and speakable hot words are controlled to respond based on the registration-based visible and speakable software development kit integrated into the vehicle application.

6. The voice interaction method according to claim 5, characterized in that, The method further includes: Identify the control type corresponding to the hot words in the visible and talkable hot word set; Based on a preset control type mapping table, determine the application internal component identifier corresponding to the control type; The internal component identifiers of the application are bound to the corresponding hot words in the set of visible and talkable hot words.

7. The voice interaction method according to claim 6, characterized in that, The method further includes: For each control type corresponding to the internal component identifier of the application, different generalization processing is performed based on the preset control type mapping table.

8. A voice interaction system, applied to the voice interaction method according to any one of claims 1 to 7, characterized in that, include: The interface scanning module is used to scan the embedded external components of the interface corresponding to the hot words in the vehicle system application based on accessibility services when there are hot words containing embedded external component identifiers in the set of visible and speakable hot words uploaded by the vehicle system application. An identifier binding module is used to determine target text based on the text set and bind the target text to the embedded external component identifier; The response execution module is used to perform interaction between the target voice and the interface of the vehicle-mounted application based on the target text, the embedded external component identifier, and the accessibility service.

9. An electronic device, comprising: The memory and processor are characterized in that the processor, when executing a computer program stored in the memory, implements the steps of the voice interaction method as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the voice interaction method as described in any one of claims 1 to 7.

Images