Visible-speaking voice interaction method and device for vehicle-mounted system

Abstract

The invention discloses a visible-to-speak voice interaction method for a vehicle-mounted system. The method comprises the following steps: converting a voice request for interaction between a user and the vehicle-mounted system into a trigger text; according to the scanning rule of the current UI interface, judging whether an application corresponding to the current UI interface supports scanning or not; when the application corresponding to the current UI supports scanning, judging whether the trigger text is matched with the text corresponding to the special function of the UI or not; if yes, corresponding click operation is executed according to the UI special function matched with the trigger text and the scanning rule of the current UI; and if not, executing the corresponding click operation according to the scanning rule of the current UI interface. Function configuration needing to be used in the scanning process is defined through a pre-loaded scanning rule of the current UI; then, in a voice interaction process, judging a click operation corresponding to the trigger text according to a scanning rule; therefore, the levels of scanning the UI interface, the scanning time and the consumption of the CPU can be reduced, and the optimization of the scanning performance consumption is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle-mounted voice interaction, and in particular to a visible-and-spoken voice interaction method and device for a vehicle-mounted system. Background Art

[0002] During driving, when users need to use certain functions of the vehicle system, manually clicking the relevant buttons is inconvenient and has safety issues. Therefore, many current vehicle systems have implemented the see-and-speak function controlled by voice. See-and-speak is an auxiliary skill of the voice operation apk under the vehicle system. Users can see the text information in the vehicle system and speak the text content through voice. The voice assistant in the vehicle system will simulate clicking on the corresponding text content to help users click this button, thereby realizing the function of users not having to click on the clickable buttons in the vehicle system by hand. With see-and-speak, you only need to communicate with the text above the button to perform related function operations. This method makes operation more convenient, especially during driving, which can reduce the distraction and attention of the driver, improve driving safety, and also provide better user experience and development convenience.

[0003] The existing visible and speakable functions are divided into visible and speakable functions for some functional events on the local UI interface of the vehicle system and visible and speakable functions for all functional events on all UI interfaces of the vehicle system. However, when some functional events defined by users on the local UI interface of the vehicle system do not need to use the visible and speakable function, since different functional events need to be controlled in different UI interfaces, dynamic control cannot be achieved, which leads to increased CPU performance consumption during the operation and scanning of the barrier-free service, increased operation time consumption, and the inability to realize the dynamic configuration of the function page scanning interface.

[0004] Based on this, a new solution is needed. Summary of the invention

[0005] The main purpose of the present invention is to provide a visible-can-be-spoken voice interaction method and device for an in-vehicle system.

[0006] To achieve the above object, the present invention provides a visible-speakable voice interaction method for an in-vehicle system, comprising the following steps:

[0007] Convert the user's voice request for interaction with the vehicle system into trigger text;

[0008] According to the scanning rules of the current UI interface, determine whether the application corresponding to the current UI interface supports scanning;

[0009] When the application corresponding to the current UI interface supports scanning, determining whether the trigger text matches the text corresponding to the special function of the UI interface;

[0010] When the trigger text matches the text corresponding to the special function of the UI interface, a corresponding click operation is performed according to the special function of the UI interface matched by the trigger text and the scanning rule of the current UI interface;

[0011] When the trigger text does not match the text corresponding to the special function of the UI interface, the corresponding click operation is performed according to the scanning rule of the current UI interface.

[0012] In the visible-can-speak voice interaction method for an in-vehicle system provided by the present invention, if it is a multi-screen vehicle machine mode, then when the application corresponding to the current UI interface supports scanning, before the step of determining whether the trigger text matches the special function of the UI interface, it also includes:

[0013] A trigger screen ID is acquired according to a user trigger position, wherein the click operation is performed on a screen corresponding to the trigger screen ID.

[0014] In the visible-can-be-spoken voice interaction method for an in-vehicle system provided by the present invention, before the step of converting the voice request for interaction between the user and the in-vehicle system into a trigger text, the method further includes:

[0015] Load the configuration file that includes the scan rules of the current UI interface.

[0016] In the visible-and-speakable voice interaction method for an in-vehicle system provided by the present invention, the special functions of the UI interface include page turning, system buttons, control ID, absolute position and relative position.

[0017] In the visible-and-speakable voice interaction method for a vehicle-mounted system provided by the present invention, when the trigger text matches the text corresponding to the page turning function, it is determined whether the current UI interface supports text scanning of invisible controls according to the scanning rules of the current UI interface. If so, click to execute sliding page turning.

[0018] In the visible-and-spoken voice interaction method for an in-vehicle system provided by the present invention, when the trigger text matches the text corresponding to the system button function, it is determined whether the current UI interface supports the use of the system function according to the scanning rules of the current UI interface. If so, the corresponding system button is clicked according to the trigger text.

[0019] In the visible-and-spoken voice interaction method for an in-vehicle system provided by the present invention, when the trigger text matches the text corresponding to the control ID function, the node information of the control is obtained according to the control ID corresponding to the trigger text, and it is determined whether the control is a clickable control based on the obtained node information. If so, a click operation is performed on the control; if not, the parent control of the control is obtained for click judgment.

[0020] In the see-and-speak voice interaction method for an in-vehicle system provided by the present invention, when the trigger text matches the text corresponding to the absolute position function, it is determined whether the current UI interface supports non-clickable text scanning according to the scanning rule of the current UI interface, and if so, the position to be clicked is obtained according to the trigger text, and a click operation is performed at the position to be clicked;

[0021] When the trigger text matches the text corresponding to the relative position function, determine whether the current UI interface supports non-clickable text scanning according to the scanning rules of the current UI interface. If so, obtain the position to be clicked and the relative control position according to the trigger text, and perform the click operation according to the position to be clicked and the relative control position.

[0022] In the visible-and-speakable voice interaction method for an in-vehicle system provided by the present invention, when the trigger text does not match the text corresponding to the special function of the UI interface, the step of performing a corresponding click operation according to the scanning rule of the current UI interface includes:

[0023] Recursively obtain the node information of the control corresponding to the trigger text on the current UI interface according to the text;

[0024] According to the scanning rules of the current UI interface and the obtained node information, it is determined whether the node is a scannable node. If so, it is determined whether the control is a clickable control according to the obtained node information. If so, a click operation is performed on the control.

[0025] In addition, to achieve the above-mentioned purpose, the present invention also provides a visible-speakable voice interaction device for an in-vehicle system, comprising:

[0026] Memory for storing computer programs;

[0027] A processor is used to implement the steps of the visible-and-speakable voice interaction method for an in-vehicle system as described above when executing the computer program.

[0028] In addition, to achieve the above-mentioned purpose, the present invention also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps of the visible-and-speakable voice interaction method for an in-vehicle system as described above are implemented.

[0029] In the present invention, the function configuration required for the scanning process is defined by pre-loading the scanning rules of the current UI interface; then, during the voice interaction process, the click operation corresponding to the triggered text is determined according to the scanning rules; thereby, the level of the scanning UI interface and the time and CPU consumption of the scanning process can be reduced, the scanning performance consumption can be optimized, and the stability of the see-and-speak execution function can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art are briefly introduced below. Obviously, the drawings in the following description are only embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on the provided drawings without creative work:

[0031] Figure 1 It is a flowchart of a visible-and-speakable voice interaction method for an in-vehicle system provided by the first embodiment of the present invention;

[0032] Figure 2 The following is a schematic diagram of the UI interface;

[0033] Figure 3 Shown is a schematic diagram of another UI interface. DETAILED DESCRIPTION

[0034] In order to facilitate understanding of the present invention, the present invention will be described more fully below with reference to the relevant drawings. Typical embodiments of the present invention are given in the drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present invention more thorough and comprehensive.

[0035] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0036] The general idea of ​​the present invention is: in view of the existing technology, when some functional events defined by users on the local UI interface of the vehicle-mounted system do not need to use the visible-and-speak function, there is an increase in CPU performance consumption during the scanning process of the barrier-free service, as well as an increase in running time consumption and an inability to realize the dynamic configuration of the function page scanning interface. A visible-and-speak voice interaction method for a vehicle-mounted system is provided, which defines the functional configuration to be used in the scanning process through pre-loaded scanning rules of the current UI interface; then, in the voice interaction process, the click operation corresponding to the trigger text is determined according to the scanning rules; thereby, the levels of the scanning UI interface and the time and CPU consumption of the scanning process can be reduced, the scanning performance consumption can be optimized, and the stability of the visible-and-speak execution function can be improved.

[0037] In order to better understand the above-mentioned technical scheme, the above-mentioned technical scheme will be described in detail below in conjunction with the accompanying drawings and specific implementation methods. It should be understood that the embodiments of the present invention and the specific features in the embodiments are detailed descriptions of the technical scheme of the present application, rather than limitations on the technical scheme of the present application. In the absence of conflict, the embodiments of the present invention and the technical features in the embodiments can be combined with each other.

[0038] Embodiment 1

[0039] Reference Figure 1 , Figure 1 The flowchart of an embodiment of the visible-can-be-spoken voice interaction method for an in-vehicle system of the present invention is shown in FIG. In one embodiment, the visible-can-be-spoken voice interaction method for an in-vehicle system includes:

[0040] Step S10, loading a configuration file including scanning rules of the current UI interface.

[0041] Specifically, in one embodiment of the present invention, a configuration file of the scanning rules of the current UI interface is loaded in advance through the configuration words written by the developer in the file, wherein the configuration file of the scanning rules of the current UI interface defines the functional configuration that needs to be used during the scanning process. The configuration file of the scanning rules of the current UI interface includes the following judgment logic: whether the application corresponding to the current UI interface needs to support scanning; whether the current UI interface needs to support text scanning of invisible controls; whether the corresponding UI node needs to be scanned; whether it needs to support non-clickable text scanning; whether it needs to support the use of system buttons such as up and down flipping, returning, etc.; and scanning the corresponding screen UI according to the user input direction in the multi-screen car mode. Through this functional configuration, functional events that developers do not need to implement during scanning can be defined to achieve non-execution of scanning operations.

[0042] Step S20: converting the voice request of the user for interacting with the vehicle-mounted system into a trigger text.

[0043] Specifically, in one embodiment of the present invention, it can be understood that converting a voice request into text information is a prior art in the art, including but not limited to steps such as preprocessing, feature extraction, and voice recognition.

[0044] Step S30, judging whether the application corresponding to the current UI interface supports scanning according to the scanning rules of the current UI interface, if the application corresponding to the current UI interface supports scanning, proceeding to step S40, if the application corresponding to the current UI interface does not support scanning, ending.

[0045] Specifically, in one embodiment of the present invention, since there are multiple applications in the vehicle system, the scanning rules of the pre-loaded UI interface can be used to define whether a certain application supports scanning. Therefore, after waking up a UI interface through a trigger text, the configuration word must first be used to query whether the application corresponding to the UI interface supports scanning. If it supports scanning, the subsequent matching process is performed. If it does not support scanning, the wake-up is terminated to reduce scanning time and unnecessary resource consumption.

[0046] Step S40, determine whether the trigger text matches the text corresponding to the special function of the UI interface. When the trigger text matches the text corresponding to the special function of the UI interface, proceed to step S50; when the trigger text does not match the text corresponding to the special function of the UI interface, proceed to step S60.

[0047] Specifically, in one embodiment of the present invention, in order to reduce the scanning time, the buttons with special functions in the UI interface are grouped together to form a text set including the special functions of the UI interface. When the trigger text matches the text corresponding to the special function of the UI interface, the click operation is performed directly according to the special function of the UI interface.

[0048] Furthermore, in one embodiment of the present invention, the special functions of the UI interface include page turning, system buttons, control IDs, absolute positions, and relative positions. For example, when the user's voice request is to turn up, even if the words "turn up" are displayed on the current UI interface, the page turning function control will be matched instead of clicking the location of the text for turning up. In this way, the special functions of the UI interface can be identified and triggered more quickly, improving the user's interaction efficiency.

[0049] Step S50, performing a corresponding click operation according to the special function of the UI interface that triggers the text matching and the scanning rules of the current UI interface.

[0050] Specifically, in one embodiment of the present invention, since the control corresponding to the page turning function is usually an invisible control, ordinary visible or audible functions can only click on text within the visible range, so it is necessary to define in the scanning rules of the loaded UI interface whether the current UI interface supports text scanning of invisible controls. Therefore, when the trigger text matches the text corresponding to the page turning function, according to the scanning rules of the current UI interface, it is determined whether the current UI interface supports text scanning of invisible controls. If so, click to execute sliding page turning. For example, the text corresponding to the page turning function is "flip up" and "flip down". When the user says "flip up", the system first determines whether the application corresponding to the current UI interface supports scanning. If it does, it is found through matching that "flip up" matches the page turning function; continue to query the scanning rules of the current UI interface, and when the current UI interface supports text scanning of invisible controls, click the page turning control.

[0051] Specifically, in one embodiment of the present invention, the system button is usually a button common to the entire system, for example, the return button of the Android system, the ordinary visible and utterable function can only click the text within the visible range, so it is necessary to define whether the current UI interface supports the system function in the scanning rules of the loaded UI interface. If the system function is supported, the system button is directly clicked when the user requests it as a system button, otherwise it is treated as a normal trigger text. Therefore, when the trigger text matches the text corresponding to the system button function, according to the scanning rules of the current UI interface, it is determined whether the current UI interface supports the use of the system function. If so, the corresponding system button is clicked according to the trigger text. For example, the text corresponding to the return function is "return" or "return to the upper layer", then when the user says "return", the system first determines whether the application corresponding to the current UI interface supports scanning. If it does, it is found through matching that "return" matches the system return function; continue to query the scanning rules of the current UI interface, and click the return control when the current UI interface supports the system function.

[0052] Specifically, in one embodiment of the present invention, in the UI interface, each control usually has a corresponding ID, so corresponding matching texts can be set for different control IDs. Therefore, when the trigger text matches the text corresponding to the control ID function, the node information of the control is obtained according to the control ID corresponding to the trigger text; then, it is determined whether the control is a clickable control based on the obtained node information. The node information of each control contains the property of whether the control is a clickable control. If the control is determined to be a clickable control through the node information, a click operation is performed on the control; if not, the parent control of the control is obtained for click judgment, that is, the node information of the parent node is used to determine whether it has a clickable property. If so, the parent node control is clicked, otherwise the parent node of the parent node is obtained, and so on.

[0053] Specifically, in one embodiment of the present invention, controls and text at some locations on the UI interface overlap (for example, Figure 2 "Driver Unlock" shown), some are not overlapped (for example, Figure 2 The "automatic unlocking" shown in the Figure 3 In addition, in the UI interface, the position of some text is not always constant, but can be relatively moved according to the UI interface. For example, Figure 3 In the "high contrast mode" shown, if the HUD switch above is turned off, the buttons below the HUD will be automatically hidden, and the "high contrast mode" will move up. Therefore, in this application, for the case where the text and the control do not overlap and the position of the text does not change, the text corresponding to the control is matched to the control through the absolute position function, that is, when the trigger text of "automatic unlocking" is recognized, the control next to it is clicked instead of the "automatic unlocking" text itself; for the case where the text and the control do not overlap and the position of the text changes, the text corresponding to the control is matched to the control through the relative position function. Therefore, when the trigger text matches the text corresponding to the absolute position function, according to the scanning rules of the current UI interface, it is determined whether the current UI interface supports non-clickable text scanning. If so, the position to be clicked is obtained according to the trigger text, and the click operation is performed at the position to be clicked. When the trigger text matches the text corresponding to the relative position function, according to the scanning rules of the current UI interface, it is determined whether the current UI interface supports non-clickable text scanning. If so, the position to be clicked and the relative control position are obtained according to the trigger text, and the click operation is performed according to the position to be clicked and the relative control position.

[0054] Step S60: executing a corresponding click operation according to the scanning rule of the current UI interface.

[0055] Specifically, in one embodiment of the present invention, when the trigger text is ordinary text (for example, Figure 2 When the "main driver unlocks" shown in the figure does not match the text corresponding to the special function of the UI interface, the node information of the control corresponding to the trigger text on the current UI interface is recursively obtained according to the text; according to the scanning rules of the current UI interface and the obtained node information, it is determined whether the node is a scannable node; if so, it is determined whether the control is a clickable control according to the obtained node information; if so, a click operation is performed on the control.

[0056] Furthermore, in one embodiment of the present invention, if a vehicle includes multiple screens, different screen IDs are defined for different screens in the scanning rules of the UI interface, and when determining that the application corresponding to the current UI interface supports scanning according to the scanning rules of the current UI interface, the trigger screen ID is first obtained according to the user trigger position, and subsequent click actions on the screen are performed on the screen corresponding to the trigger screen ID. Thus, when there are multiple screens in the vehicle, it is possible to accurately determine which screen the user is operating, and only perform the corresponding click operation on that screen. By distinguishing the IDs of different screens, we can ensure the accuracy and reliability of the operation, thereby avoiding misoperation and confusion, and improving the user experience of the multi-screen vehicle system. At the same time, it also provides convenience and flexibility for subsequent functional expansion and optimization.

[0057] In the multi-screen car mode, define whether to support the multi-screen car scanning mode.

[0058] The embodiment of the present invention further provides a visible-and-speakable voice interaction device for an in-vehicle system, which may include:

[0059] Memory for storing computer programs;

[0060] The processor can implement the following steps when executing the computer program stored in the above memory:

[0061] Convert the voice request of the user for interacting with the vehicle system into trigger text; determine whether the application corresponding to the current UI interface supports scanning according to the scanning rules of the current UI interface; when the application corresponding to the current UI interface supports scanning, determine whether the trigger text matches the text corresponding to the special function of the UI interface; when the trigger text matches the text corresponding to the special function of the UI interface, perform a corresponding click operation according to the special function of the UI interface matched by the trigger text and the scanning rules of the current UI interface; when the trigger text does not match the text corresponding to the special function of the UI interface, perform a corresponding click operation according to the scanning rules of the current UI interface.

[0062] The embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps can be implemented;

[0063] Convert the voice request of the user for interacting with the vehicle system into trigger text; determine whether the application corresponding to the current UI interface supports scanning according to the scanning rules of the current UI interface; when the application corresponding to the current UI interface supports scanning, determine whether the trigger text matches the text corresponding to the special function of the UI interface; when the trigger text matches the text corresponding to the special function of the UI interface, perform a corresponding click operation according to the special function of the UI interface matched by the trigger text and the scanning rules of the current UI interface; when the trigger text does not match the text corresponding to the special function of the UI interface, perform a corresponding click operation according to the scanning rules of the current UI interface.

[0064] The computer-readable storage medium may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM)> a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0065] In the description provided herein, a large number of specific details are described. However, it is understood that embodiments of the present invention can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0066] Similarly, it should be understood that in order to streamline the present disclosure and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the present invention, various features of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, this disclosed method should not be interpreted as reflecting the intention that the claimed invention requires more features than those explicitly recited in each claim. More specifically, as reflected in the claims below, inventive aspects lie in less than all the features of the individual embodiments disclosed above. Therefore, the claims that follow the specific embodiment are hereby expressly incorporated into the specific embodiment, with each claim itself serving as a separate embodiment of the present invention.

[0067] Those skilled in the art will appreciate that the modules in the devices in the embodiments can be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments can be combined into one module or unit or component, and in addition they can be divided into multiple submodules or subunits or subassemblies. Except that at least some of such features and / or processes or units are mutually exclusive, all features disclosed in this specification and all processes or units of any method or device disclosed in this manner can be combined in any combination. Unless otherwise clearly stated, each feature disclosed in this specification can be replaced by an alternative feature providing the same, equivalent or similar purpose.

[0068] In addition, those skilled in the art will appreciate that, although some embodiments described herein include certain features included in other embodiments but not other features, the combination of features of different embodiments is meant to be within the scope of the present invention and form different embodiments. For example, any one of the claimed embodiments may be used in any combination.

[0069] The various component embodiments of the present invention can be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It should be understood by those skilled in the art that a microprocessor or digital signal processor (DSP) can be used in practice to implement some or all of the functions of some or all of the components in the embodiments of the present invention. The present invention can also be implemented as a device or apparatus program (e.g., computer program and computer program product) for executing part or all of the methods described herein. Such a program implementing the present invention can be stored on a computer-readable medium, or can have the form of one or more signals. Such a signal can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0070] It should be noted that the above embodiments illustrate the present invention rather than limit it, and that those skilled in the art may devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets should not be constructed as a limitation to the claims. The word "one" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention may be implemented by means of hardware including a number of different elements and by means of a suitably programmed computer. Several of these devices may be embodied by the same hardware item. The use of the words first, second, and third, etc. does not indicate any order. These words may be interpreted as names.

Claims

1. A visible-speakable voice interaction method for an in-vehicle system, characterized in that: The following steps are involved: Convert the user's voice request for interaction with the vehicle system into trigger text; According to the scanning rules of the current UI interface, determine whether the application corresponding to the current UI interface supports scanning; When the application corresponding to the current UI interface supports scanning, determining whether the trigger text matches the text corresponding to the special function of the UI interface; When the trigger text matches the text corresponding to the special function of the UI interface, a corresponding click operation is performed according to the special function of the UI interface matched by the trigger text and the scanning rule of the current UI interface; When the trigger text does not match the text corresponding to the special function of the UI interface, the corresponding click operation is performed according to the scanning rule of the current UI interface.

2. The visible-and-speakable voice interaction method for an in-vehicle system according to claim 1, characterized in that: If it is a multi-screen car machine mode, then when the application corresponding to the current UI interface supports scanning, before the step of determining whether the trigger text matches the special function of the UI interface, it also includes: A trigger screen ID is acquired according to a user trigger position, wherein the click operation is performed on a screen corresponding to the trigger screen ID.

3. The visible-and-speakable voice interaction method for an in-vehicle system according to claim 1, characterized in that: Before the step of converting the voice request of the user interacting with the vehicle system into a trigger text, the method further includes: Load the configuration file that includes the scan rules of the current UI interface.

4. The visible-and-speakable voice interaction method for an in-vehicle system according to claim 1, characterized in that: The special functions of the UI interface include page turning, system buttons, control ID, absolute position and relative position.

5. The visible-and-speakable voice interaction method for an in-vehicle system according to claim 4, characterized in that: When the trigger text matches the text corresponding to the page turning function, according to the scanning rules of the current UI interface, it is determined whether the current UI interface supports text scanning of invisible controls. If so, click to execute sliding page turning.

6. The visible-and-speakable voice interaction method for an in-vehicle system according to claim 4, characterized in that: When the trigger text matches the text corresponding to the system button function, determine whether the current UI interface supports the use of system functions according to the scanning rules of the current UI interface. If so, click the corresponding system button according to the trigger text.

7. The visible-and-speakable voice interaction method for an in-vehicle system according to claim 4, characterized in that: When the trigger text matches the text corresponding to the control ID function, the node information of the control is obtained according to the control ID corresponding to the trigger text, and it is determined whether the control is a clickable control based on the obtained node information. If it is, a click operation is performed on the control. If not, the parent control of the control is obtained for click judgment.

8. The visible-and-speakable voice interaction method for an in-vehicle system as claimed in claim 4, characterized in that: When the trigger text matches the text corresponding to the absolute position function, judging whether the current UI interface supports non-clickable text scanning according to the scanning rules of the current UI interface, and if so, obtaining the position to be clicked according to the trigger text, and performing a click operation at the position to be clicked; When the trigger text matches the text corresponding to the relative position function, determine whether the current UI interface supports non-clickable text scanning according to the scanning rules of the current UI interface. If so, obtain the position to be clicked and the relative control position according to the trigger text, and perform the click operation according to the position to be clicked and the relative control position.

9. The visible-and-speakable voice interaction method for an in-vehicle system according to claim 1, characterized in that: When the trigger text does not match the text corresponding to the special function of the UI interface, the step of performing the corresponding click operation according to the scanning rule of the current UI interface includes: Recursively obtain the node information of the control corresponding to the trigger text on the current UI interface according to the text; According to the scanning rules of the current UI interface and the obtained node information, it is determined whether the node is a scannable node. If so, it is determined whether the control is a clickable control according to the obtained node information. If so, a click operation is performed on the control.

10. A visible and audible voice interaction device for an in-vehicle system, characterized in that: include: Memory for storing computer programs; A processor is used to implement the steps of the visible-and-speakable voice interaction method for an in-vehicle system as described in any one of claims 1 to 9 when executing the computer program.

Images