Voice interaction method, voice interaction device, vehicle and computer readable storage medium

Abstract

The application discloses a voice interaction method, a voice interaction device, a vehicle and a computer readable storage medium. The voice interaction method comprises the following steps: receiving a user voice request when a vehicle-mounted system displays a graphical user interface constructed by a three-dimensional engine; determining a scene event according to the user voice request to determine a target vehicle function control in a vehicle function control; and performing voice interaction with the target vehicle function control through the three-dimensional engine to display a dynamic effect corresponding to the scene event in a three-dimensional model of the vehicle. In the technical scheme of the application, the voice request of the user is received, the scene event is determined according to the user voice request, the target vehicle function control is further determined, and finally the voice interaction of the target vehicle control is performed through the three-dimensional engine to complete the display of the dynamic effect corresponding to the scene event in the three-dimensional model. In this way, the operation of the graphical user interface constructed by the three-dimensional engine can be realized through voice interaction, and the use scene is enriched.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent vehicles, and more particularly, to a voice interaction method, a voice interaction device, a vehicle, and a computer readable storage medium. BACKGROUND

[0002] With the intelligentization of vehicles, the application of vehicles and the interaction with users are more and more frequent, and the voice interaction mode improves the safety of driving compared with the interaction mode of manual operation of a graphical user interface (GUI) by a user. In the related art, there are many restrictions on the use scenario of voice interaction. SUMMARY

[0003] The present application provides a voice interaction method, a voice interaction device, a vehicle, and a computer readable storage medium.

[0004] The present application provides a voice interaction method, which comprises: receiving a user voice request when a vehicle-mounted system displays a graphical user interface constructed by a three-dimensional engine, the graphical user interface comprising a vehicle three-dimensional model and vehicle function controls, the vehicle function controls comprising voice interaction attributes, the voice interaction attributes being generated by the three-dimensional engine according to attributes of the vehicle function controls; determining a scene event according to the user voice request to determine a target vehicle function control in the vehicle function controls; performing voice interaction with the target vehicle function control by the three-dimensional engine to display a dynamic effect corresponding to the scene event in the vehicle three-dimensional model.

[0005] In this way, by receiving a user voice request and determining a scene event according to the user voice request, a target vehicle function control is further determined, and finally voice interaction of the target vehicle control is performed by the three-dimensional engine to complete display of a dynamic effect corresponding to the scene event in the three-dimensional model. In this way, the operation of the graphical user interface constructed by the three-dimensional engine can be realized through voice interaction, and the use scenario is enriched.

[0006] The determination of the scene event according to the user voice request to determine the target vehicle function control in the vehicle function controls comprises: generating a scene snapshot according to the graphical user interface and uploading the scene snapshot to a server; uploading the user voice request to the server to enable the server to obtain a semantic recognition result through language understanding according to the user voice request, find the scene snapshot according to the semantic recognition result to generate the scene event, and send the scene event to the vehicle.

[0007] In this way, by generating a scene event and sending the scene event to the vehicle, preparation can be made for determining the target vehicle function control in the vehicle function controls.

[0008] The vehicle function control includes a voice interaction control script, the voice interaction control script includes the voice interaction attribute, and the scene event is determined according to the user voice request to determine the target vehicle function control in the vehicle function control. Further comprising: determining the voice interaction control script corresponding to the target vehicle function control according to the scene event and the voice interaction attribute. The voice interaction with the target vehicle function control is executed through the three-dimensional engine to display the dynamic effect corresponding to the scene event in the vehicle three-dimensional model, comprising: calling the voice interaction control script corresponding to the target vehicle function control through the three-dimensional engine to display the dynamic effect corresponding to the scene event in the vehicle model.

[0009] In this way, the vehicle determines the target function control through the received scene event and voice interaction attribute, and the voice interaction with the target vehicle function control is executed by the three-dimensional engine, which can complete the display of the dynamic effect corresponding to the scene event in the vehicle three-dimensional model.

[0010] The graphical user interface includes a voice interaction scene script, the vehicle function control includes a voice interaction control script, the voice interaction control script includes the voice interaction attribute, and the voice interaction method further comprises: in response to the first click on the voice option, initializing the voice interaction scene script to make the graphical user interface as a scene root node, and initializing the voice interaction control script to make the vehicle function control as a scene child node; adding the voice interaction attribute corresponding to the scene child node to the parent node of the scene child node to add the scene interaction attribute to the scene root node; constructing a voice interaction scene according to the voice interaction attribute of the scene root node. The scene snapshot is generated according to the voice interaction scene and uploaded to the server.

[0011] In this way, the generation of the scene snapshot can be completed by constructing the voice interaction scene, which can prepare for determining the target vehicle function control.

[0012] In response to the first click on the voice option, the voice interaction scene script is initialized to make the graphical user interface as a scene root node, and the voice interaction control script is initialized to make the vehicle function control as a scene child node, comprising: initializing the voice interaction scene script, registering the graphical user interface and setting the state of the graphical user interface to the initialization state; initializing the voice interaction control script, and determining the parent node corresponding to the scene child node.

[0013] In this way, the parent node corresponding to the scene child node can be determined, and the scene interaction attribute can be added to the scene root node for subsequent preparation.

[0014] The voice interaction attribute includes component attributes, the initialization of the voice interaction control script, and the determination of the parent node corresponding to the scene sub-node, including: initializing the voice interaction control script to obtain the component attributes, the component attributes including at least one of an element unique identification code, an element type, an element label, an element content dynamic type, an element hit priority, and an element visibility; and determining the parent node corresponding to the scene sub-node based on the scene sub-node.

[0015] In this way, the determination of the parent node corresponding to the scene sub-node can be completed, and preparation for subsequently adding the scene interaction attribute to the scene root node can be made.

[0016] The voice interaction attribute includes element attributes, and the adding of the voice interaction attribute of the scene sub-node to the parent node of the scene sub-node to add the scene interaction attribute to the scene root node, including: calling the voice interaction control script to initialize the element visibility of the voice interaction attribute of the scene sub-node; and adding the voice interaction attribute of the scene sub-node to the element attribute of the corresponding parent node to add the voice interaction attribute to the scene root node.

[0017] In this way, by adding the voice interaction attribute to the scene root node, preparation for subsequently constructing the voice interaction scene can be made, which is more concise and efficient compared with the way of extracting the root node in the GUI and traversing the child nodes of the root node to generate the scene during system running, and the performance of program running can be improved and the speed of scene generation can be accelerated.

[0018] The construction of the voice interaction scene according to the voice interaction attribute of the scene root node includes: calling the voice interaction control script to fill the attribute value of the voice interaction attribute; calling the voice interaction scene script to add the identification of the voice interaction scene to the top of the scene stack and update the voice interaction scene to an active state; and calling the voice interaction scene script to serialize the voice interaction attribute of the scene root node to generate the voice interaction scene.

[0019] In this way, by calling the voice interaction scene script to serialize the voice interaction attribute of the scene root node, the voice interaction scene can be generated, and preparation for subsequently generating the scene snapshot can be made.

[0020] The voice interaction method further includes: when the voice interaction is activated, reading the scene stack to obtain the voice interaction scene to generate the scene snapshot according to the voice interaction scene.

[0021] In this way, the generation of the scene snapshot can be completed, and preparation for subsequently determining the target vehicle function control can be made.

[0022] This application provides a voice interaction device, comprising a receiving module, a determining module, and an execution module. The receiving module receives user voice requests when an in-vehicle system displays a graphical user interface (GUI) constructed by a 3D engine. The GUI includes a 3D vehicle model and vehicle function controls. The vehicle function controls include voice interaction attributes, which are generated by the 3D engine based on the attributes of the vehicle function controls. The determining module determines a scene event based on the user voice request to identify a target vehicle function control within the vehicle function controls. The execution module performs voice interaction with the target vehicle function control via the 3D engine to display dynamic effects corresponding to the scene event in the vehicle 3D model.

[0023] In this way, by receiving the user's voice request and determining the scene event based on the user's voice request, the target vehicle function control is further determined. Finally, the voice interaction of the target vehicle control is executed through the 3D engine to complete the display of the dynamic effect corresponding to the scene event in the 3D model. In this way, the operation of the graphical user interface built by the 3D engine can be realized through voice interaction, enriching the usage scenarios.

[0024] The determining module includes a generation unit and an uploading unit. The generation unit is used to generate a scene snapshot based on the graphical user interface and upload it to the server. The uploading unit is used to upload the user's voice request to the server, so that the server can perform language understanding based on the user's voice request to obtain semantic recognition results, locate the scene snapshot based on the semantic recognition results to generate the scene event, and send the scene event to the vehicle.

[0025] Thus, by generating scene events and sending them to the vehicle, preparations can be made to determine the target vehicle function control among the vehicle function controls.

[0026] The vehicle function controls include a voice interaction control script, which includes voice interaction attributes. The determining module further includes a determining unit, which is used to determine the voice interaction control script corresponding to the target vehicle function control based on the scene event and the voice interaction attributes. The execution module includes a calling unit, which is used to call the voice interaction control script corresponding to the target vehicle function control through the 3D engine to display the dynamic effects corresponding to the scene event in the vehicle 3D model.

[0027] In this way, the vehicle determines the target functional control by receiving scene events and voice interaction attributes, and then the 3D engine executes the voice interaction with the target vehicle functional control, which can complete the display of the dynamic effects corresponding to the scene events in the vehicle 3D model.

[0028] The graphical user interface includes a voice interaction scene script, the vehicle function controls include a voice interaction control script, the voice interaction control script includes the voice interaction attributes, and the voice interaction device further includes an initialization module, an adding module, and a building module. The initialization module is used to initialize the voice interaction scene script in response to the first click of the voice option, making the graphical user interface the scene root node, and initialize the voice interaction control script so that the vehicle function controls are scene child nodes. The adding module is used to add the voice interaction attributes corresponding to the scene child nodes to the parent node of the scene child nodes, thereby adding the scene interaction attributes to the scene root node. The building module is used to construct a voice interaction scene based on the voice interaction attributes of the scene root node. The generation unit is used to generate a scene snapshot based on the voice interaction scene and upload it to the server.

[0029] In this way, by constructing a voice interaction scenario, a scene snapshot can be generated, which can prepare for determining the functional controls of the target vehicle.

[0030] The initialization module includes a first initialization unit and a second initialization unit. The first initialization unit initializes the voice interaction scene script, registers the graphical user interface, and sets the state of the graphical user interface to an initialized state. The second initialization unit initializes the voice interaction control script and determines the parent node corresponding to the scene child node.

[0031] In this way, the parent nodes corresponding to the child nodes of the scene can be determined, and preparations can be made for adding the scene interaction attributes to the root node of the scene later.

[0032] This application provides a vehicle that includes a processor and a memory. The memory stores a computer program, which, when executed by the processor, implements the aforementioned voice interaction method.

[0033] In this way, the vehicle receives the user's voice request and determines the scene events based on the user's voice request, further determines the target vehicle function controls, and finally executes the voice interaction of the target vehicle controls through the 3D engine to complete the display of the dynamic effects corresponding to the scene events in the 3D model. This can improve the performance of the program and speed up the scene generation.

[0034] This application provides a non-volatile computer-readable storage medium for a computer program, which, when executed by one or more processors, implements the above-described voice interaction method.

[0035] In this way, by receiving the user's voice request and determining the scene events based on the user's voice request, the target vehicle function controls are further determined. Finally, the voice interaction of the target vehicle controls is executed through the 3D engine to complete the display of the dynamic effects corresponding to the scene events in the 3D model. This can improve the performance of the program and speed up the scene generation.

[0036] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0037] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description taken in conjunction with the following drawings.

[0038] in:

[0039] Figure 1 This is a flowchart illustrating the voice interaction method of this application;

[0040] Figure 2 This is a schematic diagram of the graphical user interface of this application;

[0041] Figure 3 This is a flowchart illustrating the voice interaction method of this application;

[0042] Figure 4 This is a flowchart illustrating the voice interaction method of this application;

[0043] Figure 5 This is a flowchart illustrating the voice interaction method of this application;

[0044] Figure 6 This is a flowchart illustrating the voice interaction method of this application;

[0045] Figure 7 This is a schematic diagram of the graphical user interface of this application;

[0046] Figure 8 This is a schematic diagram of the graphical user interface of this application;

[0047] Figure 9 This is a schematic diagram of the graphical user interface of this application;

[0048] Figure 10 This is a schematic diagram of the graphical user interface of this application;

[0049] Figure 11 This is a flowchart illustrating the voice interaction method of this application;

[0050] Figure 12 This is a schematic diagram of the graphical user interface of this application;

[0051] Figure 13This is a flowchart illustrating the voice interaction method of this application;

[0052] Figure 14 This is a flowchart illustrating the voice interaction method of this application;

[0053] Figure 15 This is a flowchart illustrating the voice interaction method of this application;

[0054] Figure 16 This is a flowchart illustrating the voice interaction method of this application;

[0055] Figure 17 This is a flowchart illustrating the voice interaction method of this application;

[0056] Figure 18 This is a schematic diagram of the vehicle in this application. Detailed Implementation

[0057] The present application is described in detail below. Examples of the present application are shown in the accompanying drawings. The same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The following description with reference to the accompanying drawings is exemplary and is only used to explain the present application, and should not be construed as limiting the present application.

[0058] The following disclosure provides many different examples for implementing different structures of this application. In order to simplify the disclosure of this application, the components and settings of specific examples are described below as examples, and are not intended to limit this application.

[0059] As transportation becomes increasingly intelligent, user interactions with vehicle applications are becoming more frequent. Voice interaction, compared to manual graphical user interface (GUI) operation, improves driving safety. However, the application scenarios for voice interaction within this technology are subject to several limitations.

[0060] Please see Figure 1 This application provides a voice interaction method, including:

[0061] 01: When the in-vehicle system displays a graphical user interface built by a 3D engine, it receives user voice requests. The graphical user interface includes a 3D model of the vehicle and vehicle function controls. The vehicle function controls include voice interaction attributes, which are generated by the 3D engine based on the attributes of the vehicle function controls.

[0062] 02: Determine the scene event based on the user's voice request to identify the target vehicle function control in the vehicle function controls;

[0063] 03: Use the 3D engine to perform voice interaction with the target vehicle's functional controls to display the dynamic effects of scene events in the vehicle's 3D model.

[0064] This application provides a voice interaction device, which includes a receiving module, a determining module, and an execution module. The voice interaction method of this application can be implemented by the voice interaction device of this application, wherein step 01 can be implemented by the receiving module, step 02 can be implemented by the determining module, and step 03 can be implemented by the execution module. That is, the receiving module can be used to receive user voice requests when the in-vehicle system displays a graphical user interface (GUI) constructed by a 3D engine. The GUI includes a 3D vehicle model and vehicle function controls. The vehicle function controls include voice interaction attributes, which are generated by the 3D engine based on the attributes of the vehicle function controls. The determining module can be used to determine scene events based on the user's voice request to determine the target vehicle function control within the vehicle function controls. The execution module can be used to execute voice interaction with the target vehicle function control through the 3D engine to display the dynamic effects corresponding to the scene events in the vehicle 3D model.

[0065] Step 01: When the in-vehicle system displays a graphical user interface (GUI) built with a 3D engine, it receives a user voice request. The GUI includes a 3D model of the vehicle and vehicle function controls. The vehicle function controls include voice interaction attributes, which are generated by the 3D engine based on the attributes of the vehicle function controls. Specifically, the 3D engine includes the Unity engine, the vehicle 3D model can include 3D models of vehicle components, such as a 3D model of a seat, and the vehicle function controls include... Figure 2 The button controls shown allow users to adjust the height of the 3D seat model by pressing buttons A and B, adjust its position by pressing buttons C and D, and adjust the backrest height by pressing buttons E and F. Simultaneously, the vehicle seat adjusts accordingly. The system receives user voice requests while displaying a graphical user interface built with a 3D engine. This allows for the reception of user voice requests.

[0066] Step 02: Determine the scene event based on the user's voice request to identify the target vehicle function control within the vehicle function controls. Specifically, after receiving the user's voice request, determine the scene event based on the user's voice request to identify the target vehicle function control within the vehicle function controls.

[0067] In this way, the scene event can be determined by the received user voice request, and then the target vehicle function control in the vehicle function control can be determined.

[0068] Step 03 involves using a 3D engine to perform voice interaction with the target vehicle's functional controls, thereby displaying the dynamic effects corresponding to scene events in the vehicle's 3D model. Specifically, after determining the target vehicle functional controls within the vehicle functional controls, the 3D engine performs voice interaction with the target vehicle functional controls to display the dynamic effects corresponding to scene events in the vehicle's 3D model.

[0069] In this way, dynamic effects corresponding to scene events in the 3D model can be displayed, and the graphical user interface built by the 3D engine can be operated through voice interaction, thus enriching the usage scenarios.

[0070] Please see Figure 3 Step 02 (determining the scene event based on the user's voice request to determine the target vehicle function control in the vehicle function controls) includes:

[0071] 021: Generate a scene snapshot based on the graphical user interface and upload it to the server;

[0072] 022: Upload the user's voice request to the server so that the server can perform language understanding based on the user's voice request to obtain semantic recognition results, find scene snapshots based on semantic recognition results to generate scene events, and send scene events to the vehicle.

[0073] The module comprises a generation unit and an upload unit. Step 021 can be implemented by the generation unit, and step 022 can be implemented by the upload unit. That is, the generation unit can be used to generate a scene snapshot based on the graphical user interface and upload it to the server. The upload unit can be used to upload the user's voice request to the server, so that the server can perform language understanding based on the user's voice request to obtain semantic recognition results, locate the scene snapshot based on the semantic recognition results to generate scene events, and send the scene events to the vehicle.

[0074] In this way, by uploading the graphical user interface as a scene snapshot to the server for processing, generating scene events, and sending the scene events to the vehicle, the computational burden on the local hardware can be reduced, and preparation can be made for determining the target vehicle function control in the vehicle function control.

[0075] Specifically, after receiving a user's voice request, a scene snapshot is generated based on the graphical user interface and uploaded to the server. Then, the user's voice request is uploaded to the server so that the server can perform language understanding based on the user's voice request to obtain semantic recognition results, find the scene snapshot based on the semantic recognition results to generate scene events, and send the scene events to the vehicle.

[0076] Please see Figure 4The vehicle function controls include a voice interaction control script, which includes voice interaction attributes. Step 02 (determining the scene event based on the user's voice request to determine the target vehicle function control in the vehicle function controls) also includes:

[0077] 023: Determine the voice interaction control script corresponding to the target vehicle function control based on scene events and voice interaction attributes.

[0078] Step 03 (executes voice interaction with the target vehicle's functional controls via the 3D engine to display dynamic effects corresponding to scene events in the vehicle's 3D model), includes:

[0079] 031: By calling the voice interaction control script corresponding to the target vehicle's function controls through the 3D engine, dynamic effects corresponding to scene events can be displayed in the vehicle's 3D model.

[0080] The vehicle function controls include voice interaction control scripts, which in turn include voice interaction attributes. The determination module further includes a determination unit, and the execution module includes a calling unit. Step 023 can be implemented by the determination unit, and step 031 can be implemented by the calling unit. That is, the determination unit can be used to determine the voice interaction control script corresponding to the target vehicle function control based on scene events and voice interaction attributes. The calling unit can be used to call the voice interaction control script corresponding to the target vehicle function control through the 3D engine to display the dynamic effects corresponding to the scene events in the vehicle 3D model.

[0081] In this way, the vehicle determines the target functional control by receiving scene events and voice interaction attributes, and then the 3D engine executes the voice interaction with the target vehicle functional control, which can complete the display of the dynamic effects corresponding to the scene events in the vehicle 3D model.

[0082] Specifically, after the server generates the scene event and sends it to the vehicle, the voice interaction control script corresponding to the target vehicle function control can be determined based on the scene event and voice interaction attributes. Then, the voice interaction control script corresponding to the target vehicle function control is called through the 3D engine to display the dynamic effects corresponding to the scene event in the vehicle 3D model.

[0083] Please see Figure 5 The graphical user interface includes voice interaction scenario scripts, vehicle function controls include voice interaction control scripts, voice interaction control scripts include voice interaction attributes, and voice interaction methods also include:

[0084] 04: In response to the first click of the voice option, initialize the voice interaction scene script so that the graphical user interface is the root node of the scene, and initialize the voice interaction control script so that the vehicle function controls are the child nodes of the scene.

[0085] 05: Add the voice interaction attribute corresponding to the scene child node to the parent node of the scene child node, so as to add the scene interaction attribute to the scene root node;

[0086] 06: Construct a voice interaction scene based on the voice interaction attributes of the scene root node.

[0087] Step 021 (Generating a scene snapshot based on the graphical user interface and uploading it to the server) includes:

[0088] 0211: Generate a scene snapshot based on the voice interaction scenario and upload it to the server.

[0089] The graphical user interface includes a voice interaction scenario script, the vehicle function controls include a voice interaction control script, the voice interaction control script includes voice interaction attributes, and the voice interaction device also includes an initialization module, an adding module, and a building module. Step 04 can be implemented by the initialization module, step 05 by the adding module, step 06 by the building module, and step 0211 by the generation unit. That is, the initialization module can be used to initialize the voice interaction scenario script in response to the first click of the voice option, so that the graphical user interface is the root node of the scenario, and initialize the voice interaction control script so that the vehicle function controls are the child nodes of the scenario. The adding module can be used to add the voice interaction attributes corresponding to the child nodes of the scenario to the parent node of the child nodes of the scenario, so as to add the scenario interaction attributes to the root node of the scenario. The building module can be used to construct the voice interaction scenario based on the voice interaction attributes of the root node of the scenario. The generation unit can be used to generate a scenario snapshot based on the voice interaction scenario and upload it to the server.

[0090] In this way, by constructing a voice interaction scenario, a scene snapshot can be generated, which can prepare for determining the functional controls of the target vehicle.

[0091] Specifically, after receiving a user's voice request, if the voice option is clicked for the first time, such as when the intelligent voice interface is opened for the first time, the voice interaction scene script is initialized so that the graphical user interface is the root node of the scene, the voice interaction control script is initialized so that the vehicle function controls are the child nodes of the scene, the voice interaction attributes corresponding to the child nodes of the scene are added to the parent node of the child nodes of the scene, so that the scene interaction attributes are added to the root node of the scene, the voice interaction scene is constructed according to the voice interaction attributes of the root node of the scene, and finally the scene snapshot is generated and uploaded to the server based on the voice interaction scene.

[0092] Please see Figure 6 Step 04 (in response to the first click of the voice option, initialize the voice interaction scene script so that the graphical user interface is the root node of the scene, and initialize the voice interaction control script so that the vehicle function controls are child nodes of the scene), including:

[0093] 041: Initialize the voice interaction scenario script, register the graphical user interface, and set the graphical user interface state to the initial state;

[0094] 042: Initialize the voice interaction control script and determine the parent node corresponding to the scene child node.

[0095] The initialization module includes a first initialization unit and a second initialization unit. Step 041 can be implemented by the first initialization unit, and step 042 can be implemented by the second initialization unit. That is, the first initialization unit can be used to initialize the voice interaction scene script, register the graphical user interface, and set the graphical user interface state to the initialization state. The second initialization unit can be used to initialize the voice interaction control script and determine the parent node corresponding to the scene child node.

[0096] In this way, the parent nodes corresponding to the child nodes of the scene can be determined, and preparations can be made for adding the scene interaction attributes to the root node of the scene later.

[0097] Specifically, upon receiving a user's voice request, if this is the first click on the voice option, the voice interaction scene script is first initialized, the graphical user interface is registered and its state is set to the initial state, and then the voice interaction control script is initialized. The voice interaction control script includes: a button script (VuiButton), such as... Figure 7 The image shows a back button, a close button, and a delete button; the option script (VuiToggle) is shown, such as... Figure 8 The cell boxes and checkboxes shown can have values ​​of true and false; scrollbar scripts (VuiSlider), such as... Figure 9 The numerical bars and sliders shown can have a value of 50; block scripts (VuiSegment), such as... Figure 10 As shown, its value can be 1, 2, or 3; static scrollable control (VuiScrollView) and dynamic scrollable control (VuiRecycleView). After initialization, the parent node corresponding to the scene's child nodes is determined.

[0098] Please see Figure 11 The voice interaction attributes include component attributes. Step 042 (initializing the voice interaction control script and determining the parent node corresponding to the scene child node) includes:

[0099] 0421: Initialize the voice interaction control script to obtain component properties. Component properties include at least one of the following: element unique identifier, element type, element tag, element content dynamic type, element hit priority, and element visibility.

[0100] 0422: Search upwards from the scene's child nodes to determine the parent node corresponding to the scene's child node.

[0101] The voice interaction attributes include component attributes. The second initialization unit includes an initialization subunit and a lookup subunit. Step 0421 can be implemented by the initialization subunit, and step 0422 can be implemented by the lookup subunit. That is, the initialization subunit can be used to initialize the voice interaction control script to obtain component attributes. Component attributes include at least one of the following: element unique identifier, element type, element tag, element content dynamic type, element hit priority, and element visibility. The lookup subunit can be used to search upwards from the scene child node to determine the parent node corresponding to the scene child node.

[0102] In this way, the parent nodes corresponding to the child nodes of the scene can be determined, and preparations can be made for adding the scene interaction attributes to the root node of the scene later.

[0103] Specifically, after registering the graphical user interface (GUI) and setting its state to the initial state, the voice interaction control script is first initialized to obtain component properties. These properties include at least one of the following: element unique identifier (id), element type, element label, element content dynamic type, element hit priority, and element visibility. The element unique identifier (id) is the unique identifier for each element in the Voice User Interface (VUI); the element type (type) is the type of the element in the VUI, used to identify the control type and support semantic return; for example, for a button, it allows the user to say "click" or "open" to issue an operation command for the button; the element label (label) is the entity word that the element in the VUI can support user queries, such as... Figure 12 If the label of the element in the selected area is "Back", the user can issue a command by saying "Back"; the element content dynamic type (layoutLoadable) indicates whether the content of the element in Vui is dynamic, that is, whether the content is obtained through network retrieval, such as music playlists, navigation POI lists, etc.; the element hit priority (priority) indicates the hit priority of each element in Vui, that is, when there are two or more Vui elements with the same label on the page, their hit priority can be adjusted by setting the priority value; the element visibility (Visible) indicates the visibility of the element in Vui, for example, setting the Visible value to TRUE means that the element is visible, and further, the parent node can be searched upwards based on the scene child node.

[0104] Please see Figure 13Voice interaction attributes include element attributes. Step 05 (adding the voice interaction attributes corresponding to the scene child nodes to the parent node of the scene child nodes, so as to add the scene interaction attributes to the scene root node) includes:

[0105] 051: Call the voice interaction control script to initialize the element visibility of the voice interaction attribute of the scene's child nodes;

[0106] 052: Add the voice interaction attribute of the scene child node to the element attribute of the corresponding parent node, so as to add the voice interaction attribute to the scene root node.

[0107] The voice interaction attributes include element attributes. The adding module includes a third initialization unit and a first adding unit. Step 051 can be implemented by the third initialization unit, and step 052 can be implemented by the first adding unit. That is, the third initialization unit can be used to call the voice interaction control script to initialize the element visibility of the voice interaction attributes of the scene child nodes. The first adding unit can be used to add the voice interaction attributes of the scene child nodes to the element attributes of the corresponding parent nodes, so as to add the voice interaction attributes to the scene root node.

[0108] In this way, by adding the voice interaction attribute to the root node of the scene, we can prepare for the subsequent construction of the voice interaction scene. Compared with the method of extracting the root node in the GUI and traversing its child nodes to generate the scene at system runtime, it is simpler and more efficient, which can improve the performance of program operation and speed up the scene generation.

[0109] Specifically, after determining the scene child nodes, the voice interaction control script is first called to initialize the element visibility of the voice interaction attribute of the scene child nodes, such as updating its Visible property to TRUE. Then, the voice interaction attribute of the scene child nodes is added to the element attribute of the corresponding parent node, so as to add the voice interaction attribute to the scene root node.

[0110] Please see Figure 14 Step 06 (Constructing a voice interaction scene based on the voice interaction attributes of the scene root node) includes:

[0111] 061: Call the voice interaction control script to populate the attribute values ​​of the voice interaction properties;

[0112] 062: Call the voice interaction scene script to add the voice interaction scene identifier to the top of the scene stack and update the voice interaction scene to the active state;

[0113] 063: Call the voice interaction scene script to serialize the voice interaction attributes of the scene root node and generate the voice interaction scene.

[0114] The building module includes a filling unit, a second adding unit, and a serialization unit. Step 061 can be implemented by the filling unit, step 062 by the second adding unit, and step 063 by the serialization unit. That is, the filling unit can be used to call the voice interaction control script to fill in the attribute values ​​of the voice interaction properties. The second adding unit can be used to call the voice interaction scene script to add the identifier of the voice interaction scene to the top of the scene stack and update the voice interaction scene to an active state. The serialization unit can be used to call the voice interaction scene script to serialize the voice interaction attributes of the scene root node, generating the voice interaction scene.

[0115] Thus, by calling the voice interaction scene script to serialize the voice interaction attributes of the scene root node, a voice interaction scene can be generated, preparing for the subsequent generation of scene snapshots.

[0116] Specifically, after adding the scene interaction attributes to the scene root node, the voice interaction control script is first called to populate the attribute values ​​of the voice interaction attributes. Then, the voice interaction scene script is called to add the identifier of the voice interaction scene to the top of the scene stack and update the voice interaction scene to the active state. Finally, the interaction scene script is called to serialize the voice interaction attributes of the scene root node, thereby generating the voice interaction scene.

[0117] Please see Figure 15 Voice interaction methods also include:

[0118] 07: When voice interaction is activated, read the scene stack to obtain the voice interaction scene, and generate a scene snapshot based on the voice interaction scene.

[0119] In this way, a scene snapshot can be generated, which will prepare for the subsequent determination of the target vehicle's functional controls.

[0120] Specifically, after generating the voice interaction scenario, it is determined whether the voice interaction has been activated. For example, the user can activate the voice interaction by issuing the command "Hello, XX". When the voice interaction is activated, the scenario (intelligent voice interface) in the scenario stack is read to obtain the voice interaction scenario, so as to generate a scenario snapshot based on the voice interaction scenario. In one embodiment, after the user issues a command to activate the voice interaction and complete the generation of the scene snapshot, the audio stream is uploaded to the voice cloud (server) through a long WebSocket connection between the voice assistant and the cloud. The recognition result of ASR (Automatic Speech Recognition) will directly call the scene NLU (Natural Language Understanding) for semantic recognition. The scene NLU first searches for the scene snapshot for semantic understanding, and then the cloud generates a scene command and sends it to the voice client (vehicle). The voice client parses the scene command and dispatches it to the VuiEngine SDK (3D engine SDK). In the VuiEngine SDK, the corresponding voice interaction scene is found through the scene ID mapping and dispatched to the execution event (onVuiEvent) of the business layer. The execution event of the voice interaction scene finds the 3D engine component (VuiComponent) of the execution event, and executes the simulation to select the switch of the corresponding component in VuiToggle and sets its value to true.

[0121] Please see Figure 16 and Figure 17 In one embodiment, the execution flow of the voice interaction method of this application is shown in the figure.

[0122] Please see Figure 18 The voice interaction method of this application can be implemented by the vehicle 100 of this application. Specifically, the vehicle 100 includes one or more processors 10 and a memory 20. The memory 20 stores a computer program. When the computer program is executed by the processor 10, the steps of the above-described voice interaction method are implemented.

[0123] For example, when a computer program is executed by processor 10, the following steps are implemented for voice interaction:

[0124] 01: When the in-vehicle system displays a graphical user interface built by a 3D engine, it receives user voice requests. The graphical user interface includes a 3D model of the vehicle and vehicle function controls. The vehicle function controls include voice interaction attributes, which are generated by the 3D engine based on the attributes of the vehicle function controls.

[0125] 02: Determine the scene event based on the user's voice request to identify the target vehicle function control in the vehicle function controls;

[0126] 03: Use the 3D engine to perform voice interaction with the target vehicle's functional controls to display the dynamic effects of scene events in the vehicle's 3D model.

[0127] This application provides a non-volatile computer-readable storage medium for a computer program, on which a computer program is stored, and when the program is executed by a processor 10, it implements the steps of any of the above-described voice interaction methods.

[0128] For example, when the program is executed by processor 10, the following steps are implemented for voice interaction:

[0129] 01: When the in-vehicle system displays a graphical user interface built by a 3D engine, it receives user voice requests. The graphical user interface includes a 3D model of the vehicle and vehicle function controls. The vehicle function controls include voice interaction attributes, which are generated by the 3D engine based on the attributes of the vehicle function controls.

[0130] 02: Determine the scene event based on the user's voice request to identify the target vehicle function control in the vehicle function controls;

[0131] 03: Use the 3D engine to perform voice interaction with the target vehicle's functional controls to display the dynamic effects of scene events in the vehicle's 3D model.

[0132] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the preferred scope of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the function involved, as will be understood by those skilled in the art to which embodiments of this application pertain.

[0133] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processing module, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (control method), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic device, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which programs can be printed, because programs can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0134] It should be understood that the various parts of this application can be implemented in hardware, software, firmware, or a combination thereof. The aforementioned steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0135] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0136] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0137] The storage medium mentioned above may be a read-only memory, a disk, or an optical disk, etc. Although this application has been shown and described above, it is understood that the above description of this application is exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above application within the scope of this application.

Claims

1. A voice interaction method, characterized in that, include: When the in-vehicle system displays a graphical user interface built by a 3D engine, it receives a user's voice request. The graphical user interface includes a 3D model of the vehicle and vehicle function controls. The 3D model of the vehicle includes 3D models of vehicle parts. The vehicle function controls include voice interaction attributes, which are generated by the 3D engine based on the attributes of the vehicle function controls. The scene event is determined based on the user's voice request to determine the target vehicle function control in the vehicle function control; The 3D engine performs voice interaction with the functional controls of the target vehicle to display the dynamic effects corresponding to the scene events in the 3D model of the vehicle. The dynamic effects synchronously display the state changes of the vehicle parts. The vehicle function controls include a voice interaction control script, the voice interaction control script includes the voice interaction attributes, and the step of determining the scene event based on the user's voice request to determine the target vehicle function control in the vehicle function controls further includes: The voice interaction control script corresponding to the target vehicle function control is determined based on the scene event and the voice interaction attribute. The step of performing voice interaction with the target vehicle's functional controls through the 3D engine to display the dynamic effects corresponding to the scene events in the vehicle's 3D model includes: The 3D engine calls the voice interaction control script corresponding to the function control of the target vehicle to display the dynamic effects corresponding to the scene event in the 3D model of the vehicle.

2. The voice interaction method according to claim 1, characterized in that, The step of determining the scene event based on the user's voice request to determine the target vehicle function control in the vehicle function controls includes: A scene snapshot is generated based on the graphical user interface and uploaded to the server; The user's voice request is uploaded to the server, so that the server can perform language understanding based on the user's voice request to obtain semantic recognition results, find the scene snapshot based on the semantic recognition results to generate the scene event, and send the scene event to the vehicle.

3. The voice interaction method according to claim 2, characterized in that, The graphical user interface includes a voice interaction scenario script, the vehicle function controls include a voice interaction control script, the voice interaction control script includes the voice interaction attributes, and the voice interaction method further includes: In response to the first click of the voice option, the voice interaction scene script is initialized so that the graphical user interface is the scene root node, and the voice interaction control script is initialized so that the vehicle function control is the scene child node. Add the voice interaction attribute corresponding to the scene child node to the parent node of the scene child node, so as to add the scene interaction attribute to the scene root node; A voice interaction scenario is constructed based on the voice interaction attributes of the scenario root node; The step of generating a scene snapshot based on the graphical user interface and uploading it to the server includes: A scene snapshot is generated based on the voice interaction scenario and uploaded to the server.

4. The voice interaction method according to claim 3, characterized in that, The response to the first click on the voice option, initializing the voice interaction scene script so that the graphical user interface is the scene root node, and initializing the voice interaction control script so that the vehicle function controls are scene child nodes, includes: Initialize the voice interaction scenario script, register the graphical user interface, and set the state of the graphical user interface to the initialization state; Initialize the voice interaction control script and determine the parent node corresponding to the scene child node.

5. The voice interaction method according to claim 4, characterized in that, The voice interaction attributes include component attributes. The initialization of the voice interaction control script and the determination of the parent node corresponding to the scene child node include: The component attributes are obtained by initializing the voice interaction control script. The component attributes include at least one of the following: element unique identifier, element type, element tag, element content dynamic type, element hit priority, and element visibility. Based on the child nodes of the scene, search upwards to find the parent node to determine the parent node corresponding to the child node of the scene.

6. The voice interaction method according to claim 4, characterized in that, The voice interaction attribute includes element attributes. Adding the voice interaction attribute corresponding to the scene child node to the parent node of the scene child node, so as to add the scene interaction attribute to the scene root node, includes: The script for the voice interaction control is invoked to initialize the element visibility of the voice interaction attribute of the scene child node; The voice interaction attribute of the scene child node is added to the element attribute of the corresponding parent node, so as to add the voice interaction attribute to the scene root node.

7. The voice interaction method according to claim 6, characterized in that, The step of constructing a voice interaction scene based on the voice interaction attributes of the scene root node includes: The script of the voice interaction control is invoked to populate the attribute values ​​of the voice interaction properties; The voice interaction scene script is invoked to serialize the voice interaction attributes of the scene root node and generate the voice interaction scene.

8. A voice interaction device, characterized in that, include: A receiving module is used to receive user voice requests when the in-vehicle system displays a graphical user interface constructed by a 3D engine. The graphical user interface includes a 3D model of the vehicle and vehicle function controls. The 3D model of the vehicle includes 3D models of vehicle parts. The vehicle function controls include voice interaction attributes, which are generated by the 3D engine based on the attributes of the vehicle function controls. The determining module is used to determine the scene event based on the user's voice request to determine the target vehicle function control in the vehicle function control; An execution module is used to perform voice interaction with the functional controls of the target vehicle through the 3D engine, so as to display the dynamic effects corresponding to the scene events in the 3D model of the vehicle, and the dynamic effects synchronously display the state changes of vehicle parts; The vehicle function controls include a voice interaction control script, the voice interaction control script includes the voice interaction attributes, and the determining module includes: A determining unit is configured to determine the voice interaction control script corresponding to the target vehicle function control based on the scene event and the voice interaction attribute. The execution module includes: The calling unit is used to call the voice interaction control script corresponding to the function control of the target vehicle through the 3D engine, so as to display the dynamic effect corresponding to the scene event in the 3D model of the vehicle.

9. The voice interaction device according to claim 8, characterized in that, The determining module includes: A generation unit, configured to generate a scene snapshot based on the graphical user interface and upload it to a server; An upload unit is configured to upload the user's voice request to the server, so that the server can perform language understanding based on the user's voice request to obtain semantic recognition results, find the scene snapshot based on the semantic recognition results to generate the scene event, and send the scene event to the vehicle.

10. The voice interaction device according to claim 9, characterized in that, The graphical user interface includes a voice interaction scenario script, the vehicle function controls include a voice interaction control script, the voice interaction control script includes the voice interaction attributes, and the voice interaction device further includes: An initialization module is configured to, in response to the first click of the voice option, initialize the voice interaction scene script so that the graphical user interface is the scene root node, and initialize the voice interaction control script so that the vehicle function control is the scene child node. An adding module is used to add the voice interaction attribute corresponding to the scene child node to the parent node of the scene child node, so as to add the scene interaction attribute to the scene root node; The construction module is used to construct a voice interaction scene based on the voice interaction attributes of the scene root node; The generation unit is used for: A scene snapshot is generated based on the voice interaction scenario and uploaded to the server.

11. The voice interaction device according to claim 10, characterized in that, The initialization module includes: The first initialization unit is used to initialize the voice interaction scenario script, register the graphical user interface, and set the state of the graphical user interface to the initialization state. The second initialization unit is used to initialize the voice interaction control script and determine the parent node corresponding to the scene child node.

12. A vehicle, characterized in that, The vehicle includes a processor and a memory, the memory storing a computer program, which, when executed by the processor, implements the voice interaction method according to any one of claims 1-7.

13. A non-volatile computer-readable storage medium for computer programs, characterized in that, When the computer program is executed by one or more processors, it implements the voice interaction method according to any one of claims 1-7.

Images