A vehicle-mounted voice interaction method, system and storage medium

Abstract

The application provides a vehicle-mounted voice interaction method and system and a storage medium, relates to the field of vehicle-mounted voice interaction, and aims to effectively avoid the congestion of AI inference traffic to the key interaction of the vehicle side through the double-channel design of the cloud Link module, and guarantee the real-time performance and stability of the vehicle side interaction. The cloud Link module to which the method is applied comprises a first communication module for communicating with a vehicle and a second communication module for communicating with a cloud AI inference end, the two communication modules are independent of each other, and the communication priority of the first communication module is higher than that of the second communication module; in response to receiving a voice dialogue request sent by the vehicle through the first communication module, an inference request is sent to the cloud AI inference end through the second communication module; and in response to receiving a dialogue inference result returned by the cloud AI inference end through the second communication module, the dialogue inference result is sent to the vehicle through the first communication module.

Description

Technical Field

[0001] This application relates to the field of in-vehicle voice interaction, and more particularly to an in-vehicle voice interaction method, system, and storage medium. Background Technology

[0002] With the rapid development of intelligent connected vehicle technology, the real-time performance and stability of in-vehicle voice interaction are crucial to the driving experience. However, in high-concurrency artificial intelligence (AI) inference scenarios, ensuring the real-time response of vehicle-side voice interaction has become an urgent technical problem to be solved.

[0003] To address the aforementioned needs, various in-vehicle voice interaction technologies already exist, but these solutions still have significant shortcomings: processing is limited to the vehicle itself, resulting in slow response times and low recognition accuracy; data between the vehicle and the cloud is transmitted through a single communication channel, making critical vehicle-side interactions susceptible to cloud-based congestion, compromising real-time performance and stability. Therefore, there is an urgent need to propose a novel in-vehicle voice interaction method that can effectively isolate or avoid interference from cloud AI inference traffic on critical vehicle-side interactions, thereby ensuring the real-time response and stable operation of in-vehicle voice interaction. Summary of the Invention

[0004] This application provides an in-vehicle voice interaction method, system, and storage medium, which aims to effectively avoid the blockage of AI inference traffic on key vehicle-side interactions by using the dual-channel design of the cloud Link module, thereby ensuring the real-time performance and stability of vehicle-side interactions.

[0005] In a first aspect, embodiments of this application provide an in-vehicle voice interaction method applied to a cloud-based Link module. The cloud-based Link module includes a first communication module and a second communication module. The first communication module is used to communicate with the vehicle, and the second communication module is used to communicate with a cloud-based AI inference terminal. The first and second communication modules are independent of each other, and the communication priority of the first communication module is higher than that of the second communication module. The method includes: in response to receiving a voice dialogue request sent by the vehicle through the first communication module, sending an inference request to the cloud-based AI inference terminal through the second communication module; the cloud-based AI inference terminal is used to infer a corresponding dialogue inference result based on the voice dialogue data corresponding to the inference request; in response to receiving at least a portion of the dialogue inference result returned by the cloud-based AI inference terminal through the second communication module, the cloud-based Link module sends the dialogue inference result to the vehicle through the first communication module.

[0006] Based on the aforementioned technical means, the in-vehicle voice interaction method provided by this invention, by configuring the first communication module in the cloud Link module for communication with the vehicle and the second communication module for communication with the cloud AI inference terminal, and by making the first and second communication modules independent of each other with the first communication module having a higher communication priority than the second communication module, ensures that vehicle-side communication and AI inference-side communication are isolated from each other on the channel. This allows the first communication module to still prioritize sending and receiving vehicle-side messages even when the second communication module is busy or experiencing abnormal AI inference data transmission, preventing AI inference traffic from blocking vehicle voice dialogue requests. Furthermore, when receiving a voice dialogue request from the vehicle through the first communication module, an inference request is sent to the cloud AI inference terminal through the second communication module. Since the sending of the inference request is confined to the second communication module, it does not occupy the communication resources of the first communication module, further ensuring the first communication module's ability to respond instantly to other vehicle interaction messages. Furthermore, after receiving at least a portion of the dialogue inference results returned by the cloud-based AI inference terminal via the second communication module, the dialogue inference results are sent to the vehicle via the first communication module. Since the first communication module has a higher communication priority than the second communication module, even if the second communication module is still continuously receiving inference data, the dialogue inference results can still be preferentially sent to the vehicle via the first communication module, thereby reducing the overall response latency of voice interaction. In summary, this invention achieves backpressure isolation between vehicle-side communication and AI inference-side communication through the independent setting and priority configuration of the first and second communication modules, ensuring the real-time performance and stability of key vehicle-side interactions while improving the response efficiency of voice interaction.

[0007] In one possible implementation, in response to receiving at least a portion of the dialogue inference result returned by the cloud AI inference terminal via the second communication module, the cloud Link module sends the dialogue inference result to the vehicle via the first communication module, including: receiving at least a portion of the dialogue inference result returned by the cloud AI inference terminal via the second communication module; wherein the dialogue inference result includes multiple inference text segments; the at least a portion of the dialogue inference result includes at least one inference text segment among the multiple inference text segments; during the process of receiving at least a portion of the dialogue inference result, whenever one of the inference text segments meets a preset segmentation condition, the inference text segment is identified as the target inference text segment; a synthesis request for the target inference text segment is sent to the speech synthesis server, and audio data is received; the speech synthesis server is used to convert the target inference text segment into audio data; and the audio data is sent to the vehicle via the first communication module.

[0008] Based on the aforementioned technical means, the dialogue inference result includes multiple inference text fragments, and at least a portion of the dialogue inference result includes at least one inference text fragment from the multiple inference text fragments. This allows the cloud Link module to initiate subsequent processing after receiving at least one inference text fragment through the second communication module without waiting for the entire content of the dialogue inference result to be generated. Furthermore, during the process of receiving at least a portion of the dialogue inference result, whenever one of the at least one inference text fragments meets a preset segmentation condition, that inference text fragment is identified as the target inference text fragment, and a synthesis request for the target inference text fragment is immediately sent to the speech synthesis server. This approach enables speech synthesis to begin as soon as a suitable inference text segment is obtained, without waiting for the entire dialogue inference result to be generated. This transforms AI inference and speech synthesis from serial processing to a pipelined parallel processing that proceeds segment by segment. That is, while the AI ​​inference end in the cloud continues to generate subsequent inference text segments, the already determined target inference text segment is being synthesized into speech, reducing the overall latency from the start of inference to the generation of the first audio segment and improving the response speed of voice interaction. At the same time, since the audio data is sent to the vehicle through the first communication module, the synthesized audio data can reach the vehicle side in a timely manner, and the vehicle can receive and play it simultaneously, ensuring the real-time and continuous nature of voice playback.

[0009] In one possible implementation, the preset segmentation conditions include: the reasoning text segment contains sentence-ending punctuation; or, the cumulative number of characters in the reasoning text segment reaches a preset threshold.

[0010] Based on the above technical means, since the preset segmentation conditions include both sentence-end punctuation judgment and cumulative character quantity threshold judgment, the segmentation operation can ensure the semantic integrity of the reasoning text fragments based on sentence-end punctuation, and can also segment long texts without punctuation in a timely manner according to the cumulative character quantity. This avoids the infinite waiting caused by relying solely on punctuation or the semantic fragmentation caused by relying solely on character quantity, thereby achieving a balance between initial sound response delay and speech synthesis efficiency, and improving speech synthesis quality and user experience.

[0011] In one possible implementation, the cloud-based Link module further includes a third communication module; the third communication module is used to communicate with the voice recognition server; before receiving a voice dialogue request sent by the vehicle through the first communication module, the module further includes: in response to receiving a voice recognition request sent by the vehicle through the first communication module, forwarding the voice recognition request to the voice recognition server through the third communication module; the voice recognition server is used to convert the voice data corresponding to the voice recognition request into a recognition result; in response to receiving the recognition result returned by the voice recognition server through the third communication module, sending the recognition result to the vehicle through the first communication module.

[0012] Based on the above technical means, since the cloud Link module communicates with the speech recognition server through the third communication module, the reception and forwarding of speech recognition requests, as well as the reception and distribution of recognition results, are all completed through the independent third communication module and the first communication module. The speech recognition process does not occupy the AI ​​inference communication resources of the second communication module, nor does it affect the first communication module's sending and receiving of messages from other vehicles. At the same time, the recognition results are distributed preferentially through the first communication module, ensuring that vehicles can obtain the recognized text in a timely manner, providing an accurate data foundation for subsequent voice dialogue requests based on the recognition results, and improving the coherence and real-time performance of the overall interaction process.

[0013] In one possible implementation, the method further includes: in response to receiving a dialogue cancellation request sent by the vehicle through the first communication module, interrupting the inference request sent to the cloud AI inference terminal through the second communication module, terminating the sending of dialogue inference results to the vehicle through the first communication module, maintaining the connection with the speech recognition server, and clearing the mapping record corresponding to the current session in the multidimensional distributed routing table.

[0014] Based on the above technical means, since the inference request is interrupted and the delivery of the inference result is terminated immediately upon receiving the dialogue cancellation request, cloud AI inference resources can be released in a timely manner, avoiding the continued consumption of computing power after the user cancels the conversation. At the same time, the connection with the speech recognition server is maintained, so that a new round of dialogue can be initiated immediately within the same session without rebuilding the speech recognition connection, thus improving the efficiency of interactive response. In addition, the mapping record of the current session is cleared from the multidimensional distributed routing table to avoid the routing residue of canceled sessions interfering with subsequent routing decisions and to ensure routing accuracy.

[0015] In one possible implementation, the method further includes: in response to detecting the termination of the session with the vehicle or the disconnection of the first communication module, sequentially interrupting the inference request sent to the cloud AI inference terminal through the second communication module, terminating the synthesis task with the speech synthesis server, and disconnecting the connection with the speech recognition server.

[0016] Based on the above technical means, when the session terminates or the first communication module disconnects, the inference request is interrupted first, then the synthesis task is terminated, and finally the speech recognition connection is disconnected. This ensures that the three streams of AI inference, speech synthesis, and speech recognition are released in an orderly manner according to their data dependencies. This avoids the problem of the synthesis task continuously being enqueued or the recognition result failing to trigger the dialogue due to improper release order. As a result, the atomic and consistent release of multiple resources is guaranteed when the session ends, preventing resource leakage and ghost connections, and improving the reliability of system resource management.

[0017] One possible implementation also includes: querying a multidimensional distributed routing table based on the vehicle identity carried by the new connection; the multidimensional distributed routing table is used to store the mapping relationship between the vehicle identity and the session identifier; the session identifier is used to identify a session established between the vehicle and the cloud Link module through the first communication module; when it is found that the vehicle identity is associated with an old session identifier, a session-level resource release operation is triggered for the old session identifier; after confirming that the session-level resource release operation corresponding to the old session identifier is completed, the session identifier corresponding to the new connection is updated to the multidimensional distributed routing table.

[0018] Based on the above technical means, since the routing table is queried based on the vehicle identity when a new connection is established, and the resource release operation of the old session is triggered first when it is found that it is associated with an old session identity, and then the new session identity is updated, the same vehicle identity is associated with only one active session at a time. The three-way bridge and routing records of the old session are thoroughly cleaned up, avoiding the old connection residue occupying system resources. At the same time, the routing table is updated only after the release is confirmed, ensuring the atomicity of route switching and improving the accuracy of session management and resource utilization in the cluster environment.

[0019] In one possible implementation, the multidimensional distributed routing table also includes multiple session heartbeat records; the method further includes: for any session heartbeat record among the multiple session heartbeat records: if the session heartbeat record has not been updated for more than a preset session duration threshold, it is determined as a candidate session heartbeat record; based on all candidate session heartbeat records determined at the same time, the routing information corresponding to the candidate session heartbeat records is cleaned up in sequence.

[0020] Based on the above technical means, by tracking the session activity status through session heartbeat records and identifying the corresponding session as a candidate and cleaning up its routing information after the heartbeat timeout, zombie routes left over due to instance crashes or connection anomalies can be automatically identified and cleared. This avoids the accumulation of invalid mappings in the routing table, which can lead to misdelivery or failure to deliver messages, thereby improving the reliability and accuracy of distributed session routing and ensuring the accurate delivery of downlink messages.

[0021] Secondly, embodiments of this application provide an in-vehicle voice interaction system, including a cloud Link module; the cloud Link module includes a first communication module, a second communication module, and a processing module; the first communication module is used to communicate with the vehicle; the second communication module is used to communicate with a cloud AI inference terminal; the first communication module and the second communication module are independent of each other, and the communication priority of the first communication module is higher than that of the second communication module; the processing module is configured to: in response to receiving a voice dialogue request sent by the vehicle through the first communication module, send an inference request to the cloud AI inference terminal through the second communication module; the cloud AI inference terminal is used to infer a corresponding dialogue inference result based on the voice dialogue data corresponding to the inference request; in response to receiving the dialogue inference result returned by the cloud AI inference terminal through the second communication module, send the dialogue inference result to the vehicle through the first communication module.

[0022] Thirdly, this application provides a computer-readable storage medium storing a program or instructions that, when executed on an electronic device, cause the electronic device to perform the method of the first aspect or any possible implementation thereof.

[0023] Fourthly, this application provides a computer program product, which includes a computer program or instructions that, when executed on a device, cause the device to perform the method in the first aspect or any possible implementation of the first aspect.

[0024] The second to fourth aspects described above correspond to the technical solutions of the first aspect of the embodiments of this application. The beneficial effects achieved by each aspect and its corresponding feasible implementation are similar, and will not be repeated here. It should be noted that various possible implementations of any aspect described above can be combined, provided that the solutions do not contradict each other. Attached Figure Description

[0025] Figure 1 This is a schematic diagram illustrating an application scenario of an in-vehicle voice interaction method provided in an embodiment of this application. Figure 2 This application provides an interactive schematic diagram of an in-vehicle voice interaction system according to an embodiment of the present application. Figure 3 A flowchart illustrating an in-vehicle voice interaction method provided in an embodiment of this application; Figure 4 A flowchart illustrating a method for distributing dialogue reasoning results, provided in an embodiment of this application; Figure 5 A flowchart illustrating a speech recognition process provided in an embodiment of this application; Figure 6A schematic diagram illustrating the interaction between three-way parallel bridging and cascading termination provided in an embodiment of this application; Figure 7 A complete flowchart of an in-vehicle voice interaction method provided in this application embodiment; Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0026] The embodiments of this application will now be described with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Those skilled in the art will recognize that, with the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0027] This application can be applied to scenarios where users interact with the vehicle via voice while the vehicle is in motion. This scenario involves interaction between the vehicle, the cloud-based Link module, and the cloud-based AI inference terminal. During in-vehicle voice interaction, the vehicle sends a voice dialogue request to the cloud-based Link module via a first communication module. The cloud-based Link module forwards the inference request to the cloud-based AI inference terminal via a second communication module. The cloud-based AI inference terminal generates a dialogue inference result based on the voice dialogue data, and then the cloud-based Link module sends the dialogue inference result back to the vehicle via the first communication module, thereby achieving real-time response to the voice dialogue.

[0028] In the aforementioned scenarios, the relevant technologies typically rely on a single communication channel to uniformly carry vehicle-side interaction data and cloud-based AI inference data. When the cloud-based AI inference service generates a large amount of streaming output data, this single communication channel is easily occupied by inference data, causing critical vehicle-side interaction messages to queue and resulting in issues such as voice dialogue requests not being uploaded in a timely manner and dialogue results not being delivered promptly. This leads to increased voice interaction response latency and decreased stability. Therefore, the above solutions suffer from the problem of vehicle-side communication being easily blocked by AI inference traffic, making it difficult to guarantee the real-time performance and stability of vehicle-side voice interaction in high-concurrency AI inference scenarios.

[0029] Based on this, the in-vehicle voice interaction method provided by the present invention configures the first communication module in the cloud Link module for communication with the vehicle and the second communication module for communication with the cloud AI inference terminal. The first and second communication modules are made independent of each other, with the first communication module having a higher communication priority than the second. This isolates vehicle-side communication from AI inference-side communication in the channel. Therefore, even when the second communication module is busy or experiencing abnormal AI inference data transmission, the first communication module can still prioritize sending and receiving vehicle-side messages, avoiding the blocking of AI inference traffic on vehicle voice dialogue requests. Furthermore, when a voice dialogue request is received from the vehicle through the first communication module, an inference request is sent to the cloud AI inference terminal through the second communication module. Since the sending of the inference request is confined to the second communication module, it does not occupy the communication resources of the first communication module, further ensuring the first communication module's ability to respond instantly to other vehicle interaction messages. Furthermore, after receiving the dialogue inference results returned by the cloud-based AI inference terminal via the second communication module, the dialogue inference results are sent to the vehicle via the first communication module. Since the first communication module has a higher communication priority than the second communication module, even if the second communication module is still continuously receiving inference data, the dialogue inference results can still be preferentially sent to the vehicle via the first communication module, thereby reducing the overall response latency of voice interaction. In summary, this invention achieves backpressure isolation between vehicle-side communication and AI inference-side communication through the independent setting and priority configuration of the first and second communication modules, ensuring the real-time performance and stability of key vehicle-side interactions while improving the response efficiency of voice interaction.

[0030] Among them, the back pressure isolation between vehicle-side communication and AI inference-side communication means that vehicle-side communication and AI inference-side communication do not affect each other. No matter how much or how slow the AI ​​inference data is, it will not block the timely sending and receiving of key interactive messages on the vehicle side.

[0031] Figure 1 This is a schematic diagram illustrating an application scenario of an in-vehicle voice interaction method provided in an embodiment of this application. For example... Figure 1 As shown, this application scenario involves: vehicle 101 and in-vehicle voice interaction system 102. Vehicle 101 and in-vehicle voice interaction system 102 are interconnected.

[0032] Among them, vehicle 101 can be, but is not limited to, pure electric vehicle (PEV / BEV), hybrid electric vehicle (HEV), range-extended electric vehicle (REEV), plug-in hybrid electric vehicle (PHEV), new energy vehicle, etc.

[0033] In some embodiments, the in-vehicle voice interaction system 102 includes a cloud Link module 1021, a cloud AI inference terminal 1022, a voice recognition server 1023, and a voice synthesis server 1024.

[0034] The in-vehicle voice interaction system 102 can be a service cluster deployed in the cloud, used to provide vehicle 101 with functions such as voice recognition, AI dialogue, voice synthesis and conversation management. Vehicle 101 interacts with the in-vehicle voice interaction system 102 through a communication network.

[0035] It should be understood that a communication connection is established between the vehicle 101 and the cloud Link module 1021. The cloud Link module 1021 acts as a communication middleware, interacting with the speech recognition server 1023, the cloud AI inference server 1022, and the speech synthesis server 1024 respectively, to realize pipelined processing of speech recognition, AI dialogue, and speech synthesis.

[0036] The cloud Link module 1021 includes a first communication module and a second communication module. The first communication module is used to communicate with the vehicle 101, and the second communication module is used to communicate with the cloud AI inference terminal 1022. The first communication module and the second communication module are independent of each other, and the communication priority of the first communication module is higher than that of the second communication module.

[0037] In one possible implementation, the cloud Link module 1021 can be deployed on a cloud server or edge computing node, either as a single instance or in a multi-instance cluster. When deployed in a multi-instance cluster, the instances collaborate through distributed caching and message queues. The cloud Link module 1021 acts as a communication middleware between the vehicle 101 and various cloud services, responsible for session management, stream bridging, and routing forwarding. Based on the independent configuration of the first and second communication modules, it achieves isolation between communication on the vehicle 101 side and communication on the AI ​​inference side.

[0038] In one possible implementation, the first communication module is used to establish and maintain a long-lived connection between vehicle 101 and cloud Link module 1021, carrying control signaling, voice dialogue requests, and dialogue result transmission data from vehicle 101. For example, the first communication module communicates with vehicle 101 based on the WSS protocol. Here, WSS refers to a WebSocket-based secure transmission protocol that enables the establishment of a full-duplex long-lived connection between vehicle 101 and cloud Link module 1021, simultaneously carrying the transmission of text frames and binary frames.

[0039] In one possible implementation, the second communication module is used to establish and maintain the connection between the cloud Link module 1021 and the cloud AI inference terminal 1022, carrying inference requests sent by the cloud Link module 1021 to the cloud AI inference terminal 1022, and streaming dialogue inference results returned by the cloud AI inference terminal 1022. For example, the second communication module communicates with the cloud AI inference terminal 1022 based on the server-sent events (SSE) protocol. The SSE protocol, based on HTTP, enables one-way streaming data transmission, allowing the cloud AI inference terminal 1022 to continuously push dialogue inference results to the cloud Link module 1021 in a streaming manner.

[0040] In one possible implementation, the cloud Link module 1021 may further include a third communication module. This third communication module communicates with the speech recognition server 1023, carrying the transmission of speech recognition requests and results. Exemplarily, the third communication module communicates with the speech recognition server 1023 based on the WebSocket protocol. The WebSocket protocol is a full-duplex communication protocol that enables the establishment of a persistent connection between the cloud Link module 1021 and the speech recognition server 1023, supporting real-time interaction of audio data and recognition results. The speech recognition (ASR) server is described above.

[0041] The Cloud Link module 1021 also includes a processing module, which can be a processor or a device with processing capabilities.

[0042] In one possible implementation, in the aforementioned in-vehicle voice interaction system 102, the processing module is configured such that: the vehicle sends a voice dialogue request to the cloud Link module via the first communication module; the cloud Link module sends a reasoning request to the cloud AI inference terminal via the second communication module; the cloud AI inference terminal generates a dialogue reasoning result based on the voice dialogue data corresponding to the reasoning request and returns it via the second communication module; and the cloud Link module then sends the dialogue reasoning result back to the vehicle via the first communication module. Since the first and second communication modules are independent of each other, and the communication priority of the first communication module is higher than that of the second communication module, the transmission of AI inference data will not block the sending and receiving of key interactive messages on the vehicle side, thereby ensuring the real-time performance and stability of the in-vehicle voice interaction.

[0043] In one possible implementation, the cloud-based AI inference terminal 1022 can be an independently deployed AI large-scale model inference server, which can be deployed on a cloud server to provide AI large-scale model inference services. Its function is to generate dialogue inference results based on the voice dialogue data corresponding to the inference request and return them to the cloud Link module 1021 in a streaming manner.

[0044] In one possible implementation, the speech synthesis server 1024 can be a standalone speech synthesis server, which can be deployed in the cloud or a third-party service cluster. Its function is to convert the target inference text fragment into audio data and return the audio data to the cloud Link module 1021. This includes a text-to-speech (TTS) server.

[0045] In some embodiments, the in-vehicle voice interaction system 102 further includes a cloud-based synchronous request server and a distributed session routing.

[0046] The cloud-based synchronous request server is used to send instructions to vehicle 101 from the cloud and synchronously wait for a response from vehicle 101. It completes the instruction sending and response association through the cloud-based Link module 1021. The distributed session routing is used to maintain the mapping relationship between session identifiers, vehicle identity identifiers, and instance identifiers, supporting accurate delivery and session management in the cloud-based Link module 1021 cluster environment.

[0047] In one possible implementation, the cloud synchronization request server can be a synchronization request processing service deployed in the cloud. It can be deployed in the cloud and communicate with the cloud Link module 1021 through a message queue. Its function is to process instruction requests that need to wait for a response from vehicle 101, and to achieve precise cross-instance routing based on the suspended request manager and asynchronous-to-synchronous mechanism.

[0048] As one possible implementation, distributed session routing can be a routing service based on distributed caching, which can be shared across multiple cloud Link module 1021 instances. Its function is to maintain the mapping relationship between session identifier, vehicle identity identifier, instance identifier, and session heartbeat, and support routing decisions for downlink messages, mutual kicking of vehicles 101, and cleanup of zombie routes.

[0049] based on Figure 1 Please see Figure 2 , Figure 2 This is a schematic diagram of an in-vehicle voice interaction system provided in an embodiment of this application. The vehicle-side communication module outputs text signaling and binary audio through a WSS client. The text frames and binary frames are uploaded to the session manager of the Link module in the cloud via a WSS long connection. The session manager manages the WSS connection and then distributes the data to the push routing decision service, ASR stream bridge manager, AI dialogue stream bridge manager, TTS stream bridge manager, and session-level resource cleanup service. The push routing decision service interfaces with the distributed session routing service, accesses the distributed cache multi-dimensional distributed routing table within the distributed infrastructure, and simultaneously achieves cross-instance delivery with the message queue within the distributed infrastructure. The ASR stream bridge manager... The audio stream is uploaded via WebSocket and the recognition results are downloaded via WebSocket. The AI ​​dialogue stream bridge manager relies on SSE to complete the transmission of AI requests and streaming inference results. The TTS stream bridge manager transmits text and streaming audio via HTTP streaming. The three managers forward business requests to external services, calling the speech recognition server, cloud AI inference server, and speech synthesis server in the external services in sequence. In addition, the suspended request manager under the cloud synchronization request server (i.e., the Sync module) connects to the instruction delivery service, and the message queue also completes data communication with the instruction delivery service. The whole system relies on distributed infrastructure to complete the end-to-end message flow.

[0050] It should be noted that the session manager is used to manage the creation, deletion, querying, and push of local WSS connections, which corresponds to... Figure 1 The Cloud Link module 1021 maintains the WSS long-connection function between vehicle 101 and the Cloud Link module 1021; the ASR Stream Bridge Manager is used to manage the creation and cleanup of ASR Stream Bridges, which are connected to... Figure 1 The third communication module and the speech recognition server 1023 work together; the AI ​​dialogue stream bridge manager is used to manage the creation and cleanup of AI dialogue stream bridges, which are connected to... Figure 1 The second communication module and the cloud-based AI inference terminal 1022 work together; the TTS stream bridge manager is used to manage the creation and cleanup of TTS stream bridges, which are connected to... Figure 1 It works in conjunction with the communication module used to communicate with the speech synthesis server 1024.

[0051] The system architecture and business scenarios described in the embodiments of this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided in the embodiments of this application. As those skilled in the art will know, with the evolution of in-vehicle voice interaction technology, communication protocol types and cloud AI inference services and the emergence of new business scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar vehicle-cloud collaborative voice interaction problems.

[0052] The in-vehicle voice interaction method provided in this application will be described below with reference to specific embodiments. These embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0053] Figure 3 This is a flowchart illustrating an in-vehicle voice interaction method provided in an embodiment of this application. Applied to the aforementioned cloud-based Link module, see below. Figure 3 The in-vehicle voice interaction method provided in this application embodiment may include: S301, In response to receiving a voice dialogue request sent by the vehicle through the first communication module, the cloud Link module sends an inference request to the cloud AI inference terminal through the second communication module.

[0054] In this embodiment, the cloud-based Link module is a communication middleware deployed in the cloud, used for data forwarding and session management between the vehicle and various cloud services. The cloud-based Link module includes a first communication module and a second communication module. The first communication module communicates with the vehicle, and the second communication module communicates with the cloud-based AI inference terminal. The first and second communication modules are independent of each other, and the communication priority of the first communication module is higher than that of the second communication module.

[0055] In one possible implementation, the first communication module can be a communication unit used to establish and maintain a long-term connection between the vehicle and the cloud Link module, such as a communication module based on the WSS protocol. The first communication module is used to carry data such as control signaling, voice dialogue requests, and dialogue results from the vehicle side.

[0056] In one implementation, the first communication module establishes a long-lived connection between the vehicle and the cloud-based Link module based on the WSS protocol. This WSS protocol carries both text frames and binary frames, distinguishing different data streams through frame type. The text frame carries structured JSON control signaling, including fields such as msgId, code, data, chatId, and last. msgId is a unique message identifier, code is a service code used to identify the service type of the current message, data is the service data, chatId is the dialogue ID used to associate uplink and downlink messages in the same round of dialogue, and last indicates whether it is the last frame. The binary frame carries raw audio data, using a custom frame header format. Bytes 0 to 1 are the service code used to distinguish the audio type; bytes 2 and 3 are the frame type (e.g., 0x0001 = audio data frame, 0x0002 = end frame, 0x0003 = text start identifier frame, 0x0004 = text end identifier frame), used to distinguish audio data frames, end frames, text start identifier frames, and text end identifier frames; subsequent bytes carry the audio data or the corresponding attribution identifier. By distinguishing text and audio data by frame type over the same long connection, the first communication module can simultaneously carry the transmission of control signaling and audio data, reducing the communication complexity caused by multi-channel switching.

[0057] For example, the `code` field is a business code used to identify the business type corresponding to the current message. For instance, `AI_CHAT_START` indicates initiating an AI dialogue, and `TTS_START` indicates starting speech synthesis. Bytes 0 to 1 are business codes used to distinguish audio types. For example, 0x0001 represents ASR audio, i.e., the raw audio data for speech recognition uploaded by the vehicle, and 0x0002 represents TTS audio, i.e., the speech-synthesized audio data sent from the cloud. Bytes 2 to 3 are frame types used to distinguish the specific purpose of the current binary frame. For example, 0x0001 represents an audio data frame, 0x0002 represents an end frame, 0x0003 represents a text start identifier frame, and 0x0004 represents a text end identifier frame. The attribution identifier is used to associate audio data with the corresponding message. For example, using a 2-byte length field plus an N-byte UTF-8 encoded `msgId`, the 2 bytes first indicate the byte length of `msgId`, and then the N bytes carry the content of `msgId` itself, enabling the receiver to accurately parse the message attribution corresponding to the audio data.

[0058] In one possible implementation, the second communication module can be a communication unit used to establish and maintain the connection between the cloud Link module and the cloud AI inference terminal, such as a communication module based on the SSE protocol. The second communication module is used to carry inference requests sent by the cloud Link module to the cloud AI inference terminal, as well as streaming dialogue inference results returned by the cloud AI inference terminal.

[0059] By independently carrying the WSS channel corresponding to the first communication module and the SSE bypass channel corresponding to the second communication module according to their respective data characteristics, the two can independently perform disconnection recovery and flow control, avoiding overall communication obstruction due to local congestion when a single channel carries all the data.

[0060] It should be understood that the first communication module and the second communication module are independent of each other, meaning that they are not dependent on each other in terms of communication links, data transmission, and resource usage. The data transmission status of either communication module will not directly affect the basic communication functions of the other. The first communication module has a higher communication priority than the second communication module, meaning that when both have data to be processed simultaneously, the system prioritizes scheduling the data transmission and reception of the first communication module to ensure the timely processing of critical interactive messages on the vehicle side.

[0061] For example, when streaming backpressure occurs at the cloud-based AI inference terminal, the SSE channel corresponding to the second communication module will experience limited transmission rate, while the WSS channel corresponding to the first communication module will not be affected. In this situation, the vehicle can immediately issue a cancellation command via the WSS channel and quickly receive a response from the cloud. If only a single-channel solution is used, the cancellation command will be queued along with audio and text data, preventing timely issuance and thus hindering the rapid termination of AI inference and speech synthesis tasks that are consuming computing power. Streaming backpressure refers to the phenomenon where the speed at which the cloud-based AI inference terminal generates streaming output exceeds the transmission capacity of the second communication module, causing data backlog and a decrease in transmission rate in the transmission channel.

[0062] Meanwhile, the WSS channel and SSE channel in this application support independent disconnection and reconnection logic, and they will not block each other. If the SSE channel is disconnected due to a restart of the cloud AI inference client, it can rely on Last... Event ID transmission resumes from where it left off; throughout the entire process of SSE disconnection and retransmission, the WSS channel can still transmit vehicle control signaling normally. Conversely, when a vehicle experiences a network switch, causing the WSS channel to disconnect and reconnect, the AI ​​inference task on the SSE channel side can continue to run and continuously output inference results; after the WSS channel reconnects, the generated inference results are pushed to the vehicle without triggering another round of AI inference.

[0063] In this embodiment, the cloud-based AI inference terminal is used to infer the corresponding dialogue inference result based on the voice dialogue data corresponding to the inference request.

[0064] It should be understood that the cloud-based AI inference server can be an independently deployed AI large-scale model inference server. It receives inference requests forwarded by the cloud-based Link module through a second communication module, performs inference based on the corresponding voice dialogue data, and returns the dialogue inference results in a streaming manner. The streaming manner means that during the inference process, the cloud-based AI inference server does not wait for all inference results to be generated before returning them all at once. Instead, it immediately sends a portion of the results to the cloud-based Link module through the second communication module as soon as they are generated. The cloud-based Link module can then perform further processing on the received portions of the results. This method allows the transmission of inference results to proceed synchronously with the inference process, thereby reducing the overall waiting time for the voice dialogue.

[0065] In one possible implementation, the vehicle sends a voice dialogue request to the cloud-based Link module via the first communication module. After receiving the voice dialogue request via the first communication module, the cloud-based Link module sends a corresponding inference request to the cloud-based AI inference terminal via the second communication module. Since the first and second communication modules are independent of each other, sending the inference request does not consume the communication resources of the first communication module, which can still be used to receive other messages from the vehicle or send data to the vehicle.

[0066] Based on S301, since the vehicle sends a voice dialogue request to the cloud Link module through the first communication module, and the cloud Link module receives the voice dialogue request through the first communication module, it sends a corresponding inference request to the cloud AI inference terminal through the second communication module. This ensures that the reception of vehicle-side interactive data and the transmission of AI inference requests are completed through the independent first and second communication modules, respectively, and the transmission of inference requests does not consume the communication resources of the first communication module. Furthermore, since the communication priority of the first communication module is higher than that of the second communication module, even when the second communication module is busy with AI inference data transmission, the first communication module can still prioritize sending and receiving key vehicle-side interactive messages, avoiding the reception of voice dialogue requests and the transmission of dialogue results being blocked by AI inference traffic, thereby ensuring the real-time performance and stability of vehicle-side voice interaction.

[0067] S302, In response to receiving at least a portion of the dialogue reasoning result returned by the cloud AI inference terminal through the second communication module, the cloud Link module sends the dialogue reasoning result to the vehicle through the first communication module.

[0068] The dialogue reasoning result refers to the response content generated by the cloud-based AI reasoning terminal after reasoning based on the voice dialogue data corresponding to the reasoning request. At least a portion of the dialogue reasoning result refers to a part of the response content. This at least portion of the dialogue reasoning result is returned from the cloud-based AI reasoning terminal to the cloud Link module through the second communication module.

[0069] In one possible implementation, after the cloud-based AI inference terminal completes inference on a portion of the voice dialogue data and generates at least a part of the dialogue inference result, it sends the at least part of the dialogue inference result to the cloud-based Link module via the second communication module. After receiving the at least part of the dialogue inference result via the second communication module, the cloud-based Link module sends the at least part of the dialogue inference result to the vehicle via the first communication module. The cloud-based AI inference terminal continues to generate the remaining part of the dialogue inference result, and the cloud-based Link module continues to receive the remaining part via the second communication module and send the remaining part to the vehicle via the first communication module, until the entire content of the dialogue inference result has been sent.

[0070] Since the first and second communication modules are independent of each other, the reception and transmission of dialogue inference results are executed on different communication modules. The process of the second communication module receiving dialogue inference results does not consume the communication resources of the first communication module. At the same time, the communication priority of the first communication module is higher than that of the second communication module. While the second communication module is still continuously receiving streaming dialogue inference results returned by the cloud AI inference terminal, the first communication module can prioritize the execution of the dialogue inference results being sent to the vehicle, thus preventing the dialogue inference results from being stuck in the cloud Link module due to the continuous reception by the second communication module.

[0071] In some embodiments, the cloud-based AI inference terminal returns dialogue inference results in a streaming manner, which includes multiple inference fragments arriving sequentially. The cloud-based Link module receives at least a portion of the dialogue inference results via a second communication module, the at least a portion including at least one received inference fragment; in response to receiving at least one inference fragment, the cloud-based Link module sequentially sends the received inference fragments to the vehicle via a first communication module. Since receiving and sending are performed via the second and first communication modules respectively, the sending process does not need to wait for all inference fragments to be received on the second communication module, thereby reducing the latency in sending the dialogue inference results.

[0072] For example, when the cloud-based AI inference module generates three consecutive inference segments in response to a voice dialogue request, the cloud Link module can send the received first inference segment to the vehicle through the first communication module while the second communication module receives the second inference segment. This allows the receiving and sending to be performed in parallel on two independent communication modules, improving the response speed of voice interaction.

[0073] Based on S302, since the cloud-based Link module receives at least a portion of the dialogue inference results returned by the cloud-based AI inference terminal through the second communication module, and then sends this portion to the vehicle through the first communication module, the reception and transmission of the dialogue inference results are executed through the independent first and second communication modules, respectively. The process of the second communication module receiving the dialogue inference results does not consume the communication resources of the first communication module. Simultaneously, since the communication priority of the first communication module is higher than that of the second communication module, while the second communication module continues to receive the remaining portion of the dialogue inference results, the first communication module can prioritize sending the received portion to the vehicle without waiting for the second communication module to finish receiving all the inference segments. This avoids the received content remaining in the cloud-based Link module due to the continuous reception by the second communication module, thereby reducing the response latency of voice interaction and improving the real-time delivery of dialogue results.

[0074] Based on S301-S302, the voice dialogue requests sent by the vehicle are received by the cloud-based Link module through the first communication module and forwarded to the cloud-based AI inference terminal for inference via the second communication module. The dialogue inference results returned by the cloud-based AI inference terminal are received by the cloud-based Link module through the second communication module and then sent back to the vehicle via the first communication module. This ensures that both request uploading and result distribution are completed collaboratively by the independent first and second communication modules. In this process, the transmission of AI inference data is confined to the second communication module, without consuming the communication resources of the first communication module. Furthermore, the communication priority of the first communication module is higher than that of the second communication module. When the second communication module experiences transmission limitations due to heavy AI inference traffic, the first communication module can still prioritize the reception of vehicle-side voice dialogue requests and the distribution of dialogue inference results. Therefore, channel isolation and priority protection are achieved between key vehicle-side interactions and cloud-based AI inference data, preventing AI inference traffic from blocking key vehicle-side interactions and ensuring the real-time performance and stability of in-vehicle voice interaction.

[0075] In some embodiments, please refer to Figure 4 , Figure 4 This is a flowchart illustrating a method for distributing dialogue reasoning results. Based on step S302, "in response to receiving at least a portion of the dialogue reasoning result returned by the cloud AI reasoning terminal through the second communication module, the cloud Link module sends the dialogue reasoning result to the vehicle through the first communication module," it includes: S401, Receive at least a portion of the dialogue reasoning results returned by the cloud AI inference terminal through the second communication module.

[0076] The dialogue reasoning result comprises multiple reasoning text fragments. The dialogue reasoning result refers to the response content generated by the cloud-based AI reasoning end after reasoning from the voice dialogue data corresponding to the reasoning request. The cloud-based AI reasoning end refers to the AI ​​reasoning server used to infer the dialogue reasoning result based on the voice dialogue data corresponding to the reasoning request. A reasoning text fragment refers to a text unit segmented in the dialogue reasoning result according to its generation order. At least a portion of the dialogue reasoning result includes at least one reasoning text fragment from the multiple reasoning text fragments.

[0077] It should be understood that at least one inference text fragment is the content of the dialogue inference result obtained progressively as the dialogue inference result is continuously generated. The inference text fragment is obtained by the cloud Link module segmenting the received dialogue inference result based on preset segmentation conditions during the process of receiving the dialogue inference result through the second communication module. Specifically, during the streaming reception of the dialogue inference result, the cloud Link module compares the received text content with the preset segmentation conditions; when the comparison result indicates that the text content meets the preset segmentation conditions, the cloud Link module segments the text content into an inference text fragment. This inference text fragment can be identified as the target inference text fragment and used to trigger subsequent speech synthesis operations. In this way, the segmentation action is performed on the cloud Link module side, without waiting for the cloud AI inference end to return the complete dialogue inference result; inference text fragments usable for speech synthesis can be dynamically obtained during the reception process.

[0078] In one possible implementation, the cloud-based AI inference module returns the dialogue inference results in a streaming manner during the inference process. Each time a portion of text is generated, it sends the corresponding inferred text fragment to the cloud-based Link module via a second communication module. The cloud-based Link module receives each inferred text fragment sequentially through the second communication module, allowing the dialogue inference results to arrive in a fragmented form. This eliminates the need to wait for all inference results to be generated before receiving them all at once, thus laying the data foundation for subsequent segment-based speech synthesis and data delivery, and helping to shorten the response chain of voice interaction.

[0079] For example, after the speech recognition service returns the final recognition result, the cloud-based Link module pushes the result to the vehicle. The vehicle sends an AI dialogue initiation signal, which carries the chatId and the recognized text. The cloud-based Link module creates an AI dialogue stream bridge in the same session context and initiates an inference request to the cloud-based AI inference terminal via the SSE protocol. The cloud-based AI inference terminal then begins streaming back the dialogue inference results, which the cloud-based Link module receives sequentially through the second communication module.

[0080] S402. In the process of receiving at least a portion of the dialogue reasoning results, whenever one of the reasoning text segments in at least one reasoning text segment satisfies a preset segmentation condition, the reasoning text segment is determined as the target reasoning text segment.

[0081] Among them, the target inference text segment refers to the inference text segment that meets the preset segmentation conditions and can be used to trigger subsequent speech synthesis.

[0082] As one possible implementation, the preset segmentation conditions include: the reasoning text segment contains sentence-ending punctuation; or, the cumulative number of characters in the reasoning text segment reaches a preset threshold.

[0083] In one possible implementation, during the process of receiving at least a portion of the dialogue inference results, whenever one of the inference text segments in at least one inference text segment meets a preset segmentation condition, the cloud-based Link module identifies that inference text segment as the target inference text segment. In this way, the cloud-based Link module can dynamically identify text units suitable for speech synthesis during the streaming reception of dialogue inference results, without waiting for all dialogue inference results to be returned before segmentation processing.

[0084] For example, during the process of receiving at least a portion of the dialogue inference results, whenever a text fragment of type "text" is received in an SSE event frame, the cloud-based Link module determines whether the text fragment meets the minimum granularity requirement for speech synthesis. Based on semantic integrity judgment, when the text fragment encounters sentence-ending punctuation such as a period, question mark, or exclamation mark, it is determined that the text fragment is a complete segment and identified as the target inference text fragment. For long text streams without punctuation, segmentation is performed based on the cumulative character count reaching a preset threshold. The suggested range for this preset threshold is 20 to 50 characters, with the specific value determined according to the minimum synthesis granularity requirement of the speech synthesis server. If the threshold is too short, it will lead to a decrease in speech synthesis efficiency; if the threshold is too long, it will increase the initial synthesis delay. Thus, semantic integrity can be guaranteed based on sentence-ending punctuation, while the character threshold can prevent long text streams without punctuation from waiting indefinitely, thereby achieving a balance between the initial synthesis trigger time and synthesis efficiency.

[0085] As one possible implementation, as the dialogue reasoning results continue to be returned, reasoning text fragments that do not meet the preset segmentation conditions continue to accumulate until they meet the preset segmentation conditions and are identified as target reasoning text fragments, and then enter the subsequent synthesis and sending process.

[0086] S403. Send a synthesis request for the target reasoning text segment to the speech synthesis server and receive audio data.

[0087] The speech synthesis server is used to convert the target reasoning text fragments into audio data. Audio data refers to the playable audio content obtained after speech synthesis from the target reasoning text fragments.

[0088] In one possible implementation, after determining the target inference text segment, the cloud-based Link module sends a synthesis request for that target inference text segment to the speech synthesis server. The speech synthesis server, based on the synthesis request, converts the target inference text segment into corresponding audio data and returns the audio data to the cloud-based Link module. The cloud-based Link module receives this audio data for subsequent transmission to the vehicle. Since synthesis can be triggered as soon as the target inference text segment is determined, there is no need to wait for all dialogue inference results to be received, thus reducing the waiting time for generating the first audio segment.

[0089] In one implementation, for each complete segment, the cloud-based Link module immediately encapsulates the target inference text fragment and its corresponding msgId into a synthesis task, triggering the TTS stream bridge to initiate an HTTP streaming synthesis request to the speech synthesis server, without waiting for all dialogue inference results to be received.

[0090] One implementation involves a pipelined parallel processing between AI inference and speech synthesis. While the cloud-based AI inference terminal outputs the Nth inferred text segment, speech synthesis of the (N-1)th target inferred text segment is already underway or completed, and the (N-2)th audio segment is being pushed to the vehicle. This pipeline parallelism is controlled by segment granularity: the finer the segmentation, the earlier the first segment synthesis is triggered, and the lower the end-to-end initial sound response latency; however, excessively fine segmentation results in short individual text segments, leading to decreased speech synthesis efficiency. Therefore, a balance must be struck between initial sound latency and synthesis efficiency. As a result, the end-to-end speech interaction latency is reduced from a serial mode where synthesis occurs only after all AI inference is completed to a parallel mode that proceeds segment by segment, significantly reducing initial sound response latency.

[0091] S404. Audio data is sent to the vehicle via the first communication module.

[0092] As one possible implementation, after receiving the audio data returned by the speech synthesis server, the cloud Link module sends the audio data to the vehicle through the first communication module, so that the audio data obtained from speech synthesis can be delivered to the vehicle in a timely manner.

[0093] For example, the streaming audio frames returned by the speech synthesis server are encapsulated into WSS binary frames in real time by the cloud Link module. These WSS binary frames carry a msgId attribution identifier and are pushed to the vehicle through the first communication module. After receiving the audio data, the vehicle can accurately associate the audio data with the corresponding dialogue turn based on the msgId attribution identifier, achieving correct matching of audio and text in multi-turn dialogue scenarios, avoiding audio attribution confusion, and improving the accuracy of voice interaction.

[0094] Based on S401 to S404, since the dialogue reasoning results are returned in a streaming manner and received sequentially, when a certain reasoning text segment meets the preset segmentation conditions, it is identified as the target reasoning text segment and speech synthesis is triggered, without waiting for all dialogue reasoning results to be received. Therefore, the reception of dialogue reasoning results, speech synthesis, and the distribution of audio data proceed sequentially at the granularity of the reasoning text segments, realizing the parallel pipeline of AI reasoning and speech synthesis, and reducing the waiting time for the vehicle to receive a voice response. Simultaneously, since the audio data is distributed through the first communication module and carries a msgId attribution identifier, the vehicle side can accurately match audio and text in multi-turn dialogue scenarios, thereby improving the overall real-time performance, stability, and accuracy of in-vehicle voice interaction.

[0095] In some embodiments, please refer to Figure 5 , Figure 5 This is a flowchart illustrating a speech recognition process provided in an embodiment of this application. The cloud Link module further includes a third communication module; the third communication module is used to communicate with the speech recognition server; before responding to receiving a voice dialogue request sent by the vehicle through the first communication module, it also includes: S501, In response to receiving a voice recognition request sent by the vehicle through the first communication module, the voice recognition request is forwarded to the voice recognition server through the third communication module.

[0096] The speech recognition server is used to convert the speech data corresponding to the speech recognition request into a recognition result.

[0097] The third communication module refers to the communication unit in the cloud-based Link module used for communication with the speech recognition server. A speech recognition request is a message sent by the vehicle requesting speech recognition. The speech recognition server converts the speech data corresponding to the speech recognition request into a recognition result. The first, second, and third communication modules are independent communication units in the cloud-based Link module, used for communication with the vehicle, the cloud-based AI inference terminal, and the speech recognition server, respectively.

[0098] In one possible implementation, when a vehicle needs to initiate voice recognition, the vehicle sends a voice recognition request to the cloud-based Link module via the first communication module. After receiving the voice recognition request via the first communication module, the cloud-based Link module forwards the request to the voice recognition server via the third communication module. Since the third communication module is independent of the first communication module, sending the voice recognition request does not consume the communication resources of the first communication module, and the first communication module can still be used to receive other messages from the vehicle or send data to the vehicle during this process.

[0099] As one possible implementation, the third communication module establishes a connection with the speech recognition server based on the WebSocket protocol. While the connection between the third communication module and the speech recognition server is not yet ready, the cloud Link module temporarily stores the audio data sent by the vehicle through the first communication module in a buffer queue. After the connection is established, the temporarily stored audio data is sent to the speech recognition server in batches and switched to direct transmission mode to avoid loss of the first packet of audio, thereby ensuring the integrity of speech recognition.

[0100] S502, In response to receiving the recognition result returned by the voice recognition server through the third communication module, the recognition result is sent to the vehicle through the first communication module.

[0101] The recognition result refers to the text content generated by the speech recognition server after recognizing the speech data.

[0102] It should be understood that after the voice recognition server completes the recognition, the cloud Link module receives the recognition result returned by the voice recognition server through the third communication module, and sends the recognition result to the vehicle through the first communication module.

[0103] As one possible implementation, since the reception of the recognition result is performed through the third communication module and the transmission of the recognition result is performed through the first communication module, and the first and third communication modules are independent of each other, the transmission of the recognition result does not occupy the communication resources of the third communication module, thereby avoiding any impact on data reception during the speech recognition process. Simultaneously, the communication priority of the first communication module is higher than that of the second communication module. Even when the second communication module has AI inference data to be processed, the first communication module can still prioritize sending the recognition result to the vehicle, ensuring the real-time nature of the interaction on the vehicle side.

[0104] As one possible implementation, after the cloud Link module sends the recognition result to the vehicle through the first communication module, the vehicle can initiate a voice dialogue request based on the recognition result, thereby entering the subsequent AI dialogue process.

[0105] For example, when a user says "Please turn on the air conditioning" using the vehicle's voice input function, the vehicle collects the user's voice as audio data and sends a voice recognition request along with the corresponding audio data to the cloud Link module via the first communication module. After receiving the voice recognition request via the first communication module, the cloud Link module forwards it to the voice recognition server via the third communication module. The voice recognition server recognizes the audio data, generates the recognition result "Please turn on the air conditioning," and returns it to the cloud Link module via the third communication module. The cloud Link module then sends this recognition result to the vehicle via the first communication module, allowing the vehicle to display the recognized text to the user or initiate subsequent voice dialogue requests based on the recognition result.

[0106] Based on S501-S502, since the voice recognition request is received by the first communication module and then forwarded to the voice recognition server by the third communication module, and the recognition result is received by the third communication module and then sent down by the first communication module, the vehicle-side interaction and the voice recognition server-side communication during the voice recognition process are independent of each other and do not interfere with each other, thereby ensuring the real-time performance and stability of the voice recognition process and providing an accurate recognition result basis for subsequent voice dialogue.

[0107] In some embodiments, the cloud Link module manages three independent bridges in parallel within the same WSS session context, which is identified by a unique session identifier (such as wsSessionId). The three bridges are an ASR stream bridge, an AI dialogue stream bridge, and a TTS stream bridge.

[0108] The ASR stream bridge is a bridging unit in the cloud-based Link module used to establish and maintain a WebSocket connection between the vehicle and the speech recognition server, and to forward audio data and recognition results. This ASR stream bridge works in conjunction with the third communication module in the cloud-based Link module, which is used to communicate with the speech recognition server.

[0109] In one implementation, in the uplink direction, the binary audio frame sent by the vehicle through the first communication module is forwarded to the speech recognition service via the ASR stream bridge; in the downlink direction, the ASR recognition result returned by the speech recognition service is encapsulated into a WSS text frame by the ASR stream bridge and then sent to the vehicle through the first communication module.

[0110] The AI ​​Dialogue Stream Bridge is a bridging unit within the Cloud Link module used to establish and maintain an SSE connection between the Cloud Link module and the Cloud AI inference endpoint, and to forward inference requests and streaming results. This AI Dialogue Stream Bridge works in conjunction with the second communication module.

[0111] In one implementation, in the uplink direction, the cloud Link module initiates an SSE request to the cloud AI inference terminal through the AI ​​dialogue stream bridge, and the SSE request is sent to the cloud AI inference terminal via the second communication module; in the downlink direction, the AI ​​streaming result returned by the cloud AI inference terminal is received by the second communication module, encapsulated into a WSS text frame by the AI ​​dialogue stream bridge, and then sent to the vehicle through the first communication module.

[0112] The TTS stream bridge is a bridging unit within the cloud-based Link module used to establish and maintain an HTTP streaming connection between the cloud-based Link module and the speech synthesis server, and to forward text segments and audio frames. This TTS stream bridge works in conjunction with the communication module within the cloud-based Link module used for communication with the speech synthesis server.

[0113] In one implementation, in the uplink direction, text segments are sent to the speech synthesis service via a TTS stream bridge; in the downlink direction, the TTS audio frames returned by the speech synthesis service are encapsulated into WSS binary frames by the TTS stream bridge and then sent to the vehicle through the first communication module.

[0114] In this embodiment, a three-way bridge is used to associate storage through a thread-safe mapping container under the same session dimension, so that the ASR stream, AI dialogue stream and TTS stream under the same session can establish a clear association relationship with the corresponding communication module, and support parallel processing of multiple rounds of dialogue under the same session.

[0115] For example, please refer to Figure 6 As shown, Figure 6 This application provides an interactive diagram illustrating a three-way parallel bridging and cascading termination mechanism, comprising: vehicle uplink data entering the WSS session context wsSessionId and being uniformly managed by the session manager, which is used to bind the lifecycle of the three parallel bridges. The vehicle uplink data is sent to the ASR stream bridge in the form of PCM audio with binary frame 0x0001; the AI_CHAT_START text frame is distributed to the AI ​​dialogue stream bridge.

[0116] The ASR stream bridge sends data to the speech recognition server via WebSocket to obtain recognition results. The AI ​​dialogue stream bridge connects to the cloud AI inference via SSE requests to obtain SSE streaming data of type=text. The AI ​​dialogue stream bridge segments the SSE streaming data of type=text. The TTS stream bridge receives the segmented segments and initiates HTTP streaming requests accordingly, calling the speech synthesis server to obtain streaming audio frames. The text frame ASR_RESULT, the text frame AI_CHAT, and the 0x0002 binary TTS audio frame are then aggregated to complete the vehicle's downlink output. Furthermore, the lifecycles of the three parallel bridges are interdependent and support cascading termination. Under the AI_CHAT_CANCEL command, the following operations can be executed sequentially: closing the AI ​​dialogue stream bridge (1. closing AI), closing the TTS stream bridge (2. closing TTS), and closing the ASR stream bridge (3. closing ASR). The corresponding three parallel bridges are released atomically with sequential constraints by the session-level resource cleanup service.

[0117] ASR_RESULT is the service code corresponding to the speech recognition result, used to identify that the text frame carries the recognition result returned by the speech recognition server. Specifically, ASR_RESULT_REALTIME indicates an uncertain real-time recognition result, and ASR_RESULT_FINAL indicates a certain final recognition result. Vehicles use this service code to distinguish the type of recognition result for subsequent processing.

[0118] In some embodiments, the voice interaction method of this application further includes: in response to receiving a dialogue cancellation request sent by a vehicle through a first communication module, interrupting the inference request sent to the cloud AI inference terminal through a second communication module, terminating the sending of dialogue inference results to the vehicle through the first communication module, maintaining the connection with the voice recognition server, and clearing the mapping record corresponding to the current session in the multidimensional distributed routing table.

[0119] Here, a dialogue cancellation request refers to a message sent by the vehicle to request the cancellation of the current AI dialogue. A multidimensional distributed routing table is a distributed storage structure used to store session routing information and session mapping relationships. Mapping records refer to the routing entries and session mapping entries in the multidimensional distributed routing table corresponding to the current session.

[0120] For example, when a vehicle sends a dialogue cancellation request through the first communication module, the cloud-based Link module performs the following operations: the AI ​​dialogue stream bridge manager closes the AI ​​dialogue stream bridge for the corresponding chatId, cancels the SSE request, and interrupts AI inference; pending text synthesis tasks in the TTS stream bridge are discarded; and ongoing TTS synthesis is interrupted. This avoids continued consumption of cloud computing power after cancellation. Specifically, user-initiated cancellation is a cascading termination at the single-round dialogue granularity, closing only the AI ​​dialogue stream bridge and TTS stream bridge for the corresponding chatId, while the ASR stream bridge remains available; session-level resource cleanup is a full cleanup at the session granularity, closing all three bridges. After user-initiated cancellation, the same WSS session can immediately initiate a new round of dialogue without rebuilding the ASR stream bridge.

[0121] In some embodiments, the voice interaction method of this application further includes: in response to detecting the termination of the conversation with the vehicle or the disconnection of the first communication module, sequentially interrupting the inference request sent to the cloud AI inference terminal through the second communication module, terminating the synthesis task with the speech synthesis server, and disconnecting the connection with the speech recognition server.

[0122] Session termination refers to the end of the session between the vehicle and the cloud Link module.

[0123] For example, in response to the detection of session termination with the vehicle or disconnection of the first communication module, the cloud Link module performs a multi-resource atomic release with sequential constraints: first, it closes the AI ​​dialogue stream bridge, cancels the SSE request, and interrupts AI inference; then, it closes the TTS stream bridge, interrupts ongoing TTS synthesis, and discards the text task to be synthesized; next, it closes the ASR stream bridge, disconnects the WebSocket connection with the speech recognition server, and releases audio buffer resources; finally, it cleans up distributed routing records. These steps must be performed in this order and cannot be reversed. If the TTS stream bridge is closed before the AI ​​dialogue stream bridge, the new text generated by the AI ​​will not be consumed by TTS, resulting in message loss or anomalies; if the ASR stream bridge is closed before the AI ​​dialogue stream bridge, the ASR recognition result cannot trigger AI dialogue, causing initiated voice input to become unresponsive. This sequential constraint is determined by the data dependencies between the three bridges, rather than a simple resource release, thus ensuring consistent release of the three streams in any exit scenario and avoiding resource leaks.

[0124] In some embodiments, the voice interaction method of this application further includes: querying a multidimensional distributed routing table based on the vehicle identity identifier carried by the new connection; when it is found that the vehicle identity identifier is associated with an old session identifier, triggering a session-level resource release operation for the old session identifier; and after confirming that the session-level resource release operation corresponding to the old session identifier is completed, updating the session identifier corresponding to the new connection to the multidimensional distributed routing table.

[0125] Among them, the vehicle identification identifier (VIN) is the unique identification information used to identify a vehicle. The session identifier is the unique identifier used to identify a session established between the vehicle and the cloud Link module through the first communication module. The multidimensional distributed routing table is a routing table maintained based on a distributed cache to store the mapping relationship between vehicle identification identifiers and session identifiers. This routing table also contains routing information such as session routes, session details, and live heartbeats. The old session identifier refers to the session identifier already existing in the multidimensional distributed routing table associated with the vehicle identification identifier before the new connection is established. The session identifier is used to identify a session established between the vehicle and the cloud Link module through the first communication module. The session-level resource release operation refers to the unified cleanup operation of the three bridges and routing records corresponding to the old session identifier.

[0126] In one implementation, when a vehicle initiates a new connection, the new connection carries a vehicle identification identifier. The cloud-based Link module queries the multidimensional distributed routing table based on this vehicle identification identifier to determine if an old session identifier is already associated with the vehicle identification identifier. If an old session identifier is found to be associated with the vehicle identification identifier, it indicates that the vehicle already has an old active session. At this point, a session-level resource release operation is triggered for the old session identifier to close the AI ​​dialogue stream bridge, TTS stream bridge, and ASR stream bridge corresponding to the old session, and to clear the routing records corresponding to the old session identifier. After confirming that the session-level resource release operation corresponding to the old session identifier is complete, the cloud-based Link module updates the session identifier corresponding to the new connection to the multidimensional distributed routing table, completing the route switch.

[0127] One implementation allows only one active session identifier to be associated with the same vehicle identity at a time. When a new connection is established, the cloud-based Link module looks up the old session identifier through the mapping relationship between vehicle identity identifiers and session identifiers in the multi-dimensional distributed routing table. This old session identifier is added to a mutual kick-out flag set, a thread-safe set maintained in the memory of the cloud-based Link module instance. This set stores the old session identifiers replaced by the new connection and removes them after the old session resources are cleaned up. Subsequently, the cloud-based Link module actively closes the old connection corresponding to the old session. The old connection's closing callback skips duplicate cleanup after detecting the mutual kick-out flag, and the session-level resource release operation is uniformly triggered by the new connection establishment process, completing the cascading cleanup of the three bridges and routing records. Here, an instance refers to an independent running node of the cloud-based Link module deployed in the cluster. Each instance independently maintains its own memory and connection resources, and multiple instances coordinate through distributed caching and message queues.

[0128] One implementation involves that if a new connection fails to be established or is disconnected midway, the old session identifiers corresponding to the mutual kick-out tag set will be cleaned up by the zombie route cleanup mechanism after the TTL timeout, ensuring that no residual routes will appear.

[0129] For example, vehicle A initiates a new connection, carrying a vehicle identity identifier of VIN_A. The cloud-based Link module queries the multidimensional distributed routing table based on VIN_A. If it finds that VIN_A is associated with an old session identifier wsSessionId_old, it triggers a session-level resource release operation for wsSessionId_old, closing the three-way bridge corresponding to the session and clearing the routing record. After confirming the completion of the resource release operation for the session corresponding to wsSessionId_old, the cloud-based Link module assigns a new session identifier wsSessionId_new to the new connection and updates the mapping relationship between VIN_A and wsSessionId_new to the multidimensional distributed routing table, while actively closing the old connection. If the new connection establishment process fails, the old session record corresponding to wsSessionId_old will be cleared by the zombie route cleanup mechanism after the TTL timeout.

[0130] In some embodiments, the voice interaction method of this application further includes: for any one of multiple session heartbeat records: when a session heartbeat record has not been updated for more than a preset session duration threshold, it is determined as a candidate session heartbeat record; based on all candidate session heartbeat records determined at the same time, the routing information corresponding to the candidate session heartbeat records is cleared in sequence.

[0131] The multidimensional distributed routing table also includes multiple session heartbeat records. A session heartbeat record is a record used to document the last active time of each session, and this record is updated periodically according to the session's activity status. A preset session duration threshold is a time threshold used to determine whether a session heartbeat record has expired. Candidate session heartbeat records are session heartbeat records that have not been updated after exceeding the preset session duration threshold. Routing information refers to the session-related mapping records in the multidimensional distributed routing table, including the mapping between session identifiers and instance identifiers, the mapping between vehicle identity identifiers and session identifiers, and session details.

[0132] As one possible implementation, the cloud-based Link module identifies any one of multiple session heartbeat records as a candidate session heartbeat record if that record has not been updated for more than a preset session duration threshold. Based on all candidate session heartbeat records identified at the same time, the cloud-based Link module sequentially cleans up the routing information corresponding to each candidate session heartbeat record, thereby deleting invalid mappings from the session routing table.

[0133] For example, the cloud-based Link module periodically scans live heartbeat records through the session routing service and uses a distributed lock to ensure that only one instance performs the cleanup operation at any given time, avoiding conflicts caused by concurrent cleanup by multiple instances. When a session heartbeat record is detected to have exceeded the TTL threshold without being updated, the routing record corresponding to that session heartbeat record is cleaned up. The default value for this TTL threshold is 180 seconds.

[0134] In some embodiments, the cloud-based Link module also works in conjunction with the cloud-based synchronous request server (Sync module). The cloud-based synchronous request server is a service module deployed in the cloud that manages the suspension and response association of requests when cloud-based services need to issue instructions to vehicles and synchronously wait for vehicle responses. A suspended request refers to a request temporarily suspended by the cloud-based synchronous request server while waiting for a vehicle response. The asynchronous-to-synchronous mechanism refers to the processing mechanism by which the cloud-based synchronous request server converts asynchronous vehicle responses into synchronous waiting results. A message queue is a queue component used to pass messages between service instances in the cloud. Response topics and response tags are used to identify and route response messages in the message queue.

[0135] When a cloud-based service needs to issue a command to a vehicle and synchronously await a response, the cloud-based synchronous request server generates a unique message identifier (msgId) and registers a suspended request in the suspended request manager, returning a waitable object based on an asynchronous-to-synchronous mechanism. The cloud-based synchronous request server sends this msgId and the command content to the cloud-based Link module via a message queue. The push service of the cloud-based Link module records the mapping relationship between the msgId and the response topic and tag when issuing the command, and then issues the command to the vehicle. When the vehicle responds, the cloud-based Link module queries this mapping relationship based on the replyMsgId in the response message to obtain the corresponding response topic and tag, and routes the response precisely to the corresponding cloud-based synchronous request server instance by tag via a message queue. The suspended request manager of the cloud-based synchronous request server completes the suspended request based on the response, and the caller synchronously receives the vehicle's response.

[0136] As one possible implementation, the suspend request is set with a timeout period. If no response is received from the vehicle after the timeout period, the cloud synchronization request server automatically releases the resources corresponding to the suspend request and returns a timeout response to ensure that resources are not permanently occupied in abnormal scenarios.

[0137] Based on the above synchronous request and response association mechanism, cloud services do not need to block and wait after issuing instructions. Instead, they can achieve cross-instance synchronous response by suspending requests and precise routing of message queues. This avoids the overhead of full-instance broadcasting and polling, improves the association efficiency of instruction issuance and response in a cluster environment, and avoids resource retention in abnormal situations through timeout protection.

[0138] In some embodiments, when the cloud Link module needs to push a downlink message to a vehicle, it first queries the session route in the multidimensional distributed routing table based on the session identifier corresponding to the message to obtain the instance identifier corresponding to the session identifier. The session route is used to record the cloud Link module instance where each session identifier is currently located. The cloud Link module determines whether the instance identifier points to its own instance; if it points to its own instance, it directly calls the session manager of its own instance to push the message; if it points to a remote instance, it delivers the message to the target instance corresponding to the instance identifier through a message queue, and the target instance completes the push. Thus, the downlink message does not need to be broadcast to all instances in the cluster, but is delivered to a unique target instance according to the routing table, avoiding full instance broadcast forwarding in a multi-instance cluster, reducing the network overhead of a single push from O(N) to O(1), where N is the number of cluster instances.

[0139] For example, when instance A of the cloud Link module needs to push a message to a vehicle with session identifier wsSessionId_1, instance A queries the multidimensional distributed routing table to determine that the instance identifier corresponding to wsSessionId_1 is instance B. Instance A determines that this instance identifier points to a remote instance and then delivers the message to instance B via a message queue. After receiving the message, instance B pushes the message to the corresponding vehicle through its local session manager. Compared to broadcasting the message to all instances and having each instance determine whether it holds the session, this method significantly reduces network traffic and invalid forwarding.

[0140] In some embodiments, the voice interaction method of this application further includes an ASR audio buffering mechanism. The ASR stream bridge experiences a connection establishment delay when establishing a WebSocket connection with the speech recognition server, typically ranging from 50 milliseconds to 200 milliseconds. While the WebSocket connection is not yet ready, the vehicle begins sending binary audio frames through the first communication module. To prevent audio frame loss during this period, the cloud Link module temporarily stores the received binary audio frames in a buffer queue. Once the WebSocket connection is established and the first configuration packet is sent, the cloud Link module batch refreshes the binary audio frames temporarily stored in the buffer queue to the speech recognition server, and then switches to direct transmission mode, directly sending subsequent received binary audio frames to the speech recognition server. This buffering mechanism ensures that the first packet of audio is not lost, avoiding truncation at the beginning of speech recognition due to the WebSocket connection establishment delay, thereby improving speech recognition quality.

[0141] In some embodiments, the voice interaction method of this application further includes a mechanism for associating TTS audio frames with text messages. The streaming audio frames returned by the speech synthesis server are encapsulated into WSS binary frames, which additionally carry a msgId attribution identifier in their frame header. Upon receiving the WSS binary frame, the vehicle associates the audio data with the corresponding AI text response using the msgId attribution identifier, achieving correct audio-text matching in multi-turn dialogue scenarios. When multiple turns of dialogue occur in parallel within the same WSS session, AI responses corresponding to different chatIds generate independent msgIds. The vehicle then accurately assigns the TTS audio stream to the corresponding dialogue turn, avoiding confusion between audio from different dialogue turns and improving the accuracy of voice interaction in multi-turn dialogue scenarios.

[0142] For example, please refer to Figure 7 , Figure 7 This is a complete flowchart illustrating an in-vehicle voice interaction method provided in an embodiment of this application. Specifically, it includes: S1: Speech recognition stage.

[0143] The vehicle sends a voice recognition request to the cloud Link module via the first communication module. The cloud Link module establishes a WebSocket connection with the voice recognition server via the third communication module. The vehicle continues to upload WSS binary frames carrying PCM audio (service code 0x0001 indicates ASR audio) via the first communication module. When the connection between the third communication module and the voice recognition server is not ready, the audio is temporarily cached. Once the connection is ready, it is refreshed and retransmitted in batches. The voice recognition server returns a real-time recognition result (undetermined) to the cloud Link module. The cloud Link module sends the WSS text frame corresponding to the undetermined real-time recognition result (e.g., ASR_RESULT_REALTIME indicates an undetermined real-time recognition result) to the vehicle via the first communication module. The voice recognition server returns the final recognition result to the cloud Link module. The cloud Link module sends the WSS text frame corresponding to the final recognition result (ASR_RESULT_FINAL indicates a determined final recognition result) to the vehicle via the first communication module.

[0144] S2: Parallel stage of AI dialogue and speech synthesis pipeline.

[0145] The vehicle sends a voice dialogue request (such as AI_CHAT_START signaling) carrying a chatId and recognized text through the first communication module. The cloud Link module initiates an SSE request to the cloud AI inference terminal through the second communication module. The cloud AI inference terminal streams the dialogue inference results and returns inference text segments of type "text". When an inference text segment meets the preset segmentation conditions, the cloud Link module identifies the inference text segment as the target inference text segment and initiates an HTTP streaming synthesis request to the speech synthesis server. The speech synthesis server returns the audio data corresponding to the target inference text segment. The cloud Link module encapsulates the audio data into a WSS binary frame carrying a msgId (service code 0x0002 indicates TTS audio) and sends it to the vehicle through the first communication module. The above segmentation judgment, synthesis request, and audio transmission are executed in a loop. When the cloud AI inference terminal returns an SSE event of type "end", the cloud Link module sends an AI_CHAT end text frame to the vehicle. After the speech synthesis server completes all synthesis, the cloud Link module sends a TTS end binary frame (frame type 0x0002 indicates end frame).

[0146] S3: In scenarios where the user actively cancels.

[0147] The vehicle sends a conversation cancellation request (such as AI_CHAT_CANCEL signaling) through the first communication module. The cloud Link module cancels the SSE request, interrupts the synthesis task, and discards the text task to be processed, thus completing the session interruption process.

[0148] In summary, this application's embodiments ensure the integrity of the recognition results by employing an audio buffering mechanism during the speech recognition stage, preventing the loss of the initial audio packet. During the AI ​​dialogue and speech synthesis stages, segmentation and immediate triggering of synthesis for streaming inference text segments enable parallel pipeline operation of AI inference and speech synthesis, reducing end-to-end voice interaction latency. In user-initiated cancellation scenarios, cascading termination of AI inference and synthesis tasks avoids wasting computational power. Therefore, the overall real-time performance, accuracy, and resource utilization efficiency of in-vehicle voice interaction are improved.

[0149] The in-vehicle voice interaction method according to the embodiments of this application has been described above. The apparatus for executing the above method provided in the embodiments of this application is described below. Those skilled in the art will understand that the method and electronic device can be combined and referenced with each other. The related electronic devices provided in the embodiments of this application can execute the steps in the above-described in-vehicle voice interaction method. The related electronic devices can be referred to in the following description: This application also provides an electronic device, which is described in reference to... Figure 8 The electronic device 800 may include a memory 810, a processor 820, and a communication interface 830. The memory 810, processor 820, and communication interface 830 are connected via internal connection paths. The memory 810 stores instructions, and the processor 820 executes the instructions stored in the memory 810 to control the communication interface 830 to acquire information, thereby enabling the electronic device 800 to implement the aforementioned method. Optionally, the memory 810 may be coupled to the processor 820 via an interface, or it may be integrated with the processor 820.

[0150] It should be noted that the communication interface 830 described above uses a transceiver device, such as, but not limited to, a transceiver. The communication interface 830 may also include an input / output interface.

[0151] The processor 820 stores one or more computer programs, which include instructions. When the instructions are executed by the processor 820, the electronic device 800 performs the methods described in the above embodiments.

[0152] In implementation, each step of the above method can be completed by the integrated logic circuits in the hardware of the processor 820 or by instructions in software form. The method disclosed in the embodiments of this application can be directly implemented by the hardware processor, or by a combination of hardware and software modules in the processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory 810, and the processor 820 reads the information in memory 810 and, in conjunction with its hardware, completes the steps of the above method. To avoid repetition, detailed descriptions are not provided here.

[0153] As one possible implementation, electronic device 800 can be a physical device, such as including one or more of the following modules: central processing unit, microprocessor, application-specific integrated circuit, field-programmable gate array, complex programmable logic device (CPLD), coprocessor (assisting the central processing unit in completing corresponding processing and applications), microcontroller unit (MCU), domain controller (DC), vehicle domain controller unit (VDC), electronic control unit (ECU), cockpit domain controller (CDC), vehicle integrated unit (VIU), vehicle control unit (VCU), motor control unit (MCU), etc. Furthermore, electronic device 800 includes at least one processor integrated in the form of a system-on-chip (SoC), commonly referred to by those skilled in the art as an SoC. The SoC may include at least one processor, and when the SoC includes multiple processors, the types of the multiple processors may be different.

[0154] Optionally, Figure 8 The memory 810 in the middle can store various vehicle data or cloud-based voice data. Figure 8 The processor 820 in the above embodiment can implement the steps in the in-vehicle voice interaction method.

[0155] This application also provides a computer-readable storage medium storing program code that, when run on a computer, causes the computer to perform any of the methods described in the above embodiments.

[0156] This application also provides a computer program product, which includes a computer program that, when run, causes the computer to perform any of the methods described in the above embodiments.

[0157] This application also provides a chip, including: a circuit for performing any of the methods in the above embodiments.

[0158] This application also provides a device, including... Figure 8 The electronic device 800 shown can perform any of the methods in the above embodiments.

[0159] This application also provides a vehicle, including... Figure 8 The electronic device shown is 800.

[0160] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0161] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0162] In the embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0163] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0164] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0165] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to related technologies, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0166] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0167] The terms "first," "second," etc., used in the specification, claims, and drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same attributes in the description of embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of elements is not necessarily limited to those elements, but may include other elements not explicitly listed or inherent to those processes, methods, products, or apparatuses.

[0168] It is understood that, in the embodiments of this application, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

Claims

1. A vehicle-mounted voice interaction method, characterized in that, The application is to a cloud-based Link module, which includes a first communication module and a second communication module. The first communication module is used to communicate with the vehicle, and the second communication module is used to communicate with the cloud-based AI inference terminal. The first communication module and the second communication module are independent of each other, and the communication priority of the first communication module is higher than that of the second communication module. The method includes: In response to receiving a voice dialogue request from a vehicle via the first communication module, the cloud Link module sends a reasoning request to the cloud AI inference terminal via the second communication module; the cloud AI inference terminal is used to infer the corresponding dialogue reasoning result based on the voice dialogue data corresponding to the reasoning request. In response to receiving at least a portion of the dialogue reasoning result returned by the cloud AI inference terminal through the second communication module, the cloud Link module sends the dialogue reasoning result to the vehicle through the first communication module.

2. The method according to claim 1, characterized in that, In response to receiving at least a portion of the dialogue reasoning result returned by the cloud AI inference terminal via the second communication module, the cloud Link module sends the dialogue reasoning result to the vehicle via the first communication module, including: The second communication module receives at least a portion of the dialogue reasoning results returned by the cloud AI reasoning terminal; wherein the dialogue reasoning results include multiple reasoning text fragments; and the at least portion of the dialogue reasoning results includes at least one reasoning text fragment among the multiple reasoning text fragments. During the process of receiving at least a portion of the dialogue reasoning results, whenever one of the at least one reasoning text segments satisfies a preset segmentation condition, the reasoning text segment is identified as the target reasoning text segment; A synthesis request for the target inference text segment is sent to the speech synthesis server, and audio data is received; the speech synthesis server is used to convert the target inference text segment into audio data. The audio data is sent to the vehicle via the first communication module.

3. The method according to claim 2, characterized in that, The preset segmentation conditions include: the reasoning text segment contains sentence-ending punctuation; or, the cumulative character count of the reasoning text segment reaches a preset threshold.

4. The method according to claim 2, characterized in that, The cloud-based Link module also includes a third communication module; the third communication module is used to communicate with the speech recognition server. Prior to receiving a voice dialogue request sent by the vehicle via the first communication module, the method further includes: In response to receiving a voice recognition request sent by a vehicle through the first communication module, the third communication module forwards the voice recognition request to the voice recognition server; the voice recognition server is used to convert the voice data corresponding to the voice recognition request into a recognition result. In response to receiving the recognition result returned by the voice recognition server through the third communication module, the recognition result is sent to the vehicle through the first communication module.

5. The method according to claim 4, characterized in that, Also includes: In response to receiving a dialogue cancellation request from the vehicle via the first communication module, the inference request sent to the cloud AI inference terminal via the second communication module is interrupted, and the dialogue inference result sent to the vehicle via the first communication module is terminated, while maintaining the connection with the speech recognition server.

6. The method according to claim 4, characterized in that, Also includes: In response to the detection of session termination with the vehicle or disconnection of the first communication module, the inference request sent to the cloud AI inference terminal via the second communication module is interrupted, the synthesis task with the speech synthesis server is terminated, and the connection with the speech recognition server is disconnected.

7. The method according to claim 1, characterized in that, Also includes: The multidimensional distributed routing table is queried based on the vehicle identity carried by the new connection; the multidimensional distributed routing table is used to store the mapping relationship between vehicle identity and session identity; The session identifier is used to identify a session established between the vehicle and the cloud Link module through the first communication module; When it is found that the vehicle identity identifier is associated with an old session identifier, a session-level resource release operation is triggered for the old session identifier; After confirming that the session-level resource release operation corresponding to the old session identifier is completed, the session identifier corresponding to the new connection is updated to the multidimensional distributed routing table.

8. The method according to claim 7, characterized in that, The multidimensional distributed routing table also includes multiple session heartbeat records; The method further includes: For any of the multiple session heartbeat records: if the session heartbeat record has not been updated for more than a preset session duration threshold, it is determined as a candidate session heartbeat record; Based on all candidate session heartbeat records determined at the same time, the routing information corresponding to the candidate session heartbeat records is cleared sequentially.

9. An in-vehicle voice interaction system, characterized in that, It includes a cloud-based Link module; the cloud-based Link module includes a first communication module, a second communication module, and a processing module; the first communication module is used to communicate with the vehicle; the second communication module is used to communicate with the cloud-based AI inference terminal; the first communication module and the second communication module are independent of each other, and the communication priority of the first communication module is higher than that of the second communication module. The processing module is configured as follows: In response to receiving a voice dialogue request sent by the vehicle through the first communication module, an inference request is sent to the cloud AI inference terminal through the second communication module; the cloud AI inference terminal is used to infer the corresponding dialogue inference result based on the voice dialogue data corresponding to the inference request; In response to receiving the dialogue reasoning result returned by the cloud AI inference terminal through the second communication module, the dialogue reasoning result is sent to the vehicle through the first communication module.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a program or instructions that, when executed on an electronic device, cause the device to perform the method as described in any one of claims 1-8.

Images