Wi-fi based vehicle multimedia microphone intercom system and method thereof

Abstract

The application discloses a Wi-Fi-based vehicle multimedia microphone intercom system and method, and belongs to the technical field of vehicle multimedia and wireless communication. A vehicle machine generates a one-time authentication token and encodes it into a two-dimensional code; after a terminal scans the code and accesses, the terminal carries out authentication with the token and a device capability set, and the vehicle machine issues a session identifier and audio configuration parameters; the terminal collects encoded microphone audio according to the configuration and sends the audio to the vehicle machine via RTP; the vehicle machine maps a network audio source into a virtual microphone device node, routes the virtual microphone device node to at least one of a driving area, a passenger area, a whole vehicle or an external loudspeaker according to a device identity and a real-time vehicle state, and checks vehicle speed and a parking state before external shouting; the original multimedia audio source state is saved before amplification, and is automatically restored after the amplification. The scheme is compatible with mobile phones, tablets and standard Wi-Fi microphones, replaces traditional wired and special wireless microphones, and is beneficial to improving device adaptability and maintenance convenience.

Description

Technical Field

[0001] This invention relates to the fields of vehicle multimedia audio-visual entertainment systems, wireless local area network communication, mobile terminal audio acquisition, network audio transmission, and vehicle-mounted loudspeaker control technology. In particular, it relates to a system and method applicable to commercial vehicles, buses, tourist buses, school buses, public buses, engineering vehicles, and special vehicles, which uses mobile phones, tablets, Wi-Fi microphones, or other audio acquisition devices with standard Wi-Fi access capabilities to realize vehicle multimedia microphone intercom, tour guide loudspeaker, driver's area announcements, passenger area broadcasts, in-vehicle loudspeaker, and external announcements. Background Technology

[0002] Existing commercial vehicle multimedia entertainment systems typically include microphone playback functionality in the driver's area and microphone playback functionality in the passenger area. Traditional solutions primarily utilize wired microphones or dedicated wireless pairing microphones.

[0003] Wired microphones require the installation of audio cables, dedicated sockets, and audio interfaces within the vehicle, resulting in higher installation and maintenance costs. Over long-term use, the wiring harness is easily damaged by bending or being stepped on. Furthermore, the microphone's placement is limited by cable length, preventing flexible use in any location within the vehicle. Additionally, the interface types, pin definitions, and power supply methods of wired microphones lack standardization across different vehicle models, necessitating the use of original equipment manufacturer (OEM) parts for aftermarket replacements.

[0004] Dedicated wireless microphones typically require dedicated receivers, frequency management, or proprietary pairing protocols. These solutions lead to increased material costs and inconvenient after-sales replacements. In dedicated wireless microphone systems operating in UHF / VHF bands, frequency resources are limited, and issues such as crosstalk, mispairing, and interference from neighboring vehicles can easily occur when multiple vehicles are parked densely or used simultaneously. Furthermore, if a dedicated wireless microphone is lost or damaged, users often need to purchase the same model from the original manufacturer, resulting in high replacement costs and a lengthy replacement cycle.

[0005] Some vehicles are attempting to use Bluetooth HFP / SCO to connect mobile phones as microphones. However, Bluetooth HFP / SCO links are primarily designed for narrowband calls, with sampling rates typically limited to 8kHz or 16kHz and encoding methods fixed at CVSD or mSBC. They suffer from narrow bandwidth, high latency, and poor configurability, making them unsuitable for use as a guide speaker in vehicle multimedia systems, for passenger area announcements, or for external announcements. Furthermore, Bluetooth connection distance is limited, and connection stability is significantly affected by occupant position and the vehicle's metal structure.

[0006] With the development of smartphones, tablets, Wi-Fi microphones, and in-vehicle wireless network capabilities, an increasing number of mobile terminals and audio acquisition devices now possess standard Wi-Fi access capabilities, microphone acquisition capabilities, audio encoding capabilities, and network data transmission capabilities. Vehicle multimedia hosts are also gradually acquiring Wi-Fi hotspot, Wi-Fi client, and local area network communication capabilities.

[0007] However, existing vehicle multimedia systems typically do not uniformly manage mobile phones, tablets, or Wi-Fi microphones as standard accessible vehicle microphone sources. Specifically, existing technologies have the following shortcomings: First, there is a lack of a mechanism to map external network audio sessions to the vehicle's virtual microphone channel. Local microphone input and network audio input in the vehicle's audio framework are separated at the device management level, and network audio sources cannot be uniformly routed and controlled by upper-layer applications as standard microphone devices.

[0008] Second, there is a lack of a complete access authentication and audio parameter negotiation process. Existing solutions either do not perform authentication or only perform simple password authentication, failing to form a closed loop of "one-time token generation - QR code transmission - terminal capability reporting - vehicle-mounted parameter distribution", which makes it impossible to guarantee the security of device access and the consistency of audio parameter adaptation.

[0009] Third, there is a lack of a mechanism for safe routing of speaker areas based on the vehicle's real-time status. In particular, the external announcement function, if relying solely on manual operation by the user, poses a safety hazard due to accidental triggering while driving.

[0010] Fourth, the complete process of saving the state of the original multimedia audio source, selecting processing strategies, and automatically restoring it after the microphone amplification is missing.

[0011] Fifth, there is a lack of adaptive adjustment mechanisms for fluctuations in Wi-Fi wireless network quality. Wi-Fi signals may be interfered with while the vehicle is in motion. If fixed encoding parameters and buffering strategies are used, it will lead to voice stuttering, intermittent playback, or excessive latency.

[0012] Therefore, a vehicle multimedia microphone intercom system based on standard Wi-Fi network capabilities is needed, enabling vehicles to access various audio acquisition devices such as mobile phones, tablets, and Wi-Fi microphones via a universal wireless LAN, and to input voice audio to the in-vehicle multimedia system through a universal audio transmission protocol, in order to replace traditional wired microphones and dedicated wireless pairing microphones, while solving the aforementioned access management, secure routing, state recovery, and network adaptation issues. Summary of the Invention

[0013] The technical problem this invention aims to solve is that existing vehicle microphone solutions suffer from high wiring and maintenance costs and limited usage locations for wired microphones; dedicated wireless microphones rely on dedicated receivers and are prone to crosstalk and mispairing; Bluetooth solutions have narrow bandwidth, high latency, and are unsuitable for amplification scenarios; meanwhile, existing vehicle multimedia systems lack a complete solution based on standard Wi-Fi networks to unify the access, authentication, virtual channel mapping, secure routing, state management, and network adaptation of multiple audio acquisition devices as microphone sources.

[0014] To address the aforementioned technical problems, this invention provides a vehicle multimedia microphone intercom control method based on mobile terminal Wi-Fi audio transmission, applied to a vehicle multimedia host, comprising the following core steps: The vehicle's multimedia host initiates the Wi-Fi microphone access service, generates a one-time authentication token, and encodes this token along with the vehicle's IP address, service port, and service identifier into a QR code, which is then displayed on the vehicle's screen. The wireless audio acquisition terminal obtains access information by scanning the QR code, connects to the vehicle's Wi-Fi network, and sends an authentication request to the vehicle's multimedia host, carrying the one-time authentication token, device fingerprint, device type, and audio capability set. The vehicle's multimedia host verifies the token's validity and timeliness. Upon successful verification, it issues a session identifier and audio configuration parameters, including encoding format, sampling rate, bit rate, packet duration, and RTP port. The vehicle's multimedia host creates a virtual microphone device node based on the session identifier, mapping the wireless audio acquisition terminal as a virtual microphone input device within the vehicle's infotainment system. The wireless audio acquisition terminal acquires and encodes microphone audio according to the audio configuration parameters and sends it to the vehicle's infotainment system via the RTP protocol. Upon receiving the RTP data packets, the vehicle's infotainment system performs jitter buffering, out-of-order reordering, packet loss compensation, decoding, echo suppression, howl suppression, noise suppression, and automatic gain control to obtain the processed microphone audio. Based on the device identity of the virtual microphone node, the real-time vehicle status, and preset routing rules, the vehicle's infotainment system outputs processed microphone audio to at least one of the following: the driver's area, the passenger area, the main vehicle speakers, or external speakers. Before outputting to external speakers, the system must confirm that the vehicle speed is below a preset threshold and that the vehicle is in a parking brake state or that the driver has given permission. Upon first receiving valid audio data and entering microphone amplification mode, the system saves the current multimedia audio source's audio source type, playback progress, volume status, and EQ mode; after the microphone amplification mode ends, it restores the saved playback status.

[0015] In one embodiment, the real-time vehicle status is obtained via a CAN bus, a body control module, or an on-board diagnostic interface, including at least one of vehicle speed, gear position, parking brake status, door status, ACC status, and vehicle operating mode.

[0016] In one implementation, the audio configuration parameters are sent via a TCP control channel or a WebSocket control channel, the encoding format is Opus encoding, the sampling rate is 48kHz or 16kHz, the channel is mono, the packet duration is 20ms, and the audio configuration parameters also include SSRC.

[0017] In one embodiment, the vehicle multimedia host detects the packet loss rate based on the RTP sequence number, calculates the network jitter based on the RTP timestamp, and when the packet loss rate exceeds a preset packet loss threshold or the network jitter exceeds a preset jitter threshold, it sends a bitrate reduction command, packet duration adjustment command, or encoding format switching command to the wireless audio acquisition terminal through the control channel.

[0018] In one implementation, when the vehicle multimedia host fails to receive a valid RTP audio data packet for a preset duration, it automatically mutes the microphone channel corresponding to the virtual microphone device node, restores the original multimedia audio source playback, and displays a network disconnection message on the vehicle interface.

[0019] In one implementation, when multiple wireless audio acquisition terminals request access simultaneously, the vehicle multimedia host selects a main microphone audio source based on at least one of the following mechanisms: device whitelist, service priority corresponding to device type, recent authorization record, user selection, or vehicle terminal confirmation, while the remaining devices remain in standby or are denied access.

[0020] In one implementation, if a navigation voice broadcast, a reversing radar alarm, or a vehicle safety alarm occurs during microphone amplification, the vehicle multimedia host temporarily interrupts or lowers the processed microphone audio, and resumes microphone amplification after the navigation voice broadcast, reversing radar alarm, or vehicle safety alarm ends.

[0021] In one embodiment, the vehicle multimedia host is compatible with third-party Wi-Fi microphones that conform to a preset universal access protocol, which specifies the sampling rate, encoding format, RTP payload format, device identity field, and control command format; third-party Wi-Fi microphones that conform to the preset universal access protocol can be connected without a dedicated hardware receiver.

[0022] In one implementation, the one-time authentication token is randomly generated by the vehicle multimedia host when the Wi-Fi microphone access service is started, and has a preset validity period and a single-use limit. The one-time authentication token automatically expires after being used once or after the preset validity period has expired.

[0023] In one implementation, the QR code also encodes the SSID and connection password of the Wi-Fi network. After scanning the QR code, the wireless audio acquisition terminal automatically connects to the Wi-Fi network and sends the authentication request.

[0024] In one embodiment, the device fingerprint includes at least one of the following: device model, operating system version, hardware unique identifier, and application version number of the wireless audio acquisition terminal.

[0025] In one implementation, the virtual microphone device node registers with the audio framework of the vehicle's multimedia host, enabling the virtual microphone input device and the local wired microphone input device to adopt a unified management strategy in audio routing control, volume adjustment, and priority management.

[0026] In one embodiment, during microphone amplification mode operation, the vehicle multimedia host performs at least one of the following processes on the original multimedia audio source: pausing playback, lowering the volume to a preset ratio, or mixing with microphone audio.

[0027] In one implementation, the triggering conditions for ending the microphone amplification mode include: the user actively turns off the microphone function, no valid RTP audio data packets are received for a continuous preset time, the Wi-Fi network connection is disconnected, the user manually terminates the microphone session, or the one-time authentication token expires.

[0028] This invention also provides a vehicle multimedia microphone intercom system based on mobile terminal Wi-Fi audio transmission, including a wireless audio acquisition terminal, a vehicle multimedia host, a Wi-Fi communication module, a device discovery and authentication module, a network audio receiving module, an audio decoding and processing module, a virtual microphone channel management module, an audio routing control module, a loudspeaker playback module, and a media playback status management module. The functions of each module correspond to the steps described above and will not be repeated here.

[0029] Through the above technical solution, the present invention achieves the following beneficial effects: First, by using one-time tokens and QR code authentication, a standardized access process of "scanning—authentication—parameter distribution" is established, improving the security and convenience of device access. One-time tokens avoid the risk of fixed passwords being intercepted by third parties and remaining valid for a long time, while the QR code transmission method reduces the complexity of users manually entering IP addresses and ports.

[0030] Secondly, by creating virtual microphone device nodes, external Wi-Fi audio acquisition devices are mapped to microphone input sources that can be uniformly managed within the vehicle's audio framework. This enables network microphones and local wired microphones to have a consistent behavior model in terms of routing control, volume adjustment, and priority management, avoiding the complexity of maintaining a separate set of control logic for network audio.

[0031] Third, by combining real-time vehicle status with audio routing safety control, especially by verifying vehicle speed and parking conditions before making external announcements, safety hazards caused by accidental triggering of external announcements while driving can be avoided. This mechanism combines vehicle status data with audio routing decisions, distinguishing it from simple network audio playback solutions.

[0032] Fourth, the media playback status management module saves complete status information such as audio source type, playback progress, volume status, and EQ mode before amplification and restores it afterward, achieving seamless switching between microphone amplification and multimedia playback. Compared to a simple solution that only saves volume, this solution can more accurately restore the user's previous auditory environment after amplification ends.

[0033] Fifth, through network quality monitoring and adaptive adjustment mechanisms, packet loss rate and jitter are dynamically detected based on RTP sequence number and RTP timestamp. When network quality deteriorates, a code reduction or encoding change instruction is sent to the terminal through the control channel to reduce network bandwidth pressure while ensuring voice intelligibility. When the network is interrupted, the system automatically mutes and restores the original audio source to avoid prolonged silence or howling. Attached Figure Description

[0034] Figure 1 This is a schematic diagram of the device discovery and authentication process of the present invention; Figure 2 This is a schematic diagram of the virtual microphone channel establishment and audio transmission processing flow of the present invention; Figure 3 This is a schematic diagram of the audio routing control and vehicle external announcement safety verification process of the present invention; Figure 4 This is a schematic diagram of the process for saving and restoring the original multimedia audio source state in this invention. Detailed Implementation

[0035] The technical solution of the present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. It should be noted that the following specific embodiments are for illustrative purposes only and should not be considered as limiting the scope of the invention.

[0036] like Figures 1 to 4As shown, this embodiment provides a vehicle multimedia microphone intercom system based on mobile terminal Wi-Fi audio transmission, including: a wireless audio acquisition terminal, a vehicle multimedia host, a Wi-Fi communication module, a device discovery and authentication module, a network audio receiving module, an audio decoding and processing module, a virtual microphone channel management module, an audio routing control module, a loudspeaker playback module, a media playback status management module, and a user operation module.

[0037] A wireless audio acquisition terminal is a device with Wi-Fi access and microphone acquisition capabilities, including but not limited to mobile phones, tablets, Wi-Fi microphones, Wi-Fi-enabled tour guide microphones, or smart wearable microphones. The wireless audio acquisition terminal integrates a microphone acquisition unit, an audio encoding unit, a network transmission unit, a service discovery unit, and an authentication unit.

[0038] The vehicle's multimedia head unit is a commercial vehicle audio-visual entertainment system or in-vehicle infotainment system, running Wi-Fi microphone access service. The vehicle's multimedia head unit connects to the original multimedia audio sources (such as radio, USB music, Bluetooth music, online music, etc.) and the vehicle's speaker system.

[0039] The Wi-Fi communication module is integrated into the vehicle's multimedia head unit and is used to establish an IP network connection between the wireless audio acquisition terminal and the vehicle's multimedia head unit. Access methods include a Wi-Fi hotspot provided by the vehicle's multimedia head unit, the vehicle's existing local area network, Wi-Fi Direct, or a mobile terminal hotspot enabled by the wireless audio acquisition terminal.

[0040] The device discovery and authentication module connects to the authentication information database to generate one-time authentication tokens, coded QR codes, receive and verify authentication requests, and manage device whitelists and authorization records.

[0041] The network audio receiving module listens to a designated RTP port, receives RTP audio data packets sent by the wireless audio acquisition terminal, and parses the sequence number and timestamp information in the RTP header.

[0042] The audio decoding and processing module performs jitter buffering, out-of-order reordering, packet loss compensation, and decoding on the received RTP audio data packets, and performs echo suppression, howling suppression, noise suppression, and automatic gain control on the decoded audio.

[0043] The virtual microphone channel management module creates a virtual microphone device node based on the session identifier after successful authentication, mapping the wireless audio acquisition terminal to a virtual microphone input device that can be uniformly managed within the vehicle's multimedia host.

[0044] The audio routing control module connects to the vehicle status acquisition interface, obtains the real-time vehicle status through the CAN bus, body control module, or vehicle diagnostic interface, and determines the target amplification area based on the device identity of the virtual microphone device node, user role, authorized area mask, real-time vehicle status, and user selection.

[0045] The amplification and playback module outputs the processed microphone audio from the audio routing control module through digital-to-analog conversion and power amplification to the vehicle speaker system, and can also mix it with the original multimedia audio source.

[0046] The media playback status management module saves the current playback status and volume status of the multimedia audio source before entering the microphone amplification mode, and restores them after the amplification mode ends.

[0047] The user operation module is the touch screen, physical buttons, or steering wheel buttons of the vehicle's multimedia host. Users can use the user operation module to enable or disable the Wi-Fi microphone access service, view the list of connected devices, select the main microphone device, set the audio routing zone, adjust the microphone volume, manage the device whitelist, and configure the processing strategy for the original multimedia audio source.

[0048] Device discovery and certification process like Figure 1 As shown, the device discovery and authentication process includes the following steps: Step A1: The vehicle's multimedia host initiates the Wi-Fi microphone access service, providing wireless LAN access capability through the Wi-Fi communication module. After the service is initiated, the device discovery and authentication module generates a one-time authentication token, which has a preset validity period (e.g., five minutes) and a single-use limit.

[0049] Step A2: The device discovery and authentication module encodes the one-time authentication token, vehicle IP address, service port, SSID, Wi-Fi password and service identifier into a QR code and displays it on the vehicle screen.

[0050] Step A3: The user uses the wireless audio acquisition terminal to scan the QR code on the vehicle's infotainment screen to extract the access information and one-time authentication token.

[0051] Step A4: The wireless audio acquisition terminal automatically connects to the vehicle's Wi-Fi network based on the parsed SSID and password.

[0052] Step A5: The wireless audio acquisition terminal sends an authentication request to the designated service port of the vehicle's multimedia host. The authentication request includes: a one-time authentication token, device fingerprint, device type, device name, user role identifier, and audio capability set. The device fingerprint includes the device model, operating system version, unique hardware identifier, and application version number. The audio capability set includes a list of supported encoding formats, a list of supported sampling rates, and a list of supported packet durations.

[0053] Step A6: The device discovery and authentication module verifies the validity and expiration of the one-time authentication token. If the token has been used, expired, or does not match, access is denied and a notification is displayed on the vehicle's infotainment screen.

[0054] Step A7: After successful verification, the device discovery and authentication module generates a unique session identifier and returns an authentication success response to the wireless audio acquisition terminal. The authentication success response includes the session identifier and audio configuration parameters. These parameters include the encoding format, sampling rate, number of channels, target bitrate, packet duration, RTP port, and SSRC. For example, using Opus encoding, the audio configuration parameters are: Opus encoding, 48kHz sampling rate, mono, 32kbps target bitrate, and 20ms packet duration.

[0055] Step A8: The device discovery and authentication module writes the authenticated device information into the authentication information database, including device fingerprint, device type, session identifier, authorized area mask, and authorization time.

[0056] Virtual microphone channel establishment and audio transmission processing like Figure 2 As shown, the virtual microphone channel establishment and audio transmission processing flow includes: Step B1: The virtual microphone channel management module reads the device type, device name, and authorized region mask corresponding to the session from the authentication information database.

[0057] Step B2: The virtual microphone channel management module determines the default priority of the device. The driver's area microphone has the highest priority, followed by the passenger area tour guide microphone, and then the temporary broadcasting equipment.

[0058] Step B3: The virtual microphone channel management module registers the virtual microphone device node with the vehicle multimedia host's audio framework, enabling it to be managed by the upper-layer audio routing policy as a standard microphone input source. The virtual microphone device node includes device identity, user role, authorized region mask, and priority attributes. After registration, this virtual microphone input device adopts a unified management strategy with the local wired microphone input device in terms of audio routing control, volume adjustment, and priority management.

[0059] Step B4: The user turns on the microphone function through the wireless audio acquisition terminal.

[0060] Step B5: The wireless audio acquisition terminal starts microphone acquisition and encodes the audio according to the audio configuration parameters sent by the vehicle's infotainment system. Taking Opus encoding as an example, it uses a 48kHz sampling rate, mono, 20ms frame length, and a target bit rate of 32kbps.

[0061] Step B6: The wireless audio acquisition terminal encapsulates the encoded audio data into RTP data packets. The header of each RTP data packet contains SSRC, RTP sequence number and RTP timestamp.

[0062] Step B7: The wireless audio acquisition terminal sends RTP data packets to the designated RTP port of the vehicle multimedia host via the UDP protocol.

[0063] Step B8: The network audio receiving module receives RTP data packets, and the audio decoding and processing module performs jitter buffering, out-of-order reordering, packet loss compensation and decoding. It also performs echo suppression, howling suppression, noise suppression and automatic gain control on the decoded audio to obtain the processed microphone audio.

[0064] Step B9: The processed audio is marked as a microphone amplification input source from the virtual microphone device node and sent to the audio routing control module.

[0065] Audio routing control and vehicle external announcement security verification like Figure 3 As shown, the audio routing control process includes: Step C1: The audio routing control module obtains the device identity, device type, user role, and authorized region mask from the virtual microphone device node.

[0066] Step C2: The audio routing control module reads the real-time status of the vehicle through the vehicle status acquisition interface, including vehicle speed, gear, parking brake status, door status, ACC status, and vehicle operation mode.

[0067] Step C3: The audio routing control module determines the candidate target area based on the device identity and default routing rules. Default rules include: tour guide microphones are routed to passenger area speakers by default; driver area microphones are routed to driver area speakers or the entire vehicle speakers by default; when the user selects the "whole vehicle broadcast" mode, the microphone is routed to the entire vehicle speakers; when the user selects the "external announcement" mode, the microphone is routed to external speakers.

[0068] Step C4: If the candidate area includes an external speaker, perform a safety check: check if the vehicle speed is below a preset speed threshold (e.g., 5 km / h), if the vehicle is in parking brake mode, or if the driver has confirmed permission to speak from outside the vehicle via the vehicle's infotainment system. If the vehicle speed is not below the preset threshold, the vehicle is not in parking brake mode, and the driver has not confirmed permission, then audio routing to the external speaker will be refused, and the user will be notified via the vehicle's infotainment system screen.

[0069] Step C5: After the security verification is passed, the audio routing control module sends the processed microphone audio to the amplification and playback module, which then drives the target speaker to output the audio.

[0070] Step C6: During microphone amplification, if a high-priority event such as navigation voice broadcast, reversing radar alarm, or vehicle safety alarm is detected, the microphone amplification will be temporarily interrupted or lowered, and will automatically resume after the high-priority event ends.

[0071] Saving and restoring the original multimedia audio source status like Figure 4 As shown, the workflow of the media playback status management module includes: Step D1: When a valid RTP audio data packet is received for the first time and the virtual microphone device node status changes from "standby" to "working", the media playback status management module saves the current multimedia audio source's audio source type, playback status, playback progress, volume status, EQ mode, and sound field mode.

[0072] Step D2: After saving, process the original multimedia audio source according to the user's preset strategy: pause mode (pause the original audio source playback), reduce the volume of the original audio source to a preset ratio, such as 20%, or mix mode (the original audio source continues to play and is mixed with the microphone audio).

[0073] Step D3: The microphone amplification mode ends when the user actively turns off the microphone function, fails to receive a valid RTP data packet for more than a preset time (e.g., five seconds), the Wi-Fi connection is disconnected, or the user manually terminates the microphone session.

[0074] Step D4: The virtual microphone channel management module deregisters the virtual microphone device node or sets its status to "offline".

[0075] Step D5: The media playback status management module restores the playback of the original multimedia audio source based on the status saved in step D1, including restoring the audio source type, playback progress, volume status, EQ mode, and sound field mode. If navigation voice broadcasting is incomplete during microphone amplification, the navigation broadcasting will be restored first.

[0076] Network quality monitoring and adaptive adjustment The network audio receiving module continuously monitors packet loss rate, jitter, and end-to-end latency during RTP packet reception. When the packet loss rate exceeds 5% or the jitter exceeds 30 milliseconds, the vehicle's multimedia host sends a bitrate reduction command to the wireless audio acquisition terminal via the control channel. For example, it reduces the Opus encoding bitrate from 32kbps to 16kbps, or increases the packet duration from 20ms to 40ms. When the packet loss rate exceeds 15% or the jitter exceeds 80 milliseconds, an encoding switch command is sent. Once network quality recovers, a recovery command is sent to restore the encoding parameters to their initial configuration. If no valid RTP packets are received for five consecutive seconds, the vehicle's multimedia host mutes the virtual microphone channel, resumes playback of the original multimedia audio source, and displays a network disconnection message on the vehicle's screen.

[0077] Multi-device access management When multiple wireless audio acquisition terminals request access simultaneously, the arbitration rules, ranked from highest to lowest priority, are: manual selection by the vehicle-mounted user, service priority, whitelist priority, most recent authorization record, and access time. The arbitration result is to select one primary microphone audio source, while the remaining devices remain in standby or are denied access. When the primary microphone device disconnects, if there are standby devices, the highest priority standby device is automatically promoted to primary microphone device. Specific Implementation Example 1: Using a mobile phone as a tour guide microphone The tour guide's mobile phone connects to the Wi-Fi hotspot provided by the vehicle's multimedia host and scans the QR code on the host screen to complete authentication. The host selects Opus encoding, 48kHz sampling rate, 32kbps bitrate, and 20ms packet duration based on the phone's capabilities and receives audio via RTP. The host then maps the phone as a virtual microphone device node for the tour guide in the passenger area, routing the audio to the passenger area speakers. During amplification, the original radio playback is paused and automatically resumes after the microphone is turned off.

[0079] Example 2: Tablet PC as a broadcast terminal for the driver's area The driver connects their tablet to the vehicle's Wi-Fi and selects the "Vehicle Broadcast" mode. After verifying the device type and user role, the vehicle's system routes the microphone audio to all the vehicle's speakers. During the broadcast, the original Bluetooth music volume is reduced to 20%, and automatically returns to normal after the broadcast ends.

[0080] Example 3: Standard Wi-Fi Microphone Connection The third-party Wi-Fi microphone connects according to the standard access protocol published by the vehicle's infotainment system. This protocol specifies RTP transmission, Opus encoding, a 48kHz sampling rate, mono, and a 20ms packet duration. After the vehicle's infotainment system is certified, it can be used as the audio source for the tour guide microphone in the passenger area, without the need for a dedicated receiver.

[0081] Example 4: Announcement from outside the engineering vehicle When the construction vehicle is parked for work, the workers use a Wi-Fi microphone to complete authentication and select the "external announcement" mode. The vehicle's system verifies that the vehicle speed is zero and the parking brake is engaged, and then routes the microphone audio to the external speaker. Once the vehicle releases the parking brake and starts moving, the external announcement function automatically stops.

[0082] Example 5: Automatic Recovery from Network Interruption When the tour guide used his mobile phone to amplify his voice, the Wi-Fi signal was blocked, and the packet loss rate rose to 20%. The vehicle's infotainment system sent a bitrate reduction command, and the phone reduced the bitrate from 32 kilobits per second to 16 kilobits per second. Subsequently, the phone lost network access, and the vehicle's infotainment system did not receive any RTP packets for five consecutive seconds. It then automatically muted the microphone channel, restored the original multimedia audio source playback, and displayed the message "Microphone network disconnected."

[0083] Example 6: Simultaneous Access to Arbitration by Multiple Devices Both the tour guide's and passenger's mobile phones requested access simultaneously. The vehicle's control system checked their information: the tour guide's phone was on the whitelist and had a higher service priority; the passenger's phone was not on the whitelist. The control system selected the tour guide's phone as the primary microphone source, and the passenger's phone was denied access with a "Unauthorized device" message.

Claims

1. A Wi-Fi-based vehicle multimedia microphone intercom system, characterized in that, include: Wireless audio acquisition terminal; Vehicle multimedia host; A Wi-Fi communication module is used to establish an IP network connection between the wireless audio acquisition terminal and the vehicle multimedia host. The device discovery and authentication module is used to generate a one-time authentication token, encode the one-time authentication token with the vehicle's IP address, service port, and service identifier into a QR code, receive and verify the authentication request from the wireless audio acquisition terminal, and issue a session identifier and audio configuration parameters after successful verification; the network audio receiving module is used to receive RTP audio data packets sent by the wireless audio acquisition terminal according to the audio configuration parameters; the audio decoding and processing module is used to perform jitter buffering, out-of-order reordering, packet loss compensation, decoding, echo suppression, howling suppression, noise suppression, and automatic gain control on the RTP audio data packets; the virtual microphone channel management module is used to create a virtual microphone device node according to the session identifier and map the wireless audio acquisition terminal as a virtual microphone input device in the vehicle's multimedia host. The audio routing control module is used to route the processed microphone audio to at least one of the driver's area speaker, passenger area speaker, vehicle speaker or external speaker according to the device identity corresponding to the virtual microphone device node, the real-time status of the vehicle and the preset audio routing rules. A loudspeaker playback module is used to output the processed microphone audio to the vehicle speaker system; The media playback status management module is used to save the audio source type, playback progress, volume status, and EQ mode of the current multimedia audio source when a valid RTP audio data packet is received for the first time and the microphone amplification mode is entered. It also restores the playback of the original multimedia audio source after the microphone amplification mode ends. When the preset audio routing rule determines to route the processed microphone audio to the external speaker, the audio routing control module must confirm that the current vehicle speed is lower than a preset vehicle speed threshold and that at least one of the following conditions is met: the vehicle is in parking brake mode, or the driver has confirmed permission to speak outside the vehicle via the vehicle's infotainment system. Otherwise, output to the external speaker is prohibited.

2. The system according to claim 1, characterized in that, The wireless audio acquisition terminal includes at least one of a mobile phone, tablet computer, Wi-Fi microphone, Wi-Fi-enabled tour guide microphone, or smart wearable microphone. The wireless audio acquisition terminal has a built-in microphone acquisition unit, audio encoding unit, network transmission unit, service discovery unit, and authentication unit.

3. The system according to claim 1, characterized in that, It also includes a vehicle status acquisition interface, which acquires the real-time vehicle status through a CAN bus, a body control module, or an on-board diagnostic interface. The real-time vehicle status includes at least one of vehicle speed, gear position, parking brake status, door status, ACC status, and vehicle operating mode. The device discovery and authentication module is also used to detect packet loss rate based on the RTP sequence number, calculate network jitter based on the RTP timestamp, and when the packet loss rate exceeds a preset packet loss threshold or the network jitter exceeds a preset jitter threshold, send a bitrate reduction command, packet duration adjustment command, or encoding format switching command to the wireless audio acquisition terminal through the control channel.

4. The system according to claim 1, characterized in that, When multiple wireless audio acquisition terminals request access simultaneously, the device discovery and authentication module selects a primary microphone source based on at least one mechanism among device whitelist, service priority corresponding to device type, recent authorization record, user selection, or vehicle-mounted confirmation, while the remaining devices remain in standby or are denied access. The virtual microphone channel management module is also used to register the virtual microphone device node with the audio framework of the vehicle multimedia host, so that the virtual microphone input device and the local wired microphone input device adopt a unified management strategy in audio routing control, volume adjustment, and priority management.

5. A Wi-Fi-based vehicle multimedia microphone intercom control method, applied to a vehicle multimedia host, characterized in that, include: The vehicle's multimedia host initiates the Wi-Fi microphone access service, providing wireless local area network access via the vehicle's hotspot, vehicle local area network, or Wi-Fi Direct, and generates a one-time authentication token; The vehicle multimedia host encodes the one-time authentication token, vehicle IP address, service port, and service identifier into a QR code and displays it on the vehicle screen. The vehicle multimedia host receives an authentication request from the wireless audio acquisition terminal after it accesses the system by scanning the QR code. The authentication request carries the one-time authentication token, device fingerprint, device type, and audio capability set. The audio capability set includes a list of supported encoding formats and a list of supported sampling rates. The vehicle multimedia host verifies the validity and expiration of the one-time authentication token. Upon successful verification, it sends a session identifier and audio configuration parameters to the wireless audio acquisition terminal. The audio configuration parameters include encoding format, sampling rate, bitrate, packet duration, and RTP port. The vehicle multimedia host creates a virtual microphone device node based on the session identifier, mapping the wireless audio acquisition terminal as a virtual microphone input device within the vehicle multimedia host. The virtual microphone device node includes device identity, user role, authorized area mask, and priority. The vehicle multimedia host receives RTP audio data packets sent by the wireless audio acquisition terminal according to the audio configuration parameters. The RTP audio data packets carry the session identifier, RTP sequence number, and RTP timestamp. The vehicle multimedia host performs jitter buffering, out-of-order reordering, packet loss compensation, and decoding on the RTP audio data packets. It then performs echo suppression, howl suppression, noise suppression, and automatic gain control on the decoded audio to obtain processed microphone audio. Based on the device identity corresponding to the virtual microphone device node, the vehicle's real-time status, and preset audio routing rules, the vehicle multimedia host outputs the processed microphone audio to the driver's area speakers, passenger area speakers, vehicle speakers, or external speakers. At least one of the following: When the preset audio routing rule determines to output the processed microphone audio to the external speaker, the vehicle multimedia host must confirm that the current vehicle speed is lower than a preset vehicle speed threshold and meet at least one of the following conditions: the vehicle is in a parking brake state, or the driver has confirmed through the vehicle system that permission to speak outside the vehicle is granted; otherwise, output to the external speaker is prohibited; When a valid RTP audio data packet is received for the first time and the microphone amplification mode is entered, the vehicle multimedia host saves the current multimedia audio source's audio source type, playback progress, volume status, and EQ mode; After the microphone amplification mode ends, the playback of the original multimedia audio source is restored according to the saved audio source type, playback progress, volume status, and EQ mode.

6. The method according to claim 5, characterized in that, The real-time vehicle status is obtained via CAN bus, body control module, or on-board diagnostic interface, including at least one of vehicle speed, gear position, parking brake status, door status, ACC status, and vehicle operating mode; the audio configuration parameters are sent via TCP control channel or WebSocket control channel, the encoding format is Opus encoding, the sampling rate is 48kHz or 16kHz, the channel is mono, the packet duration is 20ms, and the audio configuration parameters also include SSRC.

7. The method according to claim 5, characterized in that, The vehicle multimedia host detects the packet loss rate based on the RTP sequence number, calculates network jitter based on the RTP timestamp, and when the packet loss rate exceeds a preset packet loss threshold or the network jitter exceeds a preset jitter threshold, it sends a bitrate reduction command, packet duration adjustment command, or encoding format switching command to the wireless audio acquisition terminal through the control channel; when the vehicle multimedia host does not receive a valid RTP audio data packet for a preset duration, it automatically mutes the microphone channel corresponding to the virtual microphone device node, restores the original multimedia audio source playback, and prompts a network disconnection on the vehicle interface.

8. The method according to claim 5, characterized in that, When multiple wireless audio acquisition terminals request access simultaneously, the vehicle multimedia host selects a primary microphone audio source based on at least one of the following mechanisms: device whitelist, service priority corresponding to device type, recent authorization record, user selection, or vehicle terminal confirmation. The remaining devices remain in standby or are denied access. During microphone amplification, if navigation voice broadcast, reversing radar alarm, or vehicle safety alarm occurs, the vehicle multimedia host temporarily interrupts or lowers the processed microphone audio, and resumes microphone amplification after the navigation voice broadcast, reversing radar alarm, or vehicle safety alarm ends.

9. The method according to claim 5, characterized in that, The vehicle multimedia host is compatible with third-party Wi-Fi microphones that conform to a preset universal access protocol. This preset universal access protocol specifies the sampling rate, encoding format, RTP payload format, device identity field, and control command format. Third-party Wi-Fi microphones that conform to this preset universal access protocol can access the network without a dedicated hardware receiver. The one-time authentication token is randomly generated by the vehicle multimedia host when the Wi-Fi microphone access service is started. It has a preset validity period and a single-use limit. The one-time authentication token automatically expires after one use or after the preset validity period. The QR code also encodes the SSID and connection password of the Wi-Fi network. After scanning the QR code, the wireless audio acquisition terminal automatically connects to the Wi-Fi network and sends the authentication request.

10. The method according to claim 5, characterized in that, The device fingerprint includes at least one of the following: device model, operating system version, hardware unique identifier, and application version number of the wireless audio acquisition terminal; the virtual microphone device node registers with the audio framework of the vehicle multimedia host, so that the virtual microphone input device and the local wired microphone input device adopt a unified management strategy in audio routing control, volume adjustment, and priority management. During microphone amplification mode, the vehicle multimedia host performs at least one of the following actions on the original multimedia audio source: pause playback, lower the volume to a preset ratio, or mix with microphone audio. The trigger conditions for ending microphone amplification mode include user actively turning off the microphone function, failure to receive valid RTP audio data packets for a preset duration, Wi-Fi network connection disconnection, user manually terminating the microphone session, or expiration of the one-time authentication token.

Images