Method, device and equipment for optimizing voice playing of commercial vehicle and storage medium

Abstract

The application discloses a business vehicle voice playing optimization method and device, equipment and storage medium, relates to the technical field of voice signal processing, and the method comprises the following steps: obtaining a voice signal of a passenger initiated voice interaction request and a reference audio signal returned by a sound power amplifier assembly; performing noise elimination processing on the voice signal according to the reference audio signal to obtain a target voice signal after noise reduction; performing voice recognition according to the target voice signal to generate voice synthesis data to be broadcast; and performing partitioned broadcasting on the voice synthesis data according to a preset loudspeaker area control strategy. The application eliminates noise from the collected user voice based on the reference audio of the sound power amplifier assembly, and performs targeted regional broadcasting control, thereby solving the problem that existing business vehicle rear-row voice recognition and broadcasting are unclear, and improving the quality of voice interaction of passengers in the rear row of the business vehicle.

Description

Technical Field

[0001] This application relates to the technical field of speech signal processing, and in particular to a method, apparatus, device and storage medium for optimizing voice playback in commercial vehicles. Background Technology

[0002] With the continuous improvement of automotive intelligence, in-vehicle voice interaction systems have become one of the core configurations for enhancing the driving and riding experience. Especially in high-end business vehicles, users have placed higher demands on the accuracy of voice recognition, response speed, and user-friendliness of interaction. However, current business vehicles still face several technical bottlenecks in terms of voice function experience, which limits its actual usability for rear passengers, especially second-row passengers.

[0003] First, commercial vehicles generally adopt a 2+2+3 seating layout, which differs significantly from the 2+3 layout commonly found in sedans. This layout places second-row passengers closer to the center of the vehicle, further from the onboard microphones, resulting in decreased voice pickup quality and consequently affecting voice wake-up and recognition rates. Currently, voice recognition systems primarily rely on front-row microphones for pickup, and the domain controller fails to adequately consider the acoustic characteristics and environmental interference of rear-row passengers when processing voice commands, making the voice interaction experience for second-row passengers significantly inferior to that of front-row passengers. Second, during voice interaction, the system typically uses TTS (Text-to-Speech) for feedback. In traditional designs, TTS broadcasts are mostly executed by the front speakers, making it difficult for rear-row passengers to clearly hear the feedback while the vehicle is in motion, especially in the presence of background noise such as road noise and wind noise, further reducing the usability and user satisfaction of voice interaction. This not only affects the riding experience of second-row passengers in commercial vehicles but also contradicts the luxury and brand image pursued by high-end commercial vehicles.

[0004] Therefore, improving the quality of voice interaction for rear-seat passengers is an urgent problem that needs to be solved. Summary of the Invention

[0005] The main purpose of this application is to provide a method, device, equipment and storage medium for optimizing voice playback in commercial vehicles, aiming to solve the technical problem of how to improve the quality of voice interaction for rear passengers.

[0006] To achieve the above objectives, this application proposes a method for optimizing voice playback in commercial vehicles, the method comprising: Acquire the voice signal of the passenger initiating the voice interaction request and the reference audio signal returned by the audio amplifier assembly; The speech signal is subjected to noise reduction processing based on the reference audio signal to obtain the noise-reduced target speech signal. Speech recognition is performed based on the target speech signal to generate speech synthesis data to be broadcast; The speech synthesis data is broadcast in zones according to a preset speaker area control strategy.

[0007] In one embodiment, the step of acquiring the reference audio signal returned by the audio amplifier assembly includes: Acquire multi-channel audio data returned by the audio amplifier assembly; Determine the corresponding speaker zone identifier based on the multi-channel audio data; The multi-channel audio data are fused according to the speaker partition identifier to obtain a reference audio signal.

[0008] In one embodiment, the step of fusing the multiple audio data according to the speaker partition identifier to obtain a reference audio signal includes: The return ratio coefficient of the audio data corresponding to each zone identifier is determined based on the speaker zone identifier; The multi-channel audio data are weighted and fused according to the return ratio coefficient to obtain a reference audio signal.

[0009] In one embodiment, the step of weighting and fusing the multiple audio data according to the return scaling factor to obtain a reference audio signal includes: Extract the audio channel data corresponding to each speaker partition from the multi-channel audio data; The audio channel data is subjected to amplitude weighting calculation based on the return ratio coefficient to obtain the weighted audio components of each partition; The audio components are mixed and superimposed to obtain a reference audio signal.

[0010] In one embodiment, the step of partitioning and broadcasting the speech synthesis data according to a preset speaker area control strategy includes: Obtain the seat location information of the passenger who initiated the voice interaction request; Based on the seat position information, a target broadcast area corresponding to the seat position information is matched from a preset speaker area control strategy; When the target broadcast area is the first broadcast area, the front speaker group is controlled to broadcast the speech synthesis broadcast data. The front speaker group includes a driver's speaker group and a passenger's speaker group. When the target broadcast area is the second broadcast area, the rear speaker group is controlled to broadcast the speech synthesis broadcast data. The rear speaker group includes a left rear area speaker group and a right rear area speaker group.

[0011] In one embodiment, after the step of matching the target broadcasting area corresponding to the seat position information from a preset speaker area control strategy based on the seat position information, the method further includes: When the target broadcast area is the first broadcast area and the second broadcast area, a preset broadcast priority is obtained; When the broadcast priority of the first broadcast area is greater than that of the second broadcast area, the front speaker group and the rear speaker group are controlled to broadcast the speech synthesis data, and the broadcast volume of the front speaker group is increased and the broadcast volume of the rear speaker group is decreased. When the broadcast priority of the second broadcast area is greater than that of the first broadcast area, the front and rear speaker groups are controlled to broadcast the speech synthesis data, and the broadcast volume of the front speaker group is reduced while the broadcast volume of the rear speaker group is increased.

[0012] In one embodiment, the step of performing noise cancellation processing on the speech signal based on the reference audio signal to obtain the denoised target speech signal includes: The speech signal and the reference audio signal are time-domain aligned to obtain the target reference audio signal. Extract the environmental noise feature components from the speech signal and the speaker playback feature components from the target reference audio signal; The ambient noise feature components are filtered and canceled based on the speaker playback feature components to obtain the noise-reduced target speech signal.

[0013] Furthermore, to achieve the above objectives, this application also proposes a voice playback optimization device for commercial vehicles, the device comprising: The voice acquisition module is used to acquire the voice signal of the passenger initiating the voice interaction request and the reference audio signal returned by the audio amplifier assembly; A speech noise reduction module is used to perform noise removal processing on the speech signal based on the reference audio signal to obtain a noise-reduced target speech signal; The speech recognition module is used to perform speech recognition based on the target speech signal and generate speech synthesis data to be broadcast. The voice broadcasting module is used to broadcast the synthesized voice data in zones according to a preset speaker area control strategy.

[0014] In addition, to achieve the above objectives, this application also proposes a commercial vehicle voice playback optimization device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the commercial vehicle voice playback optimization method described above.

[0015] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the commercial vehicle voice playback optimization method described above.

[0016] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the commercial vehicle voice playback optimization method described above.

[0017] This application provides a method for optimizing voice playback in commercial vehicles. The method includes: acquiring a voice signal from a passenger initiating a voice interaction request and a reference audio signal returned by the audio amplifier assembly; performing noise cancellation processing on the voice signal based on the reference audio signal to obtain a noise-reduced target voice signal; performing speech recognition on the target voice signal to generate speech synthesis data to be played; and performing zoned playback of the speech synthesis data according to a preset speaker area control strategy. In summary, this application solves the problem of unclear voice recognition and playback in the rear seats of existing commercial vehicles by performing noise cancellation on the acquired user voice based on the reference audio from the audio amplifier assembly and performing targeted regional playback control, thereby improving the quality of voice interaction for rear passengers in commercial vehicles. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 A flowchart illustrating the first embodiment of the commercial vehicle voice playback optimization method of this application; Figure 2 This is an architecture diagram of intelligent cockpit voice recognition control in one embodiment of the commercial vehicle voice playback optimization method of this application; Figure 3 A vehicle layout scenario diagram for the commercial vehicle voice playback optimization method applied in this application; Figure 4 A flowchart illustrating the second embodiment of the commercial vehicle voice playback optimization method of this application; Figure 5This is a schematic diagram of the module structure of the commercial vehicle voice playback optimization device according to an embodiment of this application; Figure 6 This is a schematic diagram of the hardware operating environment involved in the commercial vehicle voice playback optimization method in this application embodiment.

[0021] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0022] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0023] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0024] The main solution of this application embodiment is: to acquire the voice signal of the passenger initiating a voice interaction request and the reference audio signal returned by the audio amplifier assembly; to perform noise cancellation processing on the voice signal according to the reference audio signal to obtain the noise-reduced target voice signal; to perform speech recognition according to the target voice signal to generate speech synthesis data to be broadcast; and to broadcast the speech synthesis data in zones according to a preset speaker area control strategy.

[0025] With the continuous improvement of automotive intelligence, in-vehicle voice interaction systems have become one of the core configurations for enhancing the driving and riding experience. Especially in high-end business vehicles, users have placed higher demands on the accuracy of voice recognition, response speed, and user-friendliness of interaction. However, current business vehicles still face several technical bottlenecks in terms of voice function experience, which limits its actual usability for rear passengers, especially second-row passengers.

[0026] First, commercial vehicles generally adopt a 2+2+3 seating layout, which differs significantly from the 2+3 layout commonly found in sedans. This layout places second-row passengers closer to the center of the vehicle, further from the onboard microphones, leading to decreased voice pickup quality and consequently affecting voice wake-up and recognition rates. In existing technologies, voice recognition systems primarily rely on front-row microphones for pickup, and the domain controller fails to adequately consider the acoustic characteristics and environmental interference of rear-row passengers when processing voice commands, resulting in a significantly inferior voice interaction experience for second-row passengers compared to front-row passengers. Second, during voice interaction, the system typically provides feedback via Text-to-Speech (TTS). In traditional designs, TTS broadcasts are mostly executed by front-row speakers, making it difficult for rear-row passengers to clearly hear the feedback while the vehicle is in motion, especially in the presence of road noise, wind noise, and other background noise, further reducing the usability and user satisfaction of voice interaction. This not only affects the riding experience of second-row passengers in commercial vehicles but also contradicts the luxury and brand image pursued by high-end commercial vehicles. Therefore, improving the quality of voice interaction for rear-row passengers is a pressing issue that needs to be addressed.

[0027] It should be noted that the executing entity in this embodiment can be a commercial vehicle voice playback optimization system, a computing service device with data processing, network communication, and program execution functions, or an electronic device capable of implementing the aforementioned commercial vehicle voice playback optimization function, etc. This embodiment does not specifically limit it in this way. The following uses a commercial vehicle voice playback optimization system as an example to describe this embodiment and the following embodiments.

[0028] Based on this, this application provides a method for optimizing voice playback in commercial vehicles, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the commercial vehicle voice playback optimization method of this application.

[0029] In this embodiment, the commercial vehicle voice playback optimization method includes steps S10~S40: Step S10: Obtain the voice signal of the passenger initiating the voice interaction request and the reference audio signal returned by the audio amplifier assembly.

[0030] It should be noted that the intelligent cockpit domain controller in the system monitors passenger voice commands collected by the in-vehicle microphone array in real time, and simultaneously receives processed audio data returned by the audio amplifier assembly. The voice signal is picked up by microphone arrays distributed in different locations within the vehicle (such as the front and second-row headliners), and this signal contains the passenger's wake-up word or command. The second type is the reference audio signal, which is returned in real time by the audio amplifier assembly.

[0031] Additionally, it should be noted that the audio amplifier assembly has undergone DSP (Digital Signal Processing) optimization, such as equalization, delay, and crossover, before transmitting the audio data back. Therefore, the transmitted signal is the final signal that will be emitted from the speakers after modification, which can truly reflect the acoustic environment of the car.

[0032] In one feasible implementation, the step of acquiring the reference audio signal returned by the audio power amplifier assembly includes: Step S101: Obtain the multi-channel audio data returned by the audio amplifier assembly.

[0033] It should be noted that, as Figure 2 As shown, the intelligent cockpit domain controller establishes a communication link with the audio amplifier assembly via the vehicle bus (such as Ethernet, A2B audio bus, etc.) to receive multi-channel audio data (Slots 1 to 4) from the amplifier in real time. This data is raw, unfused channel data, each corresponding to an independent physical channel of the speaker in the vehicle. It should also be noted that the multi-channel audio data refers to the multi-channel digital audio stream output after processing by the DSP module within the audio amplifier assembly. Each channel corresponds to the drive signal for a speaker zone, including independent audio data for left front, right front, center, left surround, right surround, left rear, right rear, and subwoofer channels.

[0034] Step S102: Determine the corresponding speaker partition identifier based on the multi-channel audio data.

[0035] It should be noted that speaker zone identifiers are digital tags used to distinguish different acoustic areas within the vehicle. In this step, the intelligent cockpit domain controller, while receiving multiple audio data streams, also parses the metadata carried by the data streams and determines the speaker zone identifier corresponding to each data stream based on predefined channel mapping rules. For example, through the tag in the data packet header or a fixed channel number, it can be identified whether each data stream belongs to the left front speaker or the right front speaker. Specifically, as... Figure 3 The red area is the driver's return area, the green area is the passenger's return area, the blue area is the left rear return area, and the yellow area is the right rear return area.

[0036] Step S103: The multi-channel audio data is fused according to the speaker partition identifier to obtain a reference audio signal.

[0037] It should be noted that in this step, the intelligent cockpit domain controller performs weighted fusion processing on multiple audio data streams based on the partition identifier, generating a composite reference signal that matches the microphone pickup location. This ensures that the echo reception accurately reflects the actual sound pressure level distribution at the microphone. It can be understood that the reference audio signal refers to a single or multiple composite audio signal formed after partitioned weighted fusion. Its spectral characteristics are consistent with the superposition effect of the actual sound field played inside the vehicle at the microphone location, and it can serve as a reference signal for echo cancellation and noise suppression.

[0038] In one feasible implementation, step S103 specifically includes: Step A10: Determine the return ratio coefficient of the audio data corresponding to each partition identifier based on the speaker partition identifier.

[0039] It should be noted that the return ratio coefficient refers to the weighting coefficient set for different speaker zones when sending audio data back to the domain controller as a reference tone. Since the sound emitted by speakers in different locations has varying degrees of impact on the echo picked up by the microphone (for example, speakers closer to the microphone have a greater impact), by setting different return ratios, the reference signal for echo cancellation can be constructed more accurately, avoiding over-cancellation or under-cancellation. Specifically, the domain controller has a pre-stored mapping table (as shown in Table 1) that defines the return ratio coefficient corresponding to each speaker zone identifier.

[0040] Additionally, it should be noted that the feedback ratio is not fixed; it can be dynamically adjusted according to the in-vehicle environment (such as the window opening / closing status and seat position) or sound zone settings (such as waking up only the second row).

[0041] Table 1

[0042] Step A20: Weight and fuse the multi-channel audio data according to the return ratio coefficient to obtain a reference audio signal.

[0043] It should be noted that after determining the return ratio coefficient for each partition, the domain controller will process the audio data of each channel according to these coefficients, and fuse the processed results to finally form a reference audio signal that can accurately characterize the sound field environment at the target microphone.

[0044] In one feasible implementation, step A20 specifically includes: Step A201: Extract the audio channel data corresponding to each speaker partition in the multi-channel audio data.

[0045] It should be noted that the audio channel data here is the original PCM (Pulse Code Modulation) audio stream, which represents the digital sequence of sound pressure level changes over time for that channel. In this step, the domain controller demultiplexes the multiplexed audio data stream based on the identified partition identifier, separating the independent audio channel data corresponding to each speaker partition. For example, from the received A2B data frame, the data of Slot 1 is extracted and placed into buffer 1 (corresponding to the driver's area), the data of Slot 2 is placed into buffer 2 (corresponding to the passenger's area), the data of Slot 3 is placed into buffer 3 (corresponding to the left rear area), and the data of Slot 4 is placed into buffer 4 (corresponding to the right rear area).

[0046] Step A202: Perform amplitude weighting on the audio channel data according to the return ratio coefficient to obtain the weighted audio components of each partition.

[0047] It should be noted that for each extracted audio channel data, the domain controller will calculate it with the corresponding return scaling factor. Assuming that the data of the left rear return area audio channel is a composite signal composed of three signals: left rear, left surround, and subwoofer, then the return scaling factor of each sub-channel is 0.3. After weighted calculation, the audio components output in this area are 0.3 × left rear + 0.3 × left surround + 0.3 × subwoofer.

[0048] Understandably, this step simulates the attenuation of speaker sound at different distances and positions as it reaches the microphone, allowing areas that contribute significantly to the echo (such as the left rear area for second-row passengers) to occupy a larger proportion in subsequent fusion.

[0049] Step A203: Mix and superimpose the audio components to obtain a reference audio signal.

[0050] It should be noted that the domain controller linearly superimposes all the audio components (such as the driver's area component, passenger's area component, left rear area component, right rear area component, etc.) after amplitude weighting in the time domain, that is, it adds them up point by point to generate the final reference audio signal Z (Z = driver's area component + passenger's area component + left rear area component + right rear area component). This final reference audio signal is a mathematically constructed signal that best represents the sum of the echoes generated by all the speakers in the vehicle.

[0051] Step S20: Perform noise cancellation processing on the speech signal based on the reference audio signal to obtain the noise-reduced target speech signal.

[0052] It should be noted that in this step, the system performs echo cancellation using adaptive filtering algorithms (such as LMS, NLMS, etc.) based on the speech signal and the constructed reference audio signal. The basic principle is to subtract the speaker playback component from the speech signal picked up by the microphone, thereby preserving the pure passenger voice commands. Understandably, this step significantly improves the signal-to-noise ratio, especially by eliminating or suppressing strong echoes generated by the system's own playback sounds (such as music or navigation sounds).

[0053] In one feasible implementation, step S20 specifically includes: Step S201: Perform time-domain alignment processing on the speech signal and the reference audio signal to obtain the target reference audio signal.

[0054] It should be noted that because sound travels through the air and is picked up by the microphone, there is a physical delay, and the processing links, such as the power amplifier and DSP, also introduce a fixed delay. Therefore, the speech signal (including echo) picked up by the microphone and the reference audio signal returned by the power amplifier are out of sync in time. The domain controller uses delay estimation and compensation algorithms to calculate the difference in delay between the two signals and adjusts the delay of the reference audio signal to align it with the echo component in the speech signal in time.

[0055] Step S202: Extract the environmental noise feature components from the speech signal and the speaker playback feature components from the target reference audio signal.

[0056] It should be noted that in this step, the intelligent cockpit domain controller converts the signal into a time-spectrum representation through frequency domain transformation (such as Short-Time Fourier Transform, STFT), and uses independent component analysis or deep neural networks to separate non-speech components (including environmental noise such as road noise and wind noise) and speaker linear echo components from the speech signal. The environmental noise characteristic component refers to the non-target speech spectrum components in the speech signal generated by sound sources such as the road surface outside the vehicle, the air conditioning inside the vehicle, and tires; the speaker playback characteristic component refers to the echo components in the speech signal that are picked up a second time by the microphone after the audio is played by the in-vehicle speakers and propagates along the acoustic path.

[0057] Step S203: Filter and cancel the environmental noise feature components according to the speaker playback feature components to obtain the noise-reduced target speech signal.

[0058] It should be noted that in this step, the system uses the extracted speaker playback feature components as a reference to adaptively adjust the filter coefficients to generate an estimated echo that highly matches the echo components in the microphone signal. Then, this estimated echo is subtracted from the noisy microphone signal (ambient noise feature components) in the frequency or time domain. After this filtering and cancellation process, the remaining signal is the target speech signal, primarily containing the passenger's clean speech.

[0059] Step S30: Perform speech recognition based on the target speech signal to generate speech synthesis data to be broadcast.

[0060] It should be noted that in this step, the system sends the high-quality target speech signal, after noise reduction processing, into the speech recognition engine. The engine first performs wake-up word detection; after recognizing a specific wake-up word, it begins to pick up subsequent command speech and converts it into text. Then, based on the recognized text commands, the corresponding functions (such as navigation, air conditioning control, media selection, etc.) are executed in the domain controller's logic processing unit, and the response text to be fed back to the passenger is generated based on the execution results. Finally, the response text is synthesized into natural and fluent speech data to be broadcast using a TTS (Text-to-Speech) engine.

[0061] Step S40: The speech synthesis data is broadcast in zones according to the preset speaker area control strategy.

[0062] It should be noted that in this step, the system will query the speaker area control strategy table based on the sound source localization result of the voice interaction request (such as determining that it was initiated by the passenger on the left side of the second row), and route the voice synthesis data through the audio amplifier assembly to all speakers in the vehicle (including the front left and right speakers, the center speaker, the rear left and right speakers, the surround speakers, and the subwoofer) to complete the voice feedback broadcast, so as to avoid the problem that the second row cannot hear the TTS broadcast.

[0063] Additionally, it should be noted that the speaker area control strategy refers to a set of rules for dynamically selecting the TTS broadcast area based on the voice interaction scenario and passenger location. This includes various modes such as the request source positioning strategy (broadcasting only in the same area as the requester), the whole-vehicle equalization strategy (ensuring that the second row can hear clearly by broadcasting synchronously through all speakers), and the zone enhancement strategy (increasing the volume of the area where the request source is located). This solution prioritizes the whole-vehicle equalization strategy for the second row scenario in commercial vehicles, that is, broadcasting TTS through the speakers of the entire vehicle to ensure that the second-row passengers can hear the feedback clearly, thereby solving the problem of the second-row passengers not being able to hear the TTS broadcast in traditional solutions, and thus improving the overall intelligent voice interaction experience of commercial vehicles.

[0064] This embodiment provides a method for optimizing voice playback in commercial vehicles. The method includes: acquiring a voice signal from a passenger initiating a voice interaction request and a reference audio signal returned by the audio amplifier assembly; performing noise cancellation processing on the voice signal based on the reference audio signal to obtain a noise-reduced target voice signal; performing voice recognition based on the target voice signal to generate synthesized voice data to be played; and performing zoned playback of the synthesized voice data according to a preset speaker area control strategy. In summary, this embodiment solves the problem of unclear voice recognition and playback in the rear seats of existing commercial vehicles by transmitting the audio signal back to the intelligent cockpit domain controller according to the vehicle's speaker zones and proportions as reference audio, and by performing noise cancellation on the collected user voice based on the reference audio from the audio amplifier assembly, and by performing targeted regional playback control. This improves the quality of voice interaction for rear passengers in commercial vehicles.

[0065] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in Embodiment 1 above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 4 , Figure 4 This is a flowchart illustrating the second embodiment of the commercial vehicle voice playback optimization method of this application. Step S40 specifically includes: Step S401: Obtain the seat location information of the passenger who initiated the voice interaction request.

[0066] It should be noted that in this step, the system uses sound source localization technology or in-vehicle sensor systems to determine which passenger initiated the voice interaction. For example, it uses microphone arrays distributed in different locations in the vehicle (such as front and second-row overhead microphones) to pick up voice signals. By calculating the time difference or energy difference of the sound reaching each microphone, it locates the specific direction of the sound source, thereby determining the passenger's seating area. Combining this with the in-vehicle seating layout coordinate system, it calculates the specific seat position of the passenger who initiated the request (such as the driver's seat, front passenger seat, second-row left seat, second-row right seat, etc.).

[0067] Step S402: Match the target broadcast area corresponding to the seat position information from the preset speaker area control strategy according to the seat position information.

[0068] It should be noted that in this step, the system will query a pre-stored speaker area mapping table to convert the seat position information into corresponding speaker control commands. For example, if the system identifies that the passenger is in the driver's seat, the matched target broadcast area is the first broadcast area (i.e., the front area); if the passenger is in the left or right rear seat, the matched target broadcast area is the second broadcast area (i.e., the rear area).

[0069] Additionally, it should be noted that the target broadcast area refers to the range of speaker combinations that perform TTS voice broadcasts. Based on the 2+2+3 seat layout of the commercial vehicle, it is divided into the first broadcast area (corresponding to the driver's seat and passenger seat area) and the second broadcast area (corresponding to the left rear seat and right rear seat area), with each area containing a specific speaker group configuration.

[0070] Step S403: When the target broadcast area is the first broadcast area, control the front speaker group to broadcast the voice synthesis broadcast data. The front speaker group includes the driver's speaker group and the passenger's speaker group.

[0071] It should be noted that when the interaction is determined to be initiated by someone in the front row (such as the driver or front passenger), the domain controller sends a command to the audio amplifier assembly to transmit the generated speech synthesis data to the corresponding speaker channel in the front row for playback. For example, if the initiator is the driver, the speakers near the driver (such as the left front door and the center speaker) can be used first to ensure that the sound reaches the driver's ears directly.

[0072] Additionally, it should be noted that the driver's side speaker group and the passenger side speaker group do not refer to a single speaker, but rather to a combination of speakers located near the driver's side and passenger side, including front and rear door speakers, center speaker, subwoofer, etc., with specific configurations varying depending on the vehicle model.

[0073] Step S404: When the target broadcast area is the second broadcast area, control the rear speaker group to broadcast the speech synthesis broadcast data. The rear speaker group includes a left rear area speaker group and a right rear area speaker group.

[0074] It's important to note that when the interaction initiator is determined to be located in the back row (such as the left or right rear seat), the domain controller will transmit the synthesized speech data to the corresponding speaker channel in the back row. For example, if the initiator is a passenger in the right rear seat, the right rear door speaker, the right rear surround speaker, and the subwoofer will be activated, causing the sound field center to fall on the right rear area. This step is intended to broadcast messages to passengers in the second row (i.e., the rear seats) to avoid sound being blocked by the front rows, thereby improving the voice interaction experience for second-row passengers.

[0075] In one feasible implementation, after step S402, the method further includes: Step B10: When the target broadcast area is the first broadcast area and the second broadcast area, obtain the preset broadcast priority.

[0076] It should be noted that in some scenarios, passengers in multiple areas may initiate interactions simultaneously (e.g., front and rear passengers wake up one after the other), or the system may need to broadcast important information (e.g., safety alerts) to the entire vehicle. In such cases, the target broadcast area may be set to simultaneously cover the first and second broadcast areas. The system will retrieve pre-defined broadcast priority rules, such as a default priority for front seats over rear seats, or dynamically adjust the priority based on the last area woken up. This step aims to resolve conflicts when multiple areas broadcast concurrently, ensuring that critical information is conveyed correctly.

[0077] Additionally, it should be noted that the broadcast priority can be a static preset value (such as driver's seat > passenger's seat > rear seat), or it can be dynamically calculated according to the scenario (such as based on the source of the wake word, passenger identification, etc.).

[0078] Step B20: When the broadcast priority of the first broadcast area is greater than that of the second broadcast area, control the front speaker group and the rear speaker group to broadcast the speech synthesis data, and increase the broadcast volume of the front speaker group and decrease the broadcast volume of the rear speaker group.

[0079] It should be noted that when the priority of the first broadcast area (front row) is higher than that of the second broadcast area (rear row), the system will instruct the audio amplifier assembly to output speech synthesis data to both the front and rear speaker groups simultaneously, and adjust the gain to make the front row volume significantly higher than the rear row volume. For example, the front row volume is set to 1.2 times the normal value, while the rear row volume is attenuated to 0.8 times. This ensures that front row passengers can clearly hear the feedback without completely ignoring rear row passengers, achieving a hierarchical distribution of the sound field between the front and rear rows, and preventing excessive rear row volume from interfering with the front row. Furthermore, it should be noted that the volume adjustment ratio can be flexibly set according to the actual acoustic environment and user preferences, and a gradual transition can also be used to avoid abrupt changes.

[0080] Step B30: When the broadcast priority of the second broadcast area is greater than that of the first broadcast area, control the front speaker group and the rear speaker group to broadcast the speech synthesis data, and reduce the broadcast volume of the front speaker group and increase the broadcast volume of the rear speaker group.

[0081] It's important to note that when rear-seat priority is higher (e.g., a rear passenger has just activated the system and is in the middle of a conversation), the system will control simultaneous announcements from both the front and rear seats, but shift the sound field center of gravity rearward, increasing the volume of the rear speakers and decreasing the volume of the front speakers. For example, the rear volume is increased to 1.2 times the normal value, while the front volume is attenuated to 0.8 times, ensuring that rear passengers can still clearly receive feedback even in noisy road conditions. This priority configuration is understandable for scenarios where second-row passengers initiate voice activation. By enhancing the rear-seat acoustic experience, it prioritizes the second-row passengers' auditory experience in commercial vehicles, solving the technical problem of traditional models where announcements are only made through the front speakers, resulting in unclear audio for the second row.

[0082] In this embodiment, the passenger seat position is obtained by sound source localization and matched with the target broadcast area. The volume of the front and rear speakers is dynamically adjusted in combination with the broadcast priority. This realizes the coordinated broadcast of the whole vehicle speakers and sound field optimization, which solves the technical problem that the second row passengers of the business vehicle cannot hear the TTS broadcast clearly and improves the voice interaction experience of the second row passengers.

[0083] This application also provides a voice playback optimization device for commercial vehicles; please refer to... Figure 5 The commercial vehicle voice playback optimization device includes: The voice acquisition module 10 is used to acquire the voice signal of the passenger initiating a voice interaction request and the reference audio signal returned by the audio amplifier assembly. The speech noise reduction module 20 is used to perform noise reduction processing on the speech signal based on the reference audio signal to obtain the noise-reduced target speech signal; The speech recognition module 30 is used to perform speech recognition based on the target speech signal and generate speech synthesis data to be broadcast. The voice broadcasting module 40 is used to broadcast the voice synthesis data in zones according to a preset speaker area control strategy.

[0084] The commercial vehicle voice playback optimization device provided in this application, employing the commercial vehicle voice playback optimization method in the above embodiments, can solve the technical problem of how to improve the quality of voice interaction for rear-seat passengers. Compared with the prior art, the beneficial effects of the commercial vehicle voice playback optimization device provided in this application are the same as those of the commercial vehicle voice playback optimization method provided in the above embodiments, and other technical features in the commercial vehicle voice playback optimization device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0085] In one embodiment, the voice acquisition module 10 is further configured to acquire multiple audio data transmitted back from the audio amplifier assembly; determine the corresponding speaker partition identifier based on the multiple audio data; and fuse the multiple audio data based on the speaker partition identifier to obtain a reference audio signal.

[0086] In one embodiment, the voice acquisition module 10 is further configured to determine the return ratio coefficient of the audio data corresponding to each partition identifier based on the speaker partition identifier; and to perform weighted fusion of the multiple audio data according to the return ratio coefficient to obtain a reference audio signal.

[0087] In one embodiment, the voice acquisition module 10 is further configured to extract audio channel data corresponding to each speaker partition in the multi-channel audio data; perform amplitude weighting operation on the audio channel data according to the return ratio coefficient to obtain the weighted audio components of each partition; and mix and superimpose the audio components to obtain a reference audio signal.

[0088] In one embodiment, the voice noise reduction module 20 is further configured to acquire seat position information of the passenger initiating the voice interaction request; match a target broadcast area corresponding to the seat position information from a preset speaker area control strategy according to the seat position information; when the target broadcast area is a first broadcast area, control the front speaker group to broadcast the voice-synthesized broadcast data, the front speaker group including a driver's speaker group and a passenger's speaker group; when the target broadcast area is a second broadcast area, control the rear speaker group to broadcast the voice-synthesized broadcast data, the rear speaker group including a left rear area speaker group and a right rear area speaker group.

[0089] In one embodiment, the voice broadcast module 40 is further configured to acquire seat position information of the passenger initiating the voice interaction request; match a target broadcast area corresponding to the seat position information from a preset speaker area control strategy according to the seat position information; when the target broadcast area is a first broadcast area, control the front speaker group to broadcast the voice-synthesized broadcast data, the front speaker group including a driver's speaker group and a passenger's speaker group; when the target broadcast area is a second broadcast area, control the rear speaker group to broadcast the voice-synthesized broadcast data, the rear speaker group including a left rear area speaker group and a right rear area speaker group.

[0090] In one embodiment, the voice broadcasting module 40 is further configured to: acquire a preset broadcasting priority when the target broadcasting area is a first broadcasting area and a second broadcasting area; control the front speaker group and the rear speaker group to broadcast the voice-synthesized data when the broadcasting priority of the first broadcasting area is greater than the broadcasting priority of the second broadcasting area, and increase the broadcasting volume of the front speaker group and decrease the broadcasting volume of the rear speaker group; and control the front speaker group and the rear speaker group to broadcast the voice-synthesized data when the broadcasting priority of the second broadcasting area is greater than the broadcasting priority of the first broadcasting area, and decrease the broadcasting volume of the front speaker group and increase the broadcasting volume of the rear speaker group.

[0091] This application provides a commercial vehicle voice playback optimization device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the commercial vehicle voice playback optimization method in the above embodiment 1.

[0092] The following is for reference. Figure 6 The diagram illustrates a structural schematic suitable for implementing the commercial vehicle voice playback optimization device in the embodiments of this application. The commercial vehicle voice playback optimization device in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 6 The illustrated commercial vehicle voice playback optimization device is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0093] like Figure 6As shown, the commercial vehicle voice playback optimization device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in ROM (Read Only Memory) 1002 or a program loaded from storage device 1003 into RAM (Random Access Memory) 1004. RAM 1004 also stores various programs and data required for the operation of the commercial vehicle voice playback optimization device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via bus 1005. Input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to I / O interface 1006: input devices 1007 including, for example, touch screens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, LCDs (Liquid Crystal Displays), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. The communication device 1009 allows the commercial vehicle voice playback optimization device to communicate wirelessly or wiredly with other devices to exchange data. Although the figures show commercial vehicle voice playback optimization devices with various systems, it should be understood that implementation or possession of all the systems shown is not required. More or fewer systems may be implemented alternatively.

[0094] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0095] The commercial vehicle voice playback optimization device provided in this application, employing the commercial vehicle voice playback optimization method in the above embodiments, can solve the technical problem of how to improve the quality of voice interaction for rear-seat passengers. Compared with the prior art, the beneficial effects of the commercial vehicle voice playback optimization device provided in this application are the same as those of the commercial vehicle voice playback optimization method provided in the above embodiments, and other technical features in this commercial vehicle voice playback optimization device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0096] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0097] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0098] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the commercial vehicle voice playback optimization method in the above embodiments.

[0099] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, RAM (Random Access Memory), ROM (Read Only Memory), EPROM (Erasable Programmable Read Only Memory or Flash Memory), optical fibers, CD-ROM (CD-Read Only Memory), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0100] The aforementioned computer-readable storage medium may be included in the commercial vehicle voice playback optimization device; or it may exist independently and not be installed in the commercial vehicle voice playback optimization device.

[0101] The aforementioned computer-readable storage medium carries one or more programs. When the aforementioned one or more programs are executed by the commercial vehicle voice playback optimization device, the commercial vehicle voice playback optimization device causes the following: to acquire the voice signal of the passenger initiating a voice interaction request and the reference audio signal returned by the audio amplifier assembly; to perform noise cancellation processing on the voice signal based on the reference audio signal to obtain a noise-reduced target voice signal; to perform speech recognition based on the target voice signal to generate speech synthesis data to be played; and to play the speech synthesis data in zones according to a preset speaker area control strategy.

[0102] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including LAN (Local Area Network) or WAN (Wide Area Network)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0103] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0104] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0105] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described commercial vehicle voice playback optimization method, thereby solving the technical problem of how to improve the quality of voice interaction for rear-seat passengers. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the commercial vehicle voice playback optimization method provided in the above embodiments, and will not be repeated here.

[0106] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the above-described commercial vehicle voice playback optimization method.

[0107] The computer program product provided in this application can solve the technical problem of how to improve the quality of voice interaction for rear-seat passengers. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the commercial vehicle voice playback optimization method provided in the above embodiments, and will not be repeated here.

[0108] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A method for optimizing voice playback in commercial vehicles, characterized in that, The method includes: Acquire the voice signal of the passenger initiating the voice interaction request and the reference audio signal returned by the audio amplifier assembly; The speech signal is subjected to noise reduction processing based on the reference audio signal to obtain the noise-reduced target speech signal. Speech recognition is performed based on the target speech signal to generate speech synthesis data to be broadcast; The speech synthesis data is broadcast in zones according to a preset speaker area control strategy.

2. The method as described in claim 1, characterized in that, The steps for obtaining the reference audio signal returned by the audio amplifier assembly include: Acquire multi-channel audio data returned by the audio amplifier assembly; Determine the corresponding speaker zone identifier based on the multi-channel audio data; The multi-channel audio data are fused according to the speaker partition identifier to obtain a reference audio signal.

3. The method as described in claim 2, characterized in that, The step of fusing the multiple audio data according to the speaker partition identifier to obtain a reference audio signal includes: The return ratio coefficient of the audio data corresponding to each zone identifier is determined based on the speaker zone identifier; The multi-channel audio data are weighted and fused according to the return ratio coefficient to obtain a reference audio signal.

4. The method as described in claim 3, characterized in that, The step of weighting and fusing the multi-channel audio data according to the return scaling factor to obtain a reference audio signal includes: Extract the audio channel data corresponding to each speaker partition from the multi-channel audio data; The audio channel data is subjected to amplitude weighting calculation based on the return ratio coefficient to obtain the weighted audio components of each partition; The audio components are mixed and superimposed to obtain a reference audio signal.

5. The method as described in claim 1, characterized in that, The step of partitioning and broadcasting the speech synthesis data according to a preset speaker area control strategy includes: Obtain the seat location information of the passenger who initiated the voice interaction request; Based on the seat position information, a target broadcast area corresponding to the seat position information is matched from a preset speaker area control strategy; When the target broadcast area is the first broadcast area, the front speaker group is controlled to broadcast the speech synthesis broadcast data. The front speaker group includes a driver's speaker group and a passenger's speaker group. When the target broadcast area is the second broadcast area, the rear speaker group is controlled to broadcast the speech synthesis broadcast data. The rear speaker group includes a left rear area speaker group and a right rear area speaker group.

6. The method as described in claim 5, characterized in that, After the step of matching the target broadcast area corresponding to the seat position information from the preset speaker area control strategy based on the seat position information, the method further includes: When the target broadcast area is the first broadcast area and the second broadcast area, a preset broadcast priority is obtained; When the broadcast priority of the first broadcast area is greater than that of the second broadcast area, the front speaker group and the rear speaker group are controlled to broadcast the speech synthesis data, and the broadcast volume of the front speaker group is increased and the broadcast volume of the rear speaker group is decreased. When the broadcast priority of the second broadcast area is greater than that of the first broadcast area, the front and rear speaker groups are controlled to broadcast the speech synthesis data, and the broadcast volume of the front speaker group is reduced while the broadcast volume of the rear speaker group is increased.

7. The method as described in claim 1, characterized in that, The step of performing noise removal processing on the speech signal based on the reference audio signal to obtain the noise-reduced target speech signal includes: The speech signal and the reference audio signal are time-domain aligned to obtain the target reference audio signal. Extract the environmental noise feature components from the speech signal and the speaker playback feature components from the target reference audio signal; The ambient noise feature components are filtered and canceled based on the speaker playback feature components to obtain the noise-reduced target speech signal.

8. A voice playback optimization device for commercial vehicles, characterized in that, The device includes: The voice acquisition module is used to acquire the voice signal of the passenger initiating the voice interaction request and the reference audio signal returned by the audio amplifier assembly; A speech noise reduction module is used to perform noise removal processing on the speech signal based on the reference audio signal to obtain a noise-reduced target speech signal; The speech recognition module is used to perform speech recognition based on the target speech signal and generate speech synthesis data to be broadcast. The voice broadcasting module is used to broadcast the synthesized voice data in zones according to a preset speaker area control strategy.

9. A voice playback optimization device for commercial vehicles, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the commercial vehicle voice playback optimization method as described in any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the commercial vehicle voice playback optimization method as described in any one of claims 1 to 7.

Images