Audio processing method and device of vehicle, electronic equipment and storage medium

Abstract

The invention relates to an audio processing method and device of a vehicle, electronic equipment and a storage medium, and the method comprises the steps: responding to an audio playing request of a first object, and determining a first sound region where the first object is located; under the condition that at least one second sound area except the first sound area is detected to generate the audio signal, controlling a loudspeaker corresponding to the first sound area to start a noise reduction mode; and under the condition of determining that the audio interaction signal exists between the first object of the first sound area and the second object of the second sound area, controlling the loudspeaker corresponding to the first sound area to adjust the noise reduction mode according to the audio interaction signal. According to the invention, when it is detected that the effective audio interaction signal exists between the first sound area and the second sound area, the noise reduction mode can be dynamically adjusted, the problem that user communication is blocked in the noise reduction mode is effectively solved, and users can smoothly communicate when needing communication while enjoying an exclusive sound space.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of vehicle audio processing, in particular to an audio processing method and device of a vehicle, an electronic device and a storage medium. BACKGROUND

[0002] With the rapid development of vehicle technology, in-vehicle comfort and user experience are increasingly valued. During driving and riding, noise is a common problem, including noise from the engine, wheels and the ground, as well as in-vehicle entertainment system, navigation prompts and other sounds.

[0003] Currently, in the field of intelligent cockpit, there is no clear solution on how to effectively reduce in-vehicle noise while avoiding communication barriers between drivers and passengers caused by noise reduction processing. SUMMARY

[0004] The present application provides an audio processing method and device of a vehicle, an electronic device, a vehicle and a storage medium, which can effectively reduce in-vehicle noise while avoiding communication barriers between drivers and passengers caused by noise reduction processing.

[0005] The present application provides an audio processing method of a vehicle, the interior of the cockpit of the vehicle is divided into at least two independent sound zones, each sound zone includes at least one loudspeaker, the method comprises:

[0006] In response to an audio playback request of a first object, determining a first sound zone in which the first object is located;

[0007] In the case of detecting that at least one second sound zone generates an audio signal, controlling the loudspeaker corresponding to the first sound zone to start a noise reduction mode;

[0008] In the case of determining that there is an audio interaction signal between the first object in the first sound zone and the second object in the second sound zone, controlling the loudspeaker corresponding to the first sound zone to adjust the noise reduction mode according to the audio interaction signal.

[0009] According to the above technical means, on the one hand, by dividing the cockpit of the vehicle into independent sound zones and configuring multiple loudspeakers for each sound zone, the audio interference of different areas can be effectively isolated, and the privacy of the user can be enhanced. Further, when detecting that other sound zones generate audio signals, the noise reduction function of the target sound zone can be started to better suppress the spread of cross-area noise. On the other hand, when the cross-sound zone interaction demand is identified, that is, when it is detected that there is an effective audio interaction signal between the first sound zone and the second sound zone, the noise reduction mode can be dynamically adjusted. On the basis of realizing that the sound of each sound zone of the intelligent cockpit is independent and does not interfere with each other, the problem of communication barriers between users in the noise reduction mode is effectively solved, so that the user can enjoy the exclusive sound space and communicate smoothly when needed.

[0010] In some embodiments, determining that there is an audio interaction signal between the first object in the first sound zone and the second object in the second sound zone comprises: collecting a first audio signal of the first object and a second audio signal of the second object; and determining that there is an audio interaction signal between the first object in the first sound zone and the second object in the second sound zone if a sound source direction of the first audio signal deviates from an azimuth of the spatial coordinates of the second sound zone by less than a preset angle, and a sound source direction of the second audio signal deviates from an azimuth of the spatial coordinates of the first sound zone by less than a preset angle.

[0011] According to the above technical means, by comparing the sound source direction of the first audio signal with the azimuth deviation of the spatial coordinates of the second sound zone, and the sound source direction of the second audio signal with the azimuth deviation of the spatial coordinates of the first sound zone, it can be effectively judged whether there is an audio interaction between the two sound zones. Thus, the interaction behavior between the first object and the second object can be more accurately identified, and the audio routing and environmental noise reduction strategy can be optimized.

[0012] In some embodiments, the method further comprises: controlling the loudspeaker corresponding to the first sound zone to adjust the noise reduction mode according to the audio interaction signal, including: controlling the loudspeaker corresponding to the first sound zone to turn off the noise reduction mode if it is determined that the audio interaction signal meets a first condition; and controlling the loudspeaker corresponding to the first sound zone to reduce the noise reduction intensity in the noise reduction mode if it is determined that the audio interaction signal meets a second condition; wherein the first condition is used to indicate that the first object and the second object are in a continuous interaction scenario; and the second condition is used to indicate that the first object and the second object are in a non-continuous interaction scenario.

[0013] According to the above technical means, by judging the interaction type between the first object and the second object according to the audio interaction signal, and controlling the noise reduction mode of the loudspeaker corresponding to the first sound zone, the noise reduction mode is directly turned off in the continuous interaction scenario to improve the voice clarity, and the noise reduction intensity is reduced in the non-continuous interaction scenario to balance the environmental noise and the voice quality. Thus, a more natural and comfortable in-vehicle conversation experience can be achieved, and the overall functionality and user satisfaction of the intelligent cabin can be improved.

[0014] In some embodiments, after controlling the loudspeaker corresponding to the first sound zone to turn off the noise reduction mode, the method further comprises: determining whether the audio interaction signal is detected within a preset time period; and controlling the loudspeaker corresponding to the first sound zone to turn on the noise reduction mode if the audio interaction signal is not detected within the preset time period.

[0015] According to the above technical means, by introducing a detection mechanism of a preset time period after the audio interaction ends, the environmental noise reduction function can be restored in time when the user is no longer interacting, thereby improving the overall energy efficiency and user experience.

[0016] In some embodiments, the first condition comprises one or more of: a time interval between the first audio signal and the second audio signal is less than a first preset interval; a head rotation amplitude of the first object is greater than a first preset amplitude; a head rotation amplitude of the second object is greater than a second preset amplitude; a frequency of the first audio signal is greater than a first preset frequency; a frequency of the second audio signal is greater than the first preset frequency.

[0017] The second condition comprises one or more of: a time interval between the first audio signal and the second audio signal is less than a second preset interval; a head rotation amplitude of the first object is greater than a third preset amplitude; a head rotation amplitude of the second object is greater than a fourth preset amplitude; a frequency of the first audio signal is greater than a second preset frequency; a frequency of the second audio signal is greater than the second preset frequency; wherein the second preset interval is greater than the first preset interval; the third preset amplitude is less than the first preset amplitude; the fourth preset amplitude is less than the second preset amplitude; the first preset frequency is greater than the second preset frequency.

[0018] In some embodiments, the control of the first sound area corresponding loudspeaker to open the noise reduction mode comprises: determining a superimposed audio signal; wherein the superimposed audio signal is obtained by superimposing the audio signal of at least one second sound area outside the first sound area and the external environmental noise; controlling the first sound area corresponding loudspeaker to output a reverse sound wave signal, so that the reverse sound wave signal and the superimposed audio signal occur in the first sound area. destructive interference; wherein the reverse sound wave signal is a signal with equal amplitude and opposite phase to the superimposed audio signal.

[0019] According to the above technical means, by determining the superimposed audio signal and generating the reverse sound wave signal with equal amplitude and opposite phase, and playing through the loudspeaker of the first sound area to realize the destructive interference. Through this technical solution, the interference sound from other sound areas and the environment can be effectively eliminated, so as to improve the audio clarity and user privacy protection level in the first sound area, and thus a more comfortable and personalized auditory environment can be provided for the driver and passenger.

[0020] In some embodiments, the method further comprises: obtaining the running state of the loudspeaker corresponding to the first sound area; in the case that the running state of the loudspeaker is in the working state, receiving a detection signal emitted by the loudspeaker; the detection signal represents the audio playing state of the loudspeaker; if the detection signal indicates that the loudspeaker fails, and the number of failed loudspeakers is greater than a preset number threshold, a switching signal is sent to the loudspeaker; the switching signal is used to instruct the loudspeaker to be switched to a standby loudspeaker.

[0021] According to the above technical means, by acquiring the running state of the loudspeaker corresponding to the first sound area and combining the detection signal to judge the fault, the switching mechanism can be started in time when the loudspeakers corresponding to multiple first sound areas simultaneously appear abnormal, so as to seamlessly transfer the audio playing task to the standby loudspeaker. The problem of audio interruption or quality decline can be effectively avoided, so as to improve the user experience in the intelligent cabin.

[0022] The embodiment of the application provides an audio processing device of a vehicle, and the device comprises:

[0023] A determination unit is configured to determine a first sound area in which a first object is located in response to an audio playing request of the first object;

[0024] A first control unit is configured to control a loudspeaker corresponding to the first sound area to start a noise reduction mode when it is detected that an audio signal is generated in at least one other sound area except the first sound area;

[0025] A second control unit is configured to control the loudspeaker corresponding to the first sound area to adjust the noise reduction mode according to an audio interaction signal when it is determined that the audio interaction signal exists between the first object in the first sound area and a second object in a second sound area.

[0026] The embodiment of the application provides an electronic device, which comprises a processor and a memory, and the memory stores a computer program capable of running on the processor, and the processor implements the steps in any one of the above methods when executing the computer program.

[0027] The embodiment of the application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the steps in any one of the above methods.

[0028] The embodiment of the application provides a computer program product, which comprises a computer program or instructions, and the computer program or instructions are executed by a processor to implement the steps in any one of the above methods.

[0029] The embodiment of the application provides a vehicle, which comprises the above electronic device.

[0030] The beneficial effects of the application are as follows:

[0031] (1) On the one hand, by dividing the cabin of the vehicle into independent sound zones and configuring multiple loudspeakers for each sound zone, the audio interference in different areas can be effectively isolated, enhancing the privacy of the user. Further, when detecting that the other sound zone generates an audio signal, the noise reduction function of the target sound zone can be started, which can better suppress the cross-area noise propagation. On the other hand, when identifying the cross-sound zone interaction demand, that is, detecting that there is an effective audio interaction signal between the first sound zone and the second sound zone, the noise reduction mode can be dynamically adjusted, effectively solving the problem of user communication obstruction in the noise reduction mode, allowing the user to enjoy the exclusive sound space while smoothly communicating when needed.

[0032] (2) By comparing the sound source direction of the first audio signal with the spatial coordinate azimuth deviation of the second sound zone, and the sound source direction of the second audio signal with the spatial coordinate azimuth deviation of the first sound zone, it can be effectively judged whether there is audio interaction between the two sound zones. Thus, the interaction behavior between the first object and the second object can be more accurately identified.

[0033] (3) By judging the interaction type between the first object and the second object according to the audio interaction signal, and controlling the noise reduction mode of the loudspeaker corresponding to the first sound zone, the noise reduction mode is directly turned off in the continuous interaction scenario to improve the speech clarity, and the noise reduction intensity is reduced in the non-continuous interaction scenario to balance the environmental noise and the speech quality, so that a more natural and comfortable in-vehicle conversation experience can be achieved, and the overall functionality and user satisfaction of the intelligent cabin are improved.

[0034] (4) By introducing a preset time detection mechanism after the audio interaction ends, the environmental noise reduction function can be restored in time when the user is no longer interacting, thereby improving the overall energy efficiency and user experience.

[0035] (5) By determining the superimposed audio signal and generating an anti-phase sound wave signal with equal amplitude and opposite phase, and playing it through the loudspeaker of the first sound zone to realize destructive interference. Through this technical solution, the interference sound from other sound zones and the environment can be effectively eliminated, thereby improving the audio clarity and user privacy protection level in the first sound zone.

[0036] (6) By obtaining the running state of the loudspeaker corresponding to the first sound zone and combining the detection signal for fault judgment, the switching mechanism can be started in time when multiple loudspeakers corresponding to the first sound zone simultaneously appear abnormal, thereby seamlessly transferring the audio playing task to the standby loudspeaker. The problem of audio interruption or decreased playing quality can be effectively avoided. BRIEF DESCRIPTION OF DRAWINGS

[0037] Figure 1 A flowchart of an audio processing method of a vehicle provided by the embodiments of the present application;

[0038] Figure 2 A dynamic configuration diagram of an intelligent cockpit power amplifier system provided for an embodiment of the present application;

[0039] Figure 3 A cockpit sound privacy space diagram provided for an embodiment of the present application;

[0040] Figure 4 A typical scene diagram of an intelligent cockpit power amplifier system provided for an embodiment of the present application;

[0041] Figure 5 A dynamic binding loudspeaker diagram provided for an embodiment of the present application;

[0042] Figure 6 A cockpit space dialogue perception noise reduction process diagram provided for an embodiment of the present application;

[0043] Figure 7 A component structure diagram of an audio processing device of a vehicle provided for an embodiment of the present application;

[0044] Figure 8 A hardware entity diagram of an electronic device provided for an embodiment of the present application. DETAILED DESCRIPTION

[0045] The advantages and effects of the present application can be easily understood by those skilled in the art from the description of the present application. The present application can also be implemented or applied by different specific embodiments, and the details in the description can be modified or changed based on different views and applications without departing from the spirit of the present application. It should be understood that the preferred embodiments are only for illustrating the present application, but not for limiting the protection scope of the present application.

[0046] It should be noted that the diagrams provided in the following embodiments only schematically illustrate the basic concept of the present application, and only the components related to the present application are shown in the diagrams, but not drawn according to the number, shape and size of the components in actual implementation. The shape, number and proportion of each component in actual implementation can be arbitrarily changed, and the layout pattern of the components can be more complex.

[0047] In the following description, “some embodiments” are described, which describe a subset of all possible embodiments, but it can be understood that “some embodiments” can be the same subset or different subset of all possible embodiments, and can be combined with each other without conflict.

[0048] In the following description, the terms "first", "second", "third" are only to distinguish similar objects, and do not represent a specific order of the objects. Understandably, "first", "second", "third" can be interchanged in a specific order or sequence as allowed, so that the application described herein can be implemented in an order other than that illustrated or described herein.

[0049] In the present embodiment, the term "and / or" only describes the association relationship of the associated objects, which means that there can be three relationships, for example, object A and / or object B, which can represent three cases: object A exists alone, object A and object B exist together, and object B exists alone.

[0050] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the application belongs. The terms used herein are only for the purpose of describing the embodiments of the application and are not intended to limit the application.

[0051] With the rapid development of vehicle technology, in-vehicle comfort and user experience are increasingly valued. During driving and riding, noise is a common problem, including noise from the engine, wheels and the ground, as well as sounds from in-vehicle entertainment systems, navigation prompts, etc.

[0052] To meet the different needs of users in different positions in the intelligent cabin for sound, such as the main driver wanting clearer navigation sound and smaller music sound when using navigation, while the co-driver wanting to hear only music, and the rear users possibly preferring to watch movies and wanting to hear only movie sound, etc. Or to realize different sound needs of users in different positions in the vehicle, there can be simultaneous needs to make phone calls, listen to music and watch movies without affecting each other, so that the sound space of the entire cabin is divided into four, and users in each position enjoy unique sound without interference, so the entire power amplifier system of the intelligent cabin needs to realize customizable and directional sound display to meet the needs of different users. The power amplifier system of the traditional vehicle does not decouple and bind the sound source and the entire power amplifier system, and the sound cannot be customized and directional, which cannot meet the needs of the power amplifier system of the intelligent cabin.

[0053] In related technologies, an automobile power amplifier device and a vehicle-mounted power amplifier system are proposed, which essentially deal with the quality and energy consumption of the automobile power amplifier in hardware; and a sound source directional method and system are proposed, which essentially is a directional method for identifying the source of human voice in hardware.

[0054] However, the above related technologies have the following problems: (1) cannot be customized according to user needs; (2) users affect each other when simultaneously navigating, listening to music, watching movies, etc., that is, the above solutions cannot meet the user's needs for customization and direction of the power amplifier system.

[0055] Further, even if it can be realized in the field of intelligent cockpit that users in different positions enjoy unique sound as if entering independent space without interference, when the driver and the passenger want to communicate, it may cause the driver and the passenger to be unable to communicate smoothly due to the noise reduction mode, thereby affecting the user experience.

[0056] Based on this, the embodiment of the present application provides an audio processing method of a vehicle. The interior of the cockpit of the vehicle is divided into at least two independent sound zones, and each sound zone includes at least one loudspeaker. The method comprises the following steps: in response to an audio playing request of a first object, determining a first sound zone in which the first object is located; in the case that at least one second sound zone outside the first sound zone generates an audio signal, controlling the loudspeaker corresponding to the first sound zone to start a noise reduction mode; in the case that there is an audio interaction signal between the first object in the first sound zone and a second object in the second sound zone, adjusting the noise reduction mode of the loudspeaker corresponding to the first sound zone according to the audio interaction signal. In this way, on the one hand, by dividing the cockpit of the vehicle into independent sound zones and configuring multiple loudspeakers for each sound zone, the audio interference in different areas can be effectively isolated, and the privacy of the user can be enhanced. Further, when detecting that other sound zones generate audio signals, the noise reduction function of the target sound zone is started, which can better inhibit the propagation of cross-area noise. On the other hand, when the cross-sound zone interaction demand is identified, that is, when it is detected that there is an effective audio interaction signal between the first sound zone and the second sound zone, the noise reduction mode can be dynamically adjusted. On the basis of realizing the independent sound of each sound zone in the intelligent cockpit and the non-interference, the problem that the user is blocked in communication in the noise reduction mode is effectively solved, so that the user can enjoy the exclusive sound space and communicate smoothly when needed.

[0057] In the following, the technical solutions in the embodiments of the present application will be described clearly and completely with reference to the drawings in the embodiments of the present application.

[0058] Figure 1 A flowchart of an audio processing method of a vehicle provided by the embodiment of the present application is shown in FIG. 1. Figure 1 As shown in FIG. 1, the method can be applied to an electronic control unit of a vehicle, and comprises steps 101 to 103, wherein:

[0059] Step 101: in response to an audio playing request of a first object, determining a first sound zone in which the first object is located.

[0060] It should be noted that the interior of the cockpit of the vehicle can be divided into at least two independent sound zones.

[0061] Exemplarily, if the cabin of the vehicle is divided into two sound zones, the sound zones can include a driver area and a passenger area; Exemplarily, if the vehicle is a two-row sedan, and the cabin of the vehicle is divided into multiple sound zones, the sound zones can include a left front area (driver area), a left rear area, a right front area (passenger area), and a right rear area. Exemplarily, if the vehicle is a three-row business car, and the cabin of the vehicle is divided into multiple sound zones, the multiple sound zones in the vehicle can include a driver area where the driver is located, a passenger area, a left rear first-row area, a right rear first-row area, a left rear second-row area, and a right rear second-row area. The specific areas can be divided according to the type of the vehicle, and the embodiments of the present application do not limit this.

[0062] It should be further pointed out that each sound zone can include at least one loudspeaker. That is, different sound zones can bind one or more loudspeakers at different positions.

[0063] It should be understood that the loudspeakers at different positions in the cabin can be bound to the sound sources according to different scenarios, or the binding of each loudspeaker to a specific sound source can be customized according to user needs according to a pre-configuration page.

[0064] Exemplarily, Figure 2 A dynamic configuration diagram of an intelligent cabin power amplifier system is provided in the embodiments of the present application; as Figure 2 shown, the intelligent cabin power amplifier system includes multiple sound sources, such as calls, navigation, voice assistants, media A, media B, media C, and media D. Further, the binding of each loudspeaker to a specific sound source can be customized according to user needs, such as binding the main driver loudspeaker to the three sound sources of calls, navigation, and voice assistants; binding the passenger loudspeaker to the two sound sources of voice assistants and media A; and binding the second-row left loudspeaker to the two sound sources of voice assistants and media B. By binding each loudspeaker to a specific sound source, the loudspeakers in different sound zones only play the sound source content bound thereto (such as the main driver area only outputting key information such as calls and navigation, and the passenger and rear areas independently playing media content), effectively avoiding sound aliasing when multiple sound sources are played simultaneously, and reducing interference between users in different areas.

[0065] In some embodiments, the first object can send an audio playback request through the vehicle-mounted system; the electronic control unit responds to the request, so that the first sound zone in which the first object is located can be determined.

[0066] The first object refers to a user in the cabin of the vehicle who initiates an audio playback request; the audio playback request can include but is not limited to playing navigation prompts, playing music lists, and playing rear movie audio.

[0067] It should be noted that the audio playing request can be triggered by physical interaction (button / touch screen) of the vehicle-mounted system, voice interaction (wake-up word + instruction) or wireless device projection (mobile phone Bluetooth / Wi-Fi projection) and the like.

[0068] In a possible implementation, a plurality of microphones are arranged in the cabin, and the position where the user issues the voice request, that is, the first sound area where the first object is located, is located through time difference of arrival of sound waves or beamforming technology. Illustratively, the user says "play music", and the system determines that the sound source is located in the left rear sound area through microphone analysis of the sound source direction.

[0069] In another possible implementation, the user can manually select the sound area before playing (such as popping up a prompt "please select the playing position: left front area / right rear area").

[0070] Step 102, in the case of detecting that at least one second sound area other than the first sound area generates an audio signal, controlling the loudspeaker corresponding to the first sound area to start the noise reduction mode.

[0071] Among them, the second sound area refers to other sound areas in addition to the first sound area, and it can be understood that the number of second sound areas can be one or more.

[0072] It should be noted that starting the noise reduction mode means that when at least one second sound area other than the first sound area generates an audio signal, the loudspeaker corresponding to the first sound area adopts a preset noise reduction algorithm, so as to reduce or eliminate the interference of the audio signal from the second sound area on the first sound area, and create a better audio experience environment for the user in the first sound area.

[0073] In some embodiments, controlling the loudspeaker corresponding to the first sound area to start the noise reduction mode can include the following steps:

[0074] Step 1021, determining the superimposed audio signal.

[0075] Among them, the superimposed audio signal is obtained by superimposing the audio signal of at least one second sound area other than the first sound area and the external environmental noise. Specifically, in the present application, the superimposed audio signal can include audio content played from other sound areas (such as the co-pilot or the back row) and external environmental noise (such as engine noise, wind noise, etc.) in the vehicle.

[0076] In some embodiments, the electronic control unit can collect other sound areas other than the first sound area and external environmental noise in real time through the microphone array, and fuse the collected audio signals to generate the superimposed audio signal.

[0077] In this way, by collecting and calculating the superimposed audio signal, it can be more accurately judged which audio signals need to be eliminated, so as to improve the effectiveness of noise reduction.

[0078] In step 1022, the first sound area corresponding speaker outputs the reverse sound wave signal, so that the reverse sound wave signal and the superimposed audio signal have destructive interference in the first sound area.

[0079] The reverse sound wave signal is an audio signal used to realize the active noise reduction function. The reverse sound wave signal can be a signal with equal amplitude and opposite phase of the superimposed audio signal. For example, if the peak of the superimposed audio signal occurs at a certain time, the trough of the reverse sound wave signal at the same time will cancel the peak of the superimposed audio signal.

[0080] It should be understood that when the reverse sound wave signal and the superimposed audio signal are synchronized in time and have equal amplitude and opposite phase, the reverse sound wave signal and the superimposed audio signal will have destructive interference, thereby canceling the effect of the superimposed audio signal.

[0081] In the embodiments of the present application, the corresponding reverse sound wave signal can be generated according to the superimposed audio signal of the region outside the first sound area, and the reverse sound wave signal is output through the speaker arranged in the first sound area, so as to cancel the effect of the superimposed audio signal, thereby ensuring that the user in the first sound area will not be disturbed by the content played by other sound areas and environmental noise.

[0082] It should be noted that the frequency characteristics of the superimposed audio signal can also be calculated in real time, and the parameters of the reverse sound wave signal can be dynamically adjusted to ensure that the reverse sound wave signal and the superimposed audio signal form the best destructive interference effect.

[0083] For example, Figure 3 A cockpit sound privacy space schematic diagram is provided in the embodiments of the present application; as Figure 3 As shown, the cockpit is divided into a main driver area, a co-driver area, a left rear area and a right rear area, wherein the entire cockpit sound is independent of each other. When the four position spaces are active, the reverse phase sound wave interference of the respective sound sources is sent to reduce the interference on other spaces. When there is a sound source in a single space, the reverse phase sound wave interference of the corresponding sound source is sent in other spaces to make it not disturbed by the sound source.

[0084] In the embodiments of the present application, the superimposed audio signal is determined, and the reverse sound wave signal with equal amplitude and opposite phase is generated, and the first sound area speaker is played to realize the destructive interference. Through the technical scheme, the interference sound from other sound areas and the environment can be effectively eliminated, so that the audio clarity and user privacy protection level in the first sound area can be improved, and a more comfortable and personalized auditory environment can be provided for the driver and passenger.

[0085] Step 103, in the case where it is determined that there is an audio interaction signal between the first object of the first sound area and the second object of the second sound area, the corresponding speaker of the first sound area adjusts the noise reduction mode according to the audio interaction signal.

[0086] The audio interaction signal refers to the combination of audio signals generated when there is a speech exchange behavior between the first object and the second object in two different sound areas. For example, the first object in the driver position is having a conversation with the second object in the co-driver position, or the second object in the back row position needs to communicate with the first object in the front row position, which will generate an audio interaction signal.

[0087] In the presence of an audio interaction signal, the electronic control unit can dynamically adjust the noise reduction strategy according to the audio interaction signal by recognizing the conversation behavior between the first object and the second object, to ensure the naturalness and clarity of the conversation.

[0088] It should be noted that the following methods can be used to determine whether there is an audio interaction signal between the first object of the first sound area and the second object of the second sound area.

[0089] In one possible implementation, the microphone array can be used to collect environmental audio data, and a voice activity detection algorithm can be used to identify whether there is voice activity. When it is detected that there is a speech exchange between two different sound areas, it can be automatically determined whether it is an audio interaction signal.

[0090] In another possible implementation, infrared or camera sensors can be used to monitor the head movements and mouth movements of the first object and the second object, to determine whether there is an audio interaction signal between the first object of the first sound area and the second object of the second sound area.

[0091] It should also be understood that adjusting the noise reduction mode can include turning off the noise reduction mode, or adjusting the noise reduction intensity of the noise reduction mode.

[0092] In some embodiments, when it is detected that there is a conversation behavior between the users of two sound areas, the noise reduction settings of the corresponding speaker of the first sound area can be adjusted according to the audio interaction signal between the two sound areas, to ensure smooth communication between the first object and the second object.

[0093] In some embodiments, when the first object in the first sound zone generates the first audio signal, the electronic control unit can first switch the first sound zone from the noise reduction mode to the conversation mode (the noise reduction intensity is reduced to 20%); when the second object in the second sound zone generates the second audio signal is detected within a preset time length (the conversation awareness trigger / monitoring head movement determines the speaking direction), the second sound zone is automatically switched to the conversation mode (the noise reduction intensity is reduced to 20%); further, the frequency band of the first audio signal and the second audio signal can be enhanced; after the conversation ends, the original noise reduction setting is restored.

[0094] In some embodiments, the conversation mode can be to reduce the noise reduction intensity or directly turn off the noise reduction mode; the conversation awareness trigger condition can include detecting voice activity (human voice threshold > 60 dB) and sound source direction recognition.

[0095] For example, if the independent noise reduction is turned on in each sound zone in the cabin, the driver area and the left rear area want to have a conversation; the system detects that the driver speaks (conversation awareness trigger); the driver automatically switches to the conversation mode (the noise reduction intensity is reduced to 20%); the left rear responds; the system detects that the left rear speaks (conversation awareness trigger / monitoring head movement determines the speaking direction); the left rear automatically switches to the conversation mode (the noise reduction intensity is reduced to 20%); the human voice frequency band is enhanced; after the conversation ends, the original noise reduction setting is restored.

[0096] In some embodiments of the present application, on the one hand, by dividing the cabin of the vehicle into independent sound zones and configuring multiple loudspeakers for each sound zone, the audio interference in different areas can be effectively isolated, and the user privacy and auditory independence can be ensured; further, when detecting that other sound zones generate audio signals, the noise reduction function of the target sound zone is automatically started, so that the cross-area noise propagation can be better inhibited. On the other hand, when the cross-sound zone interaction demand is identified, that is, when it is detected that there is an effective audio interaction signal between the first sound zone and the second sound zone, the noise reduction mode can be dynamically adjusted, on the basis of realizing the sound independence of each sound zone in the intelligent cabin and the mutual interference, the problem that the user communication is blocked in the noise reduction mode is effectively solved, so that the user can enjoy the exclusive sound space and can communicate smoothly when needed.

[0097] In some embodiments of the present application, determining that there is an audio interaction signal between the first object in the first sound zone and the second object in the second sound zone includes the following steps:

[0098] Step 201, collecting the first audio signal of the first object and the second audio signal of the second object.

[0099] The first object is a user in the cabin, and the first audio signal can be audio content emitted by the user. For example, the first object is a driver sitting in the driver's seat, and the first audio signal can include audio signals generated when the driver speaks or emits other sounds. Similarly, the second object is another object (user) in the cabin other than the first object, and the second audio signal is audio emitted by the second object. For example, the second object can be a passenger sitting in the front passenger seat, and the second audio signal can include sound signals generated when the passenger speaks.

[0100] In some embodiments, the electronic control unit can collect the first audio signal and the second audio signal through a microphone array deployed inside the cabin, and further, can perform signal preprocessing on the collected first audio signal and second audio signal to ensure that the audio content of the first audio signal of the first object and the second audio signal of the second object is clearer.

[0101] Step 202, if the sound source direction of the first audio signal deviates from the spatial coordinate azimuth of the second audio zone by less than a preset angle, and the sound source direction of the second audio signal deviates from the spatial coordinate azimuth of the first audio zone by less than a preset angle, it is determined that there is an audio interaction signal between the first object of the first audio zone and the second object of the second audio zone.

[0102] The sound source direction can be the position direction of the audio signal, which can be determined by acoustic positioning techniques such as beamforming. For example, in the cabin, a certain fixed point (such as the center position of the vehicle) in the cabin can be taken as a reference point, and the sound source direction can be represented by an angle to describe the direction from which the sound comes. For example, taking the front of the vehicle as 0 degrees and the back as 180 degrees, the sound source direction can be represented by an angle value between 0-360 degrees.

[0103] The spatial coordinate azimuth refers to a reference direction set based on the internal structure of the cabin. After establishing a coordinate system with the reference point as the origin in three-dimensional space, the angle is used to determine the position of the sound source in space.

[0104] In some embodiments, when the sound source direction of the first audio signal deviates from the spatial coordinate azimuth of the second audio zone by less than a preset angle, it indicates that the sound direction emitted by the first object is close to the second audio zone; at the same time, when the sound source direction of the second audio signal deviates from the spatial coordinate azimuth of the first audio zone by less than a preset angle, it indicates that the sound direction emitted by the second object is close to the first audio zone. In this case, it can be determined that there is an audio interaction signal between the first object of the first audio zone and the second object of the second audio zone, i.e., they can be engaging in audio interaction behaviors such as conversation and communication.

[0105] In the embodiments of the present application, by comparing the sound source direction of the first audio signal and the spatial coordinate azimuth deviation of the second sound zone, and the sound source direction of the second audio signal and the spatial coordinate azimuth deviation of the first sound zone, whether there is audio interaction between the two sound zones can be effectively determined. Thus, the interaction behavior between the first object and the second object can be more accurately identified, and the audio routing and environmental noise reduction strategy can be optimized.

[0106] In some embodiments of the present application, the first sound zone corresponding loudspeaker adjusts the noise reduction mode according to the audio interaction signal, including the following steps:

[0107] Step 301, in the case of determining that the audio interaction signal meets the first condition, controlling the first sound zone corresponding loudspeaker to close the noise reduction mode.

[0108] The first condition is used to indicate that the first object and the second object are in a continuous interaction scene; for example, the first object and the second object have a long time voice communication in the same space, or there is continuous voice activity. When it is detected that the audio interaction signal meets the first condition, it indicates that it is a continuous dialogue environment at present, and therefore the intelligibility of the voice needs to be prioritized.

[0109] It should be noted that closing the noise reduction mode means stopping executing the active noise reduction algorithm, so that the loudspeaker no longer emits anti-phase sound waves to cancel external noise.

[0110] In some embodiments, the first condition can include one or more of the following: the time interval between the first audio signal and the second audio signal is less than a first preset interval; the head rotation amplitude of the first object is greater than a first preset amplitude; the head rotation amplitude of the second object is greater than a second preset amplitude; the frequency of the first audio signal is greater than a first preset frequency; the frequency of the second audio signal is greater than the first preset frequency.

[0111] The time interval refers to the time difference between the adjacent occurrence of two audio signals on the time axis. The first preset interval can be set according to the system response speed, and is used to determine whether the two audio signals have continuity. If the time interval between two voice signals (the first audio signal and the second audio signal) is very short, less than the first preset interval set in advance, it indicates that the dialogue is in a continuous state.

[0112] The head rotation amplitude refers to the change in the angle of the user's head relative to the initial position when the user is performing voice interaction. The first preset amplitude is a threshold for determining whether the user is actively communicating. When the head rotation amplitude exceeds the first preset amplitude, it can be inferred that the user is participating in the dialogue.

[0113] The frequency of the audio signal represents how quickly or slowly the sound vibrates, usually measured in Hertz (Hz). The first preset frequency is a threshold for distinguishing human voice from other environmental noise. When the frequency of the audio signal is higher than the first preset frequency, it can indicate that the first object and the second object are conducting voice activity.

[0114] In some embodiments, the electronic control unit can determine whether to turn off the noise reduction mode by detecting the audio interaction signal and determining whether the signal satisfies the first condition. In a continuous interaction scenario, a clearer voice experience can be provided, thereby improving the communication efficiency between the first object and the second object, and thus optimizing the overall use experience in the intelligent cabin.

[0115] Step 302, in the case where it is determined that the audio interaction signal satisfies the second condition, the electronic control unit controls the loudspeaker corresponding to the first sound zone to reduce the noise reduction intensity in the noise reduction mode.

[0116] The second condition is used to indicate that the first object and the second object are in a non-continuous interaction scenario. For example, the first object and the second object are conducting a brief exchange, rather than a long conversation. In the case where the audio interaction signal satisfies the non-continuous interaction scenario, the electronic control unit does not completely turn off the noise reduction mode, but appropriately reduces the intensity of the noise reduction mode to maintain a certain environmental noise suppression capability while ensuring basic voice clarity.

[0117] In some embodiments, the second condition can include one or more of the following: the time interval between the first audio signal and the second audio signal is less than a second preset interval; the head rotation amplitude of the first object is greater than a third preset amplitude; the head rotation amplitude of the second object is greater than a fourth preset amplitude; the frequency of the first audio signal is greater than a second preset frequency; and the frequency of the second audio signal is greater than the second preset frequency.

[0118] The second preset interval is greater than the first preset interval; the third preset amplitude is less than the first preset amplitude; the fourth preset amplitude is less than the second preset amplitude; and the first preset frequency is greater than the second preset frequency.

[0119] It can be understood that the second preset interval is greater than the first preset interval, which indicates that the pause between the two voice signals is relatively long, and the conversation is not closely continuous, i.e., in a non-continuous interaction state.

[0120] It can also be understood that the head rotation amplitude exceeds the third preset amplitude but is less than the first preset amplitude. In this case, the head rotation can not be focused on continuous conversation with the second object, but can be an action caused by other reasons, i.e., indicating that the user can be in a non-continuous interaction scenario.

[0121] In the embodiments of the present application, the interaction type between the first object and the second object is determined according to the audio interaction signal, and the noise reduction mode of the loudspeaker corresponding to the first sound area is controlled. In the continuous interaction scene, the noise reduction mode is directly turned off to improve the voice clarity, and in the non-continuous interaction scene, the noise reduction intensity is reduced to balance the environmental noise and the voice quality. Thus, a more natural and comfortable in-vehicle conversation experience can be realized, and the overall functionality and user satisfaction of the intelligent cabin are improved.

[0122] In some embodiments of the present application, after controlling the loudspeaker corresponding to the first sound area to turn off the noise reduction mode, the method further comprises the following steps:

[0123] Step 401, determining whether an audio interaction signal is detected within a preset time length;

[0124] Step 402, in the case that no audio interaction signal is detected within the preset time length, controlling the loudspeaker corresponding to the first sound area to turn on the noise reduction mode.

[0125] The preset time length refers to a time window set by the system, which is used to determine whether the user continuously performs audio interaction activities.

[0126] It should be noted that the preset time length can be configured according to different application scenarios. For example, in the intelligent cabin, the preset time length can be set to 5 seconds, 10 seconds, 30 seconds, etc. The specific setting value depends on the requirements of the current application scenario, and the embodiments of the present application do not limit this.

[0127] In some embodiments, after the user completes a voice interaction, the system enters a waiting state, and continuously listens for a new audio interaction signal within the preset time length. If the user speaks again or triggers other audio-related operations within the preset time length after the system enters the waiting state, the system will maintain the audio interaction signal detection state. If no audio interaction signal is detected within the preset time length after the system enters the waiting state, it indicates that the user has temporarily ended the interaction behavior. At this time, the loudspeaker is controlled to turn on the noise reduction mode again to reduce the interference of other sound areas and environmental noise on the user, and to maintain a quiet and comfortable riding experience in the vehicle.

[0128] In the embodiments of the present application, by introducing the detection mechanism of the preset time length after the audio interaction ends, the environmental noise reduction function can be restored in time when the user no longer interacts, thereby improving the overall energy efficiency and user experience.

[0129] In some embodiments of the present application, the audio processing method of the vehicle further comprises the following steps:

[0130] Step 501, obtaining the running state of the loudspeaker corresponding to the first sound area.

[0131] The running state refers to whether the loudspeaker corresponding to the first sound area is currently in a normal working state. Illustratively, the running state can include a working state and a paused state.

[0132] In some embodiments, the running state of the loudspeaker corresponding to the first sound area can be acquired by the hardware detection module in real time to collect current parameters, voltage parameters, etc. of the loudspeaker, and to determine whether the loudspeaker corresponding to the first sound area is playing audio, i.e., whether the loudspeaker is in a working state.

[0133] It can be understood that the running state is acquired to confirm whether the loudspeaker corresponding to the first sound area is playing audio, so as to determine whether to receive the detection signal sent by the loudspeaker corresponding to the first sound area. That is, if the running state of the loudspeaker is a paused state, the detection signal sent by the loudspeaker does not need to be received, or in other words, the detection signal sent by the loudspeaker cannot be received. If the running state of the loudspeaker is a working state, the detection signal sent by the loudspeaker is received, and whether the loudspeaker has a fault is determined according to the detection signal.

[0134] In this way, the detection control module can ensure that the subsequent collection of the detection signal is only performed when the loudspeaker corresponding to the first sound area is in a working state, and invalid detection and misjudgment can be avoided, thereby improving the reliability of the audio system.

[0135] Step 502, in a case where the running state of the loudspeaker is a working state, receiving a detection signal sent by the loudspeaker.

[0136] Here, the detection signal refers to a signal output by the loudspeaker corresponding to the first sound area when playing audio. Illustratively, the detection signal can be audio waveform data, power output data, frequency response data, etc.

[0137] The detection signal can be used to represent the audio playing state of the loudspeaker. The audio playing state can include normal playing, no output, intermittent playing, distorted playing, etc.

[0138] In some embodiments, when the loudspeaker corresponding to the first sound area is in a working state, the output signal of the loudspeaker corresponding to the first sound area can be continuously monitored to determine whether there is an abnormal situation.

[0139] Step 503, if the detection signal indicates that the loudspeaker has a fault, and the number of loudspeakers having faults is greater than a preset number threshold, a switching signal is sent to the loudspeaker. The switching signal is used to instruct the loudspeaker to be switched to a backup loudspeaker.

[0140] The preset quantity threshold can be set according to different vehicle models or user needs, for example, in a high-fidelity sound system, the preset quantity threshold can be low, and in a general vehicle model, the preset quantity threshold can be high, and the embodiments of the application do not limit this.

[0141] The switching signal is a control signal for instructing switching of the speaker corresponding to the first sound area of the current fault to a backup speaker to maintain the audio output quality of the first sound area.

[0142] In some embodiments, when the detection signal indicates that the speaker corresponding to the first sound area fails (such as no output, sound distortion, abnormal power amplifier, etc.), and in the same sound area, the number of speakers corresponding to the first sound area of the fault exceeds the preset quantity threshold (such as greater than or equal to two), the system will trigger the fault switching mechanism to switch the faulty speaker to a backup speaker.

[0143] It can be understood that the backup speaker can be a pre-configured redundant device, and when the speaker corresponding to the first sound area fails, the system will replace the speaker corresponding to the first sound area with the backup speaker according to the switching signal, and use the backup speaker to continue to complete the audio playback task.

[0144] It should be noted that the switching process is usually controlled by the audio routing engine, including redistribution of the audio signal path, adjustment of the channel gain, and synchronous update of the noise reduction algorithm.

[0145] In the embodiments of the application, by obtaining the running state of the speaker corresponding to the first sound area and combining the detection signal for fault judgment, the switching mechanism can be started in time when multiple speakers corresponding to the first sound area simultaneously appear abnormal, so as to seamlessly transfer the audio playback task to the backup speaker. By using the above method, the problem of audio interruption or playback quality degradation can be effectively avoided, so as to improve the user experience in the intelligent cabin.

[0146] The application will be described below. The application provides an audio processing method of a vehicle in an actual scene.

[0147] The application relates to the field of intelligent cabin software and hardware intelligent power amplifier interaction design, and specifically relates to a power amplifier system technology method, in which full-vehicle speakers can be dynamically bound according to scenes, different sound sources can be played in a directional manner, and user privacy protection can be realized, and real-time environmental noise reduction can be realized in a user conversation process.

[0148] In some embodiments, to realize binding of the speaker to the specified software according to user needs, a layered software architecture can be constructed, which includes four core modules, namely an audio source management module, a scene recognition engine, an audio routing engine and a privacy protection module.

[0149] The audio source management module mainly undertakes the following functions:

[0150] Multi-channel audio stream processing: audio streams of default phone, media player, navigation, etc. application, supporting multi-channel input parallel processing.

[0151] Custom configuration: providing a configuration page, customizing the binding of each position loudspeaker with a specific audio source according to user needs.

[0152] Hardware compatibility: providing a standardized interface compatible with different hardware devices.

[0153] In some embodiments, the scene recognition engine mainly undertakes the following functions:

[0154] Scene switching trigger: triggering scene switching through user input (voice instruction, central control screen selection) or context perception (Bluetooth call state, navigation activation) or gravity sensing, etc.

[0155] Mapping logic definition: integrating a rule engine to define scene-loudspeaker mapping logic.

[0156] Exemplarily, if the current scene is a call scene, the bound main driver loudspeaker, microphone array and noise reduction algorithm can be determined according to the scene-loudspeaker mapping logic. If the current scene is a rear cinema scene, the bound loudspeaker, subwoofer and noise reduction algorithm can be determined according to the scene-loudspeaker mapping logic.

[0157] The audio routing engine mainly undertakes the following functions:

[0158] Dynamic mixing and distribution: using dynamic matrix mixing technology to distribute each channel signal to the specified loudspeaker in real time.

[0159] Time synchronization control: supporting time synchronization control to avoid multi-source audio conflict.

[0160] Audio algorithm: simultaneously starting noise reduction and shielding audio algorithms to ensure the sound clarity of each position user and mutual non-influence.

[0161] In some embodiments, as shown in Figure 3 The cabin sound space is divided into four parts, each space maintains relative independence and isolation of sound, and the privacy of the seat user is maximally protected. Further, multiple audio zones can share audio. Exemplarily, the song listened by the copilot can directly call the voice assistant to share with the rear row, directly call the rear row loudspeaker and other audio devices to play music, but does not affect other position users.

[0162] In some embodiments, exemplarily, Figure 4 A typical scene schematic diagram of an intelligent cabin power amplifier system provided by the embodiments of the present application; as shown in Figure 4As shown, the typical application scenario implements a telephone mode activation process including the following steps:

[0163] S601, the Bluetooth protocol stack detects an incoming call.

[0164] S602, the current scene is identified as a telephone scene according to the scene recognition engine, and the current scene is sent to the audio management module.

[0165] S603, the audio management module reads the user-defined configuration table according to the current scene (default if not) and determines the configuration information matched with the current scene, and sends the configuration information to the audio routing engine.

[0166] It should be noted that the user's historical operation can also be analyzed by an artificial intelligence active learning algorithm. For example, the frequency of the user manually adjusting the noise reduction intensity, volume and other parameters can be recorded; and a scene-preference mapping model (that is, a configuration table) can be established in combination with environmental sensor data (such as vehicle speed, noise type).

[0167] S604, the audio routing engine receives the configuration table.

[0168] S605, the audio routing engine starts the custom configuration speaker and channel according to the configuration table.

[0169] It should be noted that the speaker can be subjected to noise reduction by using an audio algorithm (active noise reduction algorithm, etc.).

[0170] For example, the audio routing engine can also perform the following operation sequence: channel gain adjustment; activation of the microphone array; start of echo cancellation, active noise reduction, etc.

[0171] In some embodiments, for example, Figure 5 A dynamic binding speaker schematic diagram is provided for the embodiments of the present application; as shown Figure 5 As shown, the dynamic binding speaker can include the following steps:

[0172] S701, there is a passenger in the left rear area;

[0173] S702, the rear seat is detected to have a passenger by the position gravity sensor;

[0174] S703, the passenger turns on the rear video software playback power;

[0175] S704, the current scene is dynamically identified as a rear cinema scene according to the scene recognition engine;

[0176] S705, the scene recognition engine sends the rear cinema scene to the audio management module;

[0177] S706, the audio management module reads the user-defined configuration table according to the back row cinema scene (default if not) and determines the configuration information matched with the current scene, and sends the configuration information to the audio routing engine;

[0178] S707, the audio routing engine starts the back row left speaker.

[0179] It should be noted that the audio routing engine can also perform channel gain adjustment, start echo cancellation, active noise reduction and other operations, so that the passengers in the left rear area can enjoy the cinema audio.

[0180] In some embodiments, the cabin conversation perception real-time control environmental noise function needs to be realized by the cooperation of the environmental perception layer, the computing power chip, the function cooperation mechanism and the user interaction layer.

[0181] In terms of environmental perception layer, environmental noise data can be collected in real time by using a microphone array, head movements can be monitored by using infrared or camera sensors, and noise frequency characteristics can be calculated in real time.

[0182] The computing power chip can integrate real-time noise reduction processing with extremely low delay, voice activity detection algorithms and personalized noise reduction curve learning algorithms.

[0183] In the function cooperation mechanism, the conversation perception trigger condition is to detect voice activity (human voice threshold > 60dB) and perform sound source direction recognition; the noise reduction mode switching logic is as follows: in a strong noise environment, a complete noise reduction mode is used, in a medium noise environment, an adaptive noise reduction is used, and in a conversation scene, self-position noise reduction is turned off or local noise reduction is performed.

[0184] The user interaction layer supports user control of the opening and closing of the function and the adjustment of the noise reduction degree through touch or voice control.

[0185] In some embodiments, an example of Figure 6 A cabin space conversation perception noise reduction process schematic diagram provided for the embodiments of the present application; as Figure 6 shown, can include the following steps:

[0186] S801, environmental sensor array (microphone array / accelerometer).

[0187] For example, the microphone array is used to collect sound signals in the cabin, and the accelerometer can be used to assist in detecting vibration information, providing raw data for subsequent noise reduction processing.

[0188] S802, real-time environmental noise analysis (frequency / position / intensity).

[0189] Exemplarily, the chip analyzes the collected signals in real time to determine the frequency, direction, and intensity of environmental noise, etc. These information is crucial for subsequent noise reduction control, as different noise characteristics require different processing methods.

[0190] S803, adaptive noise reduction control (active noise reduction intensity adjustment).

[0191] Exemplarily, according to the results of real-time environmental noise analysis, the system performs adaptive noise reduction control and actively adjusts the noise reduction intensity. For example, when a strong low-frequency noise is detected, the system will increase the noise reduction intensity of the corresponding frequency band.

[0192] S804, personalized volume adjustment (scene volume memory).

[0193] Exemplarily, the system can adjust the volume according to the user's personalized settings or scene memory. For example, the user may have different needs for the volume in different scenarios (such as driving, resting), and the system can remember these preferences and make corresponding adjustments.

[0194] S805, chip (noise reduction algorithm engine).

[0195] Exemplarily, the chip as the core processing unit, carries the noise reduction algorithm engine. It receives signals from the environmental sensor array and processes them, and is the calculation center of the entire noise reduction process.

[0196] S806, conversation awareness function is turned on.

[0197] Exemplarily, when the conversation awareness function is turned on, the system begins to pay attention to the conversation in the cabin in order to handle it accordingly when the conversation occurs.

[0198] S807, voice activity detection (human voice threshold > 60dB), sound source direction recognition (beamforming technology).

[0199] Exemplarily, when voice activity is detected and the intensity of human voice exceeds 60dB, the system uses beamforming technology to identify the direction of the sound source. Beamforming technology can make the microphone array more sensitive to sound in a specific direction, thus accurately identifying the location of the speaker.

[0200] S808, microphone array, conversation awareness trigger (voice activity detection).

[0201] Exemplarily, the microphone array detects whether there is voice activity in the cabin through voice activity detection technology. When voice activity (human voice threshold > 60dB) is detected, the system will trigger the subsequent conversation processing process.

[0202] S809, conversation ends, automatically restore noise reduction.

[0203] Exemplarily, after the conversation ends, the system automatically restores to the previous noise reduction state and continues to suppress the environmental noise to ensure a quiet environment in the cabin.

[0204] S810, manual intervention (touch / voice control) of the user.

[0205] Exemplarily, the user can also manually intervene in the noise reduction system through touch or voice control. For example, the user can manually adjust the noise reduction intensity, turn on or off the conversation awareness function, etc.

[0206] Exemplarily, if multiple sound zones in the cabin are all turned on with independent noise reduction, the co-driver area and the left rear area want to have a conversation; the system detects that the co-driver is speaking (conversation awareness trigger); the co-driver automatically switches to the conversation mode (the noise reduction intensity is reduced to 20%); the left rear responds; the system detects that the left rear is speaking (conversation awareness trigger / monitoring head movement to determine the speaking direction); the left rear automatically switches to the conversation mode (the noise reduction intensity is reduced to 20%); the human voice frequency band is enhanced; and after the conversation ends, the original noise reduction setting is restored.

[0207] The application can customize the sound source and sound emitting position of the loudspeaker by the user, and the user's needs and privacy will be met. For example, the conversation of the rear row user can be output by the rear row loudspeaker alone, and the loudspeakers at other positions can interfere with the sound waves in the opposite direction to reduce noise, fully protecting the privacy of the rear row user, and the conversation content cannot be clearly heard by other positions. Realize the environmental noise reduction function in real time through conversation awareness. According to the user's use scene, dynamically bind the loudspeaker, and realize dynamic, directional and customized output. When the user uses the application, the sound source of other positions can be shielded to reduce the impact on other position users and protect the user's privacy. Realize the environmental noise reduction function in real time through conversation awareness.

[0208] Based on the above embodiment, the application also provides an audio processing device of a vehicle, Figure 7 The composition structure of the audio processing device of the vehicle provided in the application is shown in the figure. Figure 7 As shown in the figure, the audio processing device 700 of the vehicle includes a determination unit 701, a first control unit 702, and a second control unit 703, wherein:

[0209] The determination unit 701 is configured to determine a first sound zone in which a first object is located in response to an audio playing request of the first object.

[0210] The first control unit 702 is configured to control the loudspeaker corresponding to the first sound zone to start a noise reduction mode when it is detected that at least one other sound zone other than the first sound zone generates an audio signal.

[0211] The second control unit 703 is configured to control the loudspeaker corresponding to the first sound area to adjust the noise reduction mode according to the audio interaction signal in a case where it is determined that the first object of the first sound area and the second object of the second sound area exist between the audio interaction signal.

[0212] In some embodiments of the present application, the second control unit 703 is further configured to collect a first audio signal of the first object and a second audio signal of the second object; and determine that the first object of the first sound area and the second object of the second sound area exist between the audio interaction signal in a case where a sound source direction of the first audio signal deviates from a spatial coordinate azimuth angle of the second sound area by less than a preset angle, and a sound source direction of the second audio signal deviates from a spatial coordinate azimuth angle of the first sound area by less than a preset angle.

[0213] In some embodiments of the present application, the second control unit 703 is further configured to control the loudspeaker corresponding to the first sound area to close the noise reduction mode in a case where it is determined that the audio interaction signal satisfies a first condition; and control the loudspeaker corresponding to the first sound area to reduce a noise reduction intensity in the noise reduction mode in a case where it is determined that the audio interaction signal satisfies a second condition; wherein the first condition is used to indicate that the first object and the second object are in a continuous interaction scenario; and the second condition is used to indicate that the first object and the second object are in a non-continuous interaction scenario.

[0214] In some embodiments of the present application, the second control unit 703 is further configured to control the loudspeaker corresponding to the first sound area to close the noise reduction mode in a case where it is determined that the audio interaction signal satisfies a first condition; and control the loudspeaker corresponding to the first sound area to reduce a noise reduction intensity in the noise reduction mode in a case where it is determined that the audio interaction signal satisfies a second condition; wherein the first condition is used to indicate that the first object and the second object are in a continuous interaction scenario; and the second condition is used to indicate that the first object and the second object are in a non-continuous interaction scenario.

[0215] The first determination unit is configured to determine whether the audio interaction signal is detected within a preset time length.

[0216] The third control unit is configured to control the loudspeaker corresponding to the first sound area to open the noise reduction mode in a case where the audio interaction signal is not detected within the preset time length. In some embodiments of the present application, the first condition includes one or more of the following: a time interval between the first audio signal and the second audio signal is less than a first preset interval; a head rotation amplitude of the first object is greater than a first preset amplitude; a head rotation amplitude of the second object is greater than a second preset amplitude; a frequency of the first audio signal is greater than a first preset frequency; and a frequency of the second audio signal is greater than the first preset frequency.

[0217] The second condition includes one or more of the following: a time interval between the first audio signal and the second audio signal is less than a second preset interval; a head rotation amplitude of the first object is greater than a third preset amplitude; a head rotation amplitude of the second object is greater than a fourth preset amplitude; a frequency of the first audio signal is greater than a second preset frequency; and a frequency of the second audio signal is greater than the second preset frequency.

[0218] The second preset interval is greater than the first preset interval; the third preset amplitude is less than the first preset amplitude; the fourth preset amplitude is less than the second preset amplitude; and the first preset frequency is greater than the second preset frequency.

[0219] In some embodiments of the present application, the first control unit 702 is further configured to determine a superimposed audio signal, wherein the superimposed audio signal is obtained by superimposing the audio signal of at least one second sound area other than the first sound area and the external environmental noise; control the loudspeaker corresponding to the first sound area to output a reverse sound wave signal, so that the reverse sound wave signal and the superimposed audio signal interfere destructively in the first sound area; wherein the reverse sound wave signal is a signal with equal amplitude and opposite phase to the superimposed audio signal.

[0220] In some embodiments of the present application, the audio processing device 700 of the vehicle further comprises an acquisition unit, a receiving unit and a sending unit, wherein:

[0221] The acquisition unit is configured to acquire the running state of the loudspeaker corresponding to the first sound area.

[0222] The receiving unit is configured to receive a detection signal emitted by the loudspeaker when the running state of the loudspeaker is a working state; the detection signal represents the audio playing state of the loudspeaker.

[0223] The sending unit is configured to send a switching signal to the loudspeaker if the detection signal indicates that the number of loudspeakers that have failed is greater than a preset number threshold; the switching signal is used to instruct the loudspeaker to be switched to a backup loudspeaker.

[0224] It should be noted that, in the embodiments of the present application, if the above-mentioned method is realized in the form of a software function module and sold or used as an independent product, it can also be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the embodiments of the present application can be embodied in the form of a software product, which is stored in a storage medium and includes a plurality of instructions for causing an electronic device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods of the embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory, a magnetic disk or an optical disk, and various storage media that can store program codes. Thus, the embodiments of the present application are not limited to any specific hardware and software combination.

[0225] The present application also provides an electronic device comprising a memory and a processor, wherein the memory stores a computer program capable of running on the processor, and the processor executes the computer program to implement the above-mentioned method.

[0226] The application further provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the method.

[0227] The application further provides a computer program product, which comprises a computer program or instructions, and the computer program or instructions are executed by a processor to implement some or all steps of the method. The computer program product can be implemented by hardware, software or a combination thereof. In an optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (SDK) or the like.

[0228] It should be noted that, Figure 8 A hardware entity schematic diagram of an electronic device provided by the application is shown in FIG. 8, which comprises a processor 801, a communication interface 802 and a memory 803, wherein: Figure 8 The processor 801 generally controls the overall operation of the electronic device 800.

[0229] The processor 801 generally controls the overall operation of the electronic device 800.

[0230] The communication interface 802 can enable the electronic device 800 to communicate with other terminals or servers through a network.

[0231] The memory 803 is configured to store instructions and applications executable by the processor 801, and can also cache data to be processed by the processor 801 and modules in the electronic device 800 (for example, image data, audio data, voice communication data and video communication data), which can be implemented by FLASH or RAM. The processor 801, the communication interface 802 and the memory 803 can transmit data through a bus 804.

[0232] Here, the electronic device can be a car machine in a vehicle.

[0233] It should be noted that: the description of the above storage medium and device embodiments is similar to the description of the above method embodiments, and has similar beneficial effects to the method embodiments. For technical details not disclosed in the storage medium and device embodiments of the application, please refer to the description of the method embodiments of the application.

[0234] It should be understood that every feature and combination of features that is described above in relation to one embodiment is applicable to at least one other embodiment, unless specifically stated otherwise. It should also be understood that every embodiment described above can be combined with any other embodiment unless specifically stated otherwise.

[0235] It should be noted that, as used in this application, the terms "comprises" or "comprising," or the like are used in the sense of "including" and not of "consisting only of," such that the process, method, article, or apparatus that includes elements in addition to those listed after such a complementing term in the claim are still within the scope of that claim. Also, the term "another" is used in the sense of "additional to or different from" rather than "one or more than one."

[0236] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The above-described device embodiments are merely illustrative. For example, the division of the above-described units is merely a logical function division, and actual implementation can have another division manner. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the various components shown or discussed can be indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other form.

[0237] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; they can be located in one place, or distributed on multiple network units; and some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0238] In addition, each functional unit in each embodiment of the present application can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be realized in the form of hardware or in the form of hardware plus software functional unit.

[0239] Those skilled in the art can understand that all or part of the steps of the above-mentioned method embodiments can be completed by program instruction related hardware, and the foregoing program can be stored in a computer readable storage medium, and the program executes the steps of the above-mentioned method embodiments when executed; and the foregoing storage medium includes various storage media that can store program codes, such as mobile storage devices, read-only memories, magnetic discs or optical discs.

[0240] Alternatively, the integrated units of the present application can be stored in a computer readable storage medium if they are realized in the form of software function modules and sold or used as independent products. Based on such understanding, the technical solutions of the present application or the parts that make contributions to the related art can be embodied in the form of software products, and the computer software products are stored in a storage medium, including a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the embodiments of the method of the present application. The foregoing storage medium includes various storage media that can store program codes, such as mobile storage devices, ROMs, magnetic discs or optical discs.

[0241] The above is only an embodiment of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered in the protection scope of the present application.

Claims

1. An audio processing method of a vehicle, characterized by, The cabin inside of the vehicle is divided into at least two independent sound zones, each sound zone comprising at least one loudspeaker, the method comprising: in response to an audio playing request of a first object, determining a first sound zone in which the first object is located; in a case where it is detected that at least one second sound zone outside the first sound zone generates an audio signal, controlling the loudspeaker corresponding to the first sound zone to start a noise reduction mode; in a case where it is determined that there is an audio interaction signal between the first object of the first sound zone and a second object of the second sound zone, controlling the loudspeaker corresponding to the first sound zone to adjust the noise reduction mode according to the audio interaction signal.

2. The method of claim 1, wherein, determining that there is an audio interaction signal between the first object of the first sound zone and the second object of the second sound zone comprises: collecting a first audio signal of the first object and a second audio signal of the second object; if a sound source direction of the first audio signal deviates from an azimuth of a spatial coordinate of the second sound zone by less than a preset angle, and a sound source direction of the second audio signal deviates from an azimuth of a spatial coordinate of the first sound zone by less than a preset angle, it is determined that there is an audio interaction signal between the first object of the first sound zone and the second object of the second sound zone.

3. The method of claim 2, wherein, controlling the loudspeaker corresponding to the first sound zone to adjust the noise reduction mode according to the audio interaction signal comprises: in a case where it is determined that the audio interaction signal satisfies a first condition, controlling the loudspeaker corresponding to the first sound zone to close the noise reduction mode; in a case where it is determined that the audio interaction signal satisfies a second condition, controlling the loudspeaker corresponding to the first sound zone to reduce a noise reduction intensity in the noise reduction mode; wherein the first condition is used to indicate that the first object and the second object are in a continuous interaction scenario; and the second condition is used to indicate that the first object and the second object are in a non-continuous interaction scenario.

4. The method of claim 3, wherein, after controlling the loudspeaker corresponding to the first sound zone to close the noise reduction mode, the method further comprises: determining whether an audio interaction signal is detected within a preset time length; in a case where the audio interaction signal is not detected within the preset time length, controlling the loudspeaker corresponding to the first sound zone to start the noise reduction mode.

5. The method of claim 3, wherein, the first condition comprises one or more of: a time interval between the first audio signal and the second audio signal is less than a first preset interval; a head rotation amplitude of the first object is greater than a first preset amplitude; a head rotation amplitude of the second object is greater than a second preset amplitude; a frequency of the first audio signal is greater than a first preset frequency; a frequency of the second audio signal is greater than a first preset frequency; the second condition comprises one or more of: a time interval between the first audio signal and the second audio signal is less than a second preset interval; a head rotation amplitude of the first object is greater than a third preset amplitude; a head rotation amplitude of the second object is greater than a fourth preset amplitude; a frequency of the first audio signal is greater than a second preset frequency; a frequency of the second audio signal is greater than the second preset frequency; The second preset interval is greater than the first preset interval; the third preset amplitude is less than the first preset amplitude; the fourth preset amplitude is less than the second preset amplitude; and the first preset frequency is greater than the second preset frequency.

6. The method according to any one of claims 1 to 5, characterized in that, The control of the first sound area corresponding to the speaker to start the noise reduction mode comprises: Determine the superposition audio signal; wherein the superposition audio signal is the audio signal of at least one second sound area outside the first sound area and the external environmental noise superposition obtained; Control the first sound area corresponding to the speaker to output the reverse sound wave signal, so that the reverse sound wave signal and the superposition audio signal in the first sound area occur destructive interference; The reverse sound wave signal is a signal with equal amplitude and opposite phase of the superposition audio signal.

7. The method according to any one of claims 1 to 5, characterized in that, The method further comprises: Obtaining the running state of the speaker corresponding to the first sound area; In the case where the running state of the speaker is the working state, receiving the detection signal emitted by the speaker; the detection signal represents the audio playing state of the speaker; If the detection signal indicates that the speaker fails, and the number of failed speakers is greater than a preset number threshold, a switching signal is sent to the speaker; the switching signal is used to indicate that the speaker is switched to a standby speaker.

8. An audio processing apparatus of a vehicle, characterized by comprising: The device comprises: A determination unit configured to determine a first sound area in which a first object is located in response to an audio playing request of the first object; A first control unit configured to control a speaker corresponding to the first sound area to start a noise reduction mode in the case where it is detected that at least one other sound area generates an audio signal; A second control unit configured to control the speaker corresponding to the first sound area to adjust the noise reduction mode according to an audio interaction signal in the case where it is determined that there is the audio interaction signal between the first object in the first sound area and a second object in a second sound area.

9. An electronic device, comprising: A processor and a memory, the memory stores a computer program that can run on the processor, and the processor implements the steps in the method of any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that, A computer program is stored thereon, and the computer program is executed by a processor to implement the steps in the method of any one of claims 1 to 7.

Images