Audio processing method, device, equipment and vehicle

Abstract

The invention provides an audio processing method, and relates to the technical field of artificial intelligence, in particular to the technical field of voice. According to the specific implementation scheme, in response to first audio data received from a mobile terminal, an audio collection device is controlled to collect second audio data in the environment where the vehicle-mounted terminal is located, and the second audio data comprises audio data generated by at least one target object in the environment where the vehicle-mounted terminal is located based on the first audio data; performing sound mixing processing on the first audio data and the second audio data to obtain sound-mixed audio data; and transmitting the mixed audio data to the vehicle-mounted terminal so as to drive a playing device of the vehicle-mounted terminal to play the mixed audio data. The invention further provides an audio processing device, equipment and a vehicle.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to the field of speech technology. More specifically, the present disclosure provides an audio processing method, device, equipment and vehicle. Background Art

[0002] With the continuous development of the Internet, human-computer interaction is becoming more and more intelligent, and voice technology is becoming more and more advanced. For example, smart speakers, smart car systems, smart phones, smart home systems, etc. all need to transmit and process voice information when interacting with users. Summary of the invention

[0003] The present disclosure provides an audio processing method, apparatus, device and vehicle.

[0004] According to one aspect of the present disclosure, an audio processing method is provided, comprising: in response to receiving first audio data from a mobile terminal, controlling an audio acquisition device to acquire second audio data in an environment in which a vehicle-mounted terminal is located, wherein the second audio data includes audio data generated by at least one target object in the environment in which the vehicle-mounted terminal is located based on the first audio data; performing mixing processing on the first audio data and the second audio data to obtain mixed audio data; and transmitting the mixed audio data to the vehicle-mounted terminal so as to drive a playback device of the vehicle-mounted terminal to play the mixed audio data.

[0005] According to another aspect of the present disclosure, an audio processing device is provided, including: an audio acquisition module, used to control an audio acquisition device to collect second audio data in an environment where a vehicle-mounted terminal is located in response to receiving first audio data from a mobile terminal, wherein the second audio data includes audio data generated by at least one target object in the environment where the vehicle-mounted terminal is located based on the first audio data; a mixing processing module, used to mix the first audio data and the second audio data to obtain mixed audio data; and an audio transmission module, used to transmit the mixed audio data to the vehicle-mounted terminal so as to drive a playback device of the vehicle-mounted terminal to play the mixed audio data.

[0006] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method provided according to the present disclosure.

[0007] According to another aspect of the present disclosure, an audio processing device is provided, including: an audio acquisition apparatus; and the electronic device provided according to the present disclosure.

[0008] According to another aspect of the present disclosure, a vehicle is provided, including: an in-vehicle terminal; and the audio processing device provided according to the present disclosure.

[0009] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided. The computer instructions are used to cause a computer to execute the method provided according to the present disclosure.

[0010] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, wherein when the computer program is executed by a processor, the method provided according to the present disclosure is implemented.

[0011] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0013] Figure 1 A diagram schematically shows an application scenario of an audio processing method, apparatus, device, and vehicle according to an embodiment of the present disclosure;

[0014] Figure 2 The flowchart of the audio processing method according to the embodiment of the present disclosure is schematically shown;

[0015] Figure 3 The schematic diagram shows a principle diagram of an audio processing method according to an embodiment of the present disclosure;

[0016] Figure 4 A flowchart of a mixing process according to an embodiment of the present disclosure is schematically shown;

[0017] Figure 5 Schematically shows a flow chart of a mixing process according to another embodiment of the present disclosure;

[0018] Figure 6 A block diagram of an audio processing device according to an embodiment of the present disclosure is schematically shown;

[0019] Figure 7 A block diagram schematically shows an electronic device to which the audio processing method according to an embodiment of the present disclosure can be applied;

[0020] Figure 8 A block diagram of an audio processing device according to an embodiment of the present disclosure is schematically shown;

[0021] Fig. 9A diagram schematically shows the relationship between the audio processing device according to an embodiment of the present disclosure and the mobile terminal and the vehicle-mounted terminal respectively; and

[0022] Fig.10 A block diagram of a vehicle according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0023] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0024] With the gradual development of smart phones, simply listening to songs can no longer meet people's needs for karaoke. In order to enjoy the fun of karaoke, various karaoke software has been developed on smart phones. However, no matter which karaoke software, it cannot achieve the effect of karaoke in KTV. This is because: first, the current karaoke software cannot achieve the effect of mixing, that is, only the sound of music can be heard from the headphones, while in actual KTV, the singer can hear his own voice from the speakers; in addition, in karaoke software, usually only headphone microphones or wired microphones are used for singing, and wireless microphones cannot be used, which greatly reduces the effect and ease of use of karaoke; thirdly, the sound of the mobile phone is usually only fed back through the headphones, or connected to ordinary computer speakers for playback, and cannot be connected to professional speakers for playback, which also greatly reduces the effect of karaoke; and limited by the headphones, based on the in-car scenario, more users in the car cannot share karaoke.

[0025] In view of this, the present disclosure provides an audio processing method, comprising: in response to receiving first audio data from a mobile terminal, controlling an audio acquisition device to acquire second audio data in an environment where a vehicle-mounted terminal is located, wherein the second audio data includes audio data generated by at least one target object in the environment where the vehicle-mounted terminal is located based on the first audio data; performing mixing processing on the first audio data and the second audio data to obtain mixed audio data; and transmitting the mixed audio data to the vehicle-mounted terminal so as to drive a playback device of the vehicle-mounted terminal to play the mixed audio data.

[0026] Compared with the current mobile phone karaoke method, the audio processing method provided by the present disclosure is no longer limited to wired headphones. Users can use mobile terminals to interact with vehicle terminals, so as to facilitate users to sing any audio data in the mobile terminal and improve the user's personalized audio experience. Since there can be multiple target objects in the environment where the vehicle terminal is located, it can not only realize individual solo, but also realize multi-person chorus and multi-person sharing. In addition, the existing hardware resources of the vehicle terminal can be used to realize high-quality karaoke in the car, which improves the car experience. Furthermore, a KTV system can be jointly established with a mobile terminal and a vehicle terminal, which can be flexibly used in various occasions such as home, office, outdoor, and vehicle, which broadens the use scenarios of karaoke applications and improves the user experience.

[0027] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision, disclosure and application of user personal information involved comply with the provisions of relevant laws and regulations, take necessary confidentiality measures, and do not violate public order and good morals.

[0028] In the technical solution of the present disclosure, the user's authorization or consent is obtained before obtaining or collecting the user's personal information.

[0029] Figure 1 The following schematically shows an application scenario diagram of the audio processing method, apparatus, device and vehicle according to an embodiment of the present disclosure. It should be noted that: Figure 1 What is shown are merely examples of application scenarios to which the embodiments of the present disclosure can be applied, in order to help those skilled in the art understand the technical content of the present disclosure, but it does not mean that the embodiments of the present disclosure cannot be used in other devices, systems, environments or scenarios.

[0030] like Figure 1 As shown, the application scenario 100 may include a mobile terminal 101, an audio processing device 102 and a vehicle terminal 103. The audio processing device 102 is wirelessly connected to the mobile terminal 101 via a smart interconnection protocol, and the audio processing device 102 is wiredly connected to the vehicle terminal 103.

[0031] The mobile terminal 101 may be any electronic device that can provide audio data, including but not limited to a smart phone, a tablet computer, a laptop computer, a smart watch, a smart bracelet, etc.

[0032] The smart interconnection protocol may be a proprietary protocol of a smart interconnection product. A user may use the mobile terminal 101 to interact with the audio processing device 102 through the smart interconnection protocol to send the first audio data to the audio processing device 102. The mobile terminal 101 and the vehicle terminal 103 may both be equipped with smart interconnection products to establish wireless communication.

[0033] The vehicle terminal 103 may be a mobile device used in a vehicle, and may support a variety of functions and applications, including but not limited to a vehicle computer, a vehicle computer, and the like.

[0034] According to an embodiment of the present disclosure, first, in response to receiving first audio data from the mobile terminal 101, the audio processing device 102 controls the audio acquisition device to acquire second audio data in the environment where the vehicle terminal 103 is located, wherein the second audio data includes audio data generated by at least one target object in the environment where the vehicle terminal is located based on the first audio data.

[0035] Secondly, the audio processing device 102 mixes the first audio data and the second audio data to obtain mixed audio data. Then, the audio processing device 102 transmits the mixed audio data to the vehicle terminal 103 so as to drive the playback device of the vehicle terminal 103 to play the mixed audio data.

[0036] Figure 2 The flowchart of the audio processing method according to the embodiment of the present disclosure is schematically shown.

[0037] like Figure 2 As shown, the audio processing method 200 may include operations S210 to S230.

[0038] In operation S210, in response to receiving first audio data from the mobile terminal, the audio collection device is controlled to collect second audio data in the environment where the vehicle terminal is located, wherein the second audio data includes audio data generated by at least one target object in the environment where the vehicle terminal is located based on the first audio data.

[0039] In operation S220, the first audio data and the second audio data are mixed to obtain mixed audio data.

[0040] In operation S230, the mixed audio data is transmitted to the vehicle terminal so as to drive a playback device of the vehicle terminal to play the mixed audio data.

[0041] Through the audio processing method of the embodiment of the present disclosure, the first audio data from the mobile terminal and the second audio data collected in the environment where the vehicle terminal is located are mixed, and the mixed audio data obtained by the processing is played by the playback device of the vehicle terminal. The user can use the mobile terminal to interact with the vehicle terminal, so as to facilitate the user to sing any audio data in the mobile terminal and improve the user's personalized audio experience. Since there can be multiple target objects in the environment where the vehicle terminal is located, it can not only realize individual solo, but also realize multi-person chorus and multi-person sharing. In addition, the existing hardware resources of the vehicle terminal can be used to realize high-quality karaoke in the car, which improves the car experience. Furthermore, a KTV system can be jointly established with a mobile terminal and a vehicle terminal, which can be flexibly used in various occasions such as home, office, outdoor, and vehicle, broadening the use scenarios of karaoke applications and improving user experience.

[0042] For example, the mobile terminal may be a smart phone, a tablet computer, a laptop computer, a smart watch, a smart bracelet, etc. The vehicle-mounted terminal may be a vehicle computer, a vehicle-mounted computer, etc. The vehicle-mounted terminal may be located in a vehicle. The audio acquisition device may be a handheld microphone, a wireless microphone, etc.

[0043] For another example, the first audio data may be a complete song, or the accompaniment of a complete song. The target object may be a passenger in a car. The second audio data may be generated by one or more passengers in the car interacting with the first audio data.

[0044] More specifically, the first audio data may be an accompaniment sound, and the second audio data may be a singing sound generated by the passengers in the car interacting with the accompaniment sound.

[0045] In an embodiment of the present disclosure, in response to receiving the first audio data from the mobile terminal, the method may include but is not limited to: obtaining the first audio data sent by the mobile terminal associated with a wired or wireless communication technology, wherein the wireless communication technology includes a WiFi network, Bluetooth, a hotspot data connection, etc.

[0046] In some embodiments, the audio processing method further includes: when the first audio data comes from an application layer of the mobile terminal, controlling a framework layer of the mobile terminal to send the first audio data through a smart interconnection protocol.

[0047] For example, the mobile terminal may run a corresponding application program to trigger an associated operation through the application program.

[0048] For example, the mobile terminal may be a smart phone, and an audio application (APP) such as a karaoke application is installed in the application layer of the smart phone. The first audio data may be an accompaniment sound in the audio application. The user may open the audio application and play the accompaniment sound. When the accompaniment sound is played, the framework layer of the smart phone is controlled to transmit the accompaniment sound through the smart interconnection protocol.

[0049] The smart interconnection protocol may be a private protocol of the smart interconnection product. The mobile terminal and the vehicle terminal may be installed with smart interconnection products that match each other so as to establish wireless communication.

[0050] For a smart interconnected product A in the hand-car interconnection scenario, taking the mobile terminal as a smart phone and the vehicle terminal as a car computer as an example, the mobile version A of the smart interconnected product A can be installed on the smart phone. 1 , and the vehicle version A of the smart interconnection product A is installed on the vehicle 2 , in order to communicate with each other wirelessly.

[0051] Compared with the existing cable connection method for mobile phone karaoke, the present invention can transmit audio data wirelessly, which can not only accurately and reliably transmit accompaniment sounds, but also does not require cable connection, avoiding the inconvenience of storing and carrying headphones, broadening the use scenarios of karaoke applications and improving the user experience.

[0052] Figure 3 The schematic diagram shows a principle diagram of an audio processing method according to an embodiment of the present disclosure.

[0053] like Figure 3 As shown, the working principle of the audio processing method of the embodiment of the present disclosure may be, for example: first, the user uses the mobile terminal 310 and browses an application 311 located in the application layer, and the application 311 may be an audio application, such as a karaoke application. When the first audio data in the application 311 is run, the audio manager (AudioFlinger) 312 of the framework layer (framework) of the mobile terminal 310 may be controlled to transmit the first audio data to the audio processing device 320 through the smart interconnection protocol 313.

[0054] Secondly, when the audio processing device 320 receives the first audio data from the mobile terminal 310, it controls the audio collection device 340 to collect the second audio data in the environment of the vehicle terminal 330. Then, the mixing processing module 322 of the audio processing device 320 mixes the first audio data and the second audio data to obtain mixed audio data. Then, the mixed audio data is transmitted to the vehicle terminal 330 so as to drive the playback device 331 of the vehicle terminal 330 to play the mixed audio data.

[0055] It should be noted that currently, mobile karaoke is mainly recorded with wired or wireless headphones. The HIFI chip added to the hardware abstraction layer (HAL) of the mobile phone synthesizes the recorded audio data and the accompaniment audio data transmitted by the audio manager (AudioFlinger) and outputs them to the headphones to achieve the ear return effect. Therefore, the existing mobile karaoke is limited to wired headphones and cannot be shared by multiple people.

[0056] Compared with the current mobile phone karaoke method, the audio processing method provided by the present disclosure is no longer limited to wired headphones. Users can use mobile terminals to interact with vehicle terminals, so as to facilitate users to sing any audio data in the mobile terminal and improve the user's personalized audio experience. Since there can be multiple target objects in the environment of the vehicle terminal, it can not only realize individual solo singing, but also realize multi-person chorus and multi-person sharing.

[0057] It is worth noting that the audio acquisition device 340 is another device parallel to the audio processing device 320 , and is not integrated or included in the audio processing device 320 .

[0058] Figure 4 The flowchart of the audio mixing process according to one embodiment of the present disclosure is schematically shown.

[0059] like Figure 4 As shown, in some embodiments, mixing the first audio data and the second audio data may include operations S431 to S433.

[0060] In operation S431, a vocal part and an accompaniment part in the first audio data are identified.

[0061] In operation S432, the vocal part is eliminated and the accompaniment part is retained to obtain processed first audio data.

[0062] In operation S433, the processed first audio data and the second audio data are mixed to obtain mixed audio data.

[0063] For example, when the first audio data is a complete song including an accompaniment part and a vocal part, in order to improve the karaoke effect, vocal suppression can be performed to eliminate or weaken the vocal part while retaining the accompaniment part.

[0064] Through the embodiments of the present disclosure, it is convenient for users to sing any audio data in the mobile terminal, ensuring that the first audio data does not affect the vocals of the second audio data, so that the user's vocals and accompaniment are harmonious when singing, thereby improving the user's personalized audio usage experience.

[0065] Figure 5 The flowchart of the mixing process according to another embodiment of the present disclosure is schematically shown.

[0066] like Figure 5 As shown, in some embodiments, mixing the first audio data and the second audio data may include operations S531 to S532.

[0067] In operation S531, at least one of volume adjustment, noise reduction, echo cancellation, and voice localization processing is performed on the second audio data to obtain processed second audio data.

[0068] In operation S532, the processed second audio data and the first audio data are mixed to obtain mixed audio data.

[0069] Considering that the vehicle terminal may be in a mobile and closed state, the second audio data collected by the audio collection device in the environment of the vehicle terminal may contain noise, murmurs, echoes, etc. These sounds will overlap the real voices of the passengers in the car and interfere with the audio collection effect. Through the embodiments of the present disclosure, the influence of volume, noise, echo and the distance of the target object on the audio quality can be avoided, the audio quality of the second audio data can be improved, the user experience can be enhanced, and high-quality karaoke can be achieved in the car.

[0070] It should be noted that operations S431 to S433 and operations S531 to S532 may be performed in parallel. However, the embodiments of the present disclosure are not limited thereto, and the two groups of operations may also be performed in other orders, for example, operations S431 to S433 may be performed first, and then operations S531 to S532 may be performed, or operations S531 to S532 may be performed first, and then operations S431 to S433 may be performed.

[0071] In some embodiments, mixing the first audio data and the second audio data includes: adjusting the audio format of at least one of the first audio data and the second audio data so that the audio formats of the first audio data and the second audio data are the same, wherein the audio format includes sampling frequency, bit depth, and number of channels. Through the embodiments of the present disclosure, since the two channels of audio data have different sources, the formats of the two channels of audio data are controlled to be the same during mixing, and slight differences between the two channels of audio data can be adjusted to improve the probability of audio synchronization and the effect of mixing.

[0072] In some embodiments, mixing the first audio data and the second audio data further includes:

[0073] In a case where the audio formats of the first audio data and the second audio data are the same, assigning corresponding weights to the first audio data and the second audio data respectively based on a predetermined mixing rule, wherein the predetermined mixing rule includes a signal-to-noise ratio optimization principle; and

[0074] The first audio data and the second audio data are mixed frame by frame using the weights to obtain mixed audio data.

[0075] It is understandable that in the field of voice technology, the optimal principle of signal-to-noise ratio (SNR or S / N) means that the higher the ratio of signal to noise in an electronic device or electronic system, the better. Generally speaking, the larger the signal-to-noise ratio, the smaller the noise mixed in the signal, and the higher the quality of sound playback, otherwise the opposite. In other embodiments, a predetermined mixing rule can be determined according to a specific usage scenario to mix two channels of audio data, which is not limited by the present disclosure.

[0076] For example, mixing can be done using the following formula:

[0077] (1)

[0078] In formula (1), i is the ordinal number of the audio frame, 0≤i≤length, and i is an integer, and length is the total number of frames; and The i-th audio frame is the first audio data and the second audio data respectively; and are weights corresponding to the first audio data and the second audio data respectively; is the i-th audio frame of the mixed audio data.

[0079] Through the embodiments of the present disclosure, different weights can be set for vocals and accompaniment based on a predetermined mixing rule. If the vocals are expected to be more prominent, a higher weight can be given to the vocals. Otherwise, if the accompaniment is expected to be more prominent, a higher weight can be given to the accompaniment.

[0080] In some embodiments, the mixed audio data includes multiple audio frames; mixing the first audio data and the second audio data also includes: when it is determined that any audio frame of the mixed audio data exceeds a predetermined audio frame length range, truncating the audio frame to a limit value of the predetermined audio frame length range.

[0081] For example, the predetermined audio frame length range is [-32768, 32767], where -32768 and 32767 are the lower limit and upper limit of the predetermined audio frame length range, respectively, and both the lower limit and the upper limit are limit values. When the frame length of any audio frame of the mixed audio data is greater than 32767, the frame length of the audio frame can be truncated to 32767; when the frame length of any audio frame of the mixed audio data is less than -32768, the frame length of the audio frame can be truncated to -32768.

[0082] Through the embodiments of the present disclosure, a truncation strategy can be adopted during mixing to prevent data overflow and ensure the accuracy and reliability of mixing processing.

[0083] At this point, the mixed audio data obtained after the mixing process is played by the playback device of the vehicle terminal. The playback device can be a vehicle audio system, a vehicle speaker, etc. Through the embodiments of the present disclosure, the existing hardware resources of the vehicle terminal can be used to achieve high-quality karaoke in the car, thereby improving the car-using experience. Furthermore, a KTV system can be established together with a mobile terminal and a vehicle terminal, which can be flexibly used in various occasions such as home, office, outdoors, and in the car, broadening the use scenarios of karaoke applications and improving the user experience.

[0084] Figure 6 The block diagram schematically shows an audio processing device according to an embodiment of the present disclosure.

[0085] like Figure 6 As shown, the audio processing device 600 may include an audio acquisition module 610 , a mixing processing module 620 and an audio transmission module 630 .

[0086] The audio acquisition module 610 is used to control the audio acquisition device to collect second audio data in the environment where the vehicle terminal is located in response to receiving the first audio data from the mobile terminal, wherein the second audio data includes audio data generated by at least one target object in the environment where the vehicle terminal is located based on the first audio data.

[0087] The audio mixing processing module 620 is used to perform audio mixing processing on the first audio data and the second audio data to obtain mixed audio data.

[0088] The audio transmission module 630 is used to transmit the mixed audio data to the vehicle terminal so as to drive the playback device of the vehicle terminal to play the mixed audio data.

[0089] According to an embodiment of the present disclosure, the audio acquisition module is also used to: when the first audio data comes from the application layer of the mobile terminal, control the framework layer of the mobile terminal to send the first audio data through the intelligent interconnection protocol.

[0090] According to an embodiment of the present disclosure, the mixing processing module is also used to: identify the vocal part and the accompaniment part in the first audio data; eliminate the vocal part and retain the accompaniment part to obtain the processed first audio data; and mix the processed first audio data and the second audio data to obtain mixed audio data.

[0091] According to an embodiment of the present disclosure, the mixing processing module is also used to: adjust the volume, reduce noise, cancel echoes and perform voice localization processing on the second audio data to obtain processed second audio data; mix the processed second audio data and the first audio data to obtain mixed audio data.

[0092] According to an embodiment of the present disclosure, the mixing processing module is also used to: adjust the audio format of at least one of the first audio data and the second audio data so that the audio formats of the first audio data and the second audio data are the same, wherein the audio format includes sampling frequency, bit depth and number of channels.

[0093] According to an embodiment of the present disclosure, the mixing processing module is also used for: when the audio formats of the first audio data and the second audio data are the same, assigning corresponding weights to the first audio data and the second audio data respectively based on predetermined mixing rules, wherein the predetermined mixing rules include an optimal signal-to-noise ratio principle; and using the weights, performing mixing processing on the first audio data and the second audio data frame by frame to obtain mixed audio data.

[0094] According to an embodiment of the present disclosure, the mixed audio data includes multiple audio frames; the mixing processing module is also used to: when it is determined that any audio frame of the mixed audio data exceeds a predetermined audio frame length range, truncate the audio frame to a limit value of the predetermined audio frame length range.

[0095] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, an audio processing device, a vehicle, a readable storage medium, and a computer program product.

[0096] Figure 7 A block diagram of an electronic device to which an audio processing method can be applied according to an embodiment of the present disclosure is schematically shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0097] like Figure 7 As shown, the electronic device 700 of this embodiment includes a computing unit 701, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 702 or a computer program loaded from a storage unit 708 to a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the electronic device 700 can also be stored. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0098] Multiple components in the electronic device 700 are connected to the I / O interface 705, including: an input unit 706, such as a keyboard, a mouse, etc.; an output unit 707, such as various types of displays, speakers, etc.; a storage unit 708, such as a disk, an optical disk, etc.; and a communication unit 709, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 709 allows the electronic device 700 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0099] The computing unit 701 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSP), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 701 performs the various methods and processes described above, such as a text processing method and / or a deployment method of a deep learning framework. For example, in some embodiments, the text processing method and / or the deployment method of a deep learning framework may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 708. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 700 via the ROM 702 and / or the communication unit 709. When the computer program is loaded into RAM 703 and executed by computing unit 701, one or more steps of the text processing method and / or the method for deploying a deep learning framework described above may be performed. Alternatively, in other embodiments, computing unit 701 may be configured to execute the text processing method and / or the method for deploying a deep learning framework in any other appropriate manner (e.g., by means of firmware).

[0100] Figure 8 The block diagram schematically shows an audio processing device according to an embodiment of the present disclosure.

[0101] like Figure 8 As shown, the audio processing device 800 of this embodiment includes an audio collection device 810 and an electronic device 820. The electronic device 820 may be an electronic device provided according to the present disclosure.

[0102] In some embodiments, the audio acquisition device is integrated with the electronic device. For example, the audio acquisition device can be a handheld microphone, and the electronic device is integrated inside, so that the audio processing device can be in the shape of a microphone. The handheld microphone is not restricted by the position and can be better used for multiple users to share karaoke in the car, improving the user's car experience. Since the audio processing device itself has an automatic sound pickup function, the device is small and portable, and does not require a lot of improvements to existing mobile phones and car computers, and has a wide range of applications.

[0103] Fig. 9 The diagram schematically shows the relationship between the audio processing device according to the embodiment of the present disclosure and the mobile terminal and the vehicle-mounted terminal respectively.

[0104] like Fig. 9 As shown, in some embodiments, the audio processing device 920 is wirelessly connected to the mobile terminal 910 via the intelligent interconnection protocol, and is wiredly connected to the vehicle terminal 930. The audio processing device 920 may be an audio processing device provided according to the present disclosure.

[0105] More specifically, the audio processing device can be connected to the vehicle terminal via a universal serial bus (USB). Considering that existing vehicle computers are all equipped with USB interfaces, the present disclosure does not require a lot of improvements to the vehicle computer, and is extremely friendly to a large number of existing vehicle users.

[0106] Fig.10 A block diagram of a vehicle according to an embodiment of the present disclosure is schematically shown.

[0107] like Fig.10 As shown, the vehicle 1000 of this embodiment includes a vehicle-mounted terminal 1010 and an audio processing device 1020. The audio processing device 1020 may be an audio processing device provided according to the present disclosure.

[0108] Various embodiments of the systems and techniques described above herein may be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard parts (ASSPs), system on chip systems (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0109] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0110] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories, read-only memories, erasable programmable read-only memories (EPROM) or flash memories, optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0111] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a cathode ray tube (CRT) display or a liquid crystal display (LCD)) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0112] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a Local Area Network (LAN), a Wide Area Network (WAN), and the Internet.

[0113] A computer system may include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship to each other.

[0114] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0115] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. An audio processing method, comprising: In response to receiving the first audio data from the mobile terminal, controlling the audio collection device to collect second audio data in the environment where the vehicle-mounted terminal is located, wherein the second audio data includes audio data generated by at least one target object in the environment where the vehicle-mounted terminal is located based on the first audio data; Mixing the first audio data and the second audio data to obtain mixed audio data; and The mixed audio data is transmitted to the vehicle-mounted terminal so as to drive a playback device of the vehicle-mounted terminal to play the mixed audio data.

2. The method according to claim 1, further comprising: In the case where the first audio data comes from the application layer of the mobile terminal, the framework layer of the mobile terminal is controlled to send the first audio data through the intelligent interconnection protocol.

3. The method according to claim 1, wherein: The first audio data and the second audio data are mixed, further comprising: Identifying a vocal part and an accompaniment part in the first audio data; Eliminating the vocal part and retaining the accompaniment part to obtain processed first audio data; and The processed first audio data and the second audio data are mixed to obtain the mixed audio data.

4. The method according to claim 1, wherein: The first audio data and the second audio data are mixed, further comprising: Perform at least one of volume adjustment, noise reduction, echo cancellation, and voice localization processing on the second audio data to obtain processed second audio data; The processed second audio data and the first audio data are mixed to obtain the mixed audio data.

5. The method according to claim 1, wherein: The first audio data and the second audio data are mixed, comprising: An audio format of at least one of the first audio data and the second audio data is adjusted so that the audio formats of the first audio data and the second audio data are the same, wherein the audio format includes a sampling frequency, a bit depth, and a number of channels.

6. The method according to claim 1 or 5, wherein: The first audio data and the second audio data are mixed, further comprising: When the first audio data and the second audio data have the same audio format, assigning corresponding weights to the first audio data and the second audio data based on a predetermined mixing rule, wherein the predetermined mixing rule includes a signal-to-noise ratio optimization principle; and The first audio data and the second audio data are mixed frame by frame using the weight to obtain the mixed audio data.

7. The method according to claim 1, wherein: The mixed audio data includes a plurality of audio frames; and the first audio data and the second audio data are mixed, further comprising: When it is determined that any audio frame of the mixed audio data exceeds a predetermined audio frame length range, the audio frame is truncated to a limit value of the predetermined audio frame length range.

8. An audio processing device, comprising: an audio acquisition module, configured to control the audio acquisition device to acquire second audio data in the environment where the vehicle-mounted terminal is located in response to receiving the first audio data from the mobile terminal, wherein the second audio data includes audio data generated by at least one target object in the environment where the vehicle-mounted terminal is located based on the first audio data; a mixing processing module, configured to perform mixing processing on the first audio data and the second audio data to obtain mixed audio data; and The audio transmission module is used to transmit the mixed audio data to the vehicle-mounted terminal so as to drive the playback device of the vehicle-mounted terminal to play the mixed audio data.

9. The audio processing device according to claim 8, wherein: The audio acquisition module is also used for: In the case where the first audio data comes from the application layer of the mobile terminal, the framework layer of the mobile terminal is controlled to send the first audio data through the intelligent interconnection protocol.

10. The audio processing device according to claim 8, wherein: The mixing processing module is also used for: Identifying a vocal part and an accompaniment part in the first audio data; Eliminating the vocal part and retaining the accompaniment part to obtain processed first audio data; as well as The processed first audio data and the second audio data are mixed to obtain the mixed audio data.

11. The audio processing device according to claim 8, wherein: The mixing processing module is also used for: Perform at least one of volume adjustment, noise reduction, echo cancellation, and voice localization processing on the second audio data to obtain processed second audio data; The processed second audio data and the first audio data are mixed to obtain the mixed audio data.

12. The audio processing device according to claim 8, wherein: The mixing processing module is also used for: An audio format of at least one of the first audio data and the second audio data is adjusted so that the audio formats of the first audio data and the second audio data are the same, wherein the audio format includes a sampling frequency, a bit depth, and a number of channels.

13. The audio processing device according to claim 8 or 12, wherein: The mixing processing module is also used for: When the first audio data and the second audio data have the same audio format, assigning corresponding weights to the first audio data and the second audio data based on a predetermined mixing rule, wherein the predetermined mixing rule includes a signal-to-noise ratio optimization principle; and The first audio data and the second audio data are mixed frame by frame using the weight to obtain the mixed audio data.

14. The audio processing device according to claim 8, wherein: The mixed audio data includes a plurality of audio frames; the mixed audio processing module is further used for: When it is determined that any audio frame of the mixed audio data exceeds a predetermined audio frame length range, the audio frame is truncated to a limit value of the predetermined audio frame length range.

15. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

16. An audio processing device, comprising: Audio acquisition device; as well as The electronic device according to claim 15.

17. The audio processing device according to claim 16, wherein: The audio processing device is wirelessly connected to the mobile terminal via the intelligent interconnection protocol, and is wiredly connected to the vehicle-mounted terminal.

18. The audio processing device according to claim 16, wherein: The audio collection device is integrated with the electronic device.

19. A vehicle comprising: Vehicle terminal; as well as An audio processing device as claimed in any one of claims 16 to 18.

20. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 7.

21. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 7.

Images