Audio data processing method, device and system and electronic equipment

Abstract

The invention provides an audio data processing method, device and system, electronic equipment, a computer readable storage medium and a computer program product, and relates to the technical field of artificial intelligence such as voice recognition, man-machine voice interaction and intelligent cabins. The method can be applied to a first terminal, and comprises the following steps: determining positioning information of a wake-up instruction in first audio data; marking the first audio data to obtain second audio data with time marking information; determining position information of the wake-up instruction in the second audio data according to the positioning information and time stamp information in the second audio data; and outputting the second audio data and the position information. And the success rate of voice wake-up is improved.

Description

Technical Field

[0001] The present disclosure relates to the fields of artificial intelligence technology such as speech recognition, human-computer speech interaction, and smart cockpits, and in particular to a method, device, system, electronic device, computer-readable storage medium, and computer program product for processing audio data. Background Art

[0002] With the continuous development of speech recognition technology, human-computer interaction has been more widely used, such as in-vehicle intelligent voice terminals, smart homes, etc. The in-vehicle terminal can recognize the voice information input by the user, enter the activation state after recognizing the wake-up word, and provide corresponding services according to the recognized voice commands, such as providing users with services such as navigation and multimedia playback while ensuring safe driving. Summary of the invention

[0003] The embodiments of the present disclosure provide a method, device, system, electronic device, computer-readable storage medium, and computer program product for processing audio data, which can improve the success rate of voice wake-up.

[0004] In the first aspect, an embodiment of the present disclosure proposes a method for processing audio data, which is applied to a first terminal, including: determining the positioning information of a wake-up instruction in first audio data; marking the first audio data to obtain second audio data with time marking information; determining the position information of the wake-up instruction in the second audio data based on the positioning information and the time marking information in the second audio data; and outputting the second audio data and the position information.

[0005] On the second aspect, the embodiment of the present disclosure also proposes another method for processing audio data, which is applied to a second terminal, including: receiving second audio data with time stamp information, and location information of a wake-up instruction in the second audio data; performing voice recognition on the second audio data according to the location information; and in response to recognizing the wake-up instruction, triggering entry into a wake-up activation state.

[0006] In the third aspect, an embodiment of the present disclosure proposes a device for processing audio data, including: a positioning information determination module, configured to determine the positioning information of a wake-up instruction in first audio data; a marking processing module, configured to perform marking processing on the first audio data to obtain second audio data with time marking information; a position information determination module, configured to determine the position information of the wake-up instruction in the second audio data based on the positioning information and the time marking information in the second audio data; and a first interface module, configured to output the second audio data and the position information.

[0007] In the fourth aspect, the embodiments of the present disclosure also propose another audio data processing device, including: a second interface module, configured to receive second audio data with time stamp information, and position information of the wake-up instruction in the second audio data; a voice wake-up module, configured to perform voice recognition on the second audio data according to the position information; a wake-up trigger module, configured to trigger entering a wake-up activation state in response to identifying the wake-up instruction; wherein the position information of the wake-up instruction in the second audio data includes the position information of a specific word in the wake-up instruction in the second audio data, and the specific word includes at least one of a starting word, a keyword, and an ending word; the time stamp information includes a timestamp corresponding to an audio frame in the second audio data.

[0008] In a fifth aspect, an embodiment of the present disclosure provides an audio data processing system, the system comprising a first terminal and a second terminal, wherein the first terminal comprises an audio data processing device as described in any implementation method in the third aspect; and the second terminal comprises an audio data processing device as described in any implementation method in the fourth aspect.

[0009] In a sixth aspect, an embodiment of the present disclosure provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that when the at least one processor executes, it is possible to implement the method for processing audio data as described in any implementation method of the first aspect or the second aspect.

[0010] In a seventh aspect, an embodiment of the present disclosure provides a non-transitory computer-readable storage medium storing computer instructions, which are used to enable a computer to implement the method for processing audio data as described in any implementation method of the first aspect or the second aspect when executed.

[0011] In an eighth aspect, an embodiment of the present disclosure provides a computer program product comprising a computer program, which, when executed by a processor, can implement the method for processing audio data as described in any implementation manner in the first aspect or the second aspect.

[0012] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Other features, objects and advantages of the present disclosure will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0014] Figure 1is an exemplary system architecture in which the present disclosure may be applied;

[0015] Figure 2 A flowchart of a method for processing audio data provided by an embodiment of the present disclosure;

[0016] Figure 3 A flowchart of another method for processing audio data provided by an embodiment of the present disclosure;

[0017] Figure 4 A flowchart of a method for processing audio data in an application scenario provided by an embodiment of the present disclosure;

[0018] Figure 5 A structural block diagram of an audio data processing device provided by an embodiment of the present disclosure;

[0019] Figure 6 A structural block diagram of another audio data processing device provided by an embodiment of the present disclosure;

[0020] Figure 7 A structural block diagram of an audio data processing system provided by an embodiment of the present disclosure;

[0021] Figure 8 A schematic diagram of the structure of an electronic device suitable for executing a method for processing audio data provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0022] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description. It should be noted that the embodiments in the present disclosure and the features in the embodiments may be combined with each other without conflict.

[0023] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0024] Figure 1 An exemplary system architecture 100 is shown to which embodiments of the audio data processing method, apparatus, system, electronic device, and computer-readable storage medium of the present disclosure can be applied.

[0025] like Figure 1As shown, the system architecture 100 may include a vehicle 101, an on-board terminal device 102 located in the vehicle 101, a terminal device 103, a network 104 and a server 105. The network 104 is used to provide a medium for a communication link between the on-board terminal device 102, the terminal device 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc. The on-board terminal device 102 or the terminal device 103 may also be connected to the cloud via the network 104, and may receive information from the cloud, or may upload local information data to the cloud. Such information may be, for example, but not limited to, information related to voice input. Among them, the server 105 should not be regarded as one or more specific devices, but may be understood as a server / server group located at a remote end that provides background computing capabilities, or may be a processing device running in a server.

[0026] The user can use the terminal device 103 to interact with the vehicle terminal device 102 through the network 104 to receive or send messages or transmit data, etc. The terminal device 103 and the vehicle terminal device 102 can be installed with various applications for realizing information communication between the two, such as voice interaction applications, data transmission applications, instant messaging applications, etc.

[0027] The terminal device 103 and the vehicle-mounted terminal device 102 can be hardware or software. When the terminal device 103 is hardware, it can be various electronic devices with display screens, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, etc.; when the terminal device 103 is software, it can be installed in the electronic devices listed above, which can be implemented as multiple software or software modules, or as a single software or software module, which is not specifically limited here. The vehicle-mounted terminal device 102 can be, for example, a vehicle-mounted system, or a third-party vehicle-mounted intelligent voice device or other electronic device integrated in the vehicle-mounted system.

[0028] The vehicle-mounted terminal device 102 and the terminal device 103 can provide various services through various built-in applications. Taking a voice interaction application that can provide voice interaction services to users in the vehicle as an example, the vehicle-mounted terminal device 102 can achieve the following effects when running the voice interaction application: first, determine the positioning information of the wake-up instruction in the first audio data; then, mark the first audio data to obtain second audio data with time mark information; then, determine the position information of the wake-up instruction in the second audio data based on the positioning information and the time mark information in the second audio data; finally, output the second audio data and the position information.

[0029] Taking a voice interaction application that can provide voice interaction services to users in a car as an example, the terminal device 103 can achieve the following effects when running the voice interaction application: first, receive the second audio data with time stamp information and the location information of the wake-up instruction in the second audio data; then, perform voice recognition on the second audio data according to the location information; finally, in response to recognizing the wake-up instruction, trigger the entry into the wake-up activation state.

[0030] The audio data processing methods provided in the subsequent embodiments of the present disclosure are generally executed by the vehicle-mounted terminal device 102 and / or the terminal device 103 , and accordingly, the audio data processing device is generally also arranged in the vehicle-mounted terminal device 102 and / or the terminal device 103 .

[0031] It should be understood that Figure 1 The number of vehicle-mounted terminal devices, terminal devices and servers in the embodiment is only for illustration. According to the implementation requirements, any number of vehicle-mounted terminal devices, terminal devices and servers may be provided.

[0032] Please refer to Figure 2 , Figure 2 A process 200 of an embodiment of a method for processing audio data according to the present disclosure is shown, which can be applied to a first terminal. The process 200 may include the following steps:

[0033] Step 201: Determine location information of a wake-up instruction in first audio data.

[0034] This step is intended to be performed by the execution subject of the audio data processing method (for example Figure 1 The vehicle terminal device 102 or other device shown in the figure) determines the positioning information of the wake-up instruction in the first audio data. The first audio data may be a continuous audio stream data obtained by collecting the user's voice input. The positioning information may be the time point information used to determine the position of the wake-up instruction in the first audio data.

[0035] Step 202: Mark the first audio data to obtain second audio data with time mark information.

[0036] In this embodiment, the first audio data is also time-marked so that the second audio data carries time-mark information. The time-mark information may be, for example, a timestamp.

[0037] Step 203: Determine the position information of the wake-up instruction in the second audio data according to the positioning information and the time mark information in the second audio data.

[0038] On the basis of steps 201 and 202, the execution subject determines the position of the wake-up instruction in the second audio data according to the positioning information of the wake-up instruction and the time stamp information in the second audio data. The position of the wake-up instruction in the second audio data may be, for example, a range of audio frames in the second audio data, for example, from a certain frame to a certain frame.

[0039] Step 204: Output the second audio data and the position information.

[0040] In this embodiment, after determining the location information of the wake-up instruction in the second audio data according to the method described in steps 203 and 204, the execution subject may further output the second audio data and the determined location information of the wake-up instruction in the second audio data to the second terminal.

[0041] The audio data processing method provided in this embodiment first determines the location information of the wake-up instruction in the first audio data, then obtains the second audio data by marking the first audio data, so that the second audio data carries time mark information, and then determines the location information of the wake-up instruction in the second audio data according to the location information of the wake-up instruction and the time mark information in the second audio data. In this way, the following problems caused by a certain delay in the transmission of the signal or audio data can be avoided: when determining the location of the wake-up instruction in the audio data according to the received location information and the original audio data, the determined location may deviate from the actual location, so that the wake-up instruction cannot be accurately located, and ultimately the wake-up fails or the activation cannot be triggered, affecting the overall product effect and the user's experience. The audio data processing method provided in this embodiment can more accurately locate the location of the wake-up instruction in the second audio data by determining the location information of the wake-up instruction in the second audio data, thereby improving the success rate and activation rate of voice wake-up and improving the user experience.

[0042] Continue to refer Figure 3 , Figure 3 A process 300 of an embodiment of another method for processing audio data according to the present disclosure is shown, which can be applied to a second terminal. The second terminal can be a terminal with voice wake-up and voice interaction functions, and the second terminal can be hardware or software.

[0043] The process 300 may include the following steps:

[0044] Step 301: Receive second audio data with time stamp information and location information of a wake-up instruction in the second audio data.

[0045] In this embodiment, the position information of the wake-up instruction received by the execution subject (e.g., the second terminal) in the second audio data may include the position information of a specific word in the wake-up instruction in the second audio data, wherein the specific word may include at least one of a starting word, a keyword, and an ending word. The time mark information in the second audio data may include a timestamp corresponding to an audio frame in the second audio data.

[0046] Step 302: Perform speech recognition on the second audio data according to the position information.

[0047] In this embodiment, the execution subject may perform voice recognition on the second audio data from the corresponding position based on the received position information, and determine whether the recognition result includes a predetermined wake-up instruction.

[0048] Step 303: In response to recognizing the wake-up instruction, triggering entry into the wake-up activation state.

[0049] In this embodiment, when the execution subject recognizes the wake-up instruction, it can be triggered to enter the wake-up activation state.

[0050] The method for processing audio data provided by the embodiment of the present disclosure first receives the second audio data and the position information of the wake-up instruction in the second audio data, then performs voice recognition on the second audio data according to the position information of the wake-up instruction in the second audio data, and triggers the entry into the wake-up activation state when the wake-up instruction is recognized. In this state, the corresponding voice control instruction is recognized and executed by collecting the voice signal input by the user, so that the intelligent voice interaction function can be realized.

[0051] The audio data processing method in this embodiment can accurately locate the voice segment containing the wake-up command in the second audio data by receiving the position information of the wake-up command in the second audio data, thereby avoiding the situation where the wake-up command is not accurately positioned due to a certain delay in signal transmission, which in turn leads to a low wake-up success rate or inability to activate. This can improve the wake-up accuracy and success rate and enhance the user experience.

[0052] Please refer to Figure 4To deepen understanding, the present disclosure also provides a system architecture diagram of a specific application scenario of another audio data processing method. The system architecture 400 may include a first terminal 401 and a second terminal 402. Among them, the first terminal 401 is a vehicle-mounted terminal, and the second terminal 402 is a mobile terminal, such as a smart phone, a smart wearable device or other terminal devices that can communicate with the vehicle-mounted terminal by wire or wirelessly. Multiple processes can be run in the first terminal 401, such as the first process and the second process and other more processes, and different processes can transmit signals and data based on the AIDL (Android Interface Definition Language) interface.

[0053] Combination Figure 4 As shown, the audio data processing method of this embodiment may include the following steps:

[0054] Step S410: acquiring first audio data.

[0055] In this embodiment, the first process running on the first terminal 401 obtains voice data of the user in the car collected by a recording device such as a microphone as the first audio data.

[0056] Step S411, mark the first audio data to obtain the second audio data.

[0057] In this embodiment, the first process makes a backup when acquiring the first audio data, one of which is used for speech recognition; the other is used for marking processing to obtain the second audio data with time stamp information. The marking processing may include marking a corresponding timestamp for each audio frame in the first audio data, so that the audio frame in the second audio data has a corresponding timestamp. After the completion of step S411, go to step S414.

[0058] Step S412, identifying a wake-up instruction.

[0059] In this embodiment, the first process identifies the acquired first audio data, and when a wake-up instruction is identified, the next step S413 is executed.

[0060] Step S413, determining the location information of the wake-up instruction.

[0061] In this embodiment, when the first process identifies the wake-up instruction, it will determine the location information of the wake-up instruction and continue to execute the next step S414.

[0062] The location information of the wake-up instruction may be time point information used to determine the location of the wake-up instruction in the first audio data. As an exemplary embodiment, the location information of the wake-up instruction is a timestamp corresponding to a specific word in the wake-up instruction. For example, if the wake-up instruction is "Hey Siri", the specific word in the wake-up instruction may be any one or more of the starting word "H", the keyword "S", and the ending word "i".

[0063] Step S414: transmitting the positioning information and the second audio data.

[0064] In this embodiment, after determining the positioning information, the first process transmits the positioning information and the second audio data to the second process.

[0065] In some optional implementations of this embodiment, the first process may transmit the positioning information and the second audio data via an AIDL interface, and the AIDL interface may be used for inter-process communication.

[0066] Step S415: receiving positioning information and second audio data.

[0067] In this embodiment, the second process running in the first terminal 401 receives the location information of the wake-up instruction and the second audio data with time stamp information from the first process.

[0068] Step S416: determine the location information of the wake-up instruction in the second audio data.

[0069] In this embodiment, the second process determines the position information of the wake-up instruction in the second audio data according to the received positioning information and the time mark information in the second audio data.

[0070] In some implementations of this embodiment, the position information of the wake-up instruction in the second audio data determined by the second process may include the position information of a specific word in the wake-up instruction in the second audio data. For example, the second process may determine the position information of the keyword "S" of the wake-up instruction in the second audio data based on the timestamp of the keyword "S" of the wake-up instruction in the positioning information and the timestamp marked in the second audio data, and the position information may be the frame in which the keyword "S" is located in the second audio data.

[0071] Step S417: output the position information and the second audio data.

[0072] In this embodiment, when the second process determines the position information of the wake-up instruction in the second audio data, the second process transmits the position information and the second audio data to the second terminal 402 .

[0073] In some implementations of this embodiment, the second process and the second terminal 402 may be connected via a USB (Universal Serial Bus) interface or wireless communication, and may perform signal and data transmission.

[0074] Step S420: receiving position information and second audio data.

[0075] In this embodiment, the second terminal 402 receives the location information of the wake-up instruction in the second audio data and the second audio data from the first terminal 401 .

[0076] Step S421: perform speech recognition on the second audio data.

[0077] In this embodiment, the second terminal 402 performs speech recognition on the second audio data according to the above-mentioned position information. For example, the position information of the keyword "S" of the wake-up instruction in the second audio data can be used to recognize the audio frames within a certain range before and after the position of the keyword "S" in the second audio data.

[0078] Step S422, identifying the wake-up instruction.

[0079] In this embodiment, when the second terminal 402 identifies the corresponding wake-up instruction, it proceeds to execute the next step S423.

[0080] Step S423: Send a wake-up activation indication.

[0081] In this embodiment, when the second terminal 402 recognizes the corresponding wake-up instruction, it may be triggered to enter the wake-up activation state, and may send a wake-up activation instruction to the second process of the first terminal 401.

[0082] In some implementations of this embodiment, the second process may perform a wake-up voice broadcast and a human-machine interface display according to the received wake-up activation indication, so as to further enhance the user experience.

[0083] In an exemplary embodiment, in the above step S417, when the second process transmits the location information and the second audio data, it also transmits the location information of the wake-up instruction (such as the timestamp of the keyword "S") to the second terminal 402; the second terminal 402 can perform corresponding verification based on the received location information of the wake-up instruction, and perform the subsequent steps S421-S423 of voice recognition and triggering the wake-up operation after the verification is passed. The verification may include redetermining the location information of the wake-up instruction in the second audio data based on the location information of the wake-up instruction and the time stamp information in the second audio data, and comparing the redetermined location information with the received location information. If they are consistent, the verification is considered to have passed, otherwise the verification is considered to have failed. In this way, the accuracy of wake-up recognition can be further improved.

[0084] In some implementations of this embodiment, the first terminal 401 is a vehicle-mounted terminal, the first process is a vehicle-mounted voice recognition process (e.g., a vehicle-mounted voice assistant), and the second process is a driving assistance process. The second terminal 402 is a mobile terminal, and a driving assistance program is installed in the second terminal 402. Signals and data can be transmitted between the first process and the second process via the AIDL protocol. The second process can output the second audio data and the location information of the wake-up instruction to the driving assistance program of the mobile terminal by wire or wirelessly, or can also output the positioning information to the driving assistance program of the mobile terminal.

[0085] The audio data processing method in this embodiment enables the first process and the second process running in the first terminal 401 to be coupled with each other, and provides an audio data processing solution with a high wake-up success rate by outputting the location information of the wake-up instruction and the second audio data to the second terminal 402. This method can support in-vehicle voice wake-up scenarios connected to external devices or driving assistance programs with voice interaction functions, and can improve user experience and driving safety.

[0086] Further references Figure 5 As an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of an audio data processing device, and the device embodiment is Figure 2 As shown and Figure 4 Corresponding to a part of the method embodiments shown in , the device can be specifically applied to the first terminal or various other electronic devices.

[0087] like Figure 5As shown, the audio data processing device 500 of this embodiment may include: a positioning information determination module 530, a tag processing module 540, a position information determination module 550 and a first interface module 560. The positioning information determination module 530 is configured to determine the positioning information of the wake-up instruction in the first audio data; the tag processing module 540 is configured to perform tag processing on the first audio data to obtain the second audio data with time tag information; the position information determination module 550 is configured to determine the position information of the wake-up instruction in the second audio data according to the positioning information and the time tag information in the second audio data; the first interface module 560 is configured to output the second audio data and the position information.

[0088] In some optional implementations of this embodiment, the audio data processing device 500 further includes an acquisition module 510 and a speech recognition module 520. The acquisition module 510 is configured to acquire the first audio data through a first process running on the first terminal; the speech recognition module 520 is configured to recognize the first audio data through the first process, and recognize the wake-up instruction in the first audio data.

[0089] In some optional implementations of this embodiment, the acquisition module 510 is further configured to back up the first audio data through the first process and send it to the marking processing module 540; the marking processing module 540 is further configured to mark the backed-up first audio data through the first process to obtain second audio data with time marking information, wherein the marking processing includes: marking the audio frames in the first audio data with corresponding timestamps.

[0090] In some optional implementations of this embodiment, the location information of the wake-up instruction determined by the location information determination module 530 includes a timestamp corresponding to a specific word in the wake-up instruction, wherein the specific word includes at least one of a starting word, a keyword, and an ending word.

[0091] In some optional implementations of this embodiment, the audio data processing device 500 also includes a transmission module (not shown), which is configured to transmit the positioning information of the wake-up instruction and the second audio data to the second process running on the first terminal.

[0092] In some optional implementations of this embodiment, the transmission module is further configured to transmit the positioning information and the second audio data to the second process based on the Android interface definition language protocol.

[0093] In some optional implementations of this embodiment, the position information determination module is further configured to determine position information of the wake-up instruction in the second audio data through a second process, and the position information includes position information of a specific word in the wake-up instruction in the second audio data.

[0094] In some optional implementations of this embodiment, the first interface module is further configured to output positioning information.

[0095] In some optional implementations of this embodiment, the audio data processing device 500 also includes an interaction module (not shown), and the first interface module is further configured to receive a wake-up activation indication; the interaction module is configured to perform a wake-up voice broadcast and a human-computer interface display according to the wake-up activation indication.

[0096] In the present embodiment, in the audio data processing device 500, other features of the positioning information determination module 530, the tag processing module 540, the position information determination module 550 and the first interface module 560 and the technical effects thereof can be referred to in Figure 2 Steps 201-204 and Figure 4 The relevant descriptions of steps S410-S417 in the corresponding embodiment are not repeated here.

[0097] This embodiment exists as an apparatus embodiment corresponding to the above-mentioned method embodiment. The audio data processing device provided in this embodiment can more accurately locate the position of the wake-up instruction in the second audio data by determining the position information of the wake-up instruction in the second audio data, thereby improving the success rate and activation rate of voice wake-up and enhancing the user experience.

[0098] Next reference Figure 6 As an implementation of the methods shown in the above figures, the present disclosure provides another embodiment of an audio data processing device, which is similar to Figure 3 As shown and Figure 4 Corresponding to a part of the method embodiments shown in , the device can be specifically applied to a second terminal or various other electronic devices.

[0099] like Figure 6 As shown, the audio data processing device 600 of this embodiment may include: a second interface module 610, a voice wake-up module 630 and a wake-up trigger module 640. The second interface module 610 is configured to receive the second audio data with time stamp information and the position information of the wake-up instruction in the second audio data; the voice wake-up module 630 is configured to perform voice recognition on the second audio data according to the received position information; the wake-up trigger module 640 is configured to trigger entering the wake-up activation state in response to recognizing the wake-up instruction.

[0100] In some optional implementations of this embodiment, the position information of the wake-up instruction in the second audio data includes the position information of a specific word in the wake-up instruction in the second audio data, and the specific word includes at least one of a starting word, a keyword, and an ending word.

[0101] In some optional implementations of this embodiment, the time marking information includes a time stamp corresponding to an audio frame in the second audio data.

[0102] In some optional implementations of this embodiment, the audio data processing device 600 may further include a verification module 620, and the second interface module 610 may be further configured to receive positioning information of the wake-up instruction, the positioning information including a timestamp corresponding to a specific word in the wake-up instruction; the verification module 620 is configured to perform verification based on the positioning information of the wake-up instruction, and after passing the verification, notify the voice wake-up module 630 to perform voice recognition on the second audio data.

[0103] In some optional implementations of this embodiment, the wake-up trigger module 640 is further configured to send a wake-up activation indication through the second interface module 610 in response to identifying the wake-up instruction.

[0104] In this embodiment, the specific processing of the second interface module 610, the verification module 620, the voice wake-up module 630 and the wake-up trigger module 640 in the audio data processing device 600 and the technical effects thereof can be referred to in Figure 3 The corresponding steps 301-303 and Figure 4 The relevant descriptions of steps S420-S423 in the corresponding embodiment are not repeated here.

[0105] This embodiment exists as an apparatus embodiment corresponding to the above-mentioned method embodiment. The audio data processing device provided in this embodiment can accurately locate the voice segment containing the wake-up instruction in the second audio data by receiving the position information of the wake-up instruction in the second audio data, thereby avoiding the situation where the wake-up instruction is not accurately positioned due to a certain delay in signal transmission, which in turn leads to a low wake-up success rate or inability to activate. This can improve the wake-up accuracy and success rate and enhance the user experience.

[0106] Further references Figure 7 The present disclosure provides a system for processing audio data. Figure 7 As shown, the processing system 700 includes a first terminal 701 and a second terminal 702. The first terminal 701 may be a vehicle-mounted terminal.

[0107] In this embodiment, the first terminal 701 may include Figure 5 The audio data processing apparatus 500 described in the illustrated embodiment.

[0108] In some optional implementations of this embodiment, the first terminal 701 includes multiple processes running therein, for example, may include an in-vehicle voice recognition process and a driving assistance process.

[0109] In some optional implementations of this embodiment, the second terminal 702 may be a mobile terminal, which may include Figure 6 The audio data processing device 600 described in the illustrated embodiment. The second terminal 702 may include a driving assistance program. The driving assistance program may be connected to the driving assistance process in the first terminal 701, and the two may transmit signals and data via wired or wireless means.

[0110] In this embodiment, the audio data processing device 500 in the first terminal 701 and the audio data processing device 600 in the second terminal 702 and the technical effects thereof can be referred to respectively. Figure 5 as well as Figure 6 The relevant descriptions in the corresponding embodiments will not be repeated here.

[0111] The audio data processing system 700 provided in this embodiment determines the position information of the wake-up instruction in the second audio data through the first terminal 701 and outputs it to the second terminal 702, so that the second terminal 702 can accurately locate the voice segment containing the wake-up instruction in the second audio data, avoiding the situation where the wake-up success rate is low or the activation cannot be performed due to the signal and audio data transmission delay between different processes and different terminals, thereby improving the wake-up accuracy and success rate and enhancing the user experience.

[0112] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can implement the audio data processing method described in any of the above embodiments when executing.

[0113] According to an embodiment of the present disclosure, the present disclosure further provides a readable storage medium, which stores computer instructions, and the computer instructions are used to enable a computer to implement the audio data processing method described in any of the above embodiments when executed.

[0114] According to an embodiment of the present disclosure, the present disclosure further provides a computer program product, which, when executed by a processor, can implement the audio data processing method described in any of the above embodiments.

[0115] Figure 8A schematic block diagram of an example electronic device 800 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0116] like Figure 8 As shown, the device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0117] A number of components in the device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0118] The computing unit 801 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 801 performs the various methods and processes described above, such as a processing method for audio data. For example, in some embodiments, the processing method for audio data may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the processing method for audio data described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform the processing method for audio data in any other appropriate manner (e.g., by means of firmware).

[0119] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0120] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0121] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0122] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0123] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0124] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and virtual private servers (VPS) services.

[0125] According to the technical solution of the embodiment of the present disclosure, by determining the position information of the wake-up instruction in the second audio data, the position of the wake-up instruction in the second audio data can be located more accurately, thereby improving the success rate and activation rate of voice wake-up and enhancing the user experience.

[0126] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0127] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for processing audio data, applied to a first terminal, comprising: Determining location information of the wake-up instruction in the first audio data; Marking the first audio data to obtain second audio data with time mark information; Determine, according to the positioning information and the time stamp information in the second audio data, the position information of the wake-up instruction in the second audio data; The second audio data and the position information are output.

2. The method according to claim 1, further comprising: The first audio data is acquired by a first process running on the first terminal.

3. The method according to claim 2, wherein: The determining the location information of the wake-up instruction in the first audio data includes: The first process performs speech recognition on the first speech data; In response to identifying the wake-up instruction, the first process determines location information of the wake-up instruction; The location information of the wake-up instruction includes a timestamp corresponding to a specific word in the wake-up instruction.

4. The method according to claim 3, wherein: The specific word includes at least one of a starting word, a keyword, and an ending word.

5. The method according to claim 3, wherein: The marking process of the first audio data to obtain the second audio data with time marking information comprises: The first process backs up the acquired first audio data, performs the marking process on the backed-up first audio data, and obtains the second audio data with time marking information.

6. The method according to claim 4, wherein: The marking process of the first audio data includes: The audio frames in the first audio data are marked with corresponding timestamps.

7. The method according to claim 5, further comprising: The first process transmits the positioning information and the second audio data to a second process running on the first terminal; The determining, according to the positioning information and the time mark information in the second audio data, the position information of the wake-up instruction in the second audio data includes: The second process determines, according to the positioning information and the time mark information in the second audio data, the position information of the wake-up instruction in the second audio data; The position information of the wake-up instruction in the second audio data includes the position information of the specific word in the wake-up instruction in the second audio data.

8. The method according to claim 7, wherein: The first process transmitting the positioning information and the second audio data to the second process running on the first terminal includes: The first process transmits the positioning information and the second audio data to the second process based on the Android interface definition language protocol.

9. The method according to claim 7 or 8, wherein: The outputting the second audio data and the position information comprises: The second process outputs the second audio data and the position information to a second terminal.

10. The method according to claim 9, further comprising: The second process outputs the positioning information to the second terminal.

11. The method according to claim 9, further comprising: The second process performs a wake-up voice broadcast and a human-machine interface display according to the received wake-up activation instruction sent by the second terminal.

12. A method for processing audio data, applied to a second terminal, comprising: receiving second audio data with time stamp information and position information of the wake-up instruction in the second audio data; performing speech recognition on the second audio data according to the position information; In response to identifying the wake-up instruction, entering into the wake-up activation state is triggered.

13. The method according to claim 12, wherein: The performing speech recognition on the second audio data according to the position information comprises: Receiving location information of the wake-up instruction, the location information including a timestamp corresponding to a specific word in the wake-up instruction; A verification is performed according to the location information of the received wake-up instruction, and after passing the verification, speech recognition is performed on the second audio data according to the location information.

14. The method according to claim 12, wherein: The position information of the wake-up instruction in the second audio data includes the position information of a specific word in the wake-up instruction in the second audio data, and the specific word includes at least one of a start word, a keyword, and an end word; The time marking information includes a time stamp corresponding to an audio frame in the second audio data.

15. The method according to any one of claims 12 to 14, further comprising: In response to identifying the wake-up instruction, a wake-up activation indication is sent to the first terminal.

16. An audio data processing device, applied to a first terminal, comprising: a positioning information determination module, configured to determine positioning information of the wake-up instruction in the first audio data; a marking processing module, configured to perform marking processing on the first audio data to obtain second audio data with time marking information; a position information determination module, configured to determine position information of the wake-up instruction in the second audio data according to the positioning information and time mark information in the second audio data; The first interface module is configured to output the second audio data and the position information.

17. The processing device according to claim 16, further comprising: an acquisition module, configured to acquire the first audio data through a first process running on the first terminal; The speech recognition module is configured to recognize the first audio data through the first process, and identify the wake-up instruction in the first audio data.

18. The processing device according to claim 17, wherein: The acquisition module is further configured to back up the first audio data through the first process and send it to the marking processing module; The marking processing module is further configured to perform the marking processing on the backed-up first audio data through the first process to obtain the second audio data with time marking information, wherein the marking processing includes: marking the audio frames in the first audio data with corresponding timestamps.

19. The processing device according to claim 17, wherein: The positioning information of the wake-up instruction determined by the positioning information determination module includes a timestamp corresponding to a specific word in the wake-up instruction, wherein the specific word includes at least one of a starting word, a keyword, and an ending word.

20. The processing device according to claim 19, further comprising: The transmission module is configured to transmit the positioning information of the wake-up instruction and the second audio data to a second process running on the first terminal.

21. The processing device according to claim 20, wherein: The transmission module is further configured to transmit the positioning information and the second audio data to the second process based on an Android interface definition language protocol.

22. The processing device according to any one of claims 19 to 21, wherein: The position information determination module is further configured to determine the position information of the wake-up instruction in the second audio data according to the positioning information and the time stamp information in the second audio data through the second process, and the position information includes the position information of the specific word in the wake-up instruction in the second audio data.

23. The processing device according to any one of claims 19 to 21, wherein: The first interface module is further configured to output the positioning information.

24. The processing device according to claim 16, further comprising an interaction module, The first interface module is further configured to receive a wake-up activation indication; The interaction module is configured to perform a wake-up voice broadcast and a human-machine interface display according to the wake-up activation indication.

25. An audio data processing device, applied to a second terminal, comprising: A second interface module is configured to receive second audio data with time mark information and position information of the wake-up instruction in the second audio data; a voice wake-up module, configured to perform voice recognition on the second audio data according to the position information; a wake-up trigger module, configured to trigger entering a wake-up activation state in response to identifying the wake-up instruction; The position information of the wake-up instruction in the second audio data includes the position information of a specific word in the wake-up instruction in the second audio data, and the specific word includes at least one of a starting word, a keyword, and an ending word; The time marking information includes a time stamp corresponding to an audio frame in the second audio data.

26. The processing device according to claim 25, further comprising a verification module, The second interface module is further configured to receive location information of the wake-up instruction, wherein the location information includes a timestamp corresponding to a specific word in the wake-up instruction; The verification module is configured to perform verification according to the location information of the received wake-up instruction, and after passing the verification, notify the voice wake-up module to perform voice recognition on the second audio data.

27. A processing device according to claim 25 or 26, wherein: The wake-up trigger module is further configured to send a wake-up activation indication through the second interface module in response to identifying the wake-up instruction.

28. A system for processing audio data, comprising a first terminal and a second terminal, wherein: The first terminal comprises an audio data processing device according to any one of claims 16-24; The second terminal comprises an audio data processing device according to any one of claims 25-27.

29. The processing system of claim 28, wherein: The first terminal includes a vehicle-mounted terminal, the second terminal includes a mobile terminal, the first process includes a vehicle-mounted voice recognition process, the second process includes a driving assistance process, and the driving assistance process outputs the second audio data and the location information of the wake-up instruction to the driving assistance program on the mobile terminal.

30. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the audio data processing method described in any one of claims 1-11, or execute the audio data processing method described in any one of claims 12-15.

31. A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable the computer to execute the audio data processing method according to any one of claims 1 to 11, or the audio data processing method according to any one of claims 12 to 15.

32. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the steps of the method for processing audio data according to any one of claims 1 to 11, or the steps of the method for processing audio data according to any one of claims 12 to 15.

Images