Audio processing method and device, readable storage medium and program product

Abstract

The invention is suitable for the technical field of audio processing, and provides an audio processing method and device, a readable storage medium and a program product. The method comprises the following steps: acquiring a first audio clip; through a pre-trained artificial intelligence model, on the basis of collection environment information corresponding to the first audio clip, noise segment filtering processing is carried out on the first audio clip, a second audio clip is obtained, voice recognition is carried out on the basis of the second audio clip, and the collection environment information comprises at least one of the collection position or time; wherein the artificial intelligence model is obtained by performing model training based on training data, the training data comprises a sample audio clip and sample information of the sample audio clip, the sample information comprises collection environment information of the sample audio clip and classification information used for indicating an audio type, and the audio type comprises a noisy type or a noiseless type. Noise reduction is carried out on the audio, the success rate of voice processing is improved, and the accuracy of a voice processing result is improved.

Description

Technical Field

[0001] This application belongs to the field of audio processing technology, and in particular relates to audio processing methods, devices, readable storage media and program products. Background Technology

[0002] In vehicle maintenance scenarios, people often use voice commands for information recording, equipment control, and other interactive processes to facilitate operation.

[0003] In existing technologies, due to the complexity of vehicle maintenance scenarios, noise often accompanies the use of voice for interaction. For example, if the maintenance point is near the road, there may be sudden honking when using voice to record information, which may cause audio processing failure or low accuracy of audio processing results (such as speech recognition results). Summary of the Invention

[0004] This application provides an audio processing method, device, readable storage medium, and program product, which can solve the problem in the prior art that the presence of noise causes audio processing to fail or the accuracy of audio processing results to be low.

[0005] In a first aspect, embodiments of this application provide an audio processing method, including:

[0006] Get the first audio segment; Using a pre-trained artificial intelligence model, based on the acquisition environment information corresponding to the first audio segment, the first audio segment is filtered for noise to obtain a second audio segment, and speech recognition is performed based on the second audio segment. The acquisition environment information includes at least one of the acquisition location or time. The artificial intelligence model is obtained by training the model based on training data. The training data includes sample audio segments and sample information of the sample audio segments. The sample information includes the acquisition environment information of the sample audio segments and classification information for indicating the audio type. The audio type includes noisy type or noise-free type.

[0007] In one possible implementation of the first aspect, obtaining the first audio segment includes: Obtain the audio to be processed; The audio to be processed is subjected to the detection of silent segments, wherein the silent segments include audio segments whose duration of silence is greater than a preset first duration; If a silent segment is detected in the audio to be processed, the silent segment in the audio to be processed is processed to obtain a first audio segment, wherein the duration of the silent segment in the audio to be processed is greater than the duration of the processed silent segment in the first audio segment.

[0008] In one possible implementation of the first aspect, processing the silent segment in the audio to be processed includes one of the following: Filter the silent segment; Increase the playback speed of the silent segment; Compress the silent segment.

[0009] In one possible implementation of the first aspect, obtaining the first audio segment includes: Based on preset segmentation conditions, the acquired audio to be processed is segmented to obtain at least two first audio segments, so that the first audio segments can be filtered for noise segments in parallel through the artificial intelligence model. The preset segmentation conditions include at least one of the following: The segmentation time interval is the preset second duration; The duration of silence is longer than the preset third duration.

[0010] In one possible implementation of the first aspect, obtaining the first audio segment includes: The acquired audio to be processed is segmented to obtain at least two first audio segments; For any two adjacent first audio segments among the at least two first audio segments, a portion of one of the two adjacent first audio segments is added to the other first audio segment, wherein the portion includes a start portion or an end portion.

[0011] In one possible implementation of the first aspect, the method further includes: In a preset acquisition environment, sample audio segments are acquired, and the sample audio segments include at least one of a first sample audio segment with noise or a second sample audio segment with human voice and no noise. Silent and / or repetitive audio segments are removed from the collected sample audio segments; The classification information and acquisition environment information of the processed sample audio segments are determined. The classification information of the first sample audio segment includes the type with noise, and the classification information of the second sample audio segment includes the type without noise.

[0012] In one possible implementation of the first aspect, the noise filtering process on the first audio segment includes: If it is determined that a first noise segment exists in the first audio segment, speech recognition is performed on the first noise segment, wherein the first noise segment includes a noise segment with a duration greater than or equal to a preset fourth duration; If the speech recognition result of the first noise segment indicates the presence of key information for constituting a sentence, the first noise segment is segmented to obtain at least two second noise segments, wherein the key information includes at least one of subject, action, state, object or modifier. The at least two second noise segments are subjected to noise segment filtering processing.

[0013] Secondly, embodiments of this application provide an audio processing apparatus, including: The acquisition module is used to acquire the first audio segment; The filtering module is used to filter the first audio segment for noise segments based on the acquisition environment information corresponding to the first audio segment using a pre-trained artificial intelligence model to obtain a second audio segment, and to perform speech recognition based on the second audio segment. The acquisition environment information includes at least one of the acquisition location or time. The artificial intelligence model is obtained by training the model based on training data. The training data includes sample audio segments and sample information of the sample audio segments. The sample information includes the acquisition environment information of the sample audio segments and classification information for indicating the audio type. The audio type includes noisy type or noise-free type.

[0014] Thirdly, embodiments of this application provide a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method described in any one of the first aspects above.

[0015] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in any one of the first aspects above.

[0016] Fifthly, embodiments of this application provide a computer program product that, when run on a computer device, causes the computer device to implement the method described in any one of the first aspects above.

[0017] The beneficial effects of the embodiments in this application compared with the prior art are: This application provides an audio processing method. Specifically, after acquiring a first audio segment, a pre-trained artificial intelligence model can filter out noise segments from the first audio segment based on the acquisition environment information corresponding to the first audio segment to obtain a second audio segment. This method can reduce noise in the first audio segment, thereby reducing the impact of noise on subsequent speech processing and improving the success rate and accuracy of speech recognition based on the second audio segment. The artificial intelligence model can be trained on training data, which may include sample audio segments and sample information of the sample audio segments. The sample information may include acquisition environment information of the sample audio segments and classification information indicating the audio type, which may include noisy or noise-free audio types. Building upon this foundation, during the training phase, AI models can learn which noise segments to filter in which environments (location and / or time), enabling them to adapt to audio processing in complex scenarios and improving their audio processing capabilities. Furthermore, preprocessing the sample audio segments included in the training data yields corresponding sample information (collection environment information, classification information). Combining these sample audio segments and their information during AI model training simplifies training costs, enhances the trained AI model's audio processing capabilities, and ultimately improves the accuracy of audio processing. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a schematic flowchart of an audio processing method provided in an embodiment of this application; Figure 2 This is a schematic diagram of audio segmentation to be processed provided in an embodiment of this application; Figure 3 This is a schematic diagram of audio segmentation to be processed provided in another embodiment of this application; Figure 4 This is a schematic diagram of audio segmentation to be processed provided in another embodiment of this application; Figure 5 This is a system architecture diagram provided in one embodiment of this application; Figure 6 This is a schematic diagram of the structure of the computer device provided in the embodiments of this application. Detailed Implementation

[0020] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0021] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0022] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0023] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0024] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0025] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0026] The following is combined with Figures 1 to 5 The audio processing method provided in the embodiments of this application will be described.

[0027] The audio processing method provided in this application can be applied to computer devices such as mobile phones, tablets, wearable devices, in-vehicle devices, augmented reality (AR) / virtual reality (VR) devices, laptops, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), and servers. This application does not impose any restrictions on the specific type of computer device.

[0028] For example, when the audio processing method is implemented through an in-vehicle device, the artificial intelligence model can be deployed on one side of the in-vehicle device. In practical applications, audio data can be directly collected through the in-vehicle device or other terminal devices, the artificial intelligence model can be called for processing, and the processing results can be stored and / or fed back through output devices (such as displays, playback devices, etc.).

[0029] For example, such as Figure 5 As shown, in automotive repair scenarios, audio processing methods can be implemented using diagnostic equipment. Artificial intelligence models can be deployed on these devices. In practical applications, the diagnostic equipment can collect audio data input by repair technicians, call the artificial intelligence model for processing, and store and / or provide feedback through output devices (such as tablet displays or in-vehicle device displays). The diagnostic equipment can be a tablet, used as a visualization device for the automotive repair process, communicating with the VCI (Vehicle Communication Interface, also known as a diagnostic box, a device used to communicate with vehicle data) via Bluetooth.

[0030] For example, when the audio processing method is implemented through a server, the artificial intelligence model can be deployed on the server side. In actual applications, the server can receive audio data uploaded by the terminal device, call the artificial intelligence model to process it, and store and / or feed back the processing results to the terminal device.

[0031] Specifically, such as Figure 1 As shown, the method provided in this application embodiment includes S101 to S102: S101, Obtain the first audio segment.

[0032] S102. Using a pre-trained artificial intelligence model, based on the acquisition environment information corresponding to the first audio segment, the first audio segment is filtered for noise to obtain a second audio segment, and speech recognition is performed based on the second audio segment. The acquisition environment information includes at least one of the acquisition location or time.

[0033] Optionally, the first audio segment can be the raw audio data acquired, such as unprocessed audio data; for example, the first audio segment can be audio data acquired when a human voice is detected, or audio data acquired when the user triggers voice input. The first audio segment can also be a portion of the raw audio data. Optionally, the first audio segment can include various sound components, such as useful speech information and any noise that may be present.

[0034] For example, the first audio segment can be acquired using various audio acquisition devices (such as microphones, recorders, mobile phones, in-vehicle devices, etc.). The acquisition environment can be diverse, such as outdoors, in a conference room, or at different times of day, such as weekdays, holidays, mornings, afternoons, and evenings. For example, the usage environment of the diagnostic equipment (i.e., the audio data acquisition environment) can be categorized, such as by the possibility of noise in the environment (e.g., location), distinguishing between workshops, enclosed indoor spaces, suburbs, etc.

[0035] Optionally, the pre-trained Artificial Intelligence (AI) model can be a model pre-trained with a large amount of data. The AI ​​model has the ability to filter noise from audio segments. For example, the AI ​​model can filter noise segments in the first audio segment based on the acquisition environment information corresponding to that segment; that is, the AI ​​model can identify and filter what kind of noise under what acquisition environment.

[0036] Optionally, the environmental information collected may include data related to the environment in which the audio was collected. This may include the location of the collection (e.g., in a vehicle repair shop, a noisy street, a quiet indoor environment, etc.) and / or the time (e.g., different times of day, night, etc.). Collecting environmental information helps artificial intelligence models to more accurately determine the noise level in the audio.

[0037] Optionally, the noise segment can be an audio segment in the first audio segment where the duration of the noise exceeds a set threshold. For example, the noise can be a non-speech part of the audio, such as a horn, music, or knocking sound.

[0038] Optionally, the second audio segment is audio data obtained by a pre-trained artificial intelligence model after filtering noise segments from the first audio segment based on the acquisition environment information corresponding to the first audio segment. Compared to the first audio segment, the noise components in the second audio segment can be effectively reduced, which is beneficial to improving the accuracy of subsequent speech recognition.

[0039] For example, in terms of processing by an artificial intelligence model, the first audio segment is input into a pre-trained artificial intelligence model. The artificial intelligence model can extract feature information from the first audio segment and first determine the acquisition environment information of the first audio segment. For example, if the feature information indicates the presence of a horn sound, and the preset acquisition location corresponding to the horn sound is workshop 1 near the roadside, then it can be determined that the first audio segment was acquired in workshop 1. In the case of workshop 1, the artificial intelligence model can filter the corresponding noise segment.

[0040] For example, the data input to the pre-trained AI model may include not only the first audio segment but also the corresponding acquisition environment information. In other words, the first audio segment and its corresponding acquisition environment information are input into the pre-trained AI model. Based on pre-learned knowledge and the acquisition environment information, the AI ​​model analyzes the sound components in the first audio segment and identifies noise segments. For example, if the acquisition environment is a noisy street, the AI ​​model can identify noise such as vehicle traffic and crowd noise; if the acquisition time is at night and the location is near an airport, the AI ​​model can identify noise such as airplane takeoff and landing.

[0041] Optionally, the filtering process for noise segments can be to remove or reduce these noise components.

[0042] The artificial intelligence model is trained based on training data, which includes sample audio segments and sample information of the sample audio segments. The sample information includes the acquisition environment information of the sample audio segments and classification information used to indicate the audio type, which includes noisy or noiseless types.

[0043] Optionally, the training data can be a dataset used to train an artificial intelligence model, which may include sample audio clips and corresponding sample information. The sample audio clips, serving as audio data for training, can be used to enable the model to learn the characteristics of different audio types. Sample information includes information related to the sample audio clips, such as information about the acquisition environment and classification information indicating the audio type. The audio type is the result of classifying the sample audio clips, and can be categorized as noisy (e.g., the audio contains noise) or noise-free (e.g., the audio contains no noise or the noise is negligible).

[0044] For example, Table 1 shows the corresponding noise levels of the acquired audio data under different acquisition environment conditions: Table 1

[0045] For example, in a vehicle repair scenario, the data collection environment can be preset to facilitate the AI ​​model's more accurate learning of the corresponding noise characteristics under the preset environment. In the example shown in Table 1, the data collection locations in the environment can be categorized into four types: workshop 1 near the roadside, sheet metal repair workshop 2, vehicle cleaning workshop 3, and enclosed indoor spaces. Furthermore, considering that the noise levels at the same collection location may differ at different times, the collection time can also be differentiated. For example, for workshop 1 near the roadside, the collection time can be set from 7:00 to 19:00, during which the corresponding noise may include horns, construction noise, and pedestrian conversations. In other collection locations, since the collection time has little impact on different noise levels, the consideration of the collection time can be ignored.

[0046] In the example shown in Table 1, when acquiring training data for model training, audio data can be collected at different collection locations and times. For example, between 7:00 and 19:00, noisy audio data 1 and noiseless audio data 2 can be collected in workshop 1 near the roadside; during working hours, noisy audio data 3 and noiseless audio data 4 can be collected in sheet metal repair workshop 2; during working hours, noisy audio data 5 and noiseless audio data 6 can be collected in vehicle cleaning workshop 3; at any time, noisy audio data 6 and noiseless audio data 7 can be collected in a closed room. Based on this, the model can learn the features of noisy and noiseless environments under different collection conditions using the training data, enabling it to filter noise.

[0047] For example, environmental information can be represented by identifiers (IDs). In workshop A, the ID of noisy audio data 1 collected in the morning is A_Noise_1, the ID of noisy audio data 2 collected at noon is A_Noise_2, and the ID of noisy audio data 3 collected in the afternoon is A_Noise_3, with the corresponding label being noise (e.g., the classification information is noisy type). Pure tone audio data (e.g., audio data that includes human voices and has no noise) has the ID A_PureTone, and the label is pure tone (e.g., the classification information is noise-free type).

[0048] Optionally, the AI ​​model can undergo semi-supervised or supervised learning. For example, sample information from audio clips can be used as real sample information. During training, as the AI ​​model processes the audio clips, it determines the predicted sample information for each clip. Then, based on the predicted and real sample information, a loss value is calculated to adjust the AI ​​model's parameters, resulting in a pre-trained AI model. The training process for the AI ​​model can be referenced from relevant technologies.

[0049] Optionally, the artificial intelligence model can employ deep learning techniques such as convolutional neural networks and recurrent neural networks. To improve the model's capabilities, better meet the needs of real-world scenarios (such as user feedback), and enhance the accuracy of speech recognition, training data can be collected periodically or intermittently as needed to update the artificial intelligence model during the implementation of audio processing methods.

[0050] In this embodiment, the second audio segment is obtained by noise filtering the first audio segment based on the acquisition environment information. This can effectively reduce noise interference in the audio. In the subsequent speech recognition process, due to the reduction or lowering of noise components, a more accurate speech recognition result can be obtained, thus improving the accuracy of speech recognition. In addition, the acquisition environment information can provide the model with additional contextual information, enabling the model to more accurately determine the noise situation in the audio according to different acquisition environments, which is beneficial to enhancing the model's adaptability and generalization ability.

[0051] In one possible implementation, the first audio segment is obtained in S101, including S101a to S101c: S101a, Obtain the audio to be processed.

[0052] S101b: Detect silence segments in the audio to be processed. Silence segments include audio segments whose duration of silence is longer than a preset first duration.

[0053] S101c: If a silent segment is detected in the audio to be processed, the silent segment in the audio to be processed is processed to obtain a first audio segment. The duration of the silent segment in the audio to be processed is greater than the duration of the processed silent segment in the first audio segment.

[0054] Optionally, if the first audio segment is processed audio data, the audio to be processed can be the initially acquired, unprocessed audio data. The audio to be processed can include various valid speech information, noise, silence segments, and other audio components.

[0055] Optionally, a silent segment refers to the audio portion where the duration of silence exceeds a preset first duration. For example, a silent segment is not necessarily one with absolutely no sound signal; it can also include audio segments with low sound intensity within a certain threshold range that can be considered silent. These include non-noise, non-speech audio segments. The first duration can be 2 seconds, but this is only an example and can be set according to actual conditions; this application does not limit this setting. For instance, in scenarios where high audio continuity is required, the first duration can be set shorter; in scenarios where a certain interval is permissible, the first duration can be set longer.

[0056] Optionally, the first audio segment is audio data obtained after processing the silent segments in the audio to be processed. Compared with the audio to be processed, the duration of the silent segments in the first audio segment is reduced, which can better meet the requirements of subsequent processing for audio continuity and effective information density.

[0057] Optionally, in the processing of silent segments, the audio to be processed can be converted into a digital signal. The sound intensity of each sampling point or sampling point within a certain time window in the digital audio signal can be calculated. The calculated sound intensity is compared with a preset silence threshold. If the sound intensity of a certain audio segment is continuously lower than the silence threshold and the duration is longer than a preset first duration, then the audio segment is determined to be a silent segment. In the implementation process, the entire audio signal can be traversed by sliding a window and processed segment by segment.

[0058] Optionally, the S101c processes the silent segments in the audio to be processed, including one of the following processing methods: Method 1: Filter out silent segments. Detected silent segments are filtered out of the audio to be processed, such as by trimming, so that the processed audio (such as the first audio segment) does not contain silent segments. This can effectively reduce the total duration of the audio and the duration occupied by silent segments.

[0059] Method 2: Increase the playback speed of silent segments. Detected silent segments are sped up, for example, by increasing their playback speed to 120% to 130%, thus shortening the duration of silent segments in the total processed audio and improving processing efficiency.

[0060] Method 3: Compress the silence segment. Compress the silence segment to shorten its duration. For example, the number of sampling points in the silence segment can be reduced by a certain proportion, thereby reducing the duration of the silence segment in the processed audio while retaining its basic characteristics.

[0061] In this embodiment, silent segments in the audio can be accurately identified, providing a clear target for subsequent silent segment processing. For example, by reasonably setting the first duration and silence threshold, the accuracy and reliability of silent segment detection can be improved, avoiding false positives or false negatives. After processing the silent segments, the total duration of the audio can be effectively shortened, improving the efficiency of audio processing. Furthermore, after removing or reducing silent segments, the effective speech information in the audio is more concentrated, which is beneficial for subsequent speech recognition processing and improves the accuracy of the processing.

[0062] In one possible implementation, obtaining the first audio segment in S101 includes: segmenting the obtained audio to be processed based on preset segmentation conditions to obtain at least two first audio segments, so as to perform noise segment filtering processing on the first audio segments in parallel through an artificial intelligence model.

[0063] The preset segmentation conditions include at least one of the following conditions: Condition 1: The time interval for segmentation is the preset second duration.

[0064] Condition 2: The duration of silence is longer than the preset third duration.

[0065] Optionally, when segmenting the audio to be processed according to the segmentation time interval, such as Figure 2 As shown, assuming a preset second duration of 30 seconds, starting from the beginning of the audio to be processed, a segmentation operation is performed every 30 seconds. An audio segment with a total length of 132 seconds can be segmented into five first audio segments. The fifth segment, being less than 30 seconds, can exist as a single segment or be merged with the fourth segment into a single segment (e.g., the merged fourth segment has a duration of 42 seconds). The segmentation of the audio to be processed can be implemented using audio processing software or programming tools. By reading the audio file to obtain the audio signal and time information, calculating the segmentation points based on the second duration, and dividing the audio signal into at least two sub-audio segments, i.e., the first audio segments, according to the segmentation points.

[0066] Optionally, when segmenting the audio to be processed according to the duration of silence, assuming a preset third duration of 300ms, silence detection can be performed on the audio using audio analysis algorithms (such as energy-based methods) to identify silent segments. When a silent segment is detected and its duration exceeds the third duration, segmentation is performed at the end of the silent segment, dividing the audio to be processed into at least two first audio segments, such as... Figure 3 As shown, if a silence duration longer than the third duration is detected before the 35th second, the audio segment is split at the 35th second to obtain the first audio segment. Subsequent audio segments will be split in the same way.

[0067] Optionally, when performing segmentation based on conditions one and two, both the segmentation time interval and the duration of silence can be considered simultaneously. For example, such as... Figure 4 As shown, if the duration of silence from the start of the audio to the 30th second is longer than the preset third duration (e.g., 300ms), then the first audio segment of the first segment can be divided into the first 30 seconds. Then, the audio segmentation begins from the 31st second. If there is silence from the 32nd second to the 35th second, and the duration of this silence is 4 seconds, which is longer than the third duration, then the first audio segment of the second segment can be divided into the 31st second second second third ...

[0068] Optionally, at least two segments of the first audio data obtained from the segmentation can be simultaneously input into a computing environment with parallel processing capabilities, ensuring that each first audio segment can be processed independently by the artificial intelligence model. The artificial intelligence model can process each received first audio segment simultaneously to improve audio processing efficiency.

[0069] Optionally, after the artificial intelligence model processes at least two first audio segments in parallel to obtain the corresponding second audio segments, during speech recognition processing, the second audio segments can be spliced ​​together in chronological order to obtain a third audio segment, and then the third audio segment can be used as a whole for speech recognition to improve the coherence and accuracy of speech recognition.

[0070] In this embodiment, by dividing the audio to be processed into at least two first audio segments and simultaneously performing noise filtering on the first audio segments using parallel processing, the overall processing time can be shortened. Furthermore, parallel processing can fully utilize computing resources and improve processing speed. In addition, the first audio segments obtained after segmentation are relatively short and have more concentrated audio content. When processing the first audio segments, the artificial intelligence model can more accurately identify and filter noise segments, improving the accuracy and effectiveness of noise filtering, resulting in higher audio quality in the second audio segment. Based on segmentation and parallel processing, this scheme can more flexibly handle audio to be processed of different lengths and complexities, enabling the processing of more audio data or more complex audio processing tasks.

[0071] In one possible implementation, obtaining the first audio segment in S101 includes: segmenting the obtained audio to be processed to obtain at least two first audio segments; for any two adjacent first audio segments among the at least two first audio segments, adding a portion of one of the two adjacent first audio segments to the other first audio segment, wherein the portion includes a start part or an end part.

[0072] Optionally, any two adjacent first audio segments can be two segments that are adjacent in chronological order within the multiple segmented first audio segments. For example, if the audio to be processed is segmented into three first audio segments A, B, and C, then A and B, and B and C are adjacent segments.

[0073] Optionally, partial segment processing refers to adding the end of the first audio segment that appears earlier in time to the first audio segment that appears later in time, and / or adding the beginning of the first audio segment that appears later in time to the first audio segment that appears earlier in time.

[0074] Optionally, the duration of a segment can be set to 200ms, allowing this segment to serve as context, providing more information and avoiding accidental boundary cuts. It is understood that the duration of a segment can be set according to actual needs, and this embodiment does not limit this.

[0075] In this embodiment of the application, considering that during the audio segmentation process, improper segmentation points may lead to discontinuity between adjacent first audio segments, such as speech being truncated or sudden changes in sound, the problem of discontinuity can be compensated by adding the beginning or end parts of adjacent first audio segments, thereby reducing the impact on speech recognition.

[0076] In one possible implementation, the method provided in this application embodiment further includes: In a preset acquisition environment, sample audio segments are acquired, including at least one of a first sample audio segment with noise or a second sample audio segment with human voice and no noise. Remove silent and / or repetitive audio segments from the collected sample audio clips; The classification information and acquisition environment information of the processed sample audio segments are determined. The classification information of the first sample audio segment includes the type with noise, and the classification information of the second sample audio segment includes the type without noise.

[0077] Optionally, the sample audio segments are audio data collected under a preset acquisition environment. They can be divided into two types: a first sample audio segment with noise and a second sample audio segment with human voice but no noise. The first sample audio segment can be used for model learning of noise features and noise filtering related processing, while the second sample audio segment can be used as a reference for clean human voice audio and as a noise-free benchmark for comparative analysis and model training.

[0078] Optionally, a silent audio segment can be a portion of the sample audio segment where there is almost no effective sound signal, the sound energy is low, and it is close to being silent. For example, a silent audio segment can be an audio segment generated during the acquisition process due to situations such as short pauses, equipment failures, or other situations resulting in no signal input.

[0079] Optionally, repeated audio segments can be parts of the sample audio segments that are completely related or highly similar in content. For example, repeated audio segments can be audio segments generated during the acquisition process due to issues such as repeated recording or device caching.

[0080] Optionally, the preset acquisition environment can be set to suit the applicable scenarios of the audio segment processing method. For example, if noise filtering is required in an indoor environment, an indoor space can be selected as the acquisition location in the acquisition environment; if the impact of outdoor traffic noise on speech is of concern, an outdoor location near a major traffic artery can be selected as the acquisition location in the acquisition environment.

[0081] Optionally, for silent segments, audio analysis algorithms can be used for detection. For example, by calculating the energy value of the audio signal and setting an energy threshold, if the energy of a segment is below this threshold for a certain duration, it is identified as a silent segment. The segment is then sliced ​​based on its start and end times and removed from the audio signal. For repetitive segments, similarity comparison algorithms can be used for detection. This involves calculating the similarity between audio signals, and if the similarity exceeds a set similarity threshold, it is identified as a repetitive segment. After identifying repetitive segments, one segment can be retained, and the remaining repetitive segments can be removed from the sample audio segments.

[0082] In this embodiment, after removing silent segments and / or repetitive segments, the interference of invalid data on subsequent processing is reduced in the sample audio segments, which helps to improve the performance of the artificial intelligence model trained on the sample audio segments. Furthermore, by collecting sample audio segments under different preset acquisition environments, including first sample audio segments with different noise types and second sample audio segments with human voices but no noise, diverse sample data can be obtained, which can provide more comprehensive data support for the training and optimization of the artificial intelligence model, enabling the model to have better adaptability and generalization ability in different scenarios.

[0083] In one possible implementation, step S102 involves filtering noise segments from the first audio segment, including steps S102a to S102c: S102a. If it is determined that there is a first noise segment in the first audio segment, speech recognition is performed on the first noise segment. The first noise segment includes a noise segment with a duration greater than or equal to a preset fourth duration.

[0084] S102b, if the speech recognition result of the first noise segment indicates the presence of key information for constituting a sentence, the first noise segment is segmented to obtain at least two second noise segments, and the key information includes at least one of subject, action, state, object or modifier.

[0085] S102c, Perform noise segment filtering on at least two second noise segments.

[0086] Optionally, the first noise segment refers to the noise portion identified in the first audio segment. The setting that the duration of the first noise segment is greater than or equal to the preset fourth duration is intended to filter out noise segments with a longer duration and further process the noise segments in that segment to avoid losing key information that would affect subsequent speech recognition due to direct filtering.

[0087] Optionally, speech recognition includes the process of converting audio signals into text information. In the speech recognition processing of the first noise segment, the aim is to determine whether the first noise segment contains key information used to constitute a sentence. Key information is the basic element constituting a sentence; if the first noise segment contains key information, it indicates that the first noise information contains content that significantly impacts subsequent speech recognition processing.

[0088] Optionally, in the key information, the subject refers to the entity performing the action, issuing the behavior, or being in a certain state in the statement; it is the core object described by the statement and can be a noun, pronoun, etc. The action is the behavior, activity, or change performed by the subject and can be a verb. The state is a description of the subject's situation or condition, explaining the subject's appearance or characteristics at a certain moment or period of time; it can be an adjective, noun, etc. The object is the target or recipient of the action; it is the object to which the action is directed and can be a noun, pronoun, etc. Modifying elements are used to modify, limit, or explain the subject, action, state, or object, making the expression of the statement more accurate, rich, and vivid; they can be attributives, adverbs, complements, etc.

[0089] Optionally, if the speech recognition result of the first noise segment indicates the presence of key information, the first noise segment can be divided into at least two second noise segments based on semantics and / or grammatical structure, or the first noise segment can be equally divided into two second noise segments. After obtaining at least two second noise segments, noise recognition can be performed on the second noise segments again. If a part of the second noise segment is determined to be a noise segment (such as a non-speech segment), it is filtered out from the first audio segment.

[0090] In this embodiment, the first noise segment, which includes key information, is segmented and further judged to retain valuable semantic information. This avoids the loss of key information due to misfiltering, making audio processing more refined and intelligent, and improving the accuracy of audio processing.

[0091] It is understood that the audio processing method provided in the above embodiments can be implemented using an artificial intelligence model.

[0092] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0093] Corresponding to the audio processing method in the above embodiments, this application provides an audio processing apparatus, which includes an acquisition module and a filtering module.

[0094] The acquisition module is used to acquire a first audio segment; the filtering module is used to filter the first audio segment for noise segments based on the acquisition environment information corresponding to the first audio segment using a pre-trained artificial intelligence model to obtain a second audio segment, so as to perform speech recognition based on the second audio segment. The acquisition environment information includes at least one of the acquisition location or time.

[0095] The artificial intelligence model is trained based on training data, which includes sample audio segments and sample information of the sample audio segments. The sample information includes the acquisition environment information of the sample audio segments and classification information used to indicate the audio type, which includes noisy or noiseless types.

[0096] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0097] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0098] This application also provides a computer device, which includes: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor. When the processor executes the computer program, it implements the steps in any of the above method embodiments.

[0099] Figure 6 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Figure 6As shown, the computer device 6 of this embodiment includes: at least one processor 60 ( Figure 6 (Only one is shown in the diagram) a processor, a memory 61, and a computer program 62 stored in the memory 61 and executable on at least one processor 60, wherein the processor 60 executes the computer program 62 to implement the steps in any of the above method embodiments.

[0100] Computer device 6 can be a desktop computer, laptop, handheld computer, cloud server, or other computing device. This computer device may include, but is not limited to, processor 60 and memory 61. Those skilled in the art will understand that... Figure 6 The computer device 6 is merely an example and does not constitute a limitation on the computer device 6. It may include more or fewer components than shown, or combine certain components, or different components, such as input / output devices, network access devices, etc.

[0101] The processor 60 may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0102] In some embodiments, memory 61 may be an internal storage unit of the computer device 6, such as a hard disk or RAM of the computer device 6. In other embodiments, memory 61 may be an external storage device of the computer device 6, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD) card, flash card, etc., provided on the computer device 6. Furthermore, memory 61 may include both internal and external storage units of the computer device 6. Memory 61 is used to store operating systems, applications, bootloaders, data, and other programs, such as program code for computer programs. Memory 61 may also be used to temporarily store data that has been output or will be output.

[0103] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps described in the various method embodiments above.

[0104] This application provides a computer program product that, when run on a mobile terminal, enables the mobile terminal to implement the steps described in the above-described method embodiments.

[0105] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. A computer-readable medium can include at least: any entity or device capable of carrying computer program code to a photographic device / terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0106] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0107] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0108] In the embodiments provided in this application, it should be understood that the disclosed apparatus / computer devices and methods can be implemented in other ways. For example, the apparatus / computer device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0109] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0110] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. An audio processing method, characterized in that, include: Get the first audio segment; Using a pre-trained artificial intelligence model, based on the acquisition environment information corresponding to the first audio segment, the first audio segment is filtered for noise to obtain a second audio segment, and speech recognition is performed based on the second audio segment. The acquisition environment information includes at least one of the acquisition location or time. The artificial intelligence model is obtained by training the model based on training data. The training data includes sample audio segments and sample information of the sample audio segments. The sample information includes the acquisition environment information of the sample audio segments and classification information for indicating the audio type. The audio type includes noisy type or noise-free type.

2. The method according to claim 1, characterized in that, The acquisition of the first audio segment includes: Obtain the audio to be processed; The audio to be processed is subjected to the detection of silent segments, wherein the silent segments include audio segments whose duration of silence is greater than a preset first duration; If a silent segment is detected in the audio to be processed, the silent segment in the audio to be processed is processed to obtain a first audio segment, wherein the duration of the silent segment in the audio to be processed is greater than the duration of the processed silent segment in the first audio segment.

3. The method according to claim 2, characterized in that, The processing of the silent segment in the audio to be processed includes one of the following: Filter the silent segment; Increase the playback speed of the silent segment; Compress the silent segment.

4. The method according to claim 1, characterized in that, The acquisition of the first audio segment includes: Based on preset segmentation conditions, the acquired audio to be processed is segmented to obtain at least two first audio segments, so that the first audio segments can be filtered for noise segments in parallel through the artificial intelligence model. The preset segmentation conditions include at least one of the following: The segmentation time interval is the preset second duration; The duration of silence is longer than the preset third duration.

5. The method according to claim 1 or 4, characterized in that, The acquisition of the first audio segment includes: The acquired audio to be processed is segmented to obtain at least two first audio segments; For any two adjacent first audio segments among the at least two first audio segments, a portion of one of the two adjacent first audio segments is added to the other first audio segment, wherein the portion includes a start portion or an end portion.

6. The method according to claim 1, characterized in that, The method further includes: In a preset acquisition environment, sample audio segments are acquired, and the sample audio segments include at least one of a first sample audio segment with noise or a second sample audio segment with human voice and no noise. Silent and / or repetitive audio segments are removed from the collected sample audio segments; The classification information and acquisition environment information of the processed sample audio segments are determined. The classification information of the first sample audio segment includes the type with noise, and the classification information of the second sample audio segment includes the type without noise.

7. The method according to claim 1, characterized in that, The noise filtering process for the first audio segment includes: If it is determined that a first noise segment exists in the first audio segment, speech recognition is performed on the first noise segment, wherein the first noise segment includes a noise segment with a duration greater than or equal to a preset fourth duration; If the speech recognition result of the first noise segment indicates the presence of key information for constituting a sentence, the first noise segment is segmented to obtain at least two second noise segments, wherein the key information includes at least one of subject, action, state, object or modifier. The at least two second noise segments are subjected to noise segment filtering processing.

8. A computer device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the method as described in any one of claims 1 to 7.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 7.

10. A computer program product, characterized in that, When the computer program product is run on a computer device, it causes the computer device to perform the method as described in any one of claims 1 to 7.

Images