Audio processing method and device, equipment, storage medium and product

Abstract

The invention belongs to the technical field of data processing, and discloses an audio processing method and device, equipment, a storage medium and a product. According to the method, the original audio of the user is collected, the optimal voiceprint is searched from the preset block chain based on the user information corresponding to the original audio, then environmental noise elimination is performed on the original audio according to the optimal voiceprint, the eliminated audio is obtained, text noise elimination is performed on the eliminated audio, and the target audio is obtained. According to the method, the optimal voiceprint is searched from the preset block chain based on the user information corresponding to the original audio, the optimal voiceprint of the user can be quickly searched from the preset block chain, and then environmental noise elimination is performed on the original audio according to the optimal voiceprint, so that the environmental noise in the original audio can be effectively recognized, and the user experience is improved. And then text noise elimination is carried out on the audio after elimination, so that the text noise in the audio after elimination can be further effectively identified, and the environment noise and the text noise in the original audio input by the user can be effectively eliminated.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to an audio processing method, device, equipment, storage medium and product. Background Art

[0002] In the scenario of digital humans or digital humans intelligent customer service or business assistants, users can interact with digital humans through voice to handle hotel and air ticket services and provide business consultation. There are many scenarios in which users use voice interaction, not limited to offices, walking in office areas, in small cars such as taxis, large vehicles such as buses or coaches, hotel entrances or lobbies, airplane cabins, airport waiting rooms, etc. These scenarios are not absolutely quiet places. Therefore, when users talk to digital humans, it is inevitable that the voices of other people around them, the noise emitted by other vehicles or objects will be picked up by the terminal device, resulting in noise interference in the voice received by the digital humans. Therefore, how to effectively eliminate the noise in the audio input by the user has become an urgent problem to be solved. Summary of the invention

[0003] The main purpose of the present application is to provide an audio processing method, device, equipment, storage medium and product, aiming to solve the technical problem of effectively eliminating noise in audio input by a user.

[0004] To achieve the above object, the present application provides an audio processing method, which comprises the following steps:

[0005] Collect the user's original audio, and search for the best voiceprint from a preset blockchain based on the user information corresponding to the original audio;

[0006] Eliminate environmental noise from the original audio according to the optimal voiceprint to obtain eliminated audio;

[0007] The text noise is removed from the eliminated audio to obtain the target audio.

[0008] Optionally, the step of performing environmental noise elimination on the original audio according to the optimal voiceprint to obtain the eliminated audio specifically includes:

[0009] The original audio is segmented according to the voiceprint length corresponding to the optimal voiceprint to obtain a plurality of audio segments;

[0010] Matching each audio segment with the optimal voiceprint to obtain a successfully matched target audio segment;

[0011] The target audio segments are spliced ​​to obtain the audio after elimination.

[0012] Optionally, the step of matching each audio segment with the optimal voiceprint to obtain a successfully matched target audio segment specifically includes:

[0013] Performing curve fitting on each audio segment and the optimal voiceprint to obtain a plurality of audio curves and voiceprint curves;

[0014] Each audio curve and the voiceprint curve are segmented to obtain a plurality of first curves and a plurality of second curves;

[0015] Determining trend similarity and curve overlap between the first curve and the second curve, and determining curve similarity according to the trend similarity and the curve overlap;

[0016] The successfully matched target audio segment is determined according to the curve similarity.

[0017] Optionally, the step of performing text noise elimination on the eliminated audio to obtain the target audio specifically includes:

[0018] Processing the modal particles in the eliminated audio to obtain processed audio;

[0019] Converting the processed audio into phonetic text;

[0020] The repeated text in the pinyin text is deleted to obtain the target audio.

[0021] Optionally, the step of deleting repeated text in the pinyin text to obtain the target audio specifically includes:

[0022] Extracting the initial consonants of each pinyin in the pinyin text, and constructing a character string according to the initial consonants;

[0023] Determine the prime numbers corresponding to the initial consonants in the character string, and use the product of the prime numbers as the first product;

[0024] Selecting a preset number of target initial consonants from the character string, and taking the product of the prime numbers corresponding to the target initial consonants as the second product;

[0025] Determine repeated text according to the first product and the second product, and delete the repeated text in the pinyin text to obtain the target audio.

[0026] Optionally, the step of collecting the original audio of the user and searching for the best voiceprint from a preset blockchain based on the user information corresponding to the original audio specifically includes:

[0027] Collect the user's original audio, and search for the voiceprint block from the preset blockchain according to the user information corresponding to the original audio;

[0028] Determine the voiceprint storage array according to the shared array address corresponding to the voiceprint block;

[0029] The user block number corresponding to the voiceprint storage array is determined according to the user information, and the best voiceprint corresponding to the user is determined according to the user block number.

[0030] In addition, to achieve the above-mentioned purpose, the present application also provides an audio processing device, the audio processing device comprising:

[0031] A voiceprint search module, used to collect the user's original audio and search for the best voiceprint from a preset blockchain based on the user information corresponding to the original audio;

[0032] An environmental noise elimination module, used to eliminate environmental noise from the original audio according to the optimal voiceprint to obtain eliminated audio;

[0033] The text noise elimination module is used to eliminate the text noise of the eliminated audio to obtain the target audio.

[0034] In addition, to achieve the above-mentioned purpose, the present application also proposes an audio processing device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the audio processing method described above.

[0035] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the audio processing method described above are implemented.

[0036] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the audio processing method described above are implemented.

[0037] This application collects the user's original audio, and searches for the best voiceprint from a preset blockchain based on the user information corresponding to the original audio, then performs environmental noise elimination on the original audio based on the best voiceprint to obtain the eliminated audio, and then performs text noise elimination on the eliminated audio to obtain the target audio. This application searches for the best voiceprint from a preset blockchain based on the user information corresponding to the original audio, can quickly find the user's best voiceprint from the preset blockchain, then performs environmental noise elimination on the original audio based on the best voiceprint, can effectively identify the environmental noise in the original audio, and then performs text noise elimination on the eliminated audio, can further effectively identify the text noise in the eliminated audio, thereby effectively eliminating the environmental noise and text noise in the original audio input by the user. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0039] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0040] Figure 1 This is a flowchart of the first embodiment of the audio processing method of the present application;

[0041] Figure 2 A schematic diagram of a voiceprint storage mode of an embodiment of the audio processing method of the present application;

[0042] Figure 3 This is a flow chart of a second embodiment of the audio processing method of the present application;

[0043] Figure 4 A schematic diagram of matching between an audio clip and an optimal voiceprint according to an embodiment of an audio processing method of the present application;

[0044] Figure 5 This is a flowchart of a third embodiment of the audio processing method of the present application;

[0045] Figure 6 This is a structural block diagram of the first embodiment of the audio processing device of the present application;

[0046] Figure 7 It is a structural diagram of an audio processing device in a hardware operating environment involved in an embodiment of the present application.

[0047] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0048] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0049] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0050] The main solution of the embodiment of the present application is: collecting the original audio of the user, and searching for the best voiceprint from the preset blockchain based on the user information corresponding to the original audio; performing environmental noise elimination on the original audio according to the best voiceprint to obtain eliminated audio; performing text noise elimination on the eliminated audio to obtain target audio.

[0051] In the scenario of digital humans or digital humans’ intelligent customer service or business assistant, users can interact with digital humans through voice to handle hotel and air ticket business and provide business consultation. There are many scenarios where users use voice interaction, not limited to offices, walking in office areas, in small cars such as taxis, large vehicles such as buses or coaches, hotel entrances or lobbies, airplane cabins, airport waiting rooms, etc. These scenarios are not absolutely quiet places. Therefore, when users talk with digital humans, it is inevitable that the voices of other people around them, the noises emitted by other vehicles or objects will be picked up by the terminal device, resulting in noise interference in the voice received by the digital humans.

[0052] This application collects the user's original audio, and searches for the best voiceprint from a preset blockchain based on the user information corresponding to the original audio, then performs environmental noise elimination on the original audio based on the best voiceprint to obtain the eliminated audio, and then performs text noise elimination on the eliminated audio to obtain the target audio. This application searches for the best voiceprint from a preset blockchain based on the user information corresponding to the original audio, can quickly find the user's best voiceprint from the preset blockchain, then performs environmental noise elimination on the original audio based on the best voiceprint, can effectively identify the environmental noise in the original audio, and then performs text noise elimination on the eliminated audio, can further effectively identify the text noise in the eliminated audio, thereby effectively eliminating the environmental noise and text noise in the original audio input by the user.

[0053] It should be noted that the execution subject of the present application may be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, etc., or an electronic device capable of realizing the above functions, an audio processing device, etc. The following takes an audio processing device as an example to illustrate this embodiment and the following embodiments.

[0054] Based on this, the present application embodiment provides an audio processing method, referring to Figure 1 , Figure 1 This is a flowchart diagram of the first embodiment of the audio processing method of the present application.

[0055] In this embodiment, the audio processing method includes the following steps:

[0056] Step S10: Collect the original audio of the user, and search for the best voiceprint from the preset blockchain based on the user information corresponding to the original audio.

[0057] It is understandable that after the user opens the Digital Intelligence application, if the scenario of this embodiment is a digital companion, the application realizes air ticket and hotel business consultation and processing through voice. The user can press and hold the voice input button to express the user's intention in words. After the expression is completed, release the voice input button. At this time, the front-end application of the Digital Intelligence application receives the original audio from the microphone, and calls the back-end service interface to transmit the original audio to the back-end. The back-end service interface receives the original audio and can realize noise elimination, voice recognition, intent recognition and other functions. The corresponding back-end service interface is called to realize data return, and the process ends.

[0058] It should be understood that the original audio of the user can be collected through the above-mentioned digital human application, and the best voiceprint corresponding to the user can be found from the preset blockchain based on the user information corresponding to the original audio. The preset blockchain can pre-store the best voiceprints of several users.

[0059] Furthermore, in order to quickly find the best voiceprint from the preset blockchain, in this embodiment, the step S10 includes: collecting the original audio of the user, and searching the voiceprint block from the preset blockchain according to the user information corresponding to the original audio; determining the voiceprint storage array according to the shared array address corresponding to the voiceprint block; determining the user block number corresponding to the voiceprint storage array according to the user information, and determining the best voiceprint corresponding to the user according to the user block number.

[0060] It is understandable that the voiceprint in this embodiment can be stored in the blockchain in the main chain + shared array mode. The voiceprint only stores one current and last voiceprint in the blockchain, that is, when a better voiceprint of the same user is uploaded to the chain, it replaces the voiceprint stored on the chain; conversely, when a voiceprint of the same user whose voiceprint effect is not as good as the existing voiceprint on the chain is uploaded to the chain, the blockchain does not process it.

[0061] It should be understood that reference Figure 2 , Figure 2 Schematic diagram of the voiceprint storage mode of an embodiment of the audio processing method of the present application. Figure 2As shown, the voiceprint storage process in this embodiment may include: Step 1: Each node of the blockchain is started, the genesis block is created, and the data is waited for to be uploaded to the chain. Step 2: The smart contract is published on the chain through the blockchain management background. After the consensus contract is reached, each blockchain node encapsulates the smart contract into a block and mounts it after the genesis block. Step 3: When the user's voiceprint data is published on the chain through the chain request interface, the user's voiceprint can be a voiceprint after noise processing. After the consensus contract is reached, each blockchain node temporarily stores the user's voiceprint locally and starts to build a local blockchain. ① If the tail block of the current blockchain is a smart contract block, it means that it is the first time that the voiceprint data is uploaded to the chain. At this time, a voiceprint shared array needs to be created to store the voiceprint information: a block representing voiceprint storage is created, a shared array (starting with an array of default length L), and the content of the block body is the address information of the shared array, that is, the voiceprint storage array can be found through the shared array address in the block body. Then the voiceprint block is mounted after the smart contract block. Each element of the shared array contains two structures: voiceprint information and the user block number (the user block number starts from 1, and the block number increases by 1 each time a new user is added). Write the user voiceprint information and user number 1 into the first element of the shared array at this time. Finally, write the user information into the block body of a newly created main chain block, and mount the user block after the voiceprint block. ② If the tail block of the current blockchain is not a smart contract block, but a user information block, it means that the voiceprint shared array has been created and there is no need to create it again. At this time, the voiceprint storage array can be found through the shared array address in the block body of the voiceprint block. The user block number n of the element can be found by traversing the array to find the last element of the shared array, that is, n user voiceprint information has been uploaded to the chain before. Write the user voiceprint information and user number n+1 into the nth element of the shared array at this time. Finally, write the user information into the block body of a newly created main chain block, and mount the user block after the user block (the user block number is n).

[0062] In this embodiment, there is also a situation where user information is processed for the second time on the chain, that is, the blockchain has already saved the user's information that has been on the chain. At this time, the quality of the two voiceprints needs to be compared. The voiceprint with good quality is saved in the blockchain. That is, if the quality of the newly chained voiceprint is better than the voiceprint already on the chain (that is, the voiceprint stored in the nth element of the shared array), the voiceprint stored in the nth element of the shared array is replaced with the newly chained voiceprint; otherwise, if the quality of the newly chained voiceprint is not better than the voiceprint already on the chain, the nth element of the shared array is not processed. The key here is the judgment of voiceprint quality, and the judgment algorithm is as follows:

[0063]

[0064] Among them, Q is the voiceprint quality parameter, the voiceprint length is L, the average time voiceprint density is M, and the effective voiceprint length is l. The meaning of the algorithm is as follows: the more effective voice content of the user contained in the voiceprint, the greater the voiceprint density (ie, the number of bytes) in the average time (such as 1 second), indicating that the voiceprint quality is better.

[0065] In addition, when the request for the user's voiceprint information to be downloaded from the chain is issued, the shared array corresponding to the user's voiceprint information will be deleted, thereby achieving capacity reduction. The processing process is as follows: Step 1: Each blockchain node receives the request for the user's voiceprint information to be downloaded from the chain. After consensus is reached through the consensus contract, each node starts the process of deleting the user's voiceprint locally. Step 2: Each blockchain node checks whether there is a corresponding user block in the local blockchain based on the user information. If the user block does not exist, it means that there is no user information, and the processing ends at this time. Otherwise, it means that the user information exists in the blockchain. Step 3: The voiceprint storage array can be found through the shared array address in the block body of the voiceprint block in the blockchain. According to the number of the user block, the shared array is traversed to find the "user block number" in a certain element in the array as the number of the user block. At this time, the element to be downloaded is found, the element is deleted from the array, and the elements following the element are sequentially moved forward by one position (the last element of the shared array is empty at this time), that is, the deletion of the element to be downloaded is realized.

[0066] In a specific implementation, to find the best voiceprint, the specific steps are as follows: Step 1: The blockchain node that receives the query request parses the user information in the query request, that is, the user information corresponding to the original audio, and proceeds to Step 2. Step 2: According to the user information described in Step 1, traverse the preset blockchain. If there is a user information block, it means that the user's voiceprint has been uploaded to the chain, and proceed to Step 3; otherwise, it means that the user's voiceprint information has never been uploaded to the chain, and the query ends. Step 3: The voiceprint storage array can be found through the shared array address in the block body of the voiceprint block in the preset blockchain. According to the user block number n, determine whether the user block number to which the nth element in the shared array belongs is n. ① If so, the voiceprint information in this element is the query object and is returned to the requester; ② If not, it means that there are off-chain voiceprints of users numbered [1, n - 1] on the chain. At this time, it is necessary to traverse from the (n - 1)th element of the shared array towards the first element to find the element with user number n. The search algorithm uses the incremental exponential search method, and the process is as follows: <1> If the user number of the (n - 1)th element is n, the required voiceprint information is found, which is the query object and is returned to the requester; otherwise, continue to execute <2>. <2> If the user number m of the (n - 1)th element > n + 1, start the incremental exponential search: Let the traversal step size be s, and initially s = 1; at this time, since m is not the search object, s = s + 1; if the user number m of the (n - s)th element = n, the required voiceprint information is found, which is the query object and is returned to the requester; otherwise, m is still > n, and at this time s = 2 × s; continue to check the relationship between the user number m of the (n - s)th element and n. If m = n, the required voiceprint information is found, which is the query object and is returned to the requester; if m is still > n, at this time s = 2 × s, and continue to check the user number m of the (n - s)th element; if m < n, at this time s = s - 1, and continue to check the user number m of the (n - s)th element until m = n is found and ended. The above query object returned to the requester is the best voiceprint corresponding to the user.

[0067] Step S20: Perform environmental noise cancellation on the original audio according to the best voiceprint to obtain the audio after cancellation.

[0068] It can be understood that the environmental noise can be non-user voices. In this embodiment, the environmental noise of the user's original audio can be cancelled according to the best voiceprint to obtain the audio after cancellation.

[0069] Step S30: Perform text noise cancellation on the audio after cancellation to obtain the target audio.

[0070] It should be understood that text noise cancellation can be to delete the modal particles and repeated text content in the audio after cancellation to obtain the target audio. The target audio refers to the audio after environmental noise cancellation and text noise cancellation of the original audio.

[0071] This embodiment collects the original audio of the user, and searches for the best voiceprint from the preset blockchain based on the user information corresponding to the original audio, then performs environmental noise elimination on the original audio according to the best voiceprint to obtain the eliminated audio, and then performs text noise elimination on the eliminated audio to obtain the target audio. This embodiment searches for the best voiceprint from the preset blockchain based on the user information corresponding to the original audio, and can quickly find the best voiceprint of the user from the preset blockchain, and then performs environmental noise elimination on the original audio according to the best voiceprint, and can effectively identify the environmental noise in the original audio, and then performs text noise elimination on the eliminated audio, and can further effectively identify the text noise in the eliminated audio, thereby effectively eliminating the environmental noise and text noise in the original audio input by the user.

[0072] refer to Figure 3 , Figure 3 This is a flowchart diagram of the second embodiment of the audio processing method of the present application.

[0073] Based on the above first embodiment, in this embodiment, step S20 includes:

[0074] Step S201: Segment the original audio according to the voiceprint length corresponding to the optimal voiceprint to obtain a plurality of audio segments.

[0075] It is understandable that the original audio can be segmented according to the voiceprint length corresponding to the optimal voiceprint to obtain several audio segments, and the length of each audio segment is the same as the above voiceprint length.

[0076] Step S202: Match each audio segment with the optimal voiceprint to obtain a successfully matched target audio segment.

[0077] It should be understood that for each audio segment, it can be matched with the best voiceprint in turn to obtain a successfully matched target audio segment, and the target audio segment is the user's effective voice.

[0078] Furthermore, in order to effectively match each audio segment with the optimal voiceprint, in this embodiment, the step S202 includes: performing curve fitting on each audio segment and the optimal voiceprint respectively to obtain a plurality of audio curves and voiceprint curves; segmenting each audio curve and the voiceprint curve respectively to obtain a plurality of first curves and a plurality of second curves; determining the trend similarity and curve overlap between the first curve and the second curve, and determining the curve similarity based on the trend similarity and the curve overlap; determining the successfully matched target audio segment based on the curve similarity.

[0079] It should be noted that each audio segment and the best voiceprint can be curve-fitted to obtain several audio curves and voiceprint curves. Each audio curve and voiceprint curve is then segmented to obtain several first curves and several second curves, and the bending trend of the first curve and the second curve is calculated. Taking the bending trend calculation of the first curve as an example, the vertical axis coordinates of the points composed of the curves from left to right on the horizontal axis are compared. If the vertical axis coordinates of the points on the left side of the horizontal axis are all lower than the vertical axis coordinates of the points on the right side of the horizontal axis, it is an upward trend; if the vertical axis coordinates of the points on the left side of the horizontal axis are all higher than the vertical axis coordinates of the points on the right side of the horizontal axis, it is a downward trend; if the vertical axis coordinates of the points on the left side of the horizontal axis are all equal to the vertical axis coordinates of the points on the right side of the horizontal axis, it is a horizontal trend. If the first curve and the second curve are both an upward trend, a downward trend or a horizontal trend, the trend similarity is calculated; otherwise, it means that a certain curve has two or three of the three trends. At this time, the curve segment needs to be further segmented according to the number of trends calculated. If there are three trends, the two trend change points in the segment are used as segmentation points to form three segments of the curve. Then the curve trend is determined again until the first curve and the second curve are both an upward trend, a downward trend or a horizontal trend.

[0080] It should be understood that the first curve and the second curve in the same curve interval can be obtained through the above steps. Assuming that the trend values ​​of the first curve and the second curve are Trend1 and Trend2 respectively, the trend value is set to 1 when the upward trend is set, the trend value is -1 when the downward trend is set, and the trend value is 0 when the horizontal trend is set. The trend similarity between the first curve and the second curve = Trend1 / Trend2 (when Trend2=0, the value is ∞). It can be seen that if the first curve and the second curve are both rising or falling, the trend similarity is 1; if the trends of the first curve and the second curve are different, the trend similarity is -1; if Trend2=0, the trend similarity is infinite.

[0081] It can be understood that, assuming that the leftmost coordinate of the first curve is (x11, y11), the leftmost coordinate of the second curve is (x21, y21), the rightmost coordinate of the first curve is (x12, y12), and the rightmost coordinate of the second curve is (x22, y22), then

[0082] In a specific implementation, the curve similarity between the first curve and the second curve = trend similarity × curve repetition. Figure 4 , Figure 4 This is a schematic diagram of the matching between an audio clip and an optimal voiceprint according to an embodiment of the audio processing method of the present application. Figure 4As shown, there are three matching situations: situation ①: the two voiceprints are completely matched, and the complete match means that the curve similarity is less than the threshold K. At this time, the audio segment is the target audio segment; situation ②: the two voiceprints are partially matched, and the partial match means that the curve similarity of some segments is less than the threshold K, and the curve similarity of other segments is greater than the threshold K. At this time, the segment less than the threshold K is cut out as the target audio segment; situation ③: the two voiceprints do not match at all, then the audio segment is environmental noise and is directly eliminated.

[0083] Step S203: splicing the target audio segments to obtain the audio after elimination.

[0084] It is understandable that all target audio clips can be spliced ​​in order to obtain the eliminated audio, and the eliminated audio can also be published to the preset blockchain.

[0085] This embodiment divides the original audio according to the voiceprint length corresponding to the best voiceprint to obtain a number of audio segments, then matches each audio segment with the best voiceprint to obtain a successfully matched target audio segment, and then splices the target audio segments to obtain the eliminated audio. This embodiment first divides the original audio according to the voiceprint length corresponding to the best voiceprint, then matches each audio segment with the best voiceprint, which can improve the matching efficiency, and removes the environmental noise of the original audio during the matching process to obtain a successfully matched target audio segment, and then splices the target audio segments to obtain the eliminated audio, so that there is no environmental noise in the eliminated audio.

[0086] refer to Figure 5 , Figure 5 This is a flowchart diagram of the third embodiment of the audio processing method of the present application.

[0087] Based on the above embodiments, in this embodiment, step S30 includes:

[0088] Step S301: Process the modal particles in the eliminated audio to obtain processed audio.

[0089] It is understandable that the modal particles in the eliminated audio may be deleted first, such as: uh, oh, ba, ha, to obtain the processed audio.

[0090] Step S302: converting the processed audio into pinyin text.

[0091] Step S303: Delete the repeated text in the pinyin text to obtain the target audio.

[0092] It should be understood that repeated text in the pinyin text can be deleted, and repeated text refers to text that is said repeatedly to obtain the target audio.

[0093] Furthermore, in order to effectively delete repeated text, in this embodiment, step S303 includes: extracting the initial consonants of each pinyin in the pinyin text, and constructing a character string based on the initial consonants; determining the prime numbers corresponding to each initial consonant in the character string, and taking the product between the prime numbers as the first product; selecting a preset number of target initial consonants from the character string, and taking the product between the prime numbers corresponding to the target initial consonants as the second product; determining repeated text based on the first product and the second product, and deleting the repeated text in the pinyin text to obtain the target audio.

[0094] It is understandable that the initial consonants of each pinyin in the pinyin text can be extracted, and all the initial consonants can be sequentially combined into a string. Each initial consonant is assigned a prime number starting from 2 in advance, and the prime numbers corresponding to each initial consonant in the string can be determined, and then all the prime numbers can be multiplied to obtain the first product J.

[0095] It should be understood that the preset number in this embodiment can be set to 2, and the initial consonants at the i-th and i+1-th positions are taken out in order in the character string in units of 2 initial consonants, i.e., the target initial consonants, i starts at 1, and i=i+2 in each processing process. The prime numbers corresponding to the target initial consonants are multiplied, i.e., the product of the prime numbers corresponding to the initial consonants at the i-th and i+1-th positions, as the second product temp. If J can divide temp×temp, it means that there are duplicate texts corresponding to the texts of the above target initial consonants. If it cannot be divided, there are no duplicate texts corresponding to the texts of the above target initial consonants.

[0096] In a specific implementation, the character string is traversed according to the pinyin content at the i-th and i+1-th positions. If it can be found, it means that duplicate text is stored. At this time, the pinyin content at the i-th and i+1-th positions in the character string is removed. Otherwise, it means that there are only two phrases with the same initial consonants but different contents. Then i=i+1 is executed until the end of the character string is processed. This embodiment can find duplicate text in the above manner, and then delete the duplicate text to obtain the target audio.

[0097] This embodiment processes the modal particles in the audio after elimination to obtain the processed audio, then converts the processed audio into phonetic text, and then deletes the repeated text in the phonetic text to obtain the target audio. This embodiment first removes the modal particles in the audio after elimination, then converts the processed audio into phonetic text, can determine the repeated text with the help of the phonetic text, and then deletes the repeated text, can effectively eliminate text noise from the audio after elimination, and obtain the target audio after environmental noise elimination and text noise elimination.

[0098] Reference Figure 6 , Figure 6 This is a structural block diagram of the first embodiment of the audio processing device of the present application.

[0099] like Figure 6 As shown, the audio processing device proposed in the embodiment of the present application includes:

[0100] A voiceprint search module 10 is used to collect the original audio of the user and search for the best voiceprint from a preset blockchain based on the user information corresponding to the original audio;

[0101] An environmental noise elimination module 20 is used to eliminate environmental noise from the original audio according to the optimal voiceprint to obtain eliminated audio;

[0102] The text noise elimination module 30 is used to eliminate text noise from the eliminated audio to obtain target audio.

[0103] This embodiment collects the original audio of the user, and searches for the best voiceprint from the preset blockchain based on the user information corresponding to the original audio, then performs environmental noise elimination on the original audio according to the best voiceprint to obtain the eliminated audio, and then performs text noise elimination on the eliminated audio to obtain the target audio. This embodiment searches for the best voiceprint from the preset blockchain based on the user information corresponding to the original audio, and can quickly find the best voiceprint of the user from the preset blockchain, and then performs environmental noise elimination on the original audio according to the best voiceprint, and can effectively identify the environmental noise in the original audio, and then performs text noise elimination on the eliminated audio, and can further effectively identify the text noise in the eliminated audio, thereby effectively eliminating the environmental noise and text noise in the original audio input by the user.

[0104] It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of the present application. In practical applications, technicians in this field can select part or all of it according to actual needs to achieve the purpose of the present embodiment, and no limitation is made here.

[0105] In addition, for technical details not fully described in this embodiment, please refer to the audio processing method provided in any embodiment of the present application, and will not be repeated here.

[0106] Based on the above-mentioned first embodiment of the audio processing device of the present application, a second embodiment of the audio processing device of the present application is proposed.

[0107] In this embodiment, the environmental noise elimination module 20 is also used to divide the original audio according to the voiceprint length corresponding to the optimal voiceprint to obtain a plurality of audio segments; match each audio segment with the optimal voiceprint to obtain a successfully matched target audio segment; and splice the target audio segments to obtain the eliminated audio.

[0108] Furthermore, the environmental noise elimination module 20 is also used to perform curve fitting on each audio segment and the optimal voiceprint, respectively, to obtain a number of audio curves and voiceprint curves; to segment each audio curve and the voiceprint curve, respectively, to obtain a number of first curves and a number of second curves; to determine the trend similarity and curve overlap between the first curve and the second curve, and to determine the curve similarity based on the trend similarity and the curve overlap; and to determine the successfully matched target audio segment based on the curve similarity.

[0109] Furthermore, the text noise elimination module 30 is also used to process the modal particles in the eliminated audio to obtain processed audio; convert the processed audio into pinyin text; delete repeated text in the pinyin text to obtain target audio.

[0110] Furthermore, the text noise elimination module 30 is also used to extract the initial consonants of each pinyin in the pinyin text, and construct a character string based on the initial consonants; determine the prime numbers corresponding to each initial consonant in the character string, and use the product between the prime numbers as the first product; select a preset number of target initial consonants from the character string, and use the product between the prime numbers corresponding to the target initial consonants as the second product; determine repeated text based on the first product and the second product, and delete the repeated text in the pinyin text to obtain the target audio.

[0111] Furthermore, the voiceprint search module 10 is also used to collect the original audio of the user, and search for a voiceprint block from a preset blockchain according to the user information corresponding to the original audio; determine a voiceprint storage array according to a shared array address corresponding to the voiceprint block; determine a user block number corresponding to the voiceprint storage array according to the user information, and determine the best voiceprint corresponding to the user according to the user block number.

[0112] Other embodiments or specific implementations of the audio processing device of the present application can refer to the above-mentioned method embodiments and will not be described in detail here.

[0113] The present application provides an audio processing device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the audio processing method in the above-mentioned embodiment 1.

[0114] Reference below Figure 7 , which shows a schematic diagram of the structure of an audio processing device suitable for implementing the embodiment of the present application. The audio processing device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 7 The audio processing device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0115] like Figure 7 As shown, the audio processing device may include a processing device 1001 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the audio processing device are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the audio processing device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows an audio processing device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have alternatively.

[0116] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0117] The audio processing device provided by the present application adopts the audio processing method in the above embodiment, which can solve the technical problem of how to effectively eliminate the noise in the audio input by the user. Compared with the prior art, the beneficial effects of the audio processing device provided by the present application are the same as the beneficial effects of the audio processing method provided by the above embodiment, and the other technical features in the audio processing device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0118] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0119] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0120] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the audio processing method in the above-mentioned embodiment.

[0121] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0122] The computer-readable storage medium may be included in the audio processing device; or may exist independently without being assembled into the audio processing device.

[0123] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the audio processing device, the audio processing device: collects the original audio of the user, and searches for the best voiceprint from the preset blockchain based on the user information corresponding to the original audio; performs environmental noise elimination on the original audio according to the best voiceprint to obtain eliminated audio; and performs text noise elimination on the eliminated audio to obtain target audio.

[0124] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0125] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0126] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0127] The readable storage medium provided in the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned audio processing method, and can solve the technical problem of how to effectively eliminate noise in the audio input by the user. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in the present application are the same as the beneficial effects of the audio processing method provided in the above-mentioned embodiment, and will not be repeated here.

[0128] The present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned audio processing method when executed by a processor.

[0129] The computer program product provided by the present application can solve the technical problem of how to effectively eliminate noise in the audio input by the user. Compared with the prior art, the beneficial effects of the computer program product provided by the present application are the same as the beneficial effects of the audio processing method provided by the above embodiment, which will not be repeated here.

[0130] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. An audio processing method, characterized in that: The audio processing method comprises the following steps: Collect the user's original audio, and search for the best voiceprint from a preset blockchain based on the user information corresponding to the original audio; Eliminate environmental noise from the original audio according to the optimal voiceprint to obtain eliminated audio; The text noise is removed from the eliminated audio to obtain the target audio.

2. The audio processing method according to claim 1, characterized in that: The step of performing environmental noise elimination on the original audio according to the optimal voiceprint to obtain the eliminated audio specifically includes: The original audio is segmented according to the voiceprint length corresponding to the optimal voiceprint to obtain a plurality of audio segments; Matching each audio segment with the optimal voiceprint to obtain a successfully matched target audio segment; The target audio segments are spliced ​​to obtain the audio after elimination.

3. The audio processing method according to claim 2, characterized in that: The step of matching each audio segment with the optimal voiceprint to obtain a successfully matched target audio segment specifically includes: Performing curve fitting on each audio segment and the optimal voiceprint to obtain a plurality of audio curves and voiceprint curves; Each audio curve and the voiceprint curve are segmented to obtain a plurality of first curves and a plurality of second curves; Determining trend similarity and curve overlap between the first curve and the second curve, and determining curve similarity according to the trend similarity and the curve overlap; The successfully matched target audio segment is determined according to the curve similarity.

4. The audio processing method according to claim 1, characterized in that: The step of performing text noise elimination on the eliminated audio to obtain the target audio specifically includes: Processing the modal particles in the eliminated audio to obtain processed audio; Converting the processed audio into phonetic text; The repeated text in the pinyin text is deleted to obtain the target audio.

5. The audio processing method according to claim 4, characterized in that: The step of deleting repeated text in the pinyin text to obtain the target audio specifically includes: Extracting the initial consonants of each pinyin in the pinyin text, and constructing a character string according to the initial consonants; Determine the prime numbers corresponding to the initial consonants in the character string, and use the product of the prime numbers as the first product; Selecting a preset number of target initial consonants from the character string, and taking the product of the prime numbers corresponding to the target initial consonants as the second product; Determine repeated text according to the first product and the second product, and delete the repeated text in the pinyin text to obtain the target audio.

6. The audio processing method according to any one of claims 1 to 5, characterized in that: The step of collecting the original audio of the user and searching for the best voiceprint from the preset blockchain based on the user information corresponding to the original audio specifically includes: Collect the user's original audio, and search for the voiceprint block from the preset blockchain according to the user information corresponding to the original audio; Determine the voiceprint storage array according to the shared array address corresponding to the voiceprint block; The user block number corresponding to the voiceprint storage array is determined according to the user information, and the best voiceprint corresponding to the user is determined according to the user block number.

7. An audio processing device, characterized in that: The audio processing device comprises: A voiceprint search module, used to collect the user's original audio and search for the best voiceprint from a preset blockchain based on the user information corresponding to the original audio; An environmental noise elimination module, used to eliminate environmental noise from the original audio according to the optimal voiceprint to obtain eliminated audio; The text noise elimination module is used to eliminate the text noise of the eliminated audio to obtain the target audio.

8. An audio processing device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the audio processing method according to any one of claims 1 to 6.

9. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the audio processing method according to any one of claims 1 to 6 are implemented.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the audio processing method according to any one of claims 1 to 6 are implemented.

Images