Audio reconstruction method, system and device and storage medium

Abstract

The embodiment of the invention provides an audio reconstruction method, system and device and a storage medium, and the method comprises the steps: determining at least one original audio segment based on a to-be-processed audio; for one original audio segment, performing interpolation processing on the original audio segment based on the target sampling rate to determine a first audio segment; smoothing the first audio segment to determine a second audio segment; and determining a target audio based on the at least one second audio segment corresponding to the at least one original audio segment.

Description

Technical Field

[0001] The present invention relates to the field of audio processing technology, and in particular to an audio reconstruction method, system, device and storage medium. Background Art

[0002] As technology continues to develop, people have higher and higher requirements for audio clarity. Sampling rate is an important factor affecting audio clarity. However, due to technical limitations and storage capacity, early audio files had low sampling rates and low clarity, which could not meet the needs of current users.

[0003] Therefore, it is necessary to provide an audio reconstruction method, system, device and storage medium to improve audio clarity and thus enhance audio quality. Summary of the invention

[0004] One or more embodiments of the present specification provide an audio reconstruction method, the method comprising: determining at least one original audio segment based on audio to be processed; for one original audio segment, based on a target sampling rate, performing interpolation processing on the original audio segment to determine a first audio segment; performing smoothing processing on the first audio segment to determine a second audio segment; and determining a target audio based on at least one second audio segment corresponding to the at least one original audio segment.

[0005] One or more embodiments of the present specification provide an audio reconstruction system, characterized in that it includes: a first determination module, configured to determine at least one original audio segment based on the audio to be processed; a second determination module, configured to perform interpolation processing on an original audio segment based on a target sampling rate to determine a first audio segment; perform smoothing processing on the first audio segment to determine a second audio segment; and a third determination module, configured to determine a target audio based on at least one second audio segment corresponding to the at least one original audio segment.

[0006] One or more embodiments of the present specification provide an audio reconstruction device, including a processor, wherein the processor is configured to execute an audio reconstruction method.

[0007] One or more embodiments of the present specification provide a computer-readable storage medium, wherein the storage medium stores computer instructions. When a computer reads the computer instructions in the storage medium, the computer executes an audio reconstruction method. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] This specification will be further described in the form of exemplary embodiments, which will be described in detail by the accompanying drawings. These embodiments are not restrictive, and in these embodiments, the same number represents the same structure, wherein:

[0009] Figure 1 is a schematic diagram of an application scenario of an audio reconstruction system according to some embodiments of this specification;

[0010] Figure 2 is an exemplary module diagram of an audio reconstruction system according to some embodiments of this specification;

[0011] Figure 3 is an exemplary flow chart of an audio reconstruction method according to some embodiments of this specification;

[0012] Figure 4 is an exemplary flow chart of determining a first audio segment according to some embodiments of the present specification;

[0013] Figure 5 is an exemplary flowchart of determining a second audio segment according to some embodiments of the present specification. DETAILED DESCRIPTION

[0014] In order to more clearly illustrate the technical solutions of the embodiments of this specification, the following is a brief introduction to the drawings required for the description of the embodiments. Obviously, the drawings described below are only some examples or embodiments of this specification. For ordinary technicians in this field, this specification can also be applied to other similar scenarios based on these drawings without creative work. Unless it is obvious from the language environment or otherwise explained, the same reference numerals in the figures represent the same structure or operation.

[0015] It should be understood that the "system", "device", "unit" and / or "module" used herein are a method for distinguishing different components, elements, parts, portions or assemblies at different levels. However, if other words can achieve the same purpose, the words can be replaced by other expressions.

[0016] As shown in this specification and claims, unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "comprise" and "include" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0017] Flowcharts are used in this specification to illustrate the operations performed by the system according to the embodiments of this specification. It should be understood that the preceding or following operations are not necessarily performed precisely in order. Instead, the steps may be processed in reverse order or simultaneously. At the same time, other operations may also be added to these processes, or one or more operations may be removed from these processes.

[0018] Figure 1 It is a schematic diagram of application scenarios of the audio reconstruction system shown in some embodiments of this specification.

[0019] In some embodiments, the application scenario 100 of the audio reconstruction system may include audio to be processed 110 , target audio 120 , a processor 130 , a network 140 , and a terminal device 150 .

[0020] The audio to be processed 110 refers to the audio that needs to be processed.

[0021] The target audio 120 is the audio obtained after processing. In some embodiments, the sampling rate of the target audio 120 is greater than the sampling rate of the audio to be processed 110.

[0022] For more information about the processed audio 110, the target audio 120, and the sampling rate, see Figure 3 and its related description.

[0023] The processor 130 may be used to process data related to the audio reconstruction system. For example, the processor 130 may determine at least one original audio segment based on the audio to be processed 110. Further, for an original audio segment, the processor 130 may interpolate the original audio segment based on the target sampling rate to determine a first audio segment, and smooth the first audio segment to determine a second audio segment. Then, the processor 130 may determine the target audio 120 based on at least one second audio segment corresponding to the at least one original audio segment.

[0024] In some embodiments, the processor 130 may be a single server or a server group. The server group may be centralized or distributed. In some embodiments, the processor 130 may be local or remote. In some embodiments, the processor 130 may be implemented on a cloud platform. As an example only, the cloud platform may include a private cloud, a public cloud, a hybrid cloud, a community cloud, a distributed cloud, an internal cloud, a multi-layer cloud, etc., or any combination thereof. In some embodiments, the processor 130 may be integrated or installed on a terminal device 150.

[0025] The network 140 can connect the components of the audio reconstruction system and / or connect the system with external resources. The network 140 allows the components to communicate with each other and with other parts outside the system to facilitate the exchange of data and / or information. In some embodiments, the network 140 can be any one or more of a wired network or a wireless network. For example, the network 140 can include a cable network, an optical fiber network, a telecommunications network, etc. or any combination thereof.

[0026] The terminal device 150 may refer to one or more terminal devices or software used by a user. In some embodiments, the terminal device 150 may be a mobile device such as a mobile phone 151, a tablet computer 152, a laptop computer 153, or any combination thereof. In some embodiments, the terminal device 150 may interact with other components in the application scenario 100 of the audio reconstruction system through the network 140. The user refers to the operator of the audio reconstruction system.

[0027] In some embodiments, the application scenario 100 of the audio reconstruction system may further include one or more other devices, such as a storage device.

[0028] The storage device may be used to store data, instructions and / or any other information. In some embodiments, the storage device may store data and / or instructions related to audio reconstruction. For example, the storage device may store the audio to be processed 110. For another example, the storage device may also store the target audio 120.

[0029] The storage device may include one or more storage components, each of which may be an independent device or a part of another device (such as the processor 130, etc.). In some embodiments, the storage device may be implemented on a cloud platform.

[0030] It should be noted that the application scenario is provided only for the purpose of illustration and is not intended to limit the scope of this specification. For those of ordinary skill in the art, various modifications or changes can be made according to the description of this specification. For example, the application scenario may also include a database. For another example, the application scenario may be implemented on other devices to achieve similar or different functions. However, changes and modifications will not deviate from the scope of this specification.

[0031] Figure 2 FIG. 2 is an exemplary module diagram of an audio reconstruction system according to some embodiments of the present specification. In some embodiments, the audio reconstruction system 200 may include a first determination module 210 , a second determination module 220 , and a third determination module 230 .

[0032] The first determination module 210 is configured to determine at least one original audio segment based on the audio to be processed.

[0033] The second determination module 220 is configured to perform an interpolation process on an original audio segment based on a target sampling rate to determine a first audio segment; and perform a smoothing process on the first audio segment to determine a second audio segment.

[0034] In some embodiments, the second determination module 220 is further configured to: determine initial interpolation parameters based on the original statistical characteristics of the original audio segment and the target sampling rate; and interpolate the original audio segment based on the initial interpolation parameters to determine the first audio segment.

[0035] In some embodiments, the second determination module 220 is further configured to: perform at least one first iterative update on the initial interpolation parameters, and interpolate the original audio segment based on the intermediate interpolation parameters after the first iterative update to determine the first intermediate segment until the first iteration completion condition is met; determine the first audio segment based on the first intermediate segment obtained from the at least one first iteration.

[0036] In some embodiments, the second determination module 220 is further configured to: determine an initial smoothing parameter based on the original statistical features; perform at least one round of second iterative update on the initial smoothing parameter, and smooth the first audio segment based on the intermediate smoothing parameter after the second iterative update to determine a second intermediate segment until the second iteration completion condition is met; determine the second audio segment based on the second intermediate segment obtained from the at least one round of second iteration.

[0037] The third determination module 230 is configured to determine the target audio based on at least one second audio segment corresponding to the at least one original audio segment.

[0038] In some embodiments, the first determination module 210 , the second determination module 220 , and the third determination module 230 may be integrated into the processor 130 .

[0039] For more details about the first determination module 210, the second determination module 220 and the third determination module 230, see Figure 3-Figure 5 and its related description.

[0040] It should be understood that Figure 2 The system and its modules shown can be implemented in various ways. It should be noted that the above description of the audio reconstruction system and its modules is only for the convenience of description and cannot limit this specification to the scope of the embodiments. It is understandable that for those skilled in the art, after understanding the principle of the system, it is possible to arbitrarily combine the modules or form a subsystem to connect with other modules without deviating from this principle. In some embodiments, Figure 2The first determination module, the second determination module, and the third determination module disclosed in the specification may be different modules in a system, or one module may realize the functions of two or more modules. For example, each module may share a storage module, or each module may have its own storage module. Such variations are within the protection scope of this specification.

[0041] Figure 3 is an exemplary flow chart of an audio reconstruction method according to some embodiments of this specification. Figure 3 As shown, process 300 includes the following steps. In some embodiments, Figure 3 One or more operations of the process 300 shown may be performed in Figure 1 The audio reconstruction system is implemented in the application scenario 100 shown. For example, Figure 3 The process 300 shown may be stored in a storage device in the form of instructions and called and / or executed by the processor 130 .

[0042] In some embodiments, the processor 130 may determine at least one original audio segment based on the audio to be processed; for an original audio segment, based on a target sampling rate, interpolate the original audio segment to determine a first audio segment; smooth the first audio segment to determine a second audio segment; and determine the target audio based on at least one second audio segment corresponding to at least one original audio segment.

[0043] Step 310 : Determine at least one original audio segment based on the audio to be processed. In some embodiments, step 310 may be performed by the first determination module 210 .

[0044] The audio to be processed refers to the audio to be processed. For example, the audio to be processed may be an audio file with a low sampling rate. Exemplarily, the low sampling rate may include 22.05kHZ, 44.1kHZ, 48kHZ, etc.

[0045] In some embodiments, the audio to be processed may be composed of sampled data obtained by sampling at an original sampling rate, wherein sampling is the process of converting a continuous signal into a discrete signal.

[0046] The sampling rate (or sampling frequency) refers to the number of times samples are taken per unit time. The higher the sampling rate, the more samples there are, the larger the amount of data is, and the closer it is to the original sound. In some embodiments, the sampling rate is generally limited by the sampling instrument.

[0047] The original sampling rate is the sampling rate of the audio to be processed. In some embodiments, the original sampling rate is a relatively low sampling rate. For example, the original sampling rate may be lower than the sampling rate threshold, and the sampling rate threshold may be set manually. Exemplarily, the sampling rate threshold may be 30kHZ, 40kHZ, 50kHZ, etc.

[0048] The original audio segment refers to the segment obtained by segmenting the audio to be processed.

[0049] In some embodiments, the processor 130 may divide the audio signal to be processed into multiple unit segments, obtain the audio features of each unit segment through spectrum analysis technology, determine the audio correlation between adjacent segments based on the audio features, and then determine multiple original audio segments based on the audio correlation.

[0050] A unit segment refers to an audio segment with a unit length (for example, the unit length may be 1 second). It is understandable that there is no overlap between multiple unit segments, and the adjacent multiple unit segments are combined to form an audio signal to be processed, and the audio of each unit segment is also composed of a corresponding portion of sampled data.

[0051] In some embodiments, spectrum analysis techniques may include, but are not limited to, Fourier transform, fast Fourier transform, and autocorrelation algorithm.

[0052] Audio features refer to specific features of audio. In some embodiments, audio features may include frequency spectrum and power spectrum. The frequency spectrum represents the amplitude distribution of the signal at different frequencies and describes the frequency characteristics of the signal; the power spectrum represents the energy distribution of each frequency component and describes the frequency energy distribution of the signal.

[0053] Audio correlation refers to a numerical value or letter that can reflect the similarity of audio. For example, audio correlation can be represented by a numerical value between 1 and 10, or letters a to f, or star rating. The larger the value, the larger the dictionary ranking, or the higher the star rating, the higher the audio correlation. It can be understood that there is an audio correlation between every two adjacent unit segments. For example, 99 audio correlations can be calculated for an audio signal to be processed with a total of 100 unit segments.

[0054] In some embodiments, the audio correlation can be obtained based on the weighted average of the spectral correlation and power spectrum correlation between adjacent unit segments. Among them, the spectral correlation and the power spectrum correlation can respectively characterize the similarity of the spectrum and the power spectrum of the two unit segments. In some embodiments, the processor 130 can use a correlation algorithm (such as one or more of waveform comparison, spectrum comparison, etc.) to determine the spectral correlation and the power spectrum correlation. The weight of the weighted average is an empirical preset value, for example, the weights can all be 0.5.

[0055] In some embodiments, the processor 130 may also use only one of the spectrum correlation and the power spectrum correlation to determine the audio correlation. For example, the processor 130 may determine the spectrum correlation or the power spectrum correlation as the audio correlation.

[0056] In some embodiments, the processor 130 divides two adjacent unit segments whose audio correlation is higher than or equal to a preset threshold into the same original audio segment. If the audio correlation between a unit segment and the two adjacent unit segments on the left and right are both lower than the preset threshold, the unit segment itself is regarded as an original audio segment. The preset threshold is determined manually.

[0057] For example, the audio correlation sequence is {0.9, 0.9, 0.8, 0.7, 0.6...}, which means that the audio correlation between unit segment 1 and unit segment 2 is 0.9, the audio correlation between unit segment 2 and unit segment 3 is 0.9... and so on; assuming the preset threshold is 0.8, it can be known that unit segments 1, 2, 3, and 4 will be divided into the same original audio segment, and the audio correlation between unit segment 4 and unit segment 5 is less than the preset threshold, then unit segment 5 will not be divided into the previous original audio segment. The audio correlation between unit segment 4 and unit segment 5 is 0.7, the audio correlation between unit segment 5 and unit segment 6 is 0.6, and the audio correlation between unit segment 5 and the two adjacent unit segments on the left and right are both lower than the preset threshold, then unit segment 5 itself is regarded as an original audio segment.

[0058] Step 320 : for one original audio segment, execute steps 321 to 322 . In some embodiments, step 320 may be executed by the second determining module 220 .

[0059] Step 321: Perform interpolation processing on the original audio segment based on the target sampling rate to determine the first audio segment.

[0060] The target sampling rate refers to the sampling rate of the audio segment that is expected to be obtained, that is, the sampling rate of the first audio segment obtained by interpolation. It can be understood that the target sampling rate is greater than the original sampling rate, and the target sampling rates of all segments are the same.

[0061] Interpolation processing is a method of calculating unknown sampling points from known sampling points to achieve sampling rate conversion. In some embodiments, interpolation processing includes but is not limited to linear interpolation, Lagrange interpolation, spline interpolation, etc.

[0062] The first audio segment is the first audio segment after the interpolation process is performed.

[0063] In some embodiments, the processor 130 may perform interpolation processing on the original audio segment based on the target sampling rate in various ways to determine the first audio segment.

[0064] For example, the processor 130 may determine initial interpolation parameters based on original statistical features of the original audio segment and a target sampling rate; and perform interpolation processing on the original audio segment based on the initial interpolation parameters to determine the first audio segment.

[0065] For another example, the processor 130 may determine initial interpolation parameters based on the original statistical features and target sampling rate of the original audio segment; perform at least one first iteration update on the intermediate interpolation parameters, and interpolate the original audio segment based on the intermediate interpolation parameters after the first iteration update to determine a first intermediate segment until the first iteration completion condition is met; and determine a first audio segment based on the first intermediate segment obtained from at least one first iteration.

[0066] For more information on determining the first audio segment, see Figure 4 and its related description.

[0067] Step 322: Smoothing the first audio segment to determine a second audio segment.

[0068] Smoothing refers to applying an algorithm to an audio signal to reduce disharmony in the audio so that the audio sounds smoother and more natural. In some embodiments, smoothing includes but is not limited to moving average filtering, Gaussian filtering, spline smoothing, and the like.

[0069] The second audio segment is an audio segment obtained by smoothing the first audio segment.

[0070] In some embodiments, the processor 130 can smooth the first audio segment in a variety of ways to determine the second audio segment. For example, the processor can directly smooth the first audio segment based on a default smoothing parameter to determine the second audio segment. The default smoothing parameter is a smoothing parameter pre-stored in the system. For more information about the smoothing parameter, see Figure 5 and related instructions.

[0071] In some embodiments, the processor 130 may determine an initial smoothing parameter based on the original statistical features; perform at least one second iteration update on the intermediate smoothing parameter, and perform smoothing on the first audio segment based on the intermediate smoothing parameter updated in the second iteration to determine a second intermediate segment until the second iteration completion condition is met; and determine the second audio segment based on the second intermediate segment obtained in at least one second iteration. For more information, see Figure 5 and its related description.

[0072] Step 330: determine a target audio based on at least one second audio segment corresponding to at least one original audio segment.

[0073] The target audio 120 is the audio obtained after processing. It can be understood that the sampling rate of the target audio 120 is higher than the original sampling rate.

[0074] In some embodiments, the processor 130 may determine the target audio based on at least one second audio segment corresponding to at least one original audio segment in a variety of ways. For example, since each second audio segment is obtained by interpolation and smoothing of the corresponding original audio segment, the order of the at least one second audio segment is consistent with the order of the corresponding at least one original audio segment. The processor 130 may combine the plurality of second audio segments in the order of the corresponding at least one original audio segment to obtain the target audio.

[0075] In some embodiments of the present specification, by dividing the audio to be processed into at least one original audio segment and performing interpolation and smoothing processing on each original audio segment, the audio clarity can be improved to make it more in line with the current listening requirements. In addition, by performing differentiated interpolation and smoothing processing on different original audio segments, the saturation of different types of audio can be increased.

[0076] It should be noted that the above description of process 300 is only for example and illustration, and does not limit the scope of application of this specification. For those skilled in the art, various modifications and changes can be made to process 300 under the guidance of this specification. However, these modifications and changes are still within the scope of this specification.

[0077] Figure 4 FIG. 1 is an exemplary flow chart of determining a first audio segment according to some embodiments of the present specification. Figure 4 As shown, process 400 includes the following steps. In some embodiments, Figure 4 One or more operations of the process 400 shown may be performed in Figure 1 The audio reconstruction system is implemented in the application scenario 100 shown. For example, Figure 4 The process 400 shown may be stored in a storage device in the form of instructions, and called and / or executed by the processor 130. In some embodiments, steps 410 to 450 may be executed by the second determination module 220.

[0078] In some embodiments, the processor 130 may determine initial interpolation parameters based on original statistical features of the original audio segment and a target sampling rate; and perform interpolation processing on the original audio segment based on the initial interpolation parameters to determine the first audio segment.

[0079] In some embodiments, the processor 130 may perform at least one round of first iterative update on the intermediate interpolation parameters, and interpolate the original audio segment based on the intermediate interpolation parameters after the first iterative update to determine the first intermediate segment until the first iteration completion condition is met, the intermediate interpolation parameters are the initial interpolation parameters in the first first iterative update, and are the intermediate interpolation parameters generated in the previous first iterative update in other first iterative updates; the first audio segment is determined based on the first intermediate segment obtained from at least one round of first iteration.

[0080] Step 410: determining initial interpolation parameters based on original statistical features of the original audio segment and a target sampling rate.

[0081] The original statistical features refer to the audio feature information of the original audio segment. For example, the original statistical features may include spectrum distribution, peak factor, number of sampling points in a segment, etc.

[0082] In some embodiments, the original statistical features may also include other features, such as the average amplitude, short-time energy, energy envelope, etc. of the original audio segment, which are not limited here. For more information about the original audio segment and the target sampling rate, see Figure 3 and related descriptions.

[0083] In some embodiments, the original statistical feature may be data in a vector form. For example, [A(0.2, 15; 3, 32; 17, 37; ...), 1.4, 27, ...] may indicate that the spectrum distribution of the original statistical feature is A(0.2, 15; 3, 32; 17, 37; ...), the peak factor is 1.4, the number of sampling points in the segment is 27, etc.

[0084] The spectrum distribution may refer to a set of information contained in the spectrum of the original audio segment. For example, the spectrum distribution may include multiple frequencies of the original audio segment and the corresponding amplitudes. The frequency distribution may be data in the form of a vector. For example, the spectrum distribution A (0.2, 15; 3, 32; 17, 37; ...) may indicate that the amplitudes corresponding to the frequencies of 0.2 kHz, 3 kHz, and 17 kHz are 15 dB, 32 dB, and 37 dB, respectively.

[0085] The crest factor refers to the ratio of the waveform peak value to the effective value of the original audio segment.

[0086] The number of sampling points in a segment refers to the number of points sampled in the original audio segment. In some embodiments, the processor 130 can determine the number of interpolation points in the interpolation parameter based on the number of sampling points in the segment and the target sampling rate. For more information on determining the number of interpolation points, see the relevant description below.

[0087] Average amplitude refers to the average value of the amplitudes corresponding to multiple frequencies in the original audio segment. Short-time energy refers to the energy accumulation value of an audio signal in a short time window (for example, a frame). Energy envelope refers to the value that reflects the energy fluctuation range of the audio signal of the original audio segment.

[0088] Interpolation parameters refer to the interpolation algorithm and its algorithm parameters related to the original audio segment interpolation process. For example, the interpolation algorithm may include linear interpolation, Lagrange interpolation, spline interpolation, etc. For example, for linear interpolation, the algorithm parameters may include the number of interpolation points, sampling rate, etc.; for spline interpolation, the algorithm parameters may include the number of interpolation points, spline order, etc.

[0089] The initial interpolation parameters refer to the interpolation parameters used when performing the first interpolation process on the original audio segment.

[0090] In some embodiments, the processor 130 may determine the initial interpolation parameters in a variety of ways. For example, the processor 130 may determine the segmentation type corresponding to the original statistical features based on the original statistical features, and determine the initial interpolation parameters based on the segmentation type and the target sampling rate. For an explanation of the target sampling rate, see Figure 3 and related descriptions.

[0091] The segment type refers to the category or type to which the original statistical feature belongs. For example, the segment type may include tenor, mezzo-soprano, children's voice, etc.

[0092] In some embodiments, the processor 130 may determine the segment type based on the original statistical features by various methods (eg, cluster analysis, searching a data table, etc.).

[0093] In some embodiments, the processor 130 can construct a historical clustering vector based on the original statistical features corresponding to multiple historical audio segments, and perform a clustering operation to obtain multiple clusters and corresponding cluster centers. Each cluster corresponds to a segment type. Among them, the segment type can be marked by the user according to the audio playback style, playback characteristics, etc., such as tenor, mezzo-soprano, children's voice, etc. The number of clusters can be set manually. For example, the historical clustering vector can be expressed as (spectral distribution, peak factor, number of sampling points in the segment, ...).

[0094] The processor 130 can construct a vector to be matched based on the original statistical features of the current original audio segment, and determine the similarity between the vector to be matched and the historical clustering vectors corresponding to each clustering center. The processor can use the segmentation type corresponding to the clustering center with the highest similarity (i.e., the smallest vector distance) as the segmentation type corresponding to the current original audio segment. The vector distance may include Euclidean distance, cosine distance, Mahalanobis distance, Chebyshev distance, Manhattan distance, etc. The vector to be matched can be expressed as (spectral distribution, peak factor, number of sampling points in the segment, ...).

[0095] In some embodiments, the processor 130 may determine the segment type in other ways, such as a machine learning model, etc., which is not limited here.

[0096] In some embodiments, the processor 130 may determine the interpolation algorithm and some algorithm parameters corresponding to the segmentation type according to the first preset correspondence relationship. Among them, some algorithm parameters refer to parameters other than the number of interpolation points. The first preset correspondence relationship refers to a data set pre-set by the user that characterizes the correspondence between the segmentation type and the interpolation algorithm and some algorithm parameters.

[0097] The first preset correspondence may include multiple segmentation types and their corresponding interpolation algorithms and some algorithm parameters. In some embodiments, the processor 130 may perform multiple interpolation experiments on historical audio segments corresponding to different cluster centers, and record the segmentation type corresponding to the cluster center, and the interpolation algorithm and some algorithm parameters with the best interpolation effect confirmed by the user as a first preset correspondence.

[0098] In some embodiments, the processor 130 may determine the number of interpolation points based on the number of sampling points in the segment in the original statistical features and the target sampling rate. For example, the processor 130 may determine that the sampling rate of the original audio segment is 48 kHz based on the number of sampling points in the segment. It is known that the target sampling rate is 192 kHz, which is four times the difference between the two. Therefore, three sampling points are inserted between two adjacent sampling points to obtain the target sampling rate, thereby obtaining the number of interpolation points. For more information about the target sampling rate, see Figure 3 and related descriptions.

[0099] Furthermore, the processor 130 may combine the interpolation algorithm, part of the algorithm parameters, and the number of interpolation points determined above to determine the initial interpolation parameters.

[0100] In some embodiments, the processor 130 may determine the initial interpolation parameters in other ways, such as a machine learning model, etc., which is not limited here.

[0101] Step 420: Perform a first iterative update on the intermediate interpolation parameters.

[0102] The first iteration refers to a process of determining a first audio segment that meets the requirements by repeatedly executing a set of steps and gradually improving the results. The first audio segment that meets the requirements refers to the first audio segment that is determined when the first iteration completion condition is met. For more information about the first iteration completion condition and determining the first audio segment, please refer to the following and related descriptions.

[0103] The first iterative update refers to updating the intermediate interpolation parameters through the first iteration. In some embodiments, the processor 130 may generate updated intermediate interpolation parameters by performing the first iterative update on the intermediate interpolation parameters.

[0104] In some embodiments, the processor may perform a first iterative update on the intermediate interpolation parameters in a variety of ways (eg, a preset algorithm, a machine learning model, etc.).

[0105] The intermediate interpolation parameters refer to the interpolation parameters in the first iterative update process. In some embodiments, the intermediate interpolation parameters may be related to the initial interpolation parameters. For example, the intermediate interpolation parameters are the initial interpolation parameters in the first first iterative update, and are the intermediate interpolation parameters generated in the last first iterative update in other first iterative updates.

[0106] In some embodiments, the processor 130 may determine an update parameter based on an interpolated audio feature and an adjacent audio feature, where the interpolated audio feature refers to an audio feature of an interpolation point, and the adjacent audio feature refers to an audio feature of a sampling point adjacent to the interpolation point; and determine an intermediate interpolation parameter based on the update parameter.

[0107] The interpolated audio feature refers to the audio feature of the interpolation point. For example, the interpolated audio feature may include frequency, amplitude, and crest factor. For more information about the crest factor, please refer to the above and its related description.

[0108] An interpolation point refers to a new sampling point inserted into a segment of audio. For example, an interpolation point may be included in the first interpolation point in the first intermediate segment obtained after the original audio segment is interpolated based on the initial interpolation parameters in the first first iterative update. For example, an interpolation point may be included in the interpolation point in the first intermediate segment obtained after the original audio segment is interpolated based on the interpolation parameters generated in the previous first iterative update in other first iterative updates. Among them, other first iterative updates refer to iterative updates other than the first first iterative update. For more information about the first intermediate segment, please refer to the following and its related description.

[0109] The adjacent audio feature refers to the audio feature of the actual sampling point adjacent to the interpolation point (for example, the two sampling points closest to the interpolation point). The content of the adjacent audio feature is similar to that of the interpolation audio feature, and will not be repeated here.

[0110] The update parameters refer to parameters related to updating the intermediate interpolation parameters. For example, the update parameters may include parameters related to changing the interpolation algorithm, changing the algorithm parameters (eg, sampling rate, etc.) of the interpolation algorithm.

[0111] In some embodiments, the processor 130 may determine the update parameter by analyzing the interpolated audio features and the adjacent audio features.

[0112] For example, the processor 130 may determine the update parameters through steps S1 to S3.

[0113] Step S1: judging unreasonable points and unreasonable features of the interpolated audio features of the interpolated points through interpolated audio features and adjacent audio features.

[0114] Unreasonable points refer to interpolation points that do not conform to common sense.

[0115] It is understandable that for the original audio segment, the time interval between two adjacent sampling points is also very short (for example, 0.05 seconds). In such a short time interval, it is difficult for the frequency of the audio segment to fluctuate dramatically. It can be considered that the audio features between the two sampling points are monotonically changing.

[0116] In some embodiments, if the interpolation audio feature of a certain interpolation point is between the adjacent audio features of the two adjacent sampling points before and after (i.e., it is a monotonic change), the processor 130 may determine the interpolation point as a reasonable point, otherwise, the processor 130 may determine the interpolation point as an unreasonable point. The interpolation audio feature of the interpolation point is between the adjacent audio features of the two adjacent sampling points before and after, which means that the interpolation audio feature of the interpolation point is greater than the minimum value of the adjacent audio features of the two adjacent sampling points before and after, and is smaller than the maximum value of the adjacent audio features of the two adjacent sampling points before and after.

[0117] For example, when the interpolated audio features include frequency, amplitude, and peak factor, the frequency of the interpolation point is greater than the minimum value of the frequencies of the two adjacent sampling points before and after, and smaller than the maximum value of the frequencies of the two adjacent sampling points before and after; the amplitude of the interpolation point is greater than the minimum value of the amplitudes of the two adjacent sampling points before and after, and smaller than the maximum value of the amplitudes of the two adjacent sampling points before and after; the peak factor of the interpolation point is greater than the minimum value of the peak factors of the two adjacent sampling points before and after, and smaller than the maximum value of the peak factors of the two adjacent sampling points before and after.

[0118] The unreasonable feature refers to the relationship feature between the interpolated audio feature of the unreasonable point and the reasonable audio interval. For example, the unreasonable feature may include the first distance between the interpolated audio feature of the unreasonable point and the reasonable audio interval.

[0119] A reasonable audio interval refers to an interval consisting of the maximum value of adjacent audio features of adjacent sampling points before and after the interpolation point as an upper limit and the minimum value as a lower limit.

[0120] The first distance refers to the distance or distance percentage between the interpolated audio feature of the unreasonable point and the upper limit or lower limit of the reasonable audio interval.

[0121] In some embodiments, the processor 130 may represent the first distance with a distance. For example, when the interpolated audio feature of an unreasonable point is greater than the upper limit of a reasonable audio interval, the interpolated audio feature of the unreasonable point is too large, and the processor 130 may use the distance between the interpolated audio feature of the unreasonable point and the upper limit of the reasonable audio interval as the first distance. For another example, when the interpolated audio feature of an unreasonable point is less than the lower limit of a reasonable audio interval, the interpolated audio feature of the unreasonable point is too small, and the processor 130 may use the distance between the interpolated audio feature of the unreasonable point and the lower limit of the reasonable audio interval as the first distance.

[0122] In some embodiments, the processor 130 may represent the first distance as a percentage of the distance. For example, when the interpolated audio feature of the unreasonable point is too large, the first distance = (interpolated audio feature - upper limit of the reasonable audio interval) / upper limit of the reasonable audio interval × 100%. For another example, when the interpolated audio feature of the unreasonable point is too small, the first distance = (interpolated audio feature - lower limit of the reasonable audio interval) / lower limit of the reasonable audio interval × 100%.

[0123] It can be understood that according to the above formula, if the interpolated audio feature of the unreasonable point is too large, and the interpolated audio feature is greater than the upper limit of the reasonable audio interval, the first distance is positive; if the interpolated audio feature of the unreasonable point is too small, and the interpolated audio feature is less than the lower limit of the reasonable audio interval, the first distance is negative.

[0124] Step S2: Based on the unreasonable points and unreasonable features, determine the update target of the audio segment.

[0125] The update target refers to the target that the audio feature corresponding to the updated intermediate interpolation parameter needs to achieve. For example, the update target can be to make the interpolated audio feature larger or smaller, and the magnitude of the increase or decrease of the interpolated audio feature (for example, +20%, etc.).

[0126] In some embodiments, the processor 130 may perform weighted summation on the unreasonable features (ie, the first distance) of a plurality of unreasonable points to determine an update target.

[0127] It is understandable that since the interpolation parameters are processing parameters for the entire audio segment (eg, the original audio segment, etc.), the interpolation parameters cannot be applied to adjust a certain interpolation point alone. Therefore, the processor 130 needs to consider the overall situation of the audio segment to comprehensively determine the update target.

[0128] In some embodiments, the processor 130 may determine the update target based on the positive or negative attribute of the value obtained by weighted summation. For example, if the value after weighted summation is positive (or negative), the update target is to increase (or decrease) the interpolated audio feature, and the increase (decrease) amplitude is the absolute value of the value obtained by weighted summation.

[0129] In some embodiments, when performing weighted summation, the smaller the absolute value of the unreasonable feature (ie, the first distance) of the unreasonable point is, the lower the weight of the unreasonable point is.

[0130] It can be understood that the smaller the absolute value of the unreasonable feature (i.e., the first distance) of the unreasonable point, the closer the unreasonable point is to the reasonable audio interval, and the smaller the impact is. Therefore, the processor 130 can give priority to adjusting the unreasonable points that are far away from the reasonable audio interval (i.e., the longer the distance, the greater the weight of the weighted summation).

[0131] It can be understood that since the interpolated audio features and the adjacent audio features can include at least one audio feature (e.g., frequency, amplitude, peak factor, etc.), for each audio feature, the processor 130 can execute steps S1-S2 to determine the update target corresponding to each audio feature, and determine the vector composed of at least one update target corresponding to at least one audio feature as the final update target. For example, if at least one update target is a frequency update target, an amplitude update target, and a peak factor update target, the final update target can be a vector: (frequency update target, amplitude update target, peak factor update target).

[0132] Step S3: Based on the intermediate interpolation parameters and the update target, the update parameters are determined by a first preset rule, and then the updated intermediate interpolation parameters are determined.

[0133] The first preset rule refers to a pre-set data set that characterizes the association relationship between the intermediate interpolation parameters, the update target and the update parameter. For example, the first preset rule may include a combination of multiple intermediate interpolation parameters and update targets, and the update parameters corresponding to the above combinations.

[0134] In some embodiments, the processor 130 may determine the first preset rule in a variety of ways. For example, the processor 130 may use a combination of multiple historical intermediate interpolation parameters and historical update targets in the historical experimental data, and historical update parameters corresponding to the multiple combinations as the first preset rule based on the historical experimental data.

[0135] In some embodiments, the processor 130 may determine the updated intermediate interpolation parameters by adjusting the interpolation algorithm of the intermediate interpolation parameters or adjusting the algorithm parameters of the interpolation algorithm according to the update parameters.

[0136] In some embodiments of the present specification, the processor 130 updates the intermediate interpolation parameters according to the interpolated audio features and the adjacent audio features, which can ensure that the updated intermediate interpolation parameters can meet the rationality requirements of each interpolation point, rather than just the audio segment as a whole meeting the interpolation requirements, thereby improving the quality of audio interpolation processing.

[0137] Step 430: perform interpolation processing on the original audio segment based on the intermediate interpolation parameters updated in the first iteration to determine a first intermediate segment.

[0138] The first intermediate segment refers to an audio segment obtained by processing the original audio segment based on the updated intermediate interpolation parameter. The content of the first intermediate segment is similar to that of the first audio segment, and will not be described in detail here.

[0139] Step 440, determining whether the first iteration completion condition is satisfied.

[0140] The first iteration completion condition refers to a condition for determining whether the first iteration is completed. For example, the number of iterations reaches a first preset number of times, the iterations converge, etc. The first preset number of times can be manually set, for example, 300 times, 500 times, 1000 times, etc. For another example, the first iteration completion condition can be that the difference between the first statistical feature of the first intermediate segment and the original statistical feature converges. For more information about the difference between the first statistical feature of the first intermediate segment and the original statistical feature, please refer to the following and its related description.

[0141] In some embodiments, the first iteration completion condition may include that a difference between a first statistical feature of the first intermediate segment and an original statistical feature is within a first preset range.

[0142] The first statistical feature refers to the audio feature of the audio segment after difference processing. For example, the audio feature of the first middle segment can be called the first statistical feature of the first middle segment. For another example, after the first iteration is completed, the audio feature of the first audio segment can also be called the first statistical feature of the first audio segment. The content of the first statistical feature is similar to that of the original statistical feature, and will not be repeated here.

[0143] The difference between the first statistical feature and the original statistical feature may include the difference in audio feature information (eg, spectrum distribution, crest factor, average amplitude, etc.) between the two.

[0144] In some embodiments, the processor 130 may calculate the similarity between the first statistical feature and the original statistical feature, and use the value obtained by subtracting the similarity from 1 as the difference between the first statistical feature and the original statistical feature. For example, the processor 130 may calculate the vector distance (e.g., cosine distance, etc.) between the vectors corresponding to the first statistical feature and the original statistical feature, and then generate the similarity through normalization. Normalization may refer to normalizing the vector distance to a value between 0 and 1.

[0145] The first preset range refers to a preset limit range value. For example, the first range value may be 0.05 to 0.2. The first range value may be preset by a user and input into the processor 130.

[0146] It is understandable that the difference between the first statistical feature and the original statistical feature cannot be too large or too small. If the difference is too large, overfitting may occur during the interpolation process, resulting in a large difference between the audio after interpolation and the original audio, thereby causing audio distortion; if the difference is too small, it means that the difference between the audio after interpolation and the original audio is very small, and the audio quality improvement effect is poor. Therefore, this difference needs to be limited to a certain range (for example, the first preset range).

[0147] In some embodiments, the first preset range may be related to the sampling rate of the original audio segment. For example, the greater the sampling rate of the original audio segment, the smaller the upper limit of the first preset range.

[0148] In some embodiments, the first preset range can be determined by looking up a range table. The range table includes sampling rates of different original audio segments and corresponding first preset ranges. The range table is determined by the processor 130 based on experimental data. For example, the processor 130 can construct a range table based on the historical sampling rates in the experimental data and the historical first preset range that ensures stable and undistorted audio quality.

[0149] The sampling rate of the original audio segment refers to the data acquisition frequency of the original audio segment, for example, 25 times / second.

[0150] It can be understood that the greater the sampling rate of the original audio segment, the closer the original statistical features are to the real audio features, and the smaller the upper limit of the first preset range is, thereby avoiding audio distortion caused by improper interpolation (for example, excessive interpolation, etc.).

[0151] In some embodiments of the present specification, the processor 130 can ensure the rationality and reliability of the judgment result related to the completion of iteration and ensure the quality of audio processing by reasonably dividing the first preset range.

[0152] In some embodiments, the processor 130 may determine whether the first iteration completion condition is satisfied in a variety of ways. For example, when the current number of iterations reaches or exceeds a first preset number of times, or when the difference between the first statistical feature of the first intermediate segment currently obtained and the original statistical feature is within a first preset range, the processor 130 may determine that the first iteration condition is satisfied.

[0153] In some embodiments, in response to the first iteration completion condition being met, the processor 130 may determine the first audio segment by executing step 450. In some embodiments, in response to the first iteration completion condition not being met, the processor 130 may use the updated intermediate interpolation parameter determined in step 420 in the current round of first iteration update as the intermediate interpolation parameter in step 420 in the next round of first iteration update, and then continue to execute steps 420-440.

[0154] Step 450: determine a first audio segment based on at least one first intermediate segment obtained in the first iteration.

[0155] In some embodiments, in response to the first iteration completion condition being that the difference between the first statistical feature of the first intermediate segment and the original statistical feature is within a first preset range, the processor 130 may use the first intermediate segment obtained in the first iteration when the first iteration completion condition is satisfied as the first audio segment.

[0156] In some embodiments, in response to the first iteration completion condition that the number of iterations reaches the first preset number, the processor 130 can use the first intermediate segment corresponding to the difference between the first statistical feature and the original statistical feature that is closest to the first preset range as the first audio segment according to the multiple first intermediate segments generated by the multiple iterations. The difference closest to the first preset range refers to the difference corresponding to the minimum value of the interpolation between the above difference and the upper limit of the upper limit of the first preset range and the interpolation between the above difference and the lower limit of the lower limit of the first preset range.

[0157] In some embodiments, the first iteration completion condition is that the difference between the first statistical feature of the first intermediate segment and the original statistical feature is within the first preset range, and the number of iterations reaches the first preset number. When the number of iterations does not reach the first preset number, and the difference between the first statistical feature of the first intermediate segment and the original statistical feature is within the first preset range, the first iteration stops, and the processor 130 can use the first intermediate segment with the difference within the first preset range as the first audio segment. When the number of iterations reaches the first preset number and still cannot satisfy the above difference within the first preset range, the first iteration stops, and the processor 130 can use the first intermediate segment corresponding to the difference between the first statistical feature and the original statistical feature that is closest to the first preset range as the first audio segment.

[0158] In some embodiments of the present specification, the processor 130 can ensure the sampling rate of the audio data and the audio quality by performing reasonable interpolation processing on the original audio data. The processor 130 can avoid the problem of audio quality degradation or instability caused by excessive interpolation by dynamically updating the interpolation parameters to try to keep the statistical characteristics of the first audio segment to meet the first iteration completion condition.

[0159] In some embodiments of the present specification, the processor 130 can improve the applicability of the interpolation processing and the pertinence of the interpolation parameters by dynamically updating the interpolation parameters of the audio interpolation processing process, thereby further improving the audio processing quality.

[0160] It should be noted that the above description of process 400 is only for example and explanation, and does not limit the scope of application of this specification. For those skilled in the art, various modifications and changes can be made to process 400 under the guidance of this specification. However, these modifications and changes are still within the scope of this specification.

[0161] In some embodiments, steps 420 to 450 may be omitted. After executing step 410 and determining the initial interpolation parameters, the processor may interpolate the original audio segment based on the initial interpolation parameters to determine the first audio segment. The process of determining the first audio segment is similar to the process of determining the first intermediate segment, and will not be repeated here.

[0162] Figure 5 FIG. 1 is an exemplary flow chart of determining a second audio segment according to some embodiments of the present specification. Figure 5 As shown, process 500 includes the following steps. In some embodiments, Figure 5 One or more operations of the process 500 shown may be performed in Figure 1 The audio reconstruction system is implemented in the application scenario 100 shown. For example, Figure 5 The process 500 shown may be stored in a storage device in the form of instructions, and called and / or executed by the processor 130. In some embodiments, steps 510 to 550 may be executed by the second determination module 220.

[0163] In some embodiments, the processor 130 can determine an initial smoothing parameter based on the original statistical features; perform at least one round of second iterative update on the intermediate smoothing parameter, and smooth the first audio segment based on the intermediate smoothing parameter after the second iterative update to determine the second intermediate segment until the second iteration completion condition is met, the intermediate smoothing parameter is the initial smoothing parameter in the first second iterative update, and is the intermediate smoothing parameter generated in the previous second iterative update in other second iterative updates; determine the second audio segment based on the second intermediate segment obtained from at least one round of second iteration.

[0164] Step 510: Determine initial smoothing parameters based on the original statistical characteristics.

[0165] The smoothing parameter refers to the smoothing algorithm and its algorithm parameters used when smoothing the first audio segment. For example, the smoothing algorithm may include moving average filtering, Gaussian filtering, spline smoothing, etc. For example, for moving average filtering, the algorithm parameters may include window size, etc.; for Gaussian filtering, the algorithm parameters may include window size, standard deviation, etc.

[0166] The initial smoothing parameter refers to a smoothing parameter used when performing the first smoothing process on the first audio segment.

[0167] In some embodiments, the processor 130 may determine the initial smoothing parameter according to the original statistical features through various methods (eg, algorithm, statistical analysis, clustering, etc.).

[0168] For example, the processor 130 may determine the initial smoothing parameter by querying the smoothing data table according to the segmentation type corresponding to the original statistical feature. The smoothing data table includes multiple segmentation types and initial smoothing parameters corresponding to the segmentation types. For more information about segmentation types, see Figure 4 and related descriptions.

[0169] In some embodiments, the smoothing data table can be constructed by the processor 130 according to the experimental data. For example, the processor 130 can construct the smoothing data table according to the historical segmentation types corresponding to the multiple original statistical features in the experimental data and the historical smoothing parameters that the user determines to have better processing effects.

[0170] It can be understood that the interpolation process does not change the segment type. The processor 130 directly determines the initial smoothing parameter through the second preset correspondence based on the segment type determined in step 410. This embodiment directly applies the segment type determined in step 410, which can improve the audio processing efficiency to a certain extent. The determination method of the second preset correspondence is similar to the determination method of the first preset correspondence, and will not be repeated here.

[0171] In some embodiments, the processor 130 may determine the initial smoothing parameter through steps P1 to P3.

[0172] Step P1: Calculate the audio difference between the first statistical feature of the adjacent audio segments and the first statistical feature of the first audio segment.

[0173] The adjacent audio segment refers to an audio segment that is adjacent to the first audio segment and has completed the smoothing process.

[0174] The audio difference includes the difference between the first statistical feature of the first audio segment and the first statistical feature of the adjacent audio segment. For example, the audio difference may include the difference in frequency, amplitude, peak factor, etc. For more information about the first statistical feature, see Figure 4 and related descriptions.

[0175] In some embodiments, the processor 130 may generate an audio difference value based on a difference percentage between the first statistical feature of the first audio segment and the first statistical feature of the adjacent audio segment. It can be understood that if the first statistical feature of the first audio segment is greater than that of the adjacent audio segment, the audio difference value may be a positive number; otherwise, the audio difference value may be a negative number.

[0176] It can be understood that, since the first statistical feature can include at least one audio feature (e.g., frequency, amplitude, peak factor, etc.), for an audio feature, by executing step P1, the difference percentage corresponding to the audio feature can be determined, and a vector consisting of at least one difference percentage corresponding to at least one audio feature is determined as the final audio difference. For example, if at least one difference percentage is a frequency difference percentage, an amplitude difference percentage, and a peak factor difference percentage, the final audio difference can be a vector: (frequency difference percentage, amplitude difference percentage, peak factor difference percentage).

[0177] Step P2: Calculate the difference between the second statistical feature of the adjacent audio segments and the original statistical feature, and the first position relationship between the difference and the second preset range.

[0178] The second statistical feature refers to the audio feature of the audio segment after smoothing. For example, the audio feature of the second intermediate segment can be called the second statistical feature of the second intermediate segment. For another example, when the second iteration is completed, the audio feature of the second audio segment can also be called the second statistical feature of the second audio segment. For another example, the audio feature of an adjacent audio segment can also be called the second statistical feature of an adjacent audio segment. The content of the second statistical feature is similar to that of the original statistical feature and will not be repeated here. For more information about the second intermediate segment and the second iteration, please refer to the following text and its related description. For more information about the second audio segment, please refer to Figure 3 and its related description.

[0179] The first position relationship refers to: the positional relationship between the difference between the second statistical feature and the original statistical feature of adjacent audio segments and the second preset range.

[0180] For example, the first positional relationship may include that the difference between the second statistical feature of the adjacent audio segments and the original statistical feature is within the second preset range, and the difference between the second statistical feature of the adjacent audio segments and the original statistical feature is outside the second preset range. The second preset range may be a numerical range preset by the user. The second preset range may be represented by Arabic numerals, mathematical symbols, etc. For example, the second preset range may be represented as [0.2, 0.5], [15%, 25%], etc. The form of the second preset range is similar to that of the first preset range, and will not be repeated here.

[0181] In some embodiments, if the difference between the second statistical feature of the adjacent audio segment and the original statistical feature is within a second preset range, the first position relationship may include a relative position value of the difference between the second statistical feature of the adjacent audio segment and the original statistical feature within the second preset range. The relative position value refers to the ratio of the distance between the difference between the second statistical feature of the adjacent audio segment and the original statistical feature and the closest upper or lower limit of the interval of the second preset range (for example, [15%, 25%]) to the length of the interval of the second preset range. The relative position value can be expressed as a fraction or percentage.

[0182] For example, the second preset range is [15%, 30%], and the difference between the second statistical feature and the original statistical feature of the adjacent audio segment is 20%, then the relative position value can be expressed as (20%-15%) / (30%-15%)=1 / 3≈33.3%.

[0183] In some embodiments, if the difference between the second statistical feature and the original statistical feature of the adjacent audio segments is outside the second preset range, the first position relationship may include a second distance between the difference between the second statistical feature and the original statistical feature of the adjacent audio segments and the second preset range.

[0184] The second distance refers to the distance or distance percentage between the difference between the second statistical feature of the adjacent audio segment and the original statistical feature and the upper limit or lower limit of the second preset range. The second distance is determined in a similar manner to the first distance, which will not be repeated here.

[0185] It can be understood that if the difference between the second statistical feature and the original statistical feature of a certain adjacent audio segment is too large, the second distance is the distance between the difference and the upper limit of the second preset range, and the first distance is positive (for example, +20%); if the difference between the second statistical feature and the original statistical feature of a certain adjacent audio segment is too small, the second distance is the distance between the difference between the second statistical feature and the original statistical feature of the adjacent audio segment and the lower limit of the second preset range, and the first distance is negative (for example, -15%).

[0186] Step P3: Determine an initial smoothing parameter based on the audio difference, the first position relationship, and the second preset rule.

[0187] The second preset rule refers to a pre-set data set that characterizes the association relationship between different audio differences, first position relationships, and first smoothing parameter adjustment parameters. The second preset rule may include multiple sub-rules, each of which may correspond to an association relationship between an audio difference, a first position relationship, and a first smoothing parameter adjustment parameter. For example, the second preset rule may include multiple audio differences and first position relationships, as well as first smoothing parameter adjustment parameters corresponding to the audio differences and the first position relationships (for example, the adjustment direction and adjustment amplitude of the smoothing algorithm parameters in the smoothing algorithm). Among them, the first smoothing parameter adjustment parameters may include the adjustment direction, adjustment amplitude, or adjustment amount of the smoothing parameters relative to adjacent audio segments.

[0188] In some embodiments, the second preset rule may be in a form including but not limited to a vector database, a function, a data table, etc.

[0189] In some embodiments, the processor 130 may perform parameter adjustment tests on different first smoothing parameter adjustment parameters, and determine the second preset rule according to the adjusted smoothing processing results.

[0190] For example, for multiple audio differences, first position relationships, and adjustment parameters of smoothing parameters that adjust adjacent audio segments multiple times, the adjustment direction and adjustment amplitude (i.e., the first smoothing parameter adjustment parameter) corresponding to the second audio segment with the best user-confirmed effect obtained after smoothing, as well as the audio difference and the first position relationship, are recorded as a sub-rule in the second preset rule.

[0191] For another example, for multiple audio differences, first positional relationships, and adjustment parameters for adjusting the smoothing parameters of adjacent audio segments multiple times, if the adjusted smoothing parameters can make the second audio segment obtained by smoothing satisfy the second iteration completion condition, then the adjustment direction and adjustment amplitude (i.e., the first smoothing parameter adjustment parameter), as well as the audio difference and the first positional relationship, are recorded as a sub-rule. For an explanation of the second iteration completion condition, see the relevant content below.

[0192] In some embodiments, the processor 130 may determine the first smoothing parameter adjustment parameter based on the audio difference and the first position relationship according to the second preset rule. Further, the processor 130 may adjust the smoothing parameters of the adjacent audio segments according to the first smoothing parameter adjustment parameter to obtain the initial smoothing parameter.

[0193] The initial smoothing parameters determined through steps P1 to P3 are adjusted based on the smoothing parameters of adjacent audio segments, which can avoid the blindness of starting the smoothing process completely from scratch and improve the efficiency of the smoothing process.

[0194] In some embodiments, the processor 130 may determine the initial smoothing parameter in other ways. For example, the processor 130 may obtain the difference between the original statistical feature and the first statistical feature, and determine the initial smoothing parameter by querying a data table, etc., which is not limited here.

[0195] Step 520: Perform a second iterative update on the intermediate smoothing parameters.

[0196] The second iteration refers to a process of determining a second audio segment that meets the requirements by repeatedly executing a set of steps and gradually improving the results. The second audio segment that meets the requirements refers to the second audio segment determined when the second iteration completion condition is met. For more information about the second iteration completion condition and determining the second audio segment, please refer to the following and related descriptions.

[0197] The second iterative update refers to updating the intermediate smoothing parameter through the second iteration. In some embodiments, the processor 130 may generate an updated intermediate smoothing parameter by performing a second iterative update on the intermediate smoothing parameter.

[0198] The intermediate smoothing parameter refers to the smoothing parameter in the second iterative update process. In some embodiments, the intermediate smoothing parameter may be related to the initial smoothing parameter. For example, the intermediate smoothing parameter is the initial smoothing parameter in the first second iterative update, and is the intermediate smoothing parameter generated in the last second iterative update in other second iterative updates.

[0199] In some embodiments, performing a second iteration update on the intermediate smoothing parameter may include: the processor 130 may only update the smoothing algorithm parameters of the intermediate smoothing parameter without changing the smoothing algorithm. For example, the smoothing algorithm parameters are updated only by adjusting the window size multiple times (e.g., increasing, decreasing, etc.). In some embodiments, if the second iteration completion condition is still not met after the iteration reaches a preset number of iterations (e.g., 100 times), the processor 130 may execute instructions for changing the smoothing algorithm of the intermediate smoothing parameter and changing the smoothing algorithm parameters. The preset number of iterations may be pre-set by the user and input into the processor 130.

[0200] In some embodiments, performing a second iterative update on the intermediate smoothing parameter may include: the processor 130 may determine an updated intermediate smoothing parameter based on the second statistical features and the original statistical features of the adjacent audio segment, where the adjacent audio segment refers to an audio segment that is adjacent to the first audio segment and has completed smoothing processing.

[0201] In some embodiments, the processor 130 may calculate the audio difference between the first statistical feature of the adjacent audio segment and the first statistical feature of the first audio segment based on step P1. Then, the processor may calculate the difference between the second statistical feature of the second intermediate segment and the original statistical feature and the second position relationship of the second preset range based on step P2. The processor 130 may determine the updated intermediate smoothing parameter based on the above audio difference, the second position relationship, and the third preset rule based on step P3.

[0202] The second intermediate segment refers to an audio segment after the first audio segment is processed based on the intermediate smoothing parameter. The third preset rule refers to a pre-set data set that characterizes the association relationship between different audio differences, second position relationships, and second smoothing parameter adjustment parameters. For example, the third preset rule may include multiple audio differences and first position relationships, and second smoothing parameter adjustment parameters corresponding to the audio differences and the first position relationships, wherein the second smoothing parameter adjustment parameters are the adjustment direction and adjustment amplitude of the smoothing parameters relative to the second intermediate segment.

[0203] The process of determining the second position relationship is similar to that of determining the first position relationship, and the process of determining the third preset rule is similar to that of determining the second preset rule, which will not be described in detail here.

[0204] Step 530: Smoothing the first audio segment based on the intermediate smoothing parameter updated in the second iteration to determine a second intermediate segment.

[0205] The second intermediate segment refers to an audio segment obtained by processing the first audio segment based on the updated intermediate smoothing parameter. The content of the second intermediate segment is similar to that of the second audio segment, and will not be described in detail here.

[0206] In some embodiments, the processor 130 may process the first audio segment based on the intermediate smoothing parameter to determine the second intermediate segment.

[0207] Step 540, determining whether the second iteration completion condition is satisfied.

[0208] The second iteration completion condition refers to a condition for determining whether the second iteration is completed. For example, the second iteration number reaches the second preset number, the iteration converges, etc. The second preset number can be manually set, for example, 300 times, 500 times, 1000 times, etc. For another example, the difference between the second statistical feature of the second intermediate segment and the original statistical feature converges. For more information about the difference between the second statistical feature of the second intermediate segment and the original statistical feature, please refer to the following and its related description.

[0209] In some embodiments, the second iteration completion condition may include: a difference between the second statistical feature of the second intermediate segment and the original statistical feature satisfies a second preset range.

[0210] The difference between the second statistical feature and the original statistical feature may include the difference in audio feature information (eg, spectrum distribution, crest factor, average amplitude, etc.) between the two.

[0211] In some embodiments, the processor 130 may calculate the similarity between the second statistical feature and the original statistical feature, and use the difference between a preset value (eg, 1) and the similarity as the difference value between the second statistical feature and the original statistical feature.

[0212] The method for determining the difference between the second statistical feature and the original statistical feature is similar to the method for determining the difference between the first statistical feature and the original statistical feature, and will not be described in detail here.

[0213] The second preset range refers to a preset limit range value related to the difference between the second statistical feature and the original statistical feature. For example, the second range value may be 0.05 to 0.2. The processor 130 may obtain a range value preset and input by the user to determine the second range value.

[0214] In some embodiments, the second preset range may be related to the number of unreasonable points in the first audio segment. For example, the greater the number of unreasonable points, the greater the lower limit of the second preset range. For example, the processor 130 may generate the second preset range based on the number of unreasonable points multiplied by the preset point coefficient. The processor 130 may obtain the coefficient value preset and input by the user to determine the preset point coefficient. For more information about unreasonable points, see Figure 4 and related descriptions.

[0215] In some embodiments, the processor 130 may control the interval length of the second preset range to remain unchanged, and after the lower limit of the second preset range is adjusted (for example, increased by 0.02), the upper limit is also adjusted accordingly (for example, increased by 0.02).

[0216] It is understandable that the greater the number of unreasonable points, the less reliable the processor 130 is in adjusting the interpolation parameters, which is more likely to result in a relatively large difference between the first statistical feature of the first audio segment and the original statistical feature. Therefore, by dynamically adjusting the second preset range, the processor 130 can be prevented from performing excessive smoothing to meet the second iteration completion condition, thereby avoiding the problem of audio distortion.

[0217] In some embodiments, the processor 130 may determine whether the second iteration completion condition is satisfied in a variety of ways. For example, when the current second iteration number reaches or exceeds the second preset number, or when the difference between the second statistical feature of the second intermediate segment currently obtained and the original statistical feature satisfies the second preset range, the processor 130 may determine that the second iteration condition is satisfied.

[0218] In some embodiments, in response to the second iteration completion condition being met, the processor 130 may determine the second audio segment by executing step 550. In some embodiments, in response to the second iteration completion condition not being met, the processor 130 may use the updated intermediate smoothing parameter determined in step 520 in the current round of second iteration update as the intermediate smoothing parameter in step 520 in the next round of second iteration update, and then continue to execute steps 520-540.

[0219] Step 550: Determine a second audio segment based on a second intermediate segment obtained by at least one second iteration.

[0220] In some embodiments, the process of the processor 130 determining the second audio segment is similar to the process of determining the first audio segment, which is not described again.

[0221] In some embodiments of the present specification, based on the processor 130 performing differentiated smoothing processing on different audio segments, the negative impact of audio interpolation processing on audio quality can be reduced, thereby further ensuring audio quality.

[0222] It should be noted that the above description of process 500 is only for example and explanation, and does not limit the scope of application of this specification. For those skilled in the art, various modifications and changes can be made to process 500 under the guidance of this specification. However, these modifications and changes are still within the scope of this specification.

[0223] In some embodiments, steps 520 to 550 may be omitted. After executing step 510 and determining the initial smoothing parameter, the processor may smooth the original audio segment based on the initial smoothing parameter to determine the second audio segment. The process of determining the second audio segment is similar to the process of determining the second intermediate segment, and will not be repeated here.

[0224] In some embodiments, an audio reconstruction device is provided, comprising a processor, wherein the processor is configured to execute the above-mentioned audio reconstruction method.

[0225] In some embodiments, a computer-readable storage medium is involved for storing computer instructions. When a computer reads the computer instructions in the storage medium, the computer runs the above-mentioned audio reconstruction method.

[0226] The beneficial effects that may be brought about by the embodiments of this specification include but are not limited to: (1) The audio clarity can be improved through oversampling rate reconstruction, which better meets the user's hearing needs. (2) Differentiated interpolation and smoothing processing are performed on different audio segments to increase the saturation of different types of audio and improve audio quality.

[0227] The basic concepts have been described above. Obviously, for those skilled in the art, the above detailed disclosure is only for example and does not constitute a limitation of this specification. Although not explicitly stated here, those skilled in the art may make various modifications, improvements and corrections to this specification. Such modifications, improvements and corrections are suggested in this specification, so such modifications, improvements and corrections still belong to the spirit and scope of the exemplary embodiments of this specification.

[0228] At the same time, this specification uses specific words to describe the embodiments of this specification. For example, "one embodiment", "an embodiment", and / or "some embodiments" refer to a certain feature, structure or characteristic related to at least one embodiment of this specification. Therefore, it should be emphasized and noted that "one embodiment" or "an embodiment" or "an alternative embodiment" mentioned twice or more in different positions in this specification does not necessarily refer to the same embodiment. In addition, certain features, structures or characteristics in one or more embodiments of this specification can be appropriately combined.

[0229] In addition, unless explicitly stated in the claims, the order of the processing elements and sequences described in this specification, the use of alphanumeric characters, or the use of other names are not intended to limit the order of the processes and methods of this specification. Although the above disclosure discusses some invention embodiments that are currently considered useful through various examples, it should be understood that such details are only for illustrative purposes, and the attached claims are not limited to the disclosed embodiments. On the contrary, the claims are intended to cover all modifications and equivalent combinations that are consistent with the essence and scope of the embodiments of this specification. For example, although the system components described above can be implemented by hardware devices, they can also be implemented only by software solutions, such as installing the described system on an existing server or mobile device.

[0230] Similarly, it should be noted that in order to simplify the description disclosed in this specification and thus help understand one or more embodiments of the invention, in the above description of the embodiments of this specification, multiple features are sometimes combined into one embodiment, figure or description thereof. However, this disclosure method does not mean that the features required by the subject matter of this specification are more than the features mentioned in the claims. In fact, the features of the embodiments are less than all the features of the single embodiment disclosed above.

[0231] In some embodiments, numbers describing the number of components and attributes are used. It should be understood that such numbers used in the description of the embodiments are modified by the modifiers "about", "approximately" or "substantially" in some examples. Unless otherwise specified, "about", "approximately" or "substantially" indicate that the numbers are allowed to vary by ±20%. Accordingly, in some embodiments, the numerical parameters used in the specification and claims are approximate values, which may change according to the required features of individual embodiments. In some embodiments, the numerical parameters should take into account the specified significant digits and adopt the general method of retaining digits. Although the numerical domains and parameters used to confirm the breadth of their range in some embodiments of this specification are approximate values, in specific embodiments, the setting of such numerical values ​​is as accurate as possible within the feasible range.

[0232] Each patent, patent application, patent application publication, and other materials, such as articles, books, specifications, publications, documents, etc., cited in this specification are hereby incorporated by reference in their entirety. Except for application history documents that are inconsistent with or conflicting with the contents of this specification, documents that limit the broadest scope of the claims of this specification (currently or later attached to this specification) are also excluded. It should be noted that if the descriptions, definitions, and / or use of terms in the materials attached to this specification are inconsistent or conflicting with the contents described in this specification, the descriptions, definitions, and / or use of terms in this specification shall prevail.

[0233] Finally, it should be understood that the embodiments described in this specification are only used to illustrate the principles of the embodiments of this specification. Other variations may also fall within the scope of this specification. Therefore, as an example and not a limitation, alternative configurations of the embodiments of this specification may be considered consistent with the teachings of this specification. Accordingly, the embodiments of this specification are not limited to the embodiments explicitly introduced and described in this specification.

Claims

1. An audio reconstruction method, characterized in that: include: Based on the audio to be processed, determining at least one original audio segment; For an original audio segment, Based on the target sampling rate, interpolate the original audio segment to determine a first audio segment; Performing smoothing processing on the first audio segment to determine a second audio segment; A target audio is determined based on at least one second audio segment corresponding to the at least one original audio segment.

2. The method according to claim 1, characterized in that: The interpolation process is performed on the original audio segment based on the target sampling rate to determine the first audio segment, which includes: Determining initial interpolation parameters based on original statistical features of the original audio segment and the target sampling rate; Based on the initial interpolation parameters, an interpolation process is performed on the original audio segment to determine the first audio segment.

3. The method according to claim 2, characterized in that The interpolation processing is performed on the original audio segment based on the initial interpolation parameter to determine the first audio segment comprises: Performing at least one first iterative update on the intermediate interpolation parameters, and interpolating the original audio segment based on the intermediate interpolation parameters updated by the first iterative update to determine the first intermediate segment until the first iterative completion condition is satisfied, the intermediate interpolation parameters being related to the initial interpolation parameters; The first audio segment is determined based on the first intermediate segment obtained by the at least one round of first iteration.

4. The method according to claim 1, characterized in that: The smoothing process on the first audio segment to determine the second audio segment comprises: Based on the original statistical characteristics, determine the initial smoothing parameters; Performing at least one second iterative update on the intermediate smoothing parameter, and smoothing the first audio segment based on the intermediate smoothing parameter after the second iterative update to determine a second intermediate segment until a second iteration completion condition is satisfied, the intermediate smoothing parameter being related to the initial smoothing parameter; The second audio segment is determined based on the second intermediate segment obtained by the at least one second iteration.

5. An audio reconstruction system, characterized in that: include: A first determination module is configured to determine at least one original audio segment based on the audio to be processed; A second determination module is configured to perform interpolation processing on an original audio segment based on a target sampling rate to determine a first audio segment; Performing smoothing processing on the first audio segment to determine a second audio segment; The third determination module is configured to determine the target audio based on at least one second audio segment corresponding to the at least one original audio segment.

6. The system according to claim 5, characterized in that The second determining module is further configured to: Determining initial interpolation parameters based on original statistical features of the original audio segment and the target sampling rate; Based on the initial interpolation parameters, an interpolation process is performed on the original audio segment to determine the first audio segment.

7. The system according to claim 6, characterized in that The second determining module is further configured to: Performing at least one first iterative update on the intermediate interpolation parameters, and interpolating the original audio segment based on the intermediate interpolation parameters updated by the first iterative update to determine the first intermediate segment until the first iterative completion condition is satisfied, the intermediate interpolation parameters being related to the initial interpolation parameters; The first audio segment is determined based on the first intermediate segment obtained by the at least one round of first iteration.

8. The system according to claim 5, characterized in that The second determining module is further configured to: Based on the original statistical characteristics, determine the initial smoothing parameters; Performing at least one second iterative update on the intermediate smoothing parameter, and smoothing the first audio segment based on the intermediate smoothing parameter after the second iterative update to determine a second intermediate segment until a second iteration completion condition is satisfied, the intermediate smoothing parameter being related to the initial smoothing parameter; The second audio segment is determined based on the second intermediate segment obtained by the at least one second iteration.

9. An audio reconstruction device, comprising a processor, characterized in that: The processor is used to execute the audio reconstruction method according to any one of claims 1 to 4.

10. A computer-readable storage medium, wherein the storage medium stores computer instructions. When a computer reads the computer instructions in the storage medium, the computer executes the audio reconstruction method according to any one of claims 1 to 4.

Images