Audio data processing device, inference device, and audio data processing method

Abstract

To apply appropriate correction with respect to audio data in which a compression system before being extended and information about a bit rate are not included.SOLUTION: An audio data processing device 20 comprises: an estimate unit 21 in which extended audio data in which compression audio data are extended are inputted to a learning model lm1 which has learned relation between input data IND including extended audio data in which the plurality of pieces of compression audio data are extended, respectively, and output data OTD including a type of a compressed format of the plurality of pieces of compression audio data, and which estimates the type of the compressed format of the compression audio data; and an estimated result use unit 24 which performs processing according to the type of the estimated compressed format.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to an audio data processing device, an inference device, and an audio data processing method. [Background technology]

[0002] The compression method and bit rate of audio data input to an audio device often differ for each input audio data, depending on communication line conditions, application settings, etc. Since the degree of degradation of audio data varies depending on the compression method and bit rate, it is desirable for the audio device to apply appropriate correction to the input audio data according to each compression method and bit rate to achieve high sound quality. For example, Patent Document 1 discloses an audio playback device that determines the audio data method from the source medium, header, etc. of the audio data, and generates a complementary audio signal to be added to the audio signal. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 4992621 Summary of the Invention [Problem to be solved by the invention]

[0004] However, audio devices may receive audio data that has already been decompressed by a codec. In this case, the audio data does not contain information about the compression method and bit rate before being decompressed. Therefore, conventional audio devices such as those disclosed in Patent Document 1 have a problem in that they are unable to apply appropriate correction to the input audio data in accordance with the compression method and bit rate of the audio data.

[0005] Taking the above circumstances into consideration, one aspect of the present disclosure aims to perform processing according to the compression format of input audio data, even if information regarding the compression method and bit rate before decompression is not included. [Means for solving the problem]

[0006] In order to solve the above problems, an audio data processing device according to one aspect of the present disclosure includes an estimation unit that estimates the type of compression format of compressed audio data by inputting decompressed audio data obtained by decompressing multiple pieces of compressed audio data into a learning model that has learned the relationship between input data including decompressed audio data obtained by decompressing each piece of compressed audio data and output data including the types of compression formats of the multiple pieces of compressed audio data, and an estimation result utilization unit that performs processing in accordance with the estimated type of compression format.

[0007] An audio data processing method according to one aspect of the present disclosure estimates the type of compression format of the compressed audio data by inputting the expanded audio data into a learning model that has learned the relationship between input data including expanded audio data obtained by expanding each of a plurality of compressed audio data and output data including the types of compression formats of the plurality of compressed audio data, and then performs processing according to the estimated type of compression format. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a configuration diagram showing an example of an acoustic system including an audio data processing device according to a first embodiment. [Figure 2] FIG. 1 is a block diagram illustrating an example of a machine learning device according to a first embodiment. [Figure 3] FIG. 2 is a schematic diagram illustrating an example of a neural network model applied to a learning model according to the first embodiment. [Figure 4] FIG. 1 is a diagram showing an example of a compression bit rate and a compression method in a typical music distribution service. [Figure 5] 2 is a diagram showing an example of a classification result by the correction parameter determination unit in FIG. 1. FIG. [Figure 6] FIG. 10 is a diagram illustrating the relationship between compression formats and the upper limit frequency of audio data in the compression formats. [Figure 7] 2 is a frequency spectrum diagram of audio data for explaining the interpolation processing performed by the correction processing unit of FIG. 1. FIG. [Figure 8] 10 is a flowchart illustrating an example of a processing operation of the audio data processing device. DETAILED DESCRIPTION OF THE INVENTION

[0009] A: First embodiment A1: Configuration of audio data processing device 1 is a configuration diagram showing an example of an audio system 1 including an audio data processing device 20 according to the first embodiment. The audio system 1 is mounted on a vehicle such as an automobile. As shown in FIG. 1, the audio system 1 includes a storage device 10, an audio data processing device 20, an amplifier device 30, and a speaker device 40.

[0010] In the acoustic system 1, an audio data processing device 20 estimates the compression format of input audio data d1 from the input audio data d1 input from a sound source 2, and corrects the input audio data d1 based on the estimated compression format. In the acoustic system 1, an amplifier device 30 converts the corrected audio data e1 from a digital signal to an analog signal, and then amplifies the converted analog signal. In the acoustic system 1, a speaker device 40 emits sound based on an amplified audio signal e2.

[0011] The storage device 10 is a computer-readable recording medium (e.g., a non-transitive recording medium readable by a computer). The storage device 10 includes a non-volatile memory and a volatile memory. The non-volatile memory is, for example, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), and an electrically erasable programmable read-only memory (EEPROM). The volatile memory is, for example, a random access memory (RAM).

[0012] The storage device 10 stores a program p1, a learning model lm1, and various information. The program p1 defines the operation of the audio data processing device 20. The storage device 10 may store the program p1 read from a storage device in a server (not shown). In this case, the storage device in the server is an example of a computer-readable recording medium. The learning model lm1 is a trained learning model used to estimate the compression format of input audio data d1.

[0013] The audio data processing device 20 reads a program p1 from the storage device 10. By executing the program p1, the audio data processing device 20 functions as an estimation unit 21, a correction parameter determination unit 22, a delay unit 23, and an estimation result utilization unit 24. At least one of the estimation unit 21, the correction parameter determination unit 22, the delay unit 23, and the estimation result utilization unit 24 may be realized by a circuit such as a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), or an FPGA (Field Programmable Gate Array). The estimation result utilization unit 24 includes a correction processing unit 241.

[0014] The estimation unit 21 and the delay unit 23 receive input audio data d1 from the sound source 2. The input audio data d1 includes expanded audio data and uncompressed audio data. The expanded audio data is data obtained by expanding compressed audio data. On the other hand, the uncompressed audio data is data that is not compressed and therefore retains the original sound without any deterioration in sound quality. The uncompressed audio data is, for example, PCM (Pulse-Coding Modulation) data known as WAV. The PCM method may be a linear PCM method or a differential PCM (DPCM) method. The PCM method may also be an adaptive differential PCM (ADPCM) method.

[0015] Compressed audio data is data obtained by compressing PCM data using a compression method such as MP3 (MPEG-1 Audio Layer-3), Opus, or AAC (Advanced Audio Coding). MP3, Opus, and AAC are compression methods known as lossy compression. Lossy compressed audio data has its high-frequency components mainly removed during compression to increase the compression rate. Therefore, even if lossy compressed audio data is decompressed, the removed high-frequency components are not restored, resulting in a deterioration in sound quality compared to the audio data before compression, i.e., the original sound. Note that compressing PCM data into compressed audio data is also called encoding, and decompressing compressed audio data into PCM data is also called decoding.

[0016] The degree of deterioration in sound quality depends on the bit rate during compression. The bit rate during compression is hereinafter referred to as the compression bit rate. The higher the compression bit rate, the smaller the degree of deterioration in sound quality. On the other hand, the higher the compression bit rate, the lower the compression ratio and the larger the data volume. In this embodiment, the compressed audio data will be described as lossy compressed data.

[0017] The decompressed audio data is data that has been decoded into the same PCM data as WAV, but as described above, the decompressed audio data that is based on lossy compressed audio data suffers from a deterioration in sound quality compared to the original sound.

[0018] The sound source 2 is a sound source that outputs PCM data. Uncompressed audio data or decompressed audio data obtained by decompressing compressed audio data is input to the sound source 2. The sound source 2 is included in, for example, a head unit in an in-vehicle audio system.

[0019] The uncompressed audio data and decompressed audio data contained in the sound source 2 do not have information about the type of compression format. Here, the type of compression format means a combination of a compression method such as MP3 or AAC and a compression bit rate, which is the bit rate at the time of compression.

[0020] When the acoustic system 1 is part of an in-vehicle audio system, it is assumed that the sound source 2 is in a head unit and the audio data processing device 20 is in an audio amplifier separate from the head unit. 2 They are interconnected by a bus conforming to the S (Inter-IC Sound) standard. 2 In the S standard, audio data modulated by the PCM method is serially transmitted between devices.

[0021] That is, in the sound system 1 according to this embodiment, PCM data is transmitted but compressed audio data is not transmitted. If the PCM data is decompressed audio data, the PCM data does not contain information about the compression method of the compressed audio data before it is decompressed into the decompressed audio data.

[0022] Therefore, the estimation unit 21 estimates the type of compression format of the compressed audio data by inputting input audio data d1 including decompressed audio data into the learning model lm1. The learning model lm1 has already learned the relationship between input data including multiple pieces of decompressed audio data and output data including the types of compression formats of multiple pieces of compressed audio data. Here, the input audio data d1 also includes uncompressed audio data. Therefore, the estimation unit 21 estimates the type of uncompressed audio data by inputting the uncompressed audio data into the learning model lm1.

[0023] The correction parameter determination unit 22 determines, in the correction processing unit 241, correction parameters for correcting the decompressed audio data included in the input audio data d1.

[0024] The delay unit 23 delays the input audio data d1.

[0025] The estimation result utilization unit 24 performs processing according to the estimated type of compression format. The correction unit 241 performs correction processing on the input decompressed audio data using the determined correction parameters.

[0026] A2: Machine learning device 2 is a block diagram showing an example of a machine learning device 8 according to the first embodiment. The machine learning device 8 includes a control unit 80, a communication unit 81, a teacher data storage unit 82, and a trained model storage unit 83. The machine learning device 8 is configured using a general-purpose or dedicated computer or the like. In FIG. 2, the teacher data storage unit 82 and the trained model storage unit 83 are realized by a storage device 10.

[0027] The control unit 80 executes a control program to function as an acquisition unit 800 and a generation unit 801. The communication unit 81 is connected to the content distribution server 6 and the terminal device 7 via the network NET, and functions as a communication interface for transmitting and receiving various data. The content distribution server 6 is, for example, a server that provides a music distribution service.

[0028] The terminal device 7 downloads and stores compressed audio data in various compression formats from the content distribution server 6. The terminal device 7 generates and stores various types of audio data, such as uncompressed audio data and compressed audio data, from recording media such as CDs (Compact Discs). The terminal device 7 records the type of compression format for each type of audio data stored in the terminal device 7, and generates multiple pieces of training data in which the various types of audio data as input data IND are associated with the type of compression format as output data OTD.

[0029] The acquisition unit 800 acquires a plurality of pieces of teacher data from the terminal device 7 via the network NET.

[0030] The teacher data storage unit 82 is a database that stores a plurality of teacher data acquired by the acquisition unit 800.

[0031] The generation unit 801 generates a trained learning model lm1 by having the learning model lm1 learn a plurality of pieces of training data. More specifically, the generation unit 801 performs machine learning using training data stored in the training data storage unit 82. That is, the generation unit 801 inputs a plurality of pieces of training data into the learning model lm1, and has the learning model lm1 learn by machine learning the correlation between the input data IND and the output data OTD that constitute the training data, thereby generating a trained learning model lm1.

[0032] The trained model storage unit 83 is a database that stores the trained training model lm1 that has been machine-learned by the generation unit 801. The trained model lm1 stored in the trained model storage unit 83 is provided to the audio data processing device 20 via a network NET, a recording medium, or the like. Note that the trained model lm1 may be provided to an external computer, such as a server-type computer or a cloud-type computer, and stored in a storage unit of the external computer. Also, although the teacher data storage unit 82 and the trained model storage unit 83 are shown as separate storage units in FIG. 2, they may be configured as a single storage unit.

[0033] The training data consists of a set of input data IND, which includes "decompressed audio data," and output data OTD, which includes "type of compression format." "Decompressed audio data" is data in which compressed audio data has been decompressed into PCM data. Note that the input data may also include PCM data that has never been compressed, i.e., uncompressed audio data.

[0034] The output data OTD is referred to as, for example, a correct label in supervised learning. The "type of compression format" included in the output data OTD is a combination of the compression method of the compressed audio data before the decompressed audio data is decompressed and the compression bit rate, which is the bit rate at the time of compression.

[0035] FIG. 3 is a schematic diagram showing an example of a neural network model 90 applied to the learning model lm1 according to the first embodiment.

[0036] The decompressed audio data as input data IND is converted into frequency domain data and input to the neural network model 90. The decompressed audio data may also be input directly to the neural network model 90 as time series data without being converted into frequency domain data. However, since the components for each frequency band are clear in the frequency domain data, it is easier to analyze the features of the frequency domain data than in the time series data.

[0037] The neural network model 90 is composed of a convolutional neural network, a recurrent neural network, and the like.

[0038] The neural network model 90 outputs output data OTD, which includes the type of compression format.

[0039] The generation unit 801 inputs training data to the neural network model 90, and causes the neural network model 90 to perform machine learning of the correlation between the decompressed audio data that is the input data IND and the type of compression format that is the output data OTD.

[0040] More specifically, the generation unit 801 inputs the decompressed audio data constituting the training data as input data IND to the neural network model 90. The data capacity of the decompressed audio data input as input data may be limited by the playback time, number of bytes, etc.

[0041] The generation unit 801 uses an evaluation function that compares the output data output as the inference result, i.e., the type of compression format, with the output data OTD that constitutes the training data, i.e., the correct label of the type of compression format, and iteratively adjusts the weights associated with each synapse so that the value of the evaluation function becomes smaller. Here, adjusting the weights associated with each synapse is called backpropagation.

[0042] When the generation unit 801 determines that a predetermined learning termination condition is satisfied, it terminates the machine learning and stores the neural network model 90 at that time as a trained learning model lm1 in the trained model storage unit 83. The predetermined learning termination condition is, for example, that the number of iterations of the series of machine learning processes reaches a predetermined number, that the value of the evaluation function becomes smaller than an allowable value, etc.

[0043] A3: Compression format for compressed audio data The estimation unit 21 outputs type information d2 indicating the estimated “type of compression format” to the correction parameter determination unit 22.

[0044] Figure 4 shows an example of compression bit rates and compression methods used in representative music distribution services. As shown in Figure 4, MP3, Opus, and AAC are used, with all music distribution services using AAC as the compression method. Meanwhile, each service uses several different compression bit rates. The most commonly used compression bit rate is 128 kbps.

[0045] The correction parameter determination unit 22 classifies the input audio data d1 into three sound quality grades according to the compression bit rate estimated by the estimation unit 21. More specifically, the correction parameter determination unit 22 classifies the input audio data d1 estimated to have been compressed at a compression bit rate of 24 kbps, 48 ​​kbps, or 80 kbps into a low sound quality grade. The correction parameter determination unit 22 classifies the input audio data d1 estimated to have been compressed at a compression bit rate of 128 kbps or 192 kbps into a medium sound quality grade. The correction parameter determination unit 22 classifies the input audio data d1 estimated to have been compressed at a compression bit rate of 256 kbps or 320 kbps into a high sound quality grade.

[0046] The correction parameter determination unit 22 regards the input audio data d1 classified as low sound quality grade as audio data compressed at a compression bit rate of 48 kbps, and determines the correction parameters to be first correction parameters, which are parameters optimal for audio data compressed at a compression bit rate of 48 kbps.

[0047] The first correction parameters include correction parameters for each compression method. For example, the first correction parameters include correction parameters for MP3 / 48 kbps, Opus / 48 kbps, and AAC / 48 kbps. Here, "MP3 / 48 kbps" indicates that the compression method is MP3 and the compression bit rate is 48 kbps.

[0048] The correction parameter determination unit 22 regards the input audio data d1 classified as medium quality as audio data compressed at a compression bit rate of 128 kbps, and determines the correction parameters to be second correction parameters, which are optimal for audio data compressed at a compression bit rate of 128 kbps.

[0049] The second correction parameters include correction parameters for each compression method, for example, correction parameters for MP3 / 128 kbps, correction parameters for Opus / 128 kbps, and correction parameters for AAC / 128 kbps.

[0050] The correction parameter determination unit 22 regards the input audio data d1 classified as high sound quality grade as uncompressed audio data. In this case, the correction parameter determination unit 22 determines not to set any correction parameters and to pass the input audio data d1 through the correction processing unit 241.

[0051] That is, the correction parameter determination unit 22 classifies the input audio data d1 into seven types, namely, MP3 / 48kbps, Opus / 48kbps, AAC / 48kbps, MP3 / 128kbps, Opus / 128kbps, AAC / 128kbps, and WAV, depending on the estimated compression format of the input audio data d1.

[0052] Fig. 5 is a diagram showing an example of a classification result by the correction parameter determination unit 22 of Fig. 1. In Fig. 5, audio data of any one of MP3 / 48 kbps, Opus / 48 kbps, AAC / 48 kbps, MP3 / 128 kbps, Opus / 128 kbps, AAC / 128 kbps, and WAV is input as input audio data d1.

[0053] For example, it can be seen that all MP3 / 128kbps audio data is classified as MP3 / 128kbps audio data. It can also be seen that 98% of Opus / 128kbps audio data is classified as Opus / 128kbps audio data, while the remaining 2% is classified as Opus / 48kbps audio data. Thus, the seven types of audio data show a high classification accuracy of over 96% overall, and the average of the seven types shows a high classification accuracy of 98.4%.

[0054] Even when audio data other than the seven types of audio data described above is input, the correction parameter determination unit 22 classifies the input audio data d1 into one of the seven types of audio data using the method shown in Fig. 4. The correction parameter determination unit 22 determines parameters corresponding to the type of classified audio data, and outputs parameter information d3 related to the determined parameters to the correction processing unit 241. The parameter information d3 includes information related to the upper limit frequency of the input audio data d1, information on whether the input audio data d1 is to be subjected to correction processing, etc.

[0055] A4: Correction processing operation Fig. 6 is a diagram showing the relationship between compression formats and the upper limit frequency of audio data in each compression format. As shown in Fig. 6, the tendency of the change in the upper limit frequency with respect to each compression bit rate differs for each of the compression formats: MP3, Opus, and AAC.

[0056] The correction processing unit 241 receives parameter information d3 from the correction parameter determination unit 22 and receives delayed audio data d4, which is the input audio data d1 delayed from the delay unit 23. The delayed audio data d4 includes decompressed audio data or uncompressed audio data.

[0057] The amount of delay by the delay unit 23 is determined taking into consideration the calculation times of the estimation unit 21 and the correction parameter determination unit 22. The delay unit 23 outputs the delayed audio data d4 to the correction processing unit 241.

[0058] If the delayed audio data d4 is to be corrected, the correction processor 241 performs a specific upsampling process on the delayed audio data d4. Whether the delayed audio data d4 is to be corrected is determined based on information, included in the parameter information d3, about whether the input audio data d1 is to be corrected. The specific upsampling process is a process of upsampling the delayed audio data d4 to a frequency range from half the upper limit frequency based on the parameter information d3 to a frequency several kilohertz higher than half the upper limit frequency, to double the frequency. In other words, the specific upsampling process is a process of generating frequency components above the upper limit frequency that were missing from the delayed audio data d4 as harmonic components of the delayed audio data d4.

[0059] After performing a specific upsampling process, the correction processing unit 241 combines the upsampled data with the delayed audio data d4. The data generated by the specific upsampling process contains components that complement frequency components that may have been lost from the delayed audio data d4 due to a previous compression process.

[0060] FIG. 7 is a frequency spectrum diagram of audio data for explaining the correction processing by the correction processing unit 241 of FIG. 1. The frequency spectrum of the input audio data d1 is indicated by a solid line. In this example, the upper limit frequency of the input audio data d1 is 17 kHz. High-frequency components higher than the upper limit frequency are lost when the audio data is compressed. The correction processing unit 241 performs upsampling processing on the input audio data d1 to complement the lost high-frequency components. More specifically, the correction processing unit 241 generates a frequency domain F2, for example, from the upper limit frequency to 24 kHz, by upsampling a frequency domain F1, which is half the frequency domain F2. The data resulting from the upsampling of the frequency domain F1 is called complementary audio data.

[0061] The frequency spectrum of the corrected audio data e1 is indicated by a dashed line. The correction processing unit 241 obtains the corrected audio data e1 by combining the input audio data d1 with the generated complementary audio data. The spectrum of the corrected audio data e1 approaches the spectrum of the original sound before compression. In this way, the correction processing by the correction processing unit 241 is processing for complementing high-frequency components missing from the compressed audio data. Note that the correction processing unit 241 may perform processing for complementing not only high-frequency components of the compressed audio data, but also mid-frequency components and / or low-frequency components.

[0062] Fig. 8 is a flowchart showing an example of the processing operation of the audio data processing device 20. The processing operation of the audio data processing device 20 will be described below with reference to Fig. 8. The routine of Fig. 8 is started, for example, when the audio data processing device 20 is started, and is executed every time a certain period of time has elapsed.

[0063] In step S11, the audio data processing device 20 functions as the estimation unit 21, and estimates the compression format of the input audio data d1 by inputting the input audio data d1 to the learning model lm1.

[0064] In step S12, the audio data processing device 20 functions as the correction parameter determination unit 22 to determine correction parameters based on the estimated compression format.

[0065] In step S13, the audio data processing device 20 functions as the correction processing unit 241 to determine whether or not correction of the delayed audio data d4 is required based on the parameter information d3 including the correction parameters.

[0066] If correction of the delayed audio data d4 is not necessary, that is, if the determination result in step S13 is negative, the audio data processing device 20 temporarily ends this routine.

[0067] On the other hand, if correction of the delayed audio data d4 is necessary, that is, if the judgment result in step S13 is positive, the audio data processing device 20 functions as the correction processing unit 241 in step S14 to generate harmonic components of the delayed audio data d4.

[0068] In step S15, the audio data processing device 20 functions as the correction processing unit 241 to synthesize the delayed audio data d4 with the harmonic components of the delayed audio data d4, and then ends this routine.

[0069] A5: Summary of the first embodiment As described above, the audio data processing device 20 according to the first embodiment includes an estimation unit 21 and an estimation result utilization unit 24. The estimation unit 21 estimates the type of compression format of compressed audio data by inputting decompressed audio data to the learning model lm1. The decompressed audio data is data obtained by decompressing compressed audio data. The learning model lm1 has already learned the relationship between input data IND including multiple pieces of decompressed audio data and output data OTD including the types of compression formats of the multiple pieces of compressed audio data. The estimation result utilization unit 24 performs processing according to the estimated type of compression format. The correction processing unit 241 performs correction processing on the input decompressed audio data according to the estimated type of compression format.

[0070] According to this aspect, even if the input audio data does not contain information about the compression format, processing can be performed according to the compression format of the input audio data.

[0071] In the audio data processing device 20 according to the first embodiment, the input data IND includes a plurality of uncompressed audio data, and the output data OTD includes a plurality of types of uncompressed audio data. The estimation unit 21 identifies the type of uncompressed audio data by inputting the uncompressed audio data to the learning model lm1.

[0072] According to this aspect, the type of audio data can be estimated, including uncompressed audio data.

[0073] Furthermore, in the audio data processing device 20 according to the first embodiment, the type of compression format is based on the combination of the compression method and bit rate of the compressed audio data.

[0074] For example, input audio data can be classified into one of several sound quality grades based on the combination of compression method and compression bit rate, such as MP3, Opus, or AAC, or the type of uncompressed audio data, such as WAV. This allows for correction appropriate to each sound quality grade. For example, by matching the sound quality grades to the sound quality grades of audio data provided by major music distribution services, the correction becomes more effective.

[0075] Moreover, in the audio data processing device 20 according to the first embodiment, the estimation result utilization unit 24 includes a correction processing unit 241. The correction processing unit 241 performs correction processing on the input decompressed audio data in accordance with the type of compression format estimated.

[0076] According to this aspect, even if the input audio data does not contain information about the compression format, it is possible to apply appropriate correction to the audio data in accordance with the compression format of the input audio data.

[0077] Furthermore, in the audio data processing device 20 according to the first embodiment, the correction process is a process for complementing high-frequency components missing from compressed audio data.

[0078] According to this aspect, the high frequency components missing from the compressed audio data can be classified into one of several sound quality grades by the input decompressed audio data, thereby making it possible to apply correction appropriate to each sound quality grade.

[0079] In addition, in the audio data processing method according to the first embodiment, decompressed audio data is input to learning model lm1, which estimates the type of compression format of the compressed audio data and performs correction processing on the input decompressed audio data according to the estimated type of compression format. Learning model lm1 has already learned the relationship between input data IND, which includes multiple decompressed audio data, and output data OTD, which includes the types of compression formats of multiple compressed audio data.

[0080] According to this aspect, even if the input audio data does not contain information about the compression format, processing can be performed according to the compression format of the input audio data.

[0081] B: Modified example The present invention is not limited to the above-described embodiment, and various modifications can be adopted within the scope of the present invention. Specific modified embodiments are exemplified below. Furthermore, two or more embodiments arbitrarily selected from the following examples may be appropriately combined within a range that does not contradict each other. In the modified embodiments exemplified below, for elements whose actions and functions are equivalent to those of the above-described embodiment, the symbols used in the above explanation will be used, and detailed explanations of each will be omitted as appropriate.

[0082] B1: First modified example In the above embodiment, the correction parameter determination unit 22 provides three sound quality grades and classifies the audio into seven compression formats, but the classification method is not limited to this. For example, the sound quality grades may be more or less than three. Alternatively, sound quality grades may not be provided, and correction parameters may be determined for each compression bit rate. The learning model lm1 may be trained to support compression formats other than MP3, Opus, and AAC.

[0083] B2: Second variant In the above embodiment, it is assumed that the compressed audio data is audio data compressed by lossy compression, but decompressed audio data obtained by decompressing audio data compressed by lossless compression may also be input to the audio data processing device 20. Examples of lossless compression include FLAC (Free Lossless Audio Codec) and ALAC (Apple Lossless Audio Codec). Like uncompressed audio data, decompressed audio data obtained by decompressing audio data compressed by lossless compression does not require correction processing, and therefore may be classified as a high-quality sound grade.

[0084] B3: Third variant The present disclosure may be provided in the form of an inference device used to infer the type of compression format of compressed audio data, instead of the audio data processing device 20 according to the above embodiment. In this case, the inference device includes a memory and a processor.

[0085] The memory can store a learning model lm1 that has learned the relationship between input data IND, which includes decompressed audio data obtained by decompressing each of a plurality of compressed audio data sets, and output data OTD, which includes the types of compression formats of the plurality of compressed audio data sets. The processor can execute a series of processes, including an acquisition process for acquiring input data, which includes decompressed audio data obtained by decompressing compressed audio data sets, and an inference process for inferring the types of compression formats of the compressed audio data sets using the learning model lm1, once the input data has been acquired in the acquisition process.

[0086] According to this aspect, by providing it in the form of an inference device, it can be more easily applied to various devices than when it is implemented as the audio data processing device 20. It will be naturally understood by those skilled in the art that when the inference device infers the type of compression format of compressed audio data, the inference method implemented by the estimation unit 21 of the audio data processing device 20 may be applied using the trained learning model lm1 generated by the machine learning device according to the above embodiment. Note that it may be provided not only in the form of an inference device, but also in the form of an inference method or an inference program.

[0087] B4: Fourth variant In the above embodiment, the correction processing unit 241 performs correction processing on the input decompressed audio data in accordance with the estimated type of compression format. However, instead of or in addition to this correction processing, a notification processing may be performed in which information about the estimated type of compression format of the compressed audio data is output to a display device (not shown) or the like to notify the user of the estimated type of compression format of the compressed audio data. The notification processing unit that performs this notification processing is an example of an estimation result utilization unit.

[0088] B5: Fifth variant In the above embodiment, the learning model lm1 is learned by machine learning using the neural network model 90, but the learning model lm1 may also be learned by multi-class classification using a method such as multinomial logistic regression.

[0089] B6: 6th variant In the above embodiment, the decompressed audio data is data modulated by a PCM method including linear PCM, differential PCM, and adaptive differential PCM. However, the decompressed audio data may also be data modulated by PDM (Pulse-Density Modulation).

[0090] B7: 7th variant The sound system 1 according to the first embodiment is mounted on a vehicle such as an automobile, but the sound system 1 is also applicable to the fields of home audio and professional audio.

[0091] C: Notes From the above-described exemplary embodiments, the following configurations can be understood, for example.

[0092] An audio data processing device according to one aspect (aspect 1) of the present disclosure includes an estimation unit that estimates the type of compression format of compressed audio data by inputting the decompressed audio data into a learning model that has learned the relationship between input data including decompressed audio data obtained by decompressing each of a plurality of compressed audio data and output data including the types of compression formats of the plurality of compressed audio data, and an estimation result utilization unit that performs processing according to the estimated type of compression format.

[0093] According to this aspect, even if the input audio data does not contain information about the compression format, processing can be performed according to the compression format of the input audio data.

[0094] In an audio data processing device according to one aspect (aspect 2) of the present disclosure, the input data includes a plurality of uncompressed audio data, the output data includes the plurality of types of uncompressed audio data, and the estimation unit estimates the type of the uncompressed audio data by inputting the uncompressed audio data into the learning model.

[0095] According to this aspect, the type of audio data can be estimated, including uncompressed audio data.

[0096] In the audio data processing device according to one aspect (aspect 3) of the present disclosure, the type of compression format is based on a combination of a compression method of the compressed audio data and a bit rate at the time of compression.

[0097] For example, input audio data can be classified into one of several sound quality grades based on the combination of compression method and compression bit rate, such as MP3, Opus, or AAC, or the type of uncompressed audio data, such as WAV. This allows for correction appropriate to each sound quality grade. For example, by matching the sound quality grades to the sound quality grades of audio data provided by major music distribution services, the correction becomes more effective.

[0098] In an audio data processing device according to one aspect (aspect 4) of the present disclosure, the estimation result utilization unit includes a correction processing unit that performs correction processing on the input decompressed audio data according to the type of compression format estimated.

[0099] According to this aspect, even if the input audio data does not contain information about the compression format, it is possible to apply appropriate correction to the audio data in accordance with the compression format of the input audio data.

[0100] In the audio data processing device according to one aspect (aspect 5) of the present disclosure, the correction process is a process for complementing high-frequency components missing from the compressed audio data.

[0101] According to this aspect, the high frequency components missing from the compressed audio data can be classified into one of several sound quality grades by the input decompressed audio data, thereby making it possible to apply correction appropriate to each sound quality grade.

[0102] An inference device according to one aspect (aspect 6) of the present disclosure includes a memory and a processor, wherein the memory stores a learning model that has learned the relationship between input data including decompressed audio data obtained by decompressing each of a plurality of compressed audio data, and output data including the types of compression formats of the plurality of compressed audio data, and the processor executes an acquisition process that acquires input data including decompressed audio data obtained by decompressing compressed audio data, and an inference process that, upon acquiring the input data in the acquisition process, infers the type of compression format of the compressed audio data using the learning model.

[0103] According to this aspect, by providing it in the form of an inference device, it can be more easily applied to various devices than when it is implemented as an audio data processing device. It will be naturally understood by those skilled in the art that when the inference device infers the type of compression format of compressed audio data, the inference method implemented by the estimation unit 21 of the audio data processing device 20 may be applied using the trained learning model lm1 generated by the machine learning device.

[0104] An audio data processing method according to one aspect (aspect 7) of the present disclosure estimates the type of compression format of the compressed audio data by inputting the expanded audio data into a learning model that has learned the relationship between input data including expanded audio data obtained by expanding each of multiple pieces of compressed audio data and output data including the types of compression formats of the multiple pieces of compressed audio data, and then performs processing according to the estimated type of compression format.

[0105] According to this aspect, even if the input audio data does not contain information about the compression format, processing can be performed according to the compression format of the input audio data. [Explanation of symbols]

[0106] 1...acoustic system, 20...audio data processing device, 21...estimation unit, 24...estimation result utilization unit, 241...correction processing unit, IND...input data, lm1...learning model, OTD...output data

Claims

1. an estimation unit that estimates the type of compression format of the compressed audio data by inputting the decompressed audio data into a learning model that has learned the relationship between input data including decompressed audio data obtained by decompressing each of a plurality of compressed audio data and output data including the types of compression formats of the plurality of compressed audio data; an estimation result utilization unit that performs processing according to the estimated type of compression format; An audio data processing device comprising:

2. the input data includes a plurality of uncompressed audio data, and the output data includes the plurality of types of uncompressed audio data; the estimation unit estimates the type of the uncompressed audio data by inputting the uncompressed audio data to the learning model; 2. The audio data processing apparatus according to claim 1.

3. The type of the compression format is based on a combination of a compression method of the compressed audio data and a bit rate at the time of compression.

3. The audio data processing device according to claim 1.

4. The estimation result utilization unit a correction processing unit that performs a correction process on the input decompressed audio data in accordance with the estimated type of compression format, 2. The audio data processing apparatus according to claim 1.

5. the correction processing is processing for complementing high-frequency components missing from the compressed audio data; 5. The audio data processing device according to claim 4.

6. A memory and a processor, The memory includes: storing a learning model that has learned the relationship between input data including decompressed audio data obtained by decompressing each of a plurality of compressed audio data and output data including the types of compression formats of the plurality of compressed audio data; The processor: an acquisition process for acquiring input data including decompressed audio data obtained by decompressing compressed audio data; an inference process for inferring the type of compression format of the compressed audio data using the learning model when the input data is acquired in the acquisition process; To execute Reasoning device.

7. estimating the type of compression format of the compressed audio data by inputting the decompressed audio data into a learning model that has learned the relationship between input data including decompressed audio data obtained by decompressing each of a plurality of compressed audio data and output data including the types of compression formats of the plurality of compressed audio data; performing processing according to the estimated type of compression format; A method for processing audio data.

Images