Electronic device and control method of electronic device

Abstract

An electronic device and a control method of the electronic device are disclosed. The electronic device comprises: a memory that stores at least one instruction; and at least one processor that executes the at least one instruction and includes a processing circuit, wherein the at least one processor may: obtain an audio signal; input the audio signal into a pre-processing module so as to obtain feature information indicating a feature included in the audio signal; input the feature information into a vector obtaining module so as to obtain a feature vector corresponding to the audio signal on the basis of the obtained feature information; and input the feature vector into a classification module so as to identify whether the audio signal has been compressed two or more times. Here, each module may include a neural network, and the feature information may include information on at least one of a feature for a frequency domain of the audio signal, a feature for a time domain of the audio signal, and a feature for a frequency-time domain of the audio signal.

Description

Electronic devices and methods for controlling electronic devices

[0001] The present disclosure relates to an electronic device and a method for controlling the electronic device, and for example, to an electronic device and a method for controlling the same that can identify whether an audio signal is multi-compressed by analyzing the characteristics of the audio signal.

[0002] Recently, financial fraud using communication media such as vishing and voice phishing has become a global problem, and there are also continuous cases of damage caused by spam calls (e.g. robocalls) that use automated voice messages or recorded voices to make a large number of calls.

[0003] Although various technical and legal measures are being taken to address the above issues, the number of victims is increasing due to the increasing diversity of techniques related to vishing, voice phishing, and spam calls.

[0004] As a typical conventional technology, spam blocking methods are based on a blacklist of phone numbers that requires regular database updates, making them ineffective against newly created spam phone numbers. They also have the limitation of having to transmit phone numbers to an external database.

[0005] Therefore, there is a growing need for a technology that can detect and block illegal audio based on the audio signal itself within the device, without the need for database maintenance.

[0006] Embodiments of the present invention provide an electronic device capable of detecting whether an audio signal is multi-compressed by analyzing the characteristics of the audio signal, and a method of controlling the same.

[0007] According to an exemplary embodiment of the present disclosure, an electronic device includes a memory storing at least one instruction and at least one processor including a processing circuit for executing the at least one instruction, wherein the at least one processor individually and / or jointly obtains an audio signal, inputs the audio signal to a preprocessing module including a neural network to obtain feature information representing a feature included in the audio signal, inputs the feature information to a vector obtaining module including a neural network to obtain a feature vector corresponding to the audio signal based on the obtained feature information, and inputs the feature vector to a classification module including a neural network to identify whether the audio signal has been compressed twice or more, wherein the feature information includes information on at least one of a feature for a frequency domain of the audio signal, a feature for a time domain of the audio signal, and a feature for a frequency-time domain of the audio signal.

[0008] Meanwhile, the preprocessing module corresponds to at least one of a plurality of codec types and includes at least one codec preprocessing module including a neural network, and the at least one processor individually and / or jointly identifies a codec type of the audio signal, and when the codec type of the audio signal is identified, inputs the audio signal to a codec preprocessing module corresponding to the identified codec type among the at least one codec preprocessing module, thereby obtaining the feature information including a feature related to the identified codec.

[0009] Meanwhile, the preprocessing module does not correspond to the plurality of codec types and further includes a default preprocessing module including a neural network, and the at least one processor can individually and / or jointly input the audio signal into the default preprocessing module to obtain the feature information if the codec type of the audio signal is not identified.

[0010] Meanwhile, the classification module corresponds to each of at least one codec preprocessing modules and includes at least one codec classification module including a neural network, and the at least one processor individually and / or jointly inputs the feature information obtained through the at least one codec preprocessing module into the at least one codec classification module corresponding to the at least one codec preprocessing module when the codec type of the audio signal is identified, thereby identifying whether the audio signal has been compressed twice or more.

[0011] Meanwhile, the classification module corresponds to the default preprocessing module and further includes a default classification module including a neural network, and the at least one processor individually and / or jointly inputs the feature information obtained through the default preprocessing module into the default classification module when the codec type of the audio signal is not identified, thereby identifying whether the audio signal has been compressed twice or more.

[0012] Meanwhile, the at least one processor may individually and / or jointly identify a codec type of the audio signal based on metadata of the audio signal or information about a transmission channel of the audio signal.

[0013] Meanwhile, the vector acquisition module corresponds to both the at least one codec preprocessing module and the default preprocessing module, and may correspond to both the at least one codec classification module and the default classification module.

[0014] Meanwhile, the classification module outputs probability information indicating whether the audio signal has been compressed twice or more, and each of the preprocessing module, the vector acquisition module, and the classification module includes a neural network and can be learned according to backpropagation of loss based on the probability information.

[0015] Meanwhile, the output unit further includes an output circuit, and the at least one processor can individually and / or jointly control the output unit to not output the audio signal and output a warning message for the audio signal when the audio signal is identified as being compressed twice or more.

[0016] According to an exemplary embodiment of the present disclosure, a control method of an electronic device includes the steps of obtaining an audio signal, inputting the audio signal into a preprocessing module including a neural network to obtain feature information representing a feature included in the audio signal, inputting the feature information into a vector obtaining module including a neural network to obtain a feature vector corresponding to the audio signal based on the obtained feature information, and inputting the feature vector into a classification module including a neural network to identify whether the audio signal has been compressed twice or more.

[0017] Meanwhile, the preprocessing module corresponds to at least one of a plurality of codec types and includes at least one codec preprocessing module including a neural network, and the step of obtaining the feature information includes the step of identifying a codec type of the audio signal and, when the codec type of the audio signal is identified, the step of inputting the audio signal to a codec preprocessing module corresponding to the identified codec type among the at least one codec preprocessing module, thereby obtaining the feature information including a feature related to the identified codec, wherein the feature information may include information on at least one of a feature for a frequency domain of the audio signal, a feature for a time domain of the audio signal, and a feature for a frequency-time domain of the audio signal.

[0018] Meanwhile, the preprocessing module does not correspond to the plurality of codec types and further includes a default preprocessing module including a neural network, and the step of obtaining the feature information may further include a step of inputting the audio signal into the default preprocessing module to obtain the feature information if the codec type of the audio signal is not identified.

[0019] Meanwhile, the classification module corresponds to each of at least one codec preprocessing module and includes at least one codec classification module including a neural network, and the step of identifying whether the audio signal has been compressed twice or more may include a step of inputting the feature information acquired through the at least one codec preprocessing module into the at least one codec classification module corresponding to the at least one codec preprocessing module, when the codec type of the audio signal is identified, to identify whether the audio signal has been compressed twice or more.

[0020] Meanwhile, the classification module corresponds to the default preprocessing module and further includes a default classification module including a neural network, and the step of identifying whether the audio signal has been compressed twice or more may further include a step of inputting the feature information acquired through the default preprocessing module into the default classification module, if the codec type of the audio signal is not identified, to identify whether the audio signal has been compressed twice or more.

[0021] Meanwhile, the step of identifying the codec type of the audio signal may include a step of identifying the codec type of the audio signal based on metadata of the audio signal or information about a transmission channel of the audio signal.

[0022] The above and other aspects, features and advantages of specific embodiments of the present disclosure will become more apparent from the following detailed description taken in conjunction with the accompanying drawings.

[0023] Figure 1 is a block diagram showing the configuration of an electronic device according to various embodiments;

[0024] Figure 2 is a block diagram showing modules according to various embodiments;

[0025] Figure 3 is a block diagram exemplarily showing the configuration of modules according to various embodiments;

[0026] FIG. 4 is a block diagram exemplarily showing the configuration of an electronic device according to various embodiments;

[0027] FIG. 5 is a flowchart exemplarily showing a method for controlling an electronic device according to various embodiments, and

[0028] FIG. 6 is a flowchart illustrating an exemplary method for controlling an electronic device in relation to subsequent processing depending on whether an audio signal has been compressed more than twice according to various embodiments.

[0029] The various exemplary embodiments of the present disclosure can be variously modified and have various types of embodiments. Accordingly, specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the scope to specific embodiments, but should be understood to encompass various modifications, equivalents, and / or alternatives of the various embodiments of the present disclosure. In connection with the description of the drawings, similar reference numerals may be used for similar components.

[0030] In the present disclosure, if it is determined that a specific description of a related public function or configuration may unnecessarily obscure the gist of the present disclosure, a detailed description thereof may be omitted.

[0031] Additionally, the various embodiments described below may be modified in various other forms, and the scope of the technical concepts of the present disclosure is not limited to the various embodiments described below. Rather, these various exemplary embodiments are provided to fully convey the technical concepts of the present disclosure to those skilled in the art.

[0032] The terminology used in this disclosure is solely for the purpose of describing various embodiments and is not intended to limit the scope of the rights. Singular expressions include plural expressions unless the context clearly dictates otherwise.

[0033] In this disclosure, expressions such as “has,” “can have,” “includes,” or “may include” indicate the presence of a corresponding feature (e.g., a component such as a number, function, operation, or part), and do not exclude the presence of additional features.

[0034] In this disclosure, expressions such as “A or B,” “at least one of A and / or B,” or “one or more of A or / and B” can include all possible combinations of the listed items. For example, “A or B,” “at least one of A and B,” or “at least one of A or B” can all refer to (1) including at least one A, (2) including at least one B, or (3) including both at least one A and at least one B.

[0035] The expressions “first,” “second,” “first,” or “second,” etc., used in this disclosure can describe various components, regardless of order and / or importance, and are only used to distinguish one component from another, but do not limit the components.

[0036] When it is said that a component (e.g., a first component) is “(operatively or communicatively) coupled with / to” or “connected to” another component (e.g., a second component), it should be understood that said component may be directly coupled to said other component, or may be coupled via another component (e.g., a third component).

[0037] On the other hand, when it is said that a component (e.g., a first component) is "directly connected" or "directly connected" to another component (e.g., a second component), it can be understood that no other component (e.g., a third component) exists between said component and said other component.

[0038] The expression "configured to" as used in the present disclosure may be used interchangeably with, for example, "suitable for," "having the capacity to," "designed to," "adapted to," "made to," or "capable of." The term "configured to" does not necessarily indicate something that is "specifically designed to" in terms of hardware.

[0039] Instead, in some contexts, the phrase "a device configured to" may indicate that the device, in conjunction with other devices or components, is "capable of" performing A, B, and C. For example, the phrase "a processor configured (or set) to perform A, B, and C" may refer to a dedicated processor (e.g., an embedded processor) for performing those operations, or a general-purpose processor (e.g., a CPU or application processor) that can perform those operations by executing one or more software programs stored in a memory device.

[0040] In various embodiments, a "module" or "part" performs at least one function or operation, and may be implemented in hardware, software, or a combination of hardware and software. Furthermore, multiple "modules" or multiple "parts" may be integrated into at least one module and implemented as at least one processor, excluding any "modules" or "parts" that require specific hardware implementation.

[0041] The various elements and areas in the drawings are schematically drawn. Therefore, the technical concept of the present invention is not limited by the relative sizes or spacings drawn in the attached drawings.

[0042] Hereinafter, embodiments according to the present disclosure will be described in detail with reference to the attached drawings to help understand the present disclosure.

[0043] FIG. 1 is a block diagram illustrating the configuration of an electronic device (100) according to various embodiments. FIG. 2 is a block diagram illustrating modules according to various embodiments. Hereinafter, various embodiments of the present disclosure will be described in detail with reference to FIG. 1 and FIG. 2 together.

[0044] The term "electronic device (100)" refers to a device capable of identifying whether an audio signal has been compressed twice or more. The electronic device (100) may be implemented as a type of device such as a smartphone, tablet PC, etc., and, by way of example and without limitation, may also be implemented as a device such as a server, edge computing device, etc. In addition, any device capable of identifying whether an audio signal has been compressed twice or more may correspond to the electronic device (100) according to the present disclosure, regardless of its type.

[0045] As illustrated in FIG. 1, the electronic device (100) may include a memory (110) and a processor (120) (e.g., including a processing circuit). However, the present disclosure is not limited thereto, and it is understood that new configurations may be added or some configurations may be omitted in addition to the configuration illustrated in FIG. 1 when implementing the present disclosure.

[0046] At least one instruction regarding the electronic device (100) may be stored in the memory (110). In addition, an O / S (Operating System) for driving the electronic device (100) may be stored in the memory (110). In addition, various software programs or applications for operating the electronic device (100) according to various embodiments of the present disclosure may be stored in the memory (110). In addition, the memory (110) may include a semiconductor memory such as a flash memory or a magnetic storage medium such as a hard disk.

[0047] For example, various software modules for operating the electronic device (100) according to various embodiments of the present disclosure may be stored in the memory (110), and the processor (120) may control the operation of the electronic device (100) by executing the various software modules stored in the memory (110). The memory (110) is accessed by the processor (120), and data reading / recording / modifying / deleting / updating, etc. may be performed by the processor (120).

[0048] In the present disclosure, the term memory (110) may include memory (110), ROM (not shown), RAM (not shown) within the processor (120), or a memory card (not shown) (e.g., micro SD card, memory stick) mounted on the electronic device (100).

[0049] In various embodiments, the memory (110) may store various data / information, such as audio signals, feature information, feature vectors, data for multiple modules, learning data for learning multiple modules, probability information, information for codec types, etc. In addition, various information may be stored in the memory (110), and the information stored in the memory (110) may be updated by being received from an external device or input by a user, for example.

[0050] The processor (120) may include various processing circuits and control the overall operation of the electronic device (100). For example, the processor (120) is connected to the configuration of the electronic device (100) including the memory (110), and may control the overall operation of the electronic device (100) by executing at least one instruction stored in the memory (110) as described above.

[0051] The processor (120) may be implemented in various ways. For example, the processor (120) may be implemented as at least one of an Application Specific Integrated Circuit (ASIC), an embedded processor, a microprocessor, hardware control logic, a hardware finite state machine (FSM), and a digital signal processor (DSP). The term "processor" in this disclosure may be used to mean a Central Processing Unit (CPU), a Graphic Processing Unit (GPU), and a Micro Processor Unit (MPU). The processor (120) may include various processing circuits and / or multiple processors. For example, as used in this disclosure, including in the claims, the term "processor" may include various processing circuits including at least one processor, wherein one or more of the at least one processor may be configured to individually and / or jointly perform various functions described in this disclosure in a distributed manner. When "processor," "at least one processor," and "one or more processors" are described herein as being configured to perform multiple functions, these terms include, but are not limited to, situations where one processor performs some of the disclosed functions, another processor performs other of the disclosed functions, and situations where a single processor can perform all of the disclosed functions. Furthermore, the at least one processor may comprise a combination of processors that perform various disclosed functions, for example, in a distributed manner. At least one processor may execute program instructions to achieve or perform various functions.

[0052] In various embodiments, the processor (120) may analyze the characteristics of an audio signal to determine whether the audio signal has been compressed more than twice. For example, the processor (120) may analyze the characteristics of an audio signal using a plurality of modules to determine whether the audio signal has been compressed more than twice. Each of the plurality of modules may include a neural network, and thus, each module or all of the modules may be referred to as a neural network model.

[0053] As illustrated in FIG. 2, the plurality of modules may include a preprocessing module (10), a vector acquisition module (20), and a classification module (30), and each module may include a neural network. Hereinafter, the description will be made on the assumption that the plurality of modules are all included in the electronic device (100) and implemented on-device by the processor (120) included in the electronic device (100). However, it is of course possible that at least one module among the plurality of modules may be implemented in a manner in which it is included in an external device.

[0054] The processor (120) can acquire an audio signal. For example, the processor (120) can receive an audio signal from an external device through a communication unit (130) included in the electronic device (100), and can also receive an audio signal through a microphone included in the electronic device (100). In addition, there are no restrictions on the method of acquiring an audio signal, and there are no special restrictions on the type of audio signal.

[0055] The example of FIG. 2 illustrates a case where a first audio signal (A1) is received from an external device through a communication unit (130) of an electronic device (100) and a case where a second audio signal (B1) is received from an external device through a communication unit (130) (e.g., including a communication circuit) of an electronic device (100) (see FIG. 4). That is, the example of FIG. 2 does not indicate that the first audio signal (A1) and the second audio signal (B1) are input together to the preprocessing module (10), but merely illustrates the processing processes for the first audio signal (A1) and the second audio signal (B1) in parallel for comparative explanation.

[0056] In FIG. 2, the processing process for the first audio signal (A1) shows that when a user of an electronic device (100) makes a phone call to a counterpart (A0) using the electronic device (100), an audio signal according to the counterpart's (A0) speech is directly transmitted to the electronic device (100). In this case, the first audio signal (A1) may be compressed once during signal transmission.

[0057] The processing process for the second audio signal (B1) indicates that when a user of an electronic device (100) makes a phone call to a counterpart using the electronic device (100), an audio signal according to the counterpart's speech is stored in an external device (B0) and then transmitted to the electronic device (100). In this case, the second audio signal (B1) is compressed not only when transmitting the signal but also when recording by the external device (B0), and thus may be compressed twice as a result.

[0058] Audio signals compressed more than twice, such as the second audio signal (B1), may experience high frequency loss or unnatural artifacts during the multiple compression process of the audio signal, resulting in poor quality. Therefore, even when audio signals are compressed using the same codec, it is possible to distinguish between multiple compression and single compression. In particular, the present disclosure can improve the accuracy, security, and efficiency of multiple compression detection through the process described below.

[0059] The processor (120) can input an audio signal into the preprocessing module (10) to obtain feature information representing features included in the audio signal. The 'preprocessing module (10)' refers to a module (e.g., including a neural network) that can perform preprocessing on the input audio signal and obtain (extract or generate) feature information on the audio signal. The preprocessing module (10) includes a neural network and can be trained as described below with reference to FIG. 3. The preprocessing module (10) may be the first (or initial or initial) step of the process according to the present disclosure, and thus may be referred to as a 'frontend', and may also be referred to as a 'low-level feature extractor', etc.

[0060] In the description of the present disclosure, "feature information" collectively refers to information about features that can be obtained by analyzing an audio signal, and may be used with the same meaning as described below. For example, feature information refers to information about low-level features that can be obtained by analyzing the audio signal itself, whereas a feature vector is obtained by converting low-level feature information into a vector and may exemplarily represent information about high-level features that can be mapped to a vector space. In this sense, the term "feature information" may be replaced with terms such as "low-level feature information" or "first-level feature information."

[0061] For example, the feature information may include information about at least one of a feature for a frequency domain of an audio signal, a feature for a time domain of an audio signal, and a feature for a frequency-time domain of an audio signal.

[0062] For example, features of the frequency domain of an audio signal may include features of energy for each frequency band of the audio signal. Features of the time domain of an audio signal may include statistical features of the audio signal, such as mean, variance, skewness, and kurtosis in the time domain. Features of the frequency-time domain of an audio signal may be expressed as a spectrogram, Mel-Frequency Cepstral Coefficients (MFCC), etc., which visually represent the spectrum of a signal according to frequency and time.

[0063] For example, FIG. 2 illustrates a first spectrogram (A2) representing feature information corresponding to a first audio signal (A1) and a second spectrogram (B2) representing feature information corresponding to a second audio signal (B1). However, the feature information for the audio signal may be diverse in addition to the examples described above, and may differ, for example, depending on the codec used to compress the audio signal, as described below. In addition, the feature information for the audio signal may be extracted after performing various transformations, such as a Fourier Transform (FT), on the audio signal.

[0064] The processor (120) can input feature information into the vector acquisition module (20) and acquire a feature vector corresponding to the audio signal based on the acquired feature information. The 'vector acquisition module (20)' refers to a module that can extract features of an audio signal and acquire (extract or generate) a feature vector corresponding to the audio signal. For example, when feature information for an audio signal is input from the preprocessing module (10), the vector acquisition module (20) can convert the feature information into a feature vector by extracting high-level features included in the feature information for the audio signal. The vector acquisition module (20) may also be referred to as a 'feature extractor' or a 'high-level feature extractor'.

[0065] A "feature vector" refers to a vector representing the characteristics of an audio signal. As mentioned above, a feature vector is the result of quantifying feature information about an audio signal and converting it into a vector. It can be distinguished from feature information in that it can represent higher-level features than feature information. In this sense, the term "feature vector" can be used interchangeably with terms such as "high-level feature information" or "second-level feature information." Once feature information is converted into a feature vector, an audio signal can be easily classified using a neural network based on the distribution of the corresponding feature vector.

[0066] As in the example of Fig. 2, when a first spectrogram (A2) representing feature information corresponding to a first audio signal (A1) is input, the vector acquisition module (20) can output a first feature vector (A3) as described in Fig. 2. When a second spectrogram (B2) representing feature information corresponding to a second audio signal (B1) is input, the vector acquisition module (20) can output a second feature vector (B3) as depicted in Fig. 2.

[0067] The processor (120) can input a feature vector into the classification module (30) to identify whether an audio signal has been compressed twice or more. The 'classification module (30)' refers to a module that can identify whether an audio signal has been compressed twice or more (e.g., whether it has been multi-compressed) based on a feature vector corresponding to the audio signal. The classification module (30) can be simply referred to as a 'classifier'.

[0068] For example, the classification module (30) can map a feature vector corresponding to an audio signal compressed once and a feature vector corresponding to an audio signal compressed twice or more into different domains (classes or categories). When a feature vector is input from the vector acquisition module (20), the classification module (30) can output probability information (or probability value) indicating whether the audio signal has been compressed twice or more according to the distribution of the feature vector.

[0069] In the example of FIG. 2, the first feature vector (A3) represents the characteristics of the first audio signal (A1) compressed once during signal transmission, and thus may exhibit a different distribution from the second feature vector (B3) representing the characteristics of the second audio signal (B1) compressed twice during recording and signal transmission.

[0070] For example, the classification module (30) can output probability information (A4) that the probability that the first audio signal (A1) has been compressed twice or more is 0.12, and can output probability information (B4) that the probability that the second audio signal (B1) has been compressed twice or more is 0.93.

[0071] The processor (120) can identify whether an audio signal is multi-compressed based on the probability information obtained through the classification module (30). In the example of FIG. 2, when the threshold probability is 0.9, the processor (120) can identify that the first audio signal (A1) has been compressed once, and can identify that the second audio signal (B1) has been compressed twice or more.

[0072] If it is determined that the audio signal has been compressed multiple times, the processor (120) can determine the security of the audio signal based on whether the audio signal has been compressed multiple times. For example, if the audio signal contains illegal elements such as security breaches, privacy invasions, and financial fraud, such as spam calls (e.g., robocalls), vishing, or voice phishing, the audio signal is likely to be pre-recorded or altered. Therefore, if it is determined that the audio signal has been compressed more than twice, the processor (120) can determine that the audio signal contains illegal elements as described above.

[0073] Additionally, the processor (120) may determine the quality of the audio signal based on whether the audio signal has been multi-compressed. For example, when converting a low-quality audio signal into a high-quality audio signal, the audio signal may be multi-compressed during the encoding process into the high-quality audio signal. Accordingly, if the audio signal is identified as having been compressed more than twice, the processor (120) may determine that the audio signal is actually low-quality, but has been converted (e.g., modulated) to have a high-quality format.

[0074] As described above, once the security or quality of an audio signal is determined, the processor (120) can provide the user with information regarding the determined security or quality of the audio signal. The subsequent steps after the security or quality of the audio signal is determined, including providing such information to the user, will be described below with reference to FIGS. 4 and 6.

[0075] According to the embodiment described above with reference to FIGS. 1 and 2, the electronic device (100) can analyze the characteristics of an audio signal using a plurality of modules including a neural network and accurately identify whether the audio signal is multi-compressed, thereby identifying spam calls, vishing, voice phishing, forgery of audio signals, audio quality, etc.

[0076] For example, the electronic device (100) can identify whether an audio signal is multi-compressed through its own analysis within the device without having to share the audio signal with an external server, thereby protecting the user's privacy and identifying whether an audio signal is multi-compressed without having to maintain a database.

[0077] Figure 3 is a block diagram exemplarily showing the configuration of modules according to various embodiments.

[0078] As illustrated in FIG. 3, a plurality of modules according to the present disclosure may further include a preprocessing module (10), a vector acquisition module (20), and a classification module (30), as well as an optimization module (40) for learning at least one of the preprocessing module (10), the vector acquisition module (20), and the classification module (30). In addition, the preprocessing module (10) and the classification module (30) may further include detailed modules, as illustrated in FIG. 3, respectively.

[0079] The preprocessing module (10) may include at least one codec preprocessing module (11) and a default preprocessing module (12). Here, the term "codec" refers to an audio codec, which is software used to compress an audio signal. The codec may be used to generate a compressed audio signal by compressing an audio signal converted into a digital signal to include at least some of the information contained in the audio signal. Each module may include all or part of a neural network.

[0080] For example, various codecs can be used to compress audio signals, and each codec can have different characteristics. For example, various audio codecs such as the Adaptive Multi-Rate (AMR) codec that can reduce data transmission bandwidth while maintaining call quality, the Advanced Audio Coding (AAC) codec used for high-quality audio compression, the Opus codec used for high-quality audio compression for Internet calls or voice chat, and the Vorbis codec that uses a lossy compression method can have different characteristics. Therefore, by considering the characteristics of each of the various audio codecs, the characteristics of the audio signal can be extracted more effectively.

[0081] The 'codec preprocessing module (11)' refers to a preprocessing module (10) corresponding to one of multiple codec types. The codec preprocessing module (11) is a preprocessing module (10) specialized for a specific codec, and can be said to be a module for extracting characteristic information of an audio signal by considering the characteristics of the codec used to compress the audio signal. The codec preprocessing module (11) may also be referred to by terms such as 'codec specific preprocessing module (10)'.

[0082] There is no limitation on the number of codec preprocessing modules (11), and a specific codec preprocessing module (11) may be updated by being added to or excluded from multiple modules.

[0083] For example, the first codec classification module (11-1) of FIG. 3 may be a codec preprocessing module (11) corresponding to an AMR codec, and the n-th codec classification module (11-n) may be a codec preprocessing module (11) corresponding to an AAC codec.

[0084] The 'default preprocessing module (12)' refers to a preprocessing module (10) that does not correspond to multiple codec types. When the codec type of an audio signal is not identified, the default preprocessing module (12) can obtain feature information by extracting general features of an audio signal without considering the characteristics of the codec.

[0085] In various embodiments, the processor (120) may identify the codec type of the audio signal. For example, the processor (120) may identify the codec type of the audio signal based on metadata of the audio signal or information about the transmission channel of the audio signal. For example, if the transmission channel of the audio signal is a GSM (global system for mobile communications) communication channel, which is one of the digital communication standards used for mobile phone communication, the processor (120) may identify the codec type of the audio signal as a GMS codec.

[0086] When the codec type of the audio signal is identified, the processor (120) inputs the audio signal to a codec preprocessing module (11) corresponding to the identified codec type among at least one codec preprocessing module (11), thereby obtaining feature information including features related to the identified codec. If the codec type of the audio signal is not identified, the processor (120) inputs the audio signal to a default preprocessing module (12), thereby obtaining feature information.

[0087] The classification module (30) may include at least one codec classification module (31) (e.g., including 31-1 to 31-n) and a default classification module (32). The 'codec classification module (31)' refers to a classification module (30) corresponding to one of a plurality of codec types, and thus may correspond to at least one codec preprocessing module (11). The codec classification module (31) is a classification module (30) specialized for a specific codec, and can identify whether an audio signal is multi-compressed by considering the characteristics of the codec used to compress the audio signal. The codec classification module (31) may also be referred to as a term such as a 'codec specific classification module (30) (codec specific preprocessing module)'.

[0088] There is no limitation on the number of codec classification modules (31), and a specific codec classification module (31) may be updated by being added to or excluded from multiple modules.

[0089] For example, the first codec classification module (31-1) of FIG. 3 may be a codec classification module (31) corresponding to an AMR codec, and the n-th codec classification module (31-n) may be a codec classification module (31) corresponding to an AAC codec.

[0090] The 'default classification module (32)' refers to a classification module (30) that does not correspond to multiple codec types, and thus can correspond to the default preprocessing module (12). The default preprocessing module (12) can identify whether an audio signal is multi-compressed without considering the characteristics of the codec, if the codec type of the audio signal is not identified.

[0091] In various embodiments, when the codec type of the audio signal is identified, the processor (120) inputs feature information obtained through at least one codec preprocessing module (11) into at least one codec classification module (31) corresponding to at least one codec preprocessing module (11), thereby identifying whether the audio signal has been compressed twice or more. If the codec type of the audio signal is not identified, the processor (120) inputs feature information obtained through the default preprocessing module (12) into the default classification module (32), thereby obtaining feature information.

[0092] By using the detailed modules of the preprocessing module (10) and classification module (30) described above, i.e., codec specialized modules, it is possible to accurately extract the features of an audio signal by considering the codec type of the audio signal, and accordingly, the accuracy of multi-compression detection of the audio signal can be improved.

[0093] In addition, if the codec type of the audio signal cannot be identified, it can be responded to first by using the default module, and furthermore, only the detailed modules related to a specific codec can be replaced / updated without changing other modules, so it has the advantage of being able to easily respond to various new types of codecs.

[0094] As illustrated in FIG. 3, the vector acquisition module (20) may be implemented as a single module. For example, in various embodiments, the vector acquisition module (20) may correspond to at least one codec preprocessing module (11) and a default preprocessing module (12), and may correspond to at least one codec classification module (31) and a default classification module (32).

[0095] For example, when feature information is acquired from an audio signal using at least one codec preprocessing module (11) and a default preprocessing module (12), the vector acquisition module (20) can acquire one feature vector by integrating the feature information acquired through each of at least one codec preprocessing module (11) and the default preprocessing module (12) and mapping it onto an integrated vector space.

[0096] Although an embodiment has been described in which multiple detailed modules included in each of the preprocessing module (10) and the classification module (30) correspond to a single vector acquisition module (20), this is merely exemplary. For example, depending on the embodiment, the vector acquisition module (20) may also include various detailed modules, such as detailed modules specialized for a specific codec.

[0097] However, as described above, if the outputs of multiple detailed modules included in the preprocessing module (10) are processed in one vector acquisition module (20), there is an advantage in that the size of the entire system can be reduced compared to using a separate module for each codec.

[0098] The optimization module (40) refers to a module capable of optimizing a plurality of modules according to the present disclosure. For example, the optimization module (40) can train the preprocessing module (10), the vector acquisition module (20), and the classification module (30) by applying various learning algorithms to the neural networks included in each of the preprocessing module (10), the vector acquisition module (20), and the classification module (30).

[0099] In various embodiments, the learning of the preprocessing module (10), the vector acquisition module (20), and the classification module (30) can be learned through backpropagation of the loss based on the probability information acquired through the classification module (30).

[0100] The learning dataset for learning the preprocessing module (10), vector acquisition module (20), and classification module (30) can be obtained based on a single compressed audio dataset generated by compressing a raw audio dataset once and a multi-compressed audio dataset generated by compressing a raw audio dataset twice or more.

[0101] For example, a single compressed audio dataset may be a dataset obtained by performing voice call encoding on a raw audio dataset, and a multi-compressed audio dataset may be a dataset obtained by performing voice call encoding after lossy encoding on a raw audio dataset. For example, various codecs may be used for voice call encoding on a raw audio dataset, and datasets for various codecs may be used for training at least one codec preprocessing module (11) and at least one codec classification module (31).

[0102] Once the training dataset is acquired, the processor (120) can train the preprocessing module (10), the vector acquisition module (20), and the classification module (30) to classify a single compressed audio dataset into a negative class (e.g., a class indicating a multi-uncompressed audio signal) and to classify a multi-compressed audio dataset into a positive class (e.g., a class indicating a multi-compressed audio signal).

[0103] For example, when probability information indicating whether or not an audio signal is multi-compressed is output through the classification module (30) during the learning process, the processor (120) obtains a loss (or loss value) based on the difference between the probability information output through the classification module (30) and the probability information of the label, and backpropagates the obtained loss to the preprocessing module (10), the vector acquisition module (20), and the classification module (30), thereby enabling the preprocessing module (10), the vector acquisition module (20), and the classification module (30) to learn.

[0104] The loss used in learning according to the present disclosure can be obtained using various loss functions such as mean squared error (MSE), cross-entropy loss, focal loss, L1 loss, etc., and there are no particular restrictions on the type of learning and the type of loss used in learning.

[0105] FIG. 4 is a block diagram exemplarily showing the configuration of an electronic device (100) according to various embodiments.

[0106] As illustrated in FIG. 4, the electronic device (100) may include not only a memory (110) and a processor (120) (e.g., including a processing circuit), but may also further include a communication unit (130) (e.g., including a communication circuit), an input unit (140) (e.g., including an input circuit), and an output unit (150) (e.g., including an output circuit). However, the configurations illustrated in FIGS. 1 and 4 are merely exemplary, and it is to be understood that new configurations may be added or some configurations may be omitted in addition to the configurations illustrated in FIGS. 1 and 4 when implementing the present disclosure.

[0107] The communication unit (130) includes a circuit and can communicate with an external device. For example, the processor (120) can receive various data or information from an external device connected via the communication unit (130) and can also transmit various data or information to the external device.

[0108] The communication unit (130) may include at least one of a WiFi module, a Bluetooth module, a wireless communication module, an NFC module, and a UWB (Ultra-Wide Band) module. Specifically, the WiFi module and the Bluetooth module may each perform communication in the WiFi or Bluetooth manner. When using a WiFi module or a Bluetooth module, various connection information, such as an SSID, may be first transmitted and received, and then communication may be established using this, after which various pieces of information may be transmitted and received.

[0109] In addition, the wireless communication module can perform communication according to various communication standards such as IEEE, Zigbee, 3G (3rd Generation), 3GPP (3rd Generation Partnership Project), LTE (Long Term Evolution), 5G (5th Generation), etc. And, the NFC module can perform communication in the NFC (Near Field Communication) method using the 13.56MHz band among various RF-ID frequency bands such as 135kHz, 13.56MHz, 433MHz, 860~960MHz, 2.45GHz, etc. In addition, the UWB module can accurately measure ToA (Time of Arrival), which is the time it takes for a pulse to reach a target, and AoA (Ange of Arrival), which is the pulse arrival angle at the transmitting device, through communication between UWB antennas, and accordingly, precise distance and location recognition is possible within an error range of several tens of centimeters indoors.

[0110] In various embodiments, the processor (120) may receive an audio signal from an external device through the communication unit (130), and may also control the communication unit (130) to transmit the audio signal to the external device. For example, when a phone call is made between the electronic device (100) and the external device, the processor (120) may allocate a frequency band for the call between the electronic device (100) and the external device through the communication unit (130), and establish a communication channel. Once the communication channel is established, the processor (120) may receive an audio signal from the external device through the established communication channel, and may control the communication unit (130) to transmit the audio signal to the external device through the established communication channel.

[0111] For example, various technologies such as VoIP (Voice over Internet Protocol), which is a technology for transmitting voice calls using Internet protocol, GSM (Global System for Mobile Communications), which is an international standard communication technology for mobile phone communication, or VoLTE (Voice over LTE), which is a voice call transmission technology based on LTE (Long-Term Evolution) network, can be used for phone calls between an electronic device (100) and an external device.

[0112] The processor (120) may control the communication unit (130) to transmit an audio signal received from an external device to an external output device (e.g., a Bluetooth speaker, wireless earphones, etc.), thereby allowing a sound corresponding to the audio signal to be output through the external output device.

[0113] When the processor (120) identifies that the audio signal has been compressed more than twice, it can transmit information about the transmitter of the audio signal to an external server, thereby registering the transmitter of the audio signal in a spam list managed by the external server.

[0114] In addition, the processor (120) can transmit and receive various data / information such as audio signals, feature information, feature vectors, data for multiple modules, learning data for learning multiple modules, probability information, information on codec types, etc., through the communication unit (130).

[0115] The input unit (140) includes a circuit, and the processor (120) can receive a user command to control the operation of the electronic device (100) through the input unit (140). For example, the input unit (140) can be configured with a microphone, a camera (not shown), and a remote control signal receiving unit (not shown). In addition, the input unit (140) can also be implemented in a form included in a display as a touch screen. For example, the microphone can receive a voice signal and convert the received voice signal into an electrical signal.

[0116] In various embodiments, the processor (120) may receive a user input for identifying whether an audio signal is multi-compressed through the input unit (140). In addition, if multi-compressed audio signal is identified through the input unit (140), the processor (120) may receive a user input for deleting data related to the audio signal or storing the transmitter of the audio signal in a blacklist.

[0117] The output unit (150) includes a circuit, and the processor (120) can output various functions that the electronic device (100) can perform through the output unit (150). In addition, the output unit (150) can include, for example, at least one of a display, a speaker, and an indicator.

[0118] The display can output image data under the control of the processor (120). For example, the display can output an image previously stored in the memory (110) under the control of the processor (120). For example, the display according to an embodiment of the present disclosure can also display a user interface stored in the memory (110). The display can be implemented as an LCD (Liquid Crystal Display Panel), an OLED (Organic Light Emitting Diodes), etc., and in some cases, the display can also be implemented as a flexible display, a transparent display, etc. However, the display according to the present disclosure is not limited to a specific type.

[0119] The speaker can output audio data under the control of the processor (120), and the indicator can be turned on under the control of the processor (120).

[0120] In various embodiments, the processor (120) may control a speaker to output an audio signal, and may also control a display to output information related to the audio signal or text corresponding to the audio signal. The text corresponding to the audio signal may be obtained using an automatic speech recognition model (ASR model).

[0121] In various embodiments, if an audio signal is identified as being compressed more than twice, the processor (120) may control the output unit (150) to output a warning message regarding the audio signal, rather than outputting the audio signal. The warning message may be output as a voice through a speaker, as text through a display, or may be displayed as a dot on an indicator.

[0122] In various embodiments, if an audio signal is identified as being compressed more than twice, the processor (120) may control the output unit (150) to output a guidance message requesting the user's selection of a follow-up action, such as deleting data related to the audio signal or storing the transmitter of the audio signal in a blacklist.

[0123] FIG. 5 is a flowchart exemplarily showing a method for controlling an electronic device (100) according to various embodiments.

[0124] The electronic device (100) can acquire an audio signal (S510). For example, the electronic device (100) can receive an audio signal from an external device through a communication unit (130) included in the electronic device (100), and can also receive an audio signal through a microphone included in the electronic device (100).

[0125] When an audio signal is acquired, the electronic device (100) inputs the audio signal into the preprocessing module (10) to acquire feature information representing features included in the audio signal (S520). The feature information may include information on at least one of features for the frequency domain of the audio signal, features for the time domain of the audio signal, and features for the frequency-time domain of the audio signal.

[0126] Once feature information is acquired, the electronic device (100) inputs the feature information into the vector acquisition module (20) to acquire a feature vector corresponding to the audio signal based on the feature information (S530). For example, once feature information for an audio signal is acquired, the electronic device (100) can convert the feature information into a feature vector by extracting high-level features included in the feature information for the audio signal.

[0127] In the present disclosure, feature information exemplarily represents information about low-level features that can be acquired by analyzing the audio signal itself, while feature vectors are obtained by converting low-level feature information into vectors and can exemplarily represent information about high-level features that can be mapped to a vector space. A feature vector is the result of converting feature information about an audio signal into a vector by digitizing it, and can be distinguished from feature information in that it can represent higher-level features than feature information.

[0128] Once the feature vector is acquired, the electronic device (100) inputs the feature vector into the classification module (30) to identify whether the audio signal has been compressed more than twice (S540). For example, the electronic device (100) can identify whether the audio signal has been compressed multiple times based on the probability information acquired through the classification module (30).

[0129] Once it is determined whether the audio signal has been multi-compressed, the electronic device (100) can determine the security of the audio signal based on whether the audio signal has been multi-compressed. For example, if it is determined that the audio signal has been compressed more than twice, the processor (120) can determine that the audio signal contains illegal elements as described above. Furthermore, the electronic device (100) can determine the quality of the audio signal based on whether the audio signal has been multi-compressed.

[0130] Once the security of the audio signal or the quality of the audio signal is determined, the electronic device (100) can provide information about the determined security of the audio signal or the quality of the audio signal to the user.

[0131] FIG. 6 is a flowchart illustrating an exemplary method for controlling an electronic device in relation to subsequent processing depending on whether an audio signal has been compressed more than twice according to various embodiments.

[0132] FIG. 6 describes an operation after step S540 of FIG. 5 is performed to identify whether an audio signal has been compressed twice or more.

[0133] As illustrated in FIG. 6, the electronic device (100) can identify whether the audio signal is multi-compressed (S610). If the audio signal is identified as not being multi-compressed (S610-N), the electronic device (100) may not take any subsequent actions. For example, when a user of the electronic device (100) is making a phone call with a user of an external device, the electronic device (100) may continue to receive audio signals from the external device and output them through a speaker or transmit them to an external output device (e.g., a Bluetooth speaker, wireless earphones, etc.).

[0134] If the audio signal is identified as being multi-compressed (S610-Y), the electronic device (100) may not output the audio signal (S620). For example, if the user of the electronic device (100) is making a phone call with the user of an external device, the electronic device (100) may stop receiving the audio signal from the external device, or even if it receives the audio signal from the external device, it may not output it through the speaker and may not transmit it to the external output device.

[0135] If the audio signal is identified as being multi-compressed (S610-Y), the electronic device (100) may output a warning message regarding the audio signal (S630). For example, the electronic device (100) may output a warning message through the speaker indicating that the audio signal may be a spam call, vishing, or voice phishing. The warning message may be output as a voice through the speaker or as text through the display.

[0136] Although FIG. 6 illustrates that when an audio signal is identified as being multi-compressed, the step of not outputting the audio signal (S620) and the step of outputting a warning message for the audio signal (S630) are both sequentially performed, the present disclosure is not limited thereto. For example, when an audio signal is identified as being multi-compressed, one of the step of not outputting the audio signal (S620) and the step of outputting a warning message for the audio signal (S630) may not be performed, and the step of outputting a warning message for the audio signal (S630) may be performed prior to the step of not outputting the audio signal (S620).

[0137] The electronic device (100) can receive a user input (S640). If no user input is received (S640-N), the electronic device (100) may not take any subsequent action. If a user input is received (S640-Y), the electronic device (100) can delete data related to the audio signal (S650) and store the transmitter of the audio signal in a blacklist (S660). For example, if it is identified that the audio signal received while the user of the electronic device (100) is making a phone call with the user of an external device is multi-compressed, the electronic device (100) can delete the audio signal received during the phone call and the data related thereto and store the phone number of the user of the external device in a blacklist.

[0138] When the transmitting entity of the audio signal is stored in the blacklist, the electronic device (100) can store the phone number of the user of the external device in the blacklist and then block phone calls coming from that phone number.

[0139] User input may be received not only after identifying whether the audio signal is multi-compressed as illustrated in FIG. 6, but may also be received in advance during a user setting process regarding what action to take when multi-compressed audio signals are identified.

[0140] For example, if multiple compression of an audio signal is identified, the step of not outputting the audio signal (S620) and the step of outputting a warning message for the audio signal (S630) can be automatically performed without user input. In addition, if user settings have been made in advance based on user input, not only the step of not outputting the audio signal (S620) and the step of outputting a warning message for the audio signal (S630), but also the step of deleting data related to the audio signal (S650) and the step of storing the transmitter of the audio signal in a blacklist (S660) can be automatically performed.

[0141] It goes without saying that the step of deleting data related to the audio signal (S650) and the step of storing the transmitting entity of the audio signal in a blacklist (S660) do not necessarily have to be performed sequentially.

[0142] The method for controlling an electronic device (100) according to the described embodiment may be implemented as a program and provided to the electronic device (100). In particular, the program including the method for controlling an electronic device (100) may be stored and provided in a non-transitory computer readable medium.

[0143] For example, in a non-transitory computer-readable recording medium including a program for executing a method for controlling an electronic device (100), the method for controlling an electronic device (100) may include a step of obtaining an audio signal, a step of inputting the audio signal to a preprocessing module (10) to obtain feature information representing a feature included in the audio signal, a step of inputting the feature information to a vector obtaining module (20) to obtain a feature vector corresponding to the audio signal based on the obtained feature information, and a step of inputting the feature vector to a classification module (30) to identify whether the audio signal has been compressed twice or more. The feature information may include information on at least one of a feature for a frequency domain of the audio signal, a feature for a time domain of the audio signal, and a feature for a frequency-time domain of the audio signal.

[0144] In the above, a method for controlling an electronic device (100) and a computer-readable recording medium including a program for executing the method for controlling an electronic device (100) have been briefly described, but this is only to omit redundant descriptions, and it goes without saying that various embodiments of the electronic device (100) can also be applied to a method for controlling an electronic device (100) and a computer-readable recording medium including a program for executing the method for controlling an electronic device (100).

[0145] According to various embodiments of the present disclosure as described above, the electronic device (100) can analyze the characteristics of an audio signal using a plurality of modules including a neural network and accurately identify whether the audio signal is multi-compressed, thereby identifying spam calls, vishing, voice phishing, forgery of audio signals, audio quality, etc.

[0146] For example, the electronic device (100) can identify whether an audio signal is multi-compressed through its own analysis within the device without having to share the audio signal with an external server, thereby protecting the user's privacy and identifying whether an audio signal is multi-compressed without having to maintain a database.

[0147] In addition, by using the detailed modules of the preprocessing module (10) and the classification module (30), i.e., the codec-specific modules, it is possible to accurately extract the features of the audio signal by considering the codec type of the audio signal, and accordingly, the accuracy of multi-compression detection of the audio signal can be improved.

[0148] If the codec type of the audio signal cannot be identified, it can be responded to first by using the default module, and furthermore, only the detailed modules related to a specific codec can be replaced / updated without changing other modules, so it has the advantage of being able to easily respond to various new types of codecs.

[0149] In addition, if the outputs of multiple detailed modules included in the preprocessing module (10) are processed in one vector acquisition module (20), there is an advantage in that the size of the entire system can be reduced compared to using a separate module for each codec.

[0150] The artificial intelligence-related function according to the present disclosure is operated through the processor (120) and memory (110) of the electronic device (100).

[0151] The processor (120) may be composed of one or more processors (120). At this time, the one or more processors (120) may include at least one of a CPU (Central Processing Unit), a GPU (Graphic Processing Unit), and an NPU (Neural Processing Unit), but is not limited to the examples of the processors (120) described above.

[0152] The CPU is a general-purpose processor (120) capable of performing not only general calculations but also artificial intelligence calculations. Its multi-layer cache structure allows for the efficient execution of complex programs. The CPU is advantageous in a serial processing method, enabling organic linking of previous and subsequent calculation results through sequential calculations. The general-purpose processor (120) is not limited to the examples described above, except in cases where it is specifically referred to as a CPU.

[0153] A GPU is a processor (120) for large-scale calculations such as floating-point operations used in graphics processing, and can perform large-scale calculations in parallel by integrating a large number of cores. For example, a GPU may be advantageous compared to a CPU in parallel processing methods such as convolution operations. In addition, a GPU may be used as a co-processor (120) to supplement the functions of a CPU. The processor (120) for large-scale calculations is not limited to the above-described examples, except in cases where it is specified as a GPU as described above.

[0154] An NPU is a processor (120) specialized in artificial intelligence operations using an artificial neural network, and each layer constituting the artificial neural network can be implemented with hardware (e.g., silicon). At this time, since the NPU is designed specifically according to the required specifications of the company, it has a lower degree of freedom compared to a CPU or GPU, but it can efficiently process the artificial intelligence operations requested by the company. As a processor (120) specialized in artificial intelligence operations, the NPU can be implemented in various forms such as a TPU (Tensor Processing Unit), an IPU (Intelligence Processing Unit), a VPU (Vision processing unit), etc. The artificial intelligence processor (120) is not limited to the above-described examples, except in cases where it is specified as the above-described NPU.

[0155] Additionally, one or more processors (120) may be implemented as a System on Chip (SoC). In addition to one or more processors (120), the SoC may further include a memory (110) and a network interface such as a bus for data communication between the processor (120) and the memory (110).

[0156] When a plurality of processors (120) are included in a SoC (System on Chip) included in an electronic device (100), the electronic device (100) may perform operations related to artificial intelligence (e.g., operations related to learning or inference of an artificial intelligence model) by using some of the processors (120) among the plurality of processors (120). For example, the electronic device (100) may perform operations related to artificial intelligence by using at least one of a GPU, an NPU, a VPU, a TPU, and a hardware accelerator specialized in artificial intelligence operations such as convolution operations and matrix multiplication operations among the plurality of processors (120). However, this is merely an exemplary embodiment, and it is of course possible to process operations related to artificial intelligence by using a CPU or a general-purpose processor (120).

[0157] In addition, the electronic device (100) can perform operations related to functions related to artificial intelligence by utilizing multiple cores (e.g., dual cores, quad cores, etc.) included in one processor (120). In particular, the electronic device (100) can perform artificial intelligence operations such as convolution operations, matrix multiplication operations, etc. in parallel by utilizing multiple cores included in the processor (120).

[0158] One or more processors (120) are controlled to process input data according to predefined operation rules or artificial intelligence models stored in the memory (110). The predefined operation rules or artificial intelligence models are characterized by being created through learning.

[0159] "Creating through learning" means applying a learning algorithm to a large number of learning data sets to create predefined behavioral rules or an AI model with desired characteristics. This learning can occur on the device itself, where the AI ​​according to the present disclosure is implemented, or through a separate server / system.

[0160] An artificial intelligence model may be composed of multiple neural network layers. At least one layer has at least one weight value and performs its operation through the operation result of the previous layer and at least one defined operation. Examples of neural networks include a convolutional neural network (CNN), a deep neural network (DNN), a recurrent neural network (RNN), a restricted boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), deep Q-networks, and a transformer. The neural networks in the present disclosure are not limited to the above-described examples unless otherwise specified.

[0161] A learning algorithm may include a method for training a given target device (e.g., a robot) using a plurality of learning data sets to enable the given target device to make decisions or predictions on its own. Examples of learning algorithms include supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning. Unless otherwise specified, the learning algorithms in this disclosure are not limited to the aforementioned examples.

[0162] A device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, a "non-transitory storage medium" refers to a tangible device and may not contain signals (e.g., electromagnetic waves). The term does not distinguish between cases where data is permanently stored in the storage medium and cases where data is temporarily stored. For example, a "non-transitory storage medium" may include a buffer in which data is temporarily stored.

[0163] According to one embodiment, the method according to the various embodiments disclosed in the present document may be provided in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) via an application store (e.g., Play Store™) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product (e.g., a downloadable app) may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0164] Each of the components (e.g., modules or programs) according to the various embodiments of the present disclosure as described above may be composed of a single or multiple entities, and some of the sub-components described above may be omitted, or other sub-components may be further included in the various embodiments. Alternatively or additionally, some components (e.g., modules or programs) may be integrated into a single entity, which may perform the same or similar functions as those performed by each of the respective components prior to integration.

[0165] According to various embodiments, operations performed by a module, program or other component may be executed sequentially, in parallel, iteratively or heuristically, or at least some operations may be executed in a different order, omitted, or other operations may be added.

[0166] The terms "part" or "module" as used herein include a unit composed of hardware, software, or firmware, or any combination thereof, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A "part" or "module" may be an integrally formed component, or a minimum unit or part thereof that performs one or more functions. For example, a module may be composed of an application-specific integrated circuit (ASIC).

[0167] Various embodiments of the present disclosure may be implemented as software including instructions stored in a machine-readable storage medium that can be read by a machine (e.g., a computer). The device may include an electronic device (e.g., an electronic device (100)) according to various embodiments of the disclosure, which is a device capable of calling instructions stored in the storage medium and operating according to the called instructions.

[0168] When the above instruction is executed by the processor, the processor may perform the function corresponding to the instruction directly or by using other components under the control of the processor. The instruction may include code generated or executed by a compiler or interpreter.

[0169] Although the preferred embodiments of the present disclosure have been illustrated and described above, the present disclosure is not limited to the specific embodiments described above, and various modifications may be made by those skilled in the art to which the present disclosure pertains without departing from the gist of the present disclosure as claimed in the claims, and such modifications should not be understood individually from the technical idea or prospect of the present disclosure. It will also be understood that any of the embodiments described in the present disclosure can be utilized in conjunction with other embodiments described in the present disclosure.

Claims

1. In electronic devices, a memory storing at least one instruction; and At least one processor comprising a processing circuit, executing at least one instruction; The above at least one processor individually and / or jointly, Acquire an audio signal, By inputting the above audio signal into a preprocessing module including a neural network, feature information representing the features included in the audio signal is obtained, By inputting the above feature information into a vector acquisition module including a neural network, a feature vector corresponding to the audio signal is acquired based on the acquired feature information, By inputting the above feature vector into a classification module including a neural network, it is identified whether the audio signal has been compressed twice or more, An electronic device wherein the feature information includes information on at least one of a feature for a frequency domain of the audio signal, a feature for a time domain of the audio signal, and a feature for a frequency-time domain of the audio signal.

2. In paragraph 1, The above preprocessing module corresponds to at least one of a plurality of codec types and includes at least one codec preprocessing module including a neural network, The above at least one processor individually and / or jointly, Identify the codec type of the above audio signal, An electronic device that, when the codec type of the audio signal is identified, inputs the audio signal to a codec preprocessing module corresponding to the identified codec type among the at least one codec preprocessing module, thereby obtaining feature information including features related to the identified codec.

3. In paragraph 2, The above preprocessing module does not correspond to the above multiple codec types and further includes a default preprocessing module including a neural network, The above at least one processor individually and / or jointly, An electronic device that inputs the audio signal to the default preprocessing module to obtain the feature information if the codec type of the audio signal is not identified.

4. In paragraph 3, The above classification module corresponds to at least one codec preprocessing module, and includes at least one codec classification module including a neural network, The above at least one processor individually and / or jointly, An electronic device that identifies whether the audio signal has been compressed twice or more by inputting the feature information obtained through the at least one codec preprocessing module into the at least one codec classification module corresponding to the at least one codec preprocessing module when the codec type of the audio signal is identified.

5. In paragraph 4, The above classification module corresponds to the above default preprocessing module and further includes a default classification module including a neural network, The above at least one processor individually and / or jointly, An electronic device that inputs the feature information obtained through the default preprocessing module into the default classification module if the codec type of the audio signal is not identified, thereby identifying whether the audio signal has been compressed twice or more.

6. In paragraph 5, The above at least one processor individually and / or jointly, An electronic device that identifies a codec type of an audio signal based on metadata of the audio signal or information about a transmission channel of the audio signal.

7. In paragraph 6, An electronic device wherein the above vector acquisition module corresponds to both the at least one codec preprocessing module and the default preprocessing module, and corresponds to both the at least one codec classification module and the default classification module.

8. In paragraph 1, The above classification module outputs probability information indicating whether the audio signal has been compressed twice or more, An electronic device wherein each of the preprocessing module, the vector acquisition module, and the classification module includes a neural network and learns according to backpropagation of loss based on the probability information.

9. In paragraph 1, An output section including an output circuit; further comprising: The above at least one processor individually and / or jointly, An electronic device that controls the output unit to output a warning message for the audio signal without outputting the audio signal if the audio signal is identified as being compressed twice or more.

10. In a method for controlling an electronic device, Step of acquiring an audio signal; A step of inputting the audio signal into a preprocessing module including a neural network to obtain feature information representing features included in the audio signal; A step of inputting the above feature information into a vector acquisition module including a neural network, and acquiring a feature vector corresponding to the audio signal based on the acquired feature information; and A control method of an electronic device, comprising the step of inputting the above feature vector into a classification module including a neural network to identify whether the audio signal has been compressed twice or more.

11. In Article 11, The above preprocessing module corresponds to at least one of a plurality of codec types and includes at least one codec preprocessing module including a neural network, The step of obtaining the above characteristic information is: A step of identifying the codec type of the above audio signal; and When the codec type of the above audio signal is identified, the method comprises the step of inputting the audio signal to a codec preprocessing module corresponding to the identified codec type among the at least one codec preprocessing module, thereby obtaining the feature information including the feature related to the identified codec. A control method of an electronic device, wherein the feature information includes information on at least one of a feature for a frequency domain of the audio signal, a feature for a time domain of the audio signal, and a feature for a frequency-time domain of the audio signal.

12. In paragraph 11, The above preprocessing module does not correspond to the above multiple codec types and further includes a default preprocessing module including a neural network, The step of obtaining the above characteristic information is: A control method of an electronic device, further comprising the step of inputting the audio signal to the default preprocessing module to obtain the feature information if the codec type of the audio signal is not identified.

13. In paragraph 12, The above classification module corresponds to at least one codec preprocessing module, and includes at least one codec classification module including a neural network, The step of identifying whether the above audio signal has been compressed more than twice is: A control method of an electronic device, comprising the step of inputting the feature information obtained through the at least one codec preprocessing module into the at least one codec classification module corresponding to the at least one codec preprocessing module, when the codec type of the audio signal is identified, to identify whether the audio signal has been compressed twice or more.

14. In paragraph 13, The above classification module corresponds to the above default preprocessing module and further includes a default classification module including a neural network, The step of identifying whether the above audio signal has been compressed more than twice is: A control method of an electronic device, further comprising the step of inputting the feature information obtained through the default preprocessing module into the default classification module to identify whether the audio signal has been compressed twice or more, if the codec type of the audio signal is not identified.

15. In paragraph 14, The step of identifying the codec type of the above audio signal is: A control method of an electronic device, comprising the step of identifying a codec type of the audio signal based on metadata of the audio signal or information about a transmission channel of the audio signal.

Images