Method, device and computer program for providing acoustic recognition result with improved reliability

Abstract

To provide a method, device and computer program for providing acoustic recognition results with improved accuracy according to various surrounding conditions, by dividing acoustic data into time-based segments and reducing the amount of calculation while improving a recognition rate in process of performing recognition for each segment.SOLUTION: In a system in which a user terminal, which is a wireless communication device that ensures portability and mobility, and an external server are connected to a server through a network to improve the recognition accuracy of acoustic data, a method implemented by the server includes the steps of: acquiring acoustic data using an artificial intelligence model; dividing the acoustic data to generate multiple pieces of acoustic sub-data; and processing the multiple pieces of acoustic sub-data as input to an acoustic recognition model to generate acoustic recognition result information corresponding to each acoustic sub-data.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to a method for improving a recognition rate of audio data, and more particularly, to a method for improving a recognition rate while reducing the amount of calculation in the process of dividing audio data into time-unit intervals and performing recognition for each interval, and for providing an audio recognition result with improved accuracy in response to various surrounding conditions. [Background technology]

[0002] Hearing-impaired people who cannot hear sounds at all or cannot distinguish sounds well have a hard time in daily life because they cannot judge situations by hearing sounds, and they cannot immediately respond because they cannot recognize urgent situations in indoor and outdoor environments using sound information. In situations where the hearing-impaired, pedestrians wearing earphones, elderly people, and others have no or limited hearing, the sounds generated around the user may be blocked. In addition, in situations where it is difficult for the user to detect sounds, such as when sleeping, the user may not be able to recognize the surrounding situation and may be placed in an urgent situation or have an accident. Meanwhile, there is an increasing need to develop technology to detect and recognize acoustic events in such environments. Technology to detect and recognize acoustic events is applicable to a variety of fields, such as context recognition in real-life environments, urgent situation recognition, media content recognition, and situation analysis in wired communications, and is therefore being continuously researched. As for acoustic event recognition technology, the main research is on extracting various feature values ​​such as MFCC, energy, spectral flux, zero crossing rate from audio signals and verifying excellent features, and on Gaussian mixture model or rule-based classification methods, and recently, deep learning-based machine learning methods have been researched to improve the above methods. However, such methods have limitations in that the accuracy of sound detection is guaranteed at low signal-to-noise ratios and it is difficult to distinguish between surrounding noise and the sound of an event.

[0003] That is, in a real-life environment including various ambient noises, it may be difficult to detect a sound event with high reliability. Specifically, in order to detect a valid sound event, it is necessary to determine whether a sound event has occurred from sound data acquired in a time series (i.e., continuously), and also to recognize what event class has occurred, so it may be difficult to ensure high reliability. In addition, when two or more events occur simultaneously, it is necessary to solve not only a single event (monophonic) but also a polyphonic event recognition problem, which may further reduce the sound event recognition rate. Therefore, there may be a demand for improving the recognition rate in response to acoustic data acquired in a time-series manner in a real-life environment, thereby providing acoustic recognition with improved reliability. [Patent Document 1] Republic of Korea Registered Patent 10-2014-0143069 Summary of the Invention [Problem to be solved by the invention]

[0004] The problem to be solved by the present invention is to solve the above-mentioned problems, and to improve the recognition rate while reducing the amount of calculation in the process of dividing sound data into time-unit intervals and performing recognition for each interval, and to provide a sound recognition result with improved accuracy in response to various surrounding conditions. The problems to be solved by the present invention are not limited to those mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the following description. [Means for solving the problem]

[0005] In order to solve the above problems, a method for providing an audio recognition result with improved reliability according to various embodiments of the present invention is disclosed, which includes the steps of acquiring audio data, dividing the audio data to generate a plurality of audio sub-data, and processing the plurality of audio sub-data as inputs of an audio recognition model to generate audio recognition result information corresponding to each of the audio sub-data. In an alternative embodiment, the step of generating the plurality of pieces of audio sub-data includes a step of generating the plurality of pieces of audio sub-data by dividing the audio data into units of a predetermined size, and the audio recognition model may include a first recognition model that receives each of the plurality of pieces of audio sub-data as an input and provides a plurality of outputs corresponding to each piece of audio sub-data, and a second recognition model that receives combined re-verification audio sub-data generated through a combination between the plurality of pieces of audio sub-data as an input and provides an output corresponding to the combined re-verification audio sub-data. In an alternative embodiment, the step of generating the acoustic recognition result information may include a step of selecting re-verification acoustic sub-data that requires re-verification of the acoustic sub-data based on the acoustic recognition result information output by the first recognition model corresponding to each of the acoustic sub-data, a step of generating combined re-verification acoustic sub-data based on the selected re-verification acoustic sub-data, and a step of processing the combined re-verification acoustic sub-data as an input to the second recognition model to generate acoustic recognition result information, and the second recognition model may be implemented through a cloud API. In an alternative embodiment, the step of selecting the re-verification acoustic sub-data may include deriving a similarity score between recognition items associated with the output of the first recognition model, and selecting the re-verification acoustic sub-data based on the calculated similarity score. In an alternative embodiment, the step of selecting the re-verification acoustic sub-data may include a step of identifying whether acoustic recognition result information related to the output of the first recognition model is included in a pre-set re-verification item, and if the acoustic recognition result information is included in the re-verification item, selecting the acoustic sub-data on which the acoustic recognition result information is calculated as re-verification acoustic sub-data that requires re-verification. In an alternative embodiment, the first recognition model outputs a probability value for each recognition item, and generates acoustic recognition result information based on the recognition item corresponding to the maximum probability value among the calculated probability values. The step of selecting the re-verification acoustic sub-data may include a step of selecting the acoustic sub-data on which the acoustic recognition result information is calculated as re-verification acoustic sub-data that requires re-verification when there are a plurality of recognition items that exceed a predetermined critical reference value among the probability values ​​for each recognition item calculated through the first recognition model. In an alternative embodiment, the step of generating the acoustic recognition result information may include the steps of: generating correlation information between the acoustic recognition result information; and performing correction on at least one of the acoustic recognition result information corresponding to each acoustic sub-data based on the correlation information. In an alternative embodiment, the step of correcting the at least one piece of acoustic recognition result information may include, when the first acoustic recognition result information and the second acoustic recognition result information are generated within a preset time, correcting at least one of the first acoustic recognition result and the second acoustic recognition result based on association information corresponding to the first acoustic recognition result and the second acoustic recognition result information. In an alternative embodiment, the method may further include generating mood information corresponding to the acoustic recognition result information by utilizing a mood detection model, the mood information being prediction information for an atmosphere related to a space in which acoustic data is acquired, and including location prediction information and emotion prediction information; generating the acoustic recognition result information may include generating association information between first acoustic recognition result information corresponding to first acoustic sub-data and the mood information corresponding to the first acoustic sub-data, not performing correction on the first acoustic recognition result information if the association information is equal to or greater than a preset reference value, and performing correction on the first acoustic recognition result information if the association information is less than a preset reference value; and the mood detection model may be a neural network model trained to recognize the acoustic recognition result information and output the mood information corresponding to an ambient situation at each time point.

[0006] According to another embodiment of the present invention, there is disclosed an apparatus for providing an audio recognition result with improved reliability, the apparatus including a memory for storing one or more instructions and a processor for executing the one or more instructions stored in the memory, the processor executing the one or more instructions to perform the method for providing an audio recognition result with improved reliability. According to yet another embodiment of the present invention, there is disclosed a computer program stored on a computer-readable recording medium, which is capable of performing the method for providing an audio recognition result having improved reliability as described above when combined with a computer, which is hardware. Further details of the invention are contained in the detailed description and the drawings. Effect of the Invention

[0007] According to various embodiments of the present invention, it is possible to reduce the amount of calculations while improving the recognition rate in the process of dividing sound data into time-unit intervals and performing recognition for each interval, and to provide sound recognition results with improved accuracy in response to various surrounding conditions. The effects of the present invention are not limited to those mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description. [Brief description of the drawings]

[0008] [Figure 1] 1 is a diagram illustrating a system for performing a method for providing an audio recognition result with improved reliability according to an embodiment of the present invention. [Diagram 2] 1 is a hardware configuration diagram of a server that performs a method for providing an audio recognition result with improved reliability according to an embodiment of the present invention. [Diagram 3] 1 shows a flow chart illustrating an exemplary method for providing audio recognition results with increased confidence in accordance with an embodiment of the present invention. [Figure 4A] 4 is an exemplary diagram illustrating a process of dividing audio data according to an embodiment of the present invention; FIG. [Figure 4B] 4 is an exemplary diagram illustrating a process of dividing audio data according to an embodiment of the present invention; FIG. [Diagram 5] 1 is a flowchart illustrating an example of a process of generating audio recognition result information through re-verification using a plurality of artificial intelligence models according to an embodiment of the present invention. [Figure 6] 4 is an exemplary diagram illustrating a process in which an audio recognition model according to an embodiment of the present invention derives an audio recognition result; [Figure 7] 4 is an exemplary diagram illustrating a process of recognizing acoustic data using a first recognition model and a second recognition model according to an embodiment of the present invention. FIG. [Figure 8] 10 is a flowchart illustrating an example of a process of correcting audio recognition result information according to an association relationship between audio sub-data according to an embodiment of the present invention. [Figure 9A] 4 is an exemplary diagram illustrating a process of correcting audio recognition result information related to an embodiment of the present invention; [Figure 9B]4 is an exemplary diagram illustrating a process of correcting audio recognition result information related to an embodiment of the present invention; [Figure 10] 4 is a flowchart illustrating an example of a process for generating audio recognition result information in consideration of a surrounding environment and an atmosphere according to an embodiment of the present invention. [Figure 11] 1 is an exemplary diagram illustrating a limit point of audio recognition result information derived through a general recognition process according to an embodiment of the present invention; [Figure 12] FIG. 13 is an exemplary diagram illustrating a process of grasping an overall atmosphere through corresponding audio sub-data before and after a specific audio sub-data in accordance with an embodiment of the present invention and outputting audio recognition result information with improved accuracy in consideration of the grasped atmosphere. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Various embodiments will now be described with reference to the drawings. Various descriptions are presented herein to provide an understanding of the present invention. However, it will be apparent that such embodiments may be practiced without such specific descriptions. As used herein, the terms "component," "module," "system," and the like refer to computer-related entities, hardware, firmware, software, a combination of software and hardware, or software implementations. For example, a component may be, but is not limited to, a procedure running on a processor, a processor, an object, a thread of execution, a program, and / or a computer. For example, an application running on a computing device and the computing device may all be a component. One or more components may reside within a processor and / or thread of execution. A component may be localized within one computer. A component may be distributed among two or more computers. Also, such components may execute from various computer readable media having various data structures stored therein. Components may communicate through local and / or remote processes, for example, by signals comprising one or more data packets (e.g., data from one component interacting with other components in a local system, a distributed system, and / or data transmitted over a network such as the Internet to other systems via signals). Additionally, the term "or" is intended to mean an inclusive "or" rather than an exclusive "or." That is, unless otherwise specified or clear from the context, it is to be understood as meaning one of the natural inclusive permutations of "X utilizes A or B." That is, if X utilizes A; X utilizes B; or X utilizes both A and B, then "X utilizes A or B" may apply to any of these cases. Additionally, the term "and / or" as used herein should be understood to refer to and include all possible combinations of one or more of the associated listed items. Additionally, the terms "comprise" and / or "comprises" should be understood to mean the presence of the indicated feature and / or component. However, the terms "comprise" and / or "comprises" should be understood not to exclude the presence or addition of one or more other features, components and / or groups thereof. Additionally, in the present specification and claims, the singular should generally be construed to mean "one or more," unless otherwise specified or clear from the context to indicate the singular form.

[0010] Those skilled in the art should further appreciate that the various exemplary logical blocks, configurations, modules, circuits, means, logic, and algorithm steps described in connection with the embodiments disclosed herein may be embodied in electronic hardware, computer software, or any combination of both. To clearly illustrate the interchangeability of hardware and software, the various exemplary components, blocks, configurations, means, logic, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is embodied in hardware or software depends on the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in various ways for each particular application. However, such implementation decisions should not be interpreted as causing a departure from the scope of the present invention. The description of the embodiments presented is provided to enable one of ordinary skill in the art to make and practice the present invention. Various modifications to the embodiments will be apparent to those of ordinary skill in the art. The generic principles defined herein may be applied to other embodiments without departing from the scope of the present invention, and the present invention is not limited to the embodiments presented herein. The present invention is to be accorded the widest scope consistent with the principles and novel features disclosed herein. In this specification, the term "computer" refers to any type of hardware device including at least one processor, and may also include software configurations that operate on the hardware device in accordance with the embodiment. For example, the term "computer" refers to any type of device, including, but not limited to, a smartphone, tablet PC, desktop, notebook computer, and user clients and applications that run on each device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. Although each step described in this specification is described as being performed by a computer, the subject of each step is not limited to this, and depending on the embodiment, at least some of each step may be performed by different devices. The method for providing an improved reliable audio recognition result according to various embodiments of the present invention may include a method for performing additional verification on recognized audio data so as to improve the recognition rate of the audio data, a method for correcting at least a part of the audio recognition result based on the audio recognition result output within a certain time, or a method for grasping the surrounding environment and atmosphere in real time and deriving the audio recognition result by considering the surrounding environment and atmosphere together with the audio data. In the embodiment, the improvement of the recognition accuracy of the audio data may mean that the recognition accuracy of detecting a specific event from the audio data is improved.

[0011] The method of performing additional verification on the acoustic data may mean, for example, a method of improving the reliability of the finally derived acoustic recognition result by deriving and verifying additional acoustic recognition results a plurality of times using a plurality of neural network models (e.g., acoustic recognition models). In this case, each of the plurality of neural network models may be a neural network model having a different output process. For example, each neural network model is a model having a different length section of acoustic data used as an input, and one model may be a model that derives an acoustic recognition result based on acoustic data of a 1-second section, and another model may be a model that derives an acoustic recognition result based on acoustic data of a 5-second section. However, without being limited thereto, the plurality of neural network models may be acoustic recognition models having different performances. As a specific example, acoustic recognition may be primarily performed using a first recognition model having relatively low performance to output the acoustic recognition result. Here, if it is determined that the reliability of the primarily output acoustic recognition result is somewhat low, acoustic recognition may be performed again using a second recognition model having higher performance than the first recognition model, and re-verification of the first acoustic recognition result may be performed. This is a configuration in which the first recognition model is normally used, and only re-verification is performed by using the second recognition model when necessary, and has the advantage of effectively reducing the amount of calculation while improving the reliability of the recognition result. Also, the method of correcting at least a part of the audio recognition results based on the audio recognition results output within a certain time period may mean, for example, post-processing correction of the audio recognition results corresponding to the audio data. The audio data may be divided and input at a certain time interval, and the present invention outputs the audio recognition results corresponding to each divided data. In this case, a specific audio recognition result may be corrected to another audio recognition result by grasping the correlation between the audio recognition results output within a certain time period. As a specific example, if an audio recognition result of sound A (e.g., gunshot) is output and then an audio recognition result of sound B (e.g., applause) is output within n seconds, the recognition result of sound A may be corrected to sound C (e.g., firecracker sound). In other words, the recognition accuracy may be improved by performing correction on at least one of the audio recognition results in consideration of the correlation between the audio recognition results output (or recognized) within a certain time period. In addition, the present invention can improve the accuracy of recognition of audio data by grasping the surrounding environment and atmosphere in real time and deriving an audio recognition result by considering the surrounding environment and atmosphere together with the acquired audio data. By using a separate neural network model (e.g., mood detection model) that detects the surrounding environment and atmosphere at the time when the audio data is acquired, the situation before and after the input of the audio data can be recognized, and by considering the situation together with the audio data in the audio analysis process, the reliability of the output audio recognition result can be improved. As a specific example, sound A (e.g., firecracker sound) can be recognized as a gunshot in a general audio recognition model. In the case of a short explosion sound, it is difficult to output an audio recognition result with high accuracy. The present invention can grasp the atmosphere corresponding to the situation in which the audio data is acquired based on the audio recognition results detected before and after the corresponding sound is detected, and derive an audio recognition result by considering the atmosphere together with the audio data. For example, by identifying that singing, clapping, and cheering were detected before and after the recognition of the corresponding sound A, the atmosphere associated with the corresponding sound A is determined to be a celebratory atmosphere (or a party, bright, or happy atmosphere), and the corresponding explosion sound is recognized as a 'firecracker sound' rather than a 'gunshot.' In other words, the overall atmosphere is determined based on the sound recognition results output before and after the sound data to be recognized, and the sound data is recognized in consideration of the determined atmosphere, thereby improving the output accuracy of the sound recognition results. A more detailed description of a method for improving the recognition accuracy of sound data will be provided below.

[0012] Fig. 1 is a diagram illustrating a system for performing a method for providing an audio recognition result with improved reliability according to an embodiment of the present invention. As illustrated in Fig. 1, the system for performing the method for providing an audio recognition result with improved reliability according to an embodiment of the present invention may include a server 100 for improving recognition accuracy of audio data, a user terminal 200, and an external server 300. Here, the system for performing the method for providing an audio recognition result with improved reliability illustrated in Fig. 1 is according to one embodiment, and its components are not limited to the embodiment illustrated in Fig. 1, and may be added, changed, or deleted as necessary. In one embodiment, the server 100 for improving the recognition accuracy of sound data may determine whether a specific event has occurred based on the sound data. Specifically, the server 100 for improving the recognition accuracy of sound data may acquire sound data related to real life and determine whether a specific event has occurred through analysis of the acquired sound data. In one embodiment, the specific event may be related to security, safety, or the occurrence of a danger, for example, an alarm sound, a child's crying, a glass breaking sound, a tire puncture sound, etc. The specific description of the sound related to the specific event described above is merely an example, and the present invention is not limited thereto. According to the embodiment, the server 100 for improving the recognition accuracy of sound data may include any server implemented by an API (Application Programming Interface). For example, the user terminal 200 may acquire sound data and transmit it to the server 100 through a Cloud API. For example, the server 100 may acquire sound data from the user terminal 200 and determine that an emergency alarm sound (e.g., a siren sound) has occurred through analysis of the sound data. In the embodiment, the server 100 for improving the recognition accuracy of sound data may perform analysis of the sound data through a sound recognition model (e.g., an artificial intelligence model). In one embodiment, an acoustic recognition model (e.g., an artificial intelligence model) may be comprised of one or more network functions, which may be comprised of a collection of interconnected computational units that may generally be referred to as "nodes." Such "nodes" may also be referred to as "neurons." One or more network functions may be comprised of at least one or more nodes. The nodes (or neurons) that make up one or more network functions may be connected to each other by one or more "links."

[0013] In an artificial intelligence model, one or more nodes connected through links can form a relationship between input nodes and output nodes relative to each other. The concepts of input nodes and output nodes are relative, and any node that has an output node relationship with one node can also have an input node relationship with another node, and vice versa. As mentioned above, the input node to output node relationship can be generated around links. One or more output nodes can be connected to one input node through links, and vice versa. In the relationship between an input node and an output node connected through a link, the value of the output node may be determined based on data input to the input node. Here, the node connecting the input node and the output node may have a weight. The weight may be variable and may be changed by a user or an algorithm so that the artificial intelligence model performs a desired function. For example, when one or more input nodes are connected to one output node through respective links, the output node may determine an output node value based on the value input to the input node connected to the output node and the weight set for the link corresponding to each input node. As described above, in an AI model, one or more nodes are connected to each other through one or more links to form a relationship between input nodes and output nodes within the AI ​​model. The characteristics of the AI ​​model can be determined by the number of nodes and links, the association between the nodes and links, and the weighted values ​​assigned to each link within the AI ​​model. For example, if there are two AI models that have the same number of nodes and links but different weighted values ​​between the links, the two AI models can be recognized as different from each other. Some of the nodes constituting the artificial intelligence model may constitute a layer based on the distance from the first input node. For example, a set of nodes whose distance from the first input node is n may constitute an n-th layer. The distance from the first input node may be defined by the minimum number of links that must be traversed to reach the corresponding node from the first input node. However, such a definition of a layer is arbitrary for the purpose of explanation, and the degree of a layer in an artificial intelligence model may be defined in a manner different from that described above. For example, the layer of a node may be defined by the distance from the final output node. The first input node may refer to one or more nodes to which data is directly input without passing through a link in relation to other nodes in the artificial intelligence model. Or, in relation between nodes based on a link in the artificial intelligence model network, it may refer to a node that does not have other input nodes connected to a link. Similarly, the final output node may refer to one or more nodes that do not have an output node in relation to other nodes in the artificial intelligence model. Also, the hidden node may refer to a node that constitutes the artificial intelligence model other than the first input node and the final output node. The artificial intelligence model according to an embodiment of the present invention may have more nodes in the input layer than the nodes in the hidden layer close to the output layer, and may be an artificial intelligence model in which the number of nodes decreases as it progresses from the input layer to the hidden layer.

[0014] The artificial intelligence model may include one or more hidden layers. The hidden nodes of the hidden layers may receive the output of the previous layer and the output of the surrounding hidden nodes as input. The number of hidden nodes in each hidden layer may be the same or different. The number of nodes in the input layer may be determined based on the number of data fields of the input data, and may be the same or different from the number of hidden nodes. The input data input to the input layer may be operated by the hidden nodes of the hidden layer and may be output by a fully connected layer (FCL), which is an output layer. In various embodiments, the artificial intelligence model may be subjected to supervised learning using a plurality of acoustic data and feature information corresponding to each acoustic data as learning data, but is not limited thereto and various learning methods may be applied. Here, supervised learning is a method of labeling specific data and information related to the specific data to generate learning data and learning using the same, and refers to a method of labeling two pieces of data that have a causal relationship to generate learning data and learning through the generated learning data. In one embodiment, the server 100 for improving the recognition accuracy of acoustic data may determine whether to stop the learning using verification data when the learning of one or more network functions is performed for a predetermined epoch or more. The predetermined epoch may be a part of the entire learning target epoch. The verification data may be composed of at least a part of the labeled learning data. That is, the server 100 for improving the recognition accuracy of acoustic data may perform learning of an artificial intelligence model through the learning data, and after the learning of the artificial intelligence model is repeated for a predetermined epoch or more, may determine whether the learning effect of the artificial intelligence model is equal to or higher than a predetermined level using the verification data. For example, when performing learning with a target number of repeated learning times of 10 times using 100 pieces of learning data, the server 100 for improving the recognition accuracy of acoustic data may perform 3 repeated learning times using 10 pieces of verification data after performing 10 repeated learning times, which are the predetermined epochs, and if the change in the output of the artificial intelligence model during the three repeated learning times is equal to or lower than a predetermined level, may determine that further learning is meaningless and terminate the learning. That is, the validation data may be used to determine the completion of learning based on whether the effect of the learning per epoch is above or below a certain level in the iterative learning of the artificial intelligence model. The numbers of learning data, validation data, and the number of iterations described above are merely examples, and the present invention is not limited thereto.

[0015] The server 100 for improving the recognition accuracy of acoustic data may generate an artificial intelligence model by testing the performance of one or more network functions using test data and determining whether or not to activate one or more network functions. The test data may be used to verify the performance of the artificial intelligence model and may be composed of at least a portion of the learning data. For example, 70% of the learning data may be used for learning the artificial intelligence model (i.e., learning to adjust weights to output result values ​​similar to the labels), and 30% may be used as test data for verifying the performance of the artificial intelligence model. The server 100 for improving the recognition accuracy of acoustic data may input the test data to the artificial intelligence model that has completed learning, measure an error, and determine whether or not to activate the artificial intelligence model depending on whether the performance is equal to or higher than a predetermined performance. The server 100 for improving the accuracy of recognizing acoustic data verifies the performance of the trained artificial intelligence model by using test data for the trained artificial intelligence model, and if the performance of the trained artificial intelligence model is above a predetermined standard, the corresponding artificial intelligence model can be activated for use in other applications. In addition, the server 100 for improving the recognition accuracy of acoustic data may deactivate and discard the AI ​​model if the performance of the AI ​​model that has completed learning is below a predetermined standard. For example, the server 100 for improving the recognition accuracy of acoustic data may determine the performance of the AI ​​model generated based on factors such as accuracy, precision, and recall. The above-mentioned performance evaluation criteria are merely examples, and the present invention is not limited thereto. The server 100 for improving the recognition accuracy of acoustic data may generate a plurality of AI models by independently learning each AI model, and may evaluate the performance to use only AI models with a certain level of performance or higher. However, the present invention is not limited thereto. Throughout this specification, the terms computational model, neural network, network function, and neural network may be used interchangeably (hereinafter, the term neural network will be used interchangeably). The data structure may include a neural network. The data structure including the neural network may be stored on a computer-readable medium. The data structure including the neural network may also include data input to the neural network, weights of the neural network, hyperparameters of the neural network, data obtained from the neural network, activation functions associated with each node or layer of the neural network, and loss functions for training the neural network. The data structure including the neural network may include any of the components disclosed above. That is, the data structure including the neural network may include all or any combination of data input to the neural network, weights of the neural network, hyperparameters of the neural network, data obtained from the neural network, activation functions associated with each node or layer of the neural network, and loss functions for training the neural network. In addition to the above-mentioned configurations, the data structure including the neural network may include any other information that determines the characteristics of the neural network. Furthermore, the data structure may include all types of data used or generated in the computational process of the neural network, and is not limited to the above. The computer readable medium may include a computer readable recording medium and / or a computer readable transmission medium. A neural network may be comprised of a collection of interconnected computational units that may generally be referred to as nodes. Such nodes may be referred to as neurons. A neural network is comprised of at least one or more nodes.

[0016] According to an embodiment of the present invention, the server 100 for improving the recognition accuracy of acoustic data may be a server that provides a cloud computing service. More specifically, the server 100 for improving the recognition accuracy of acoustic data may be a server that provides a cloud computing service, which is a type of Internet-based computing and processes information not on a user's computer but on another computer connected to the Internet. The cloud computing service may be a service that stores data on the Internet and allows a user to use the data or program he or she needs through an Internet connection anytime and anywhere without installing it on his or her computer, and allows the data stored on the Internet to be easily shared and transmitted with simple operations and clicks. In addition, the cloud computing service may be a service that not only simply stores data on a server on the Internet, but also allows a user to perform a desired task using a function of an application provided on the web without installing a separate program, and allows many people to share documents and work at the same time. In addition, the cloud computing service may be embodied in at least one form of Infrastructure as a Service (IaaS), Platform as a Service (PaaS), Software as a Service (SaaS), a virtual machine-based cloud server, and a container-based cloud server. That is, the server 100 for improving the recognition accuracy of acoustic data according to the present invention may be embodied in the form of at least one of the above-mentioned cloud computing services. The above-mentioned specific description of the cloud computing service is merely exemplary, and the present invention may include any platform for constructing a cloud computing environment. In various embodiments, a server 100 for improving the recognition accuracy of acoustic data may be connected to a user terminal 200 via a network and may generate and provide an acoustic recognition model for analyzing acoustic data, and may also provide information on the acoustic data analyzed through the acoustic recognition model (e.g., acoustic recognition result information) to the user terminal. Here, the network may refer to a connection structure that allows information exchange between multiple nodes such as terminals and servers, etc. For example, the network may include a local area network (LAN), a wide area network (WAN), the Internet (WWW), wired / wireless data communication networks, telephone networks, wired / wireless television communication networks, etc. In addition, the wireless data communication network includes, but is not limited to, 3G, 4G, 5G, 3GPP (3rd Generation Partnership Project), 5GPP (5th Generation Partnership Project), LTE (Long Term Evolution), WIMAX (World Interoperability for Microwave Access), Wi-Fi, the Internet, LAN (Local Area Network), Wireless LAN (Wireless Local Area Network), WAN (Wide Area Network), PAN (Personal Area Network), RF (Radio Frequency), Bluetooth network, NFC (Near-Field Communication) network, satellite broadcasting network, analog broadcasting network, DMB (Digital Multimedia Broadcasting) network, etc.

[0017] In one embodiment, the user terminal 200 may be connected to a server 100 for improving the accuracy of recognition of sound data via a network, and may provide sound data to the server 100 for improving the accuracy of recognition of sound data, and may receive sound recognition result information regarding the occurrence of various events (e.g., the occurrence of gunshots, alarm sounds, children crying, glass breaking sounds, flat tire sounds, etc.) in response to the provided sound data. Here, the user terminal 200 is a wireless communication device that is guaranteed to be portable and mobile, and may include, but is not limited to, all kinds of handheld-based wireless communication devices such as navigation, personal communication system (PCS), global system for mobile communications (GSM), personal digital cellular (PDC), personal handyphone system (PHS), personal digital assistant (PDA), international mobile telecommunication (IMT)-2000, code division multiple access (CDMA)-2000, W-code division multiple access (W-CDMA), wireless broadband internet (Wibro) terminal, smartphone, smartpad, tablet PC, etc. For example, the user terminal 200 may be installed in a specific area to perform sensing related to the specific area. For example, the user terminal 200 may be installed in a vehicle and acquire sound data generated while the vehicle is parked or running. As another example, the user terminal 200 may be provided for detecting an urgent situation corresponding to an area where children are located (e.g., a playground or a space within a facility). The above description of the specific location or place where the user terminal is provided is merely an example, and the present invention is not limited thereto. In one embodiment, the external server 300 may be connected to the server 100 for improving the recognition accuracy of acoustic data through a network, and may provide various information / data necessary for the server 100 for improving the recognition accuracy of acoustic data to analyze acoustic data using an artificial intelligence model, or may receive, store, and manage result data derived as the server 100 for improving the recognition accuracy of acoustic data performs the acoustic data analysis using an artificial intelligence model. For example, the external server 300 may be, but is not limited to, a storage server separately provided outside the server 100 for improving the recognition accuracy of acoustic data. Hereinafter, a hardware configuration of the server 100 for improving the recognition accuracy of acoustic data will be described with reference to FIG. 2. FIG. 2 is a hardware configuration diagram of a server for improving the recognition accuracy of acoustic data according to an embodiment of the present invention. Referring to Fig. 2, a server 100 for improving recognition accuracy of acoustic data according to an embodiment of the present invention (hereinafter, "server 100") may include one or more processors 110, a memory 120 for loading a computer program 151 executed by the processor 110, a bus 130, a communication interface 140, and a storage 150 for storing the computer program 151. Here, Fig. 2 illustrates only components related to the embodiment of the present invention. Therefore, a person skilled in the art to which the present invention belongs will understand that other general components may be included in addition to the components illustrated in Fig. 2.

[0018] The processor 110 controls the overall operation of each component of the server 100. The processor 110 may include a central processing unit (CPU), a microprocessor unit (MPU), a microcontroller unit (MCU), a graphic processing unit (GPU), or any other type of processor commonly known in the technical field of the present invention. The processor 110 can read a computer program stored in the memory 120 and perform data processing for an artificial intelligence model according to an embodiment of the present invention. According to an embodiment of the present invention, the processor 110 can perform calculations for learning a neural network. The processor 110 can perform calculations for learning a neural network, such as processing input data for learning using deep learning (DL), extracting features from the input data, calculating errors, and updating weights of the neural network using backpropagation. In addition, at least one of the processor 110, a CPU, a GPGPU, and a TPU, may process the learning of a network function. For example, the CPU and the GPGPU may both process the learning of a network function and data classification using the network function. In addition, in one embodiment of the present invention, the processors of a plurality of computing devices may be used together to process the learning of a network function and data classification using the network function. In addition, the computer program executed in the computing device according to one embodiment of the present invention may be a CPU, GPGPU, or TPU executable program. As used herein, a network function may be used interchangeably with an artificial neural network or a neural network. As used herein, a network function may include one or more neural networks, in which case the output of the network function may be an ensemble of the outputs of the one or more neural networks. The processor 110 may provide an acoustic recognition model according to an embodiment of the present invention by reading a computer program stored in the memory 120. According to an embodiment of the present invention, the processor 110 may perform calculations for training the acoustic recognition model. According to one embodiment of the present invention, the processor 110 may generally process the overall operation of the server 100. The processor 110 may process signals, data, information, etc. input or output through the components detailed above, or may run applications stored in the memory 120, thereby providing or processing appropriate information or functions to a user or a user terminal. Furthermore, the processor 110 may execute operations for at least one application or program for executing a method according to an embodiment of the present invention, and the server 100 may include one or more processors. In various embodiments, the processor 110 may further include a random access memory (RAM, not shown) and a read-only memory (ROM, not shown) for temporarily and / or permanently storing signals (or data) processed within the processor 110. The processor 110 may also be implemented in the form of a system on chip (SoC) including at least one of a graphics processing unit, a RAM, and a ROM. The memory 120 stores various data, instructions and / or information. The memory 120 can load a computer program 151 from the storage 150 to perform the methods / operations according to various embodiments of the present invention. When the computer program 151 is loaded into the memory 120, the processor 110 can perform the methods / operations by executing one or more instructions constituting the computer program 151. The memory 120 may be embodied as a volatile memory such as a RAM, although the scope of the present invention is not limited in this respect.

[0019] The bus 130 provides a communication function between the components of the server 100. The bus 130 may be implemented as various types of buses such as an address bus, a data bus, and a control bus. The communication interface 140 supports wired / wireless Internet communication of the server 100. The communication interface 140 may also support various communication methods other than Internet communication. To this end, the communication interface 140 may be configured to include a communication module that is widely known in the technical field of the present invention. In some embodiments, the communication interface 140 may be omitted. The storage 150 may non-temporarily store a computer program 151. When a process for improving the recognition accuracy of acoustic data is performed through the server 100, the storage 150 may store various information required to provide the process for improving the recognition accuracy of acoustic data. Storage 150 may be configured to include non-volatile memory such as ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), flash memory, etc., a hard disk, a removable disk, or any form of computer-readable recording medium commonly known in the technical field to which the present invention belongs. The computer program 151, when loaded into the memory 120, may include one or more instructions that cause the processor 110 to perform the methods / operations according to various embodiments of the present invention. That is, the processor 110 may execute the one or more instructions to perform the methods / operations according to various embodiments of the present invention. In one embodiment, the computer program 151 may include one or more instructions for performing a method for providing an acoustic recognition result with improved reliability, including the steps of acquiring acoustic data, dividing the acoustic data to generate a plurality of acoustic sub-data, and processing the plurality of acoustic sub-data as inputs to an acoustic recognition model to generate acoustic recognition result information corresponding to each acoustic sub-data. The steps of a method or algorithm described in connection with the embodiments of the present invention may be embodied directly in hardware, in a software module executed by hardware, or in a combination thereof. The software module may reside in a Random Access Memory (RAM), a Read Only Memory (ROM), an Erasable Programmable ROM (EPROM), an Electrically Erasable Programmable ROM (EEPROM), a Flash Memory, a hard disk, a removable disk, a CD-ROM, or any other form of computer readable recording medium commonly known in the art to which the present invention pertains.

[0020] The components of the present invention may be embodied as a program (or application) and stored on a medium to be executed in combination with a computer, which is hardware. The components of the present invention may be implemented as software programming or software elements, and similarly, embodiments include various algorithms embodied as a combination of data structures, processes, routines or other programming constructs, and may be embodied in programming or scripting languages ​​such as C, C++, Java, assembler, etc. Functional aspects may be embodied as algorithms executed by one or more processors. Hereinafter, a method for providing an audio recognition result with improved reliability performed by the server 100 will be described with reference to Figures 3 to 12. 3 is a flow chart showing an example of a method for providing an audio recognition result with improved reliability according to an embodiment of the present invention. The order of the steps shown in FIG. 3 may be changed as necessary, and at least one step may be omitted or added. That is, the following steps are merely an embodiment of the present invention, and the scope of the present invention is not limited thereto. According to an embodiment of the present invention, a method for providing an audio recognition result with improved reliability may include acquiring audio data (S100). A server 100 may acquire audio data 10. The audio data 10 may include information related to audio acquired in real life. The acquisition of audio data according to an embodiment of the present invention may involve receiving or loading the audio data 10 stored in a memory 120. Furthermore, the acquisition of audio data may involve receiving or loading data from another storage medium, another computing device, or a separate processing module in the same computing device based on wired / wireless communication means. According to an embodiment, the acoustic data 10 may be acquired through a user terminal 200 associated with a user. For example, the user terminal 200 associated with a user may include any kind of handheld-based wireless communication device such as a smartphone, a smartpad, a tablet PC, etc., or an electronic device (e.g., a device capable of receiving acoustic data through a microphone) provided in a specific space (e.g., a user's residential space), etc. According to an embodiment of the present invention, a method for providing an audio recognition result with improved reliability may include dividing audio data to generate a plurality of audio sub-data (S200). In the embodiment, the step of generating a plurality of audio sub-data may include dividing the audio data into a predetermined size unit to generate a plurality of audio sub-data. The server 100 can divide the audio data 10 into a certain time unit to generate a plurality of audio sub-data. The plurality of audio sub-data can be obtained by dividing the audio data, which is time-series information, into a plurality of frames based on a specific time unit. Specifically, the server 100 can divide the audio data to have a size of a first predetermined time unit to generate a plurality of audio sub-data. In the case of audio (or sound), since it is time-series data that is continuously generated, it must be divided into a certain time interval and recognition must be performed for each interval. For example, if the first acoustic data is acoustic data acquired corresponding to a time period of one minute, the server 100 may set the first time unit to one second and divide the first acoustic data to configure 60 acoustic sub-data (or acoustic sub-frames). The above-mentioned specific numerical values ​​related to the first time unit and one or more acoustic sub-data are merely examples, and the present invention is not limited thereto. In the embodiment, in the process of dividing the acoustic data into a plurality of acoustic sub-data, the smaller the time unit is, the less the amount of calculation of the neural network (i.e., acoustic recognition model) is, and the performance is improved. When the amount of calculation is significantly reduced due to the shortening of the length of the acoustic sub-data corresponding to the input, the acoustic recognition model can be used as a short section recognition model, and therefore can be used as an Edge SDK. That is, by dividing the acoustic data into a plurality of short sections, the acoustic recognition model can be driven through one application executable by the server itself. In the embodiment, in the case of a specific sound, it may be difficult to derive sound recognition result information with improved reliability through a sound recognition model that performs short-duration recognition. For example, in the case of a sound with a short duration, it may be difficult to accurately recognize the corresponding sound by referring to only a specific duration. 4A, the first sound data 10a related to a gunshot having a short duration may be divided into the first sound subdata 11a to the eighth sound subdata 18a in units of one second. In this case, the gunshot occurring in the third sound subdata 13a is entirely included in the section corresponding to the third sound subdata 13a, so accurate recognition may be possible even if only one section (i.e., the third sound subdata 13a) is taken into consideration.

[0021] Meanwhile, referring to FIG. 4B, the second sound data 10b related to a gunshot having a short duration may also be divided into the first sound sub-data 11b to the eighth sound sub-data 18b in units of 1 second. In this case, the gunshot may exist across the second sound sub-data 12b and the third sound sub-data 13b as shown in FIG. 4B. Since the sound data is data acquired in a time series and is divided at a certain size, the explosion sound and the echo of the gunshot may exist in different sections as described above. In this case, when recognizing the second sound sub-data 12b and the third sound sub-data 13b through the sound recognition model, accurate recognition may be impossible because the complete gunshot is not included, and as a result, the accuracy of the output (i.e., the recognition accuracy) may be reduced. That is, the accuracy of sound recognition may decrease because short-lasting sounds are divided into different sections. To overcome this, the time unit for dividing the sound can be made larger (e.g., 2 seconds), but this increases the amount of calculation required for the sound recognition model, which is inefficient and cannot be used in low-cost devices. Accordingly, the server 100 of the present invention can perform audio recognition again in a situation where the recognition accuracy is lacking by additionally configuring a separate neural network model. That is, by using a plurality of neural network models to derive additional audio recognition results a plurality of times and performing re-verification, the reliability of the finally derived audio recognition result can be improved. In particular, by performing additional verification in response to a specific situation where the recognition accuracy is predicted to be somewhat lacking, rather than simply using a plurality of neural network models a plurality of times, the reliability of the recognition result can be improved while effectively reducing the amount of calculation. The configuration for improving the recognition accuracy by performing re-verification using a plurality of neural network models will be described in detail below with reference to FIG. 5. 5 is a flow chart showing an example of a process of generating audio recognition result information through re-verification using a plurality of artificial intelligence models according to an embodiment of the present invention. The order of the steps shown in FIG. 5 may be changed as necessary, and at least one step may be omitted or added. That is, the following steps are merely an embodiment of the present invention, and the scope of the present invention is not limited thereto. According to one embodiment, the step of generating acoustic recognition result information may include a step (S311) of selecting re-verification acoustic sub-data that requires re-verification of the acoustic sub-data based on the acoustic recognition result information output by the first recognition model 210 corresponding to each acoustic sub-data. According to an embodiment, the acoustic recognition model of the present invention may include a first recognition model 210 that receives a plurality of acoustic sub-data as inputs and provides a plurality of outputs corresponding to the respective acoustic sub-data. According to an embodiment, the server 100 can train the autoencoder through an unsupervised learning method. Specifically, the server 100 can train a dimension reduction network function (e.g., an encoder) and a dimension restoration network function (e.g., a decoder) that configure the autoencoder to output output data similar to the input data. In more detail, only the core feature data (or features) of the audio data input during the encoding process through the dimension reduction network function can be learned through a hidden layer, and the remaining information can be lost. In this case, the output data of the hidden layer during the decoding process through the dimension restoration network function can be an approximation of the input data (i.e., audio data) rather than a perfect copy value. That is, the server 100 can train the autoencoder by adjusting the weights so that the output data and the input data are as similar as possible.

[0022] An autoencoder may be a type of neural network for outputting output data similar to input data. An autoencoder may include at least one hidden layer, and an odd number of hidden layers may be disposed between input and output layers. The number of nodes in each layer may be reduced from the number of nodes in the input layer to an intermediate layer called a bottleneck layer (encoding), and then expanded symmetrically from the bottleneck layer to an output layer (symmetric to the input layer). The number of input layers and output layers may correspond to the number of items of input data remaining after preprocessing of the input data. In the autoencoder structure, the number of nodes in the hidden layer included in the encoder may decrease as it moves away from the input layer. The number of nodes in the bottleneck layer (the layer with the fewest nodes located between the encoder and the decoder) may be maintained at a certain number or more (e.g., more than half of the input layer) because if the number is too small, a sufficient amount of information may not be transmitted. The server 100 may use a training data set including a plurality of training data tagged with object information as an input of a trained dimension reduction network to match and store the output feature data for each object with the tagged object information. Specifically, the server 100 may use a dimension reduction network function to obtain feature data of a first object for training data included in the first training data subset by using a first training data subset tagged with first sound discrimination information (e.g., the sound of breaking glass) as an input of the dimension reduction network function. The obtained feature data may be expressed as a vector. In this case, the feature data output corresponding to each of the plurality of training data included in the first training data subset is an output through the training data related to the first sound, so that they may be located relatively close to each other in the vector space. The server 100 may match and store the feature data related to the first sound represented as a vector with the first sound discrimination information (i.e., the sound of breaking glass). In the case of a trained autoencoder dimensionality reduction network function, the dimensionality restoration network function can be trained to extract features well that allow the input data to be well restored. Also, for example, a plurality of training data included in each of the first training data subsets tagged with the second sound discrimination information (e.g., the sound of a siren) may be converted into feature data (i.e., features) through a dimensionality reduction network function and displayed in a vector space. In this case, since the feature data is an output through the training data related to the second sound discrimination information (i.e., the sound of a siren), it may be located relatively close to each other in the vector space. In this case, the feature data corresponding to the second sound discrimination information may be displayed in a vector space different from the feature data corresponding to the first sound discrimination information (e.g., the sound of glass breaking). In an embodiment, the server 100 may configure an audio recognition model including a dimension reduction network function in the trained autoencoder. That is, when audio subdata is inputted into the audio recognition model including the dimension reduction network function generated through the training process, the audio subdata is operated using the dimension reduction network function to extract feature information (i.e., features) corresponding to the audio subdata. In this case, the audio recognition model can evaluate the similarity of the audio style through a distance comparison between an area in which a feature corresponding to the audio subdata is displayed and the object-specific feature data in a vector space, and can output audio recognition result information corresponding to the audio subdata based on the similarity evaluation. In one embodiment, the audio recognition result information can include a plurality of recognition items and a probability value corresponding to each of the plurality of recognition items.

[0023] Specifically, the acoustic recognition model (or the first recognition model) can output feature information (i.e., features) by computing the acoustic subdata using a dimension reduction network function. In this case, the acoustic recognition model can output a plurality of recognition items corresponding to the acoustic subdata and a probability value corresponding to each recognition item based on the feature information output corresponding to the acoustic subdata and the position between the feature data of each acoustic identification information pre-recorded on the vector space through learning. The plurality of recognition items are for identifying which items the sound is related to, and may include, but are not limited to, gunshots, firecrackers, screams, the sound of a flat tire, sirens, a barking puppy, rain, etc. Such a plurality of recognition items may be generated based on sound discrimination information that is close to the feature information outputted corresponding to the sound subdata in a vector space. For example, the sound recognition model may configure the plurality of recognition items through sound discrimination information that is matched to the feature information that is close to the first feature information outputted corresponding to the first sound subdata. The above-mentioned specific description related to the plurality of recognition items is merely an example, and the present invention is not limited thereto. The probability value corresponding to each recognition item may be information about the accuracy predicted for each recognition item. For example, the acoustic recognition model may configure a plurality of recognition item information through acoustic discrimination information matched with feature information located close to the first feature information outputted corresponding to the first acoustic subdata. In this case, a higher probability value may be calculated as the first feature information is closer to the feature information corresponding to each acoustic discrimination information, and a lower probability value may be calculated as the feature information corresponding to each acoustic discrimination information is farther from the first feature information. As a specific example, as shown in FIG. 6, the audio recognition model (or the first recognition model) may output a plurality of recognition items 310, such as "a siren sound", "a scream", "a glass breaking sound" and "other sounds", in response to the first audio sub-data. The audio recognition model may also output probability values ​​320 of "1", "95", "3" and "2" in response to each recognition item 310. That is, the audio recognition model may output probability values ​​320 in which the probability of being related to the siren sound is "1", the probability of being related to the scream is "95", the probability of being related to the glass breaking sound is "3", and the probability of being related to other sounds is "2" in response to the first audio sub-data. The above descriptions of specific values ​​for each of the recognition items and probability values ​​are merely examples, and the present invention is not limited thereto. That is, the server 100 can output a recognition item corresponding to each of a plurality of acoustic subdata generated based on acoustic data through the acoustic recognition model and a probability value corresponding to each recognition item. For example, the acoustic recognition model can output a first recognition item and a probability value corresponding to each of the first recognition items in response to the first acoustic subdata, and can output a second recognition item and a probability value corresponding to each of the second recognition items in response to the second acoustic subdata. According to an embodiment, the first recognition model 210 may be characterized in that it calculates a probability value for each recognition item and generates audio recognition result information based on the recognition item corresponding to the maximum probability value among the calculated probability values. For example, referring to Fig. 6, audio recognition result information is generated that the recognition result of the audio sub-data is scream through the recognition item corresponding to the maximum probability value of 95%. In addition, in the embodiment, the first recognition model 210 may be characterized in that it calculates a probability value for each recognition item, and generates acoustic recognition result information depending on whether the calculated probability value exceeds a critical probability value previously set for each recognition item. According to various embodiments, the present invention may be provided with a plurality of acoustic recognition models. The acoustic recognition models may include a first recognition model 210 and a second recognition model 220. In this case, the second recognition model 220 may be a model in which the length of the section of the acoustic sub-data used for input is different from that of the first recognition model 210. The second recognition model 220 may be a neural network model having higher performance than the first recognition model 210. For example, the first recognition model 210 may be a neural network model trained to derive an acoustic recognition result based on acoustic sub-data of a 1-second section, and the second recognition model 220 may be a neural network model that derives an acoustic recognition result based on acoustic sub-data of a 2-second section. That is, the first recognition model 210 and the second recognition model 220 are neural network models having different performances, and the second recognition model 220 with better performance is used for re-verification. In an embodiment, the first recognition model 210 may be a neural network model that recognizes a relatively short section of audio sub-data and outputs the result, and the second recognition model 220 may be a neural network model that recognizes a relatively long section of audio sub-data and outputs the result.

[0024] The first recognition model 210 has an advantage that the amount of calculation is small and the associated consumption of computing power is small because it analyzes a short section of sound. On the other hand, the second recognition model 220 has a disadvantage that the amount of calculation increases because it analyzes a relatively long section of sound, making it difficult to install and use it as an application program in a low-cost device, but has an advantage that the accuracy of the output is high because the recognition section is long. In various embodiments, the server 100 may determine whether to perform re-verification using another neural network model (e.g., a second recognition model) based on the output result, i.e., acoustic recognition result information, corresponding to the first recognition model 210. This is to reduce the amount of calculation and the computing power used by using the light first recognition model 210 under normal circumstances, but using an additional neural network model only when the output accuracy is lacking. The server 100 may select re-verification audio sub-data that requires re-verification of the audio sub-data based on the audio recognition result information output corresponding to each audio sub-data by the first recognition model 210. The server 100 may perform re-verification and select the audio sub-data to be re-verified when the audio recognition result information is related to a pre-classified item (e.g., an item that requires re-verification) or when it is determined that the accuracy of the audio recognition result information is unclear. In a specific embodiment, the server 100 may calculate a similarity score between each recognition item calculated through the first recognition model 210, and select the re-verification audio sub-data based on the calculated similarity score. For example, the first recognition model may output recognition items such as "gunshots," "balloon bursting," and "fireworks" in response to specific sound sub-data, and may calculate the probability values ​​of each recognition item to be 85%, 90%, and 94%. In this case, sound recognition result information may be generated based on "fireworks" having the highest probability value. Meanwhile, the server 100 may calculate a similarity score between the recognition items output by the first recognition model 210. In an embodiment, the similarity score between the recognition items may be calculated based on a similarity evaluation between sounds corresponding to each item. As an example, an embedding corresponding to each recognition item may be obtained by inputting representative sound sub-data corresponding to each recognition item into a dimension reduction network function, and the similarity between the recognition items may be evaluated through the embedding similarity. In addition, the similarity between each recognition item may be evaluated using a cosine similarity that measures the similarity by calculating the cosine angle between embedded word vectors. As another example, a Siamese network and a transformer-based model may be used to determine semantic text similarity. The above-mentioned detailed description of the method for evaluating the similarity between keywords is merely an example, and the present invention is not limited thereto. In a further embodiment, the similarity between keywords or words may be measured using Jaccard similarity, Euclidean distance, and WordNet. A similarity score is derived based on the similarity evaluation between the recognition items output by the first recognition model 210. The higher the similarity between the recognition items, the higher the similarity score calculated, and the lower the similarity between the recognition items, the lower the similarity score calculated. According to an embodiment, the higher the similarity between the recognition items, the lower the output accuracy of the first recognition model 210 may be. For example, when the recognition items output by the first recognition model 210 in response to the A sound sub-data are "scream", "crash sound" and "sound of falling rain", the acoustic similarity between the corresponding items is low, and the similarity score is calculated to be low. That is, the more different the recognition items that are candidates for result prediction are, the more clearly the features of each item are different, and the reliability of the output of the first recognition model is guaranteed. On the other hand, when the recognition items output by the first recognition model 210 in response to the B sound sub-data are "gunshot", "sound of balloon bursting" and "sound of fireworks", the acoustic similarity between the corresponding items is high, and the similarity score can be calculated to be high. That is, the recognition items that are candidates for result prediction are very similar, and it is difficult to distinguish the features of each item, and the reliability of the output of the first recognition model may be somewhat low. In other words, when recognition items whose acoustic features are completely dissimilar (e.g., screams, cracking sounds, and falling rain sounds) are output, the final recognition result is likely to be accurate (there is a high probability that the neural network will provide a correct output because the acoustic features of each recognition item are completely different), but when recognition items whose acoustic features are highly similar (e.g., gunshots, firecrackers, and fireworks) are output, the final recognition result is likely to be somewhat inaccurate.

[0025] That is, the server 100 can determine whether the acoustic recognition result information is reliable by calculating the similarity scores of the recognition items output by the first recognition model 210. If the similarity scores between the recognition items are low (i.e., below a certain reference value), the server 100 can determine that the acoustic recognition result information is reliable and not perform a separate reverification of the acoustic sub-data, and if the similarity scores between the recognition items are high (i.e., exceeding a certain reference value), the server 100 can determine that the reliability of the acoustic recognition result information is low and select the acoustic sub-data as reverification acoustic sub-data that requires reverification. In an embodiment, re-validation of the acoustic sub-data may mean re-deriving the acoustic recognition result information using the second recognition model 220 . In addition, if there are multiple recognition items that exceed a preset critical standard value among the probability values ​​for each recognition item calculated through the first recognition model 210, the server 100 can select the audio sub-data on which the audio recognition result information is calculated as re-verification audio sub-data that requires re-verification. For example, the first recognition model may output the recognition items "gunshot", "balloon bursting sound" and "fireworks sound" corresponding to the specific sound sub-data, and may calculate the probability values ​​of each recognition item to be 85%, 90% and 94%. In this case, sound recognition result information is generated based on the fireworks sound having the highest probability value. However, all of the other recognition items may exceed the critical reference value (e.g., 82%, 85% and 88%) previously set for each recognition item. This may mean that the reliability of the sound recognition result information is somewhat reduced because the first recognition model 210 has determined that the other recognition items are also highly probable. For example, when the probability values ​​of the recognition items a, b and c are 97%, 6% and 3%, respectively, only the recognition item a has a probability value that exceeds the critical reference value (e.g., 80%) previously set, and therefore the reliability of the sound recognition result information may be high. In other words, the fewer the number of recognition items whose probability values ​​exceed the corresponding critical criterion value, the more likely it is that the final recognition result output will be accurate (e.g., if there is only one recognition item whose probability value exceeds the corresponding critical criterion value, the recognition result will likely have the highest reliability), but the more the number of recognition items whose probability values ​​exceed the corresponding critical criterion value, the more likely it is that the final recognition result output will be somewhat inaccurate. That is, the server 100 can determine whether the audio recognition result information is reliable by identifying whether there are multiple recognition items exceeding a preset critical reference value among the probability values ​​of each recognition item calculated through the first recognition model 210. If there is one recognition item whose probability value exceeds the critical reference value, the server 100 can determine that the audio recognition result information is reliable and not perform a separate reverification of the audio sub-data, and if there are multiple recognition items (i.e., two or more) whose probability values ​​exceed the critical reference value, the server 100 can determine that the audio recognition result information is low and select the audio sub-data as reverification audio sub-data that requires reverification. In addition, in an embodiment, the step of selecting re-verification acoustic sub-data may include a step of identifying whether acoustic recognition result information related to the output of the first recognition model 210 is included in a pre-set re-verification item, and if the acoustic recognition result information is included in the re-verification item, selecting the acoustic sub-data on which the acoustic recognition result information is calculated as re-verification acoustic sub-data that requires re-verification. The preset re-verification items may be a combination of items predefined in advance that are expected to reduce the accuracy of the output to some extent. For example, in the case of an explosion sound, it is difficult to distinguish between a gunshot, a balloon bursting, a firecracker exploding, and a fireworks display because the acoustic features are similar, so if the audio recognition result information is generated from one of the corresponding sounds, it is difficult to ensure sufficient reliability. Therefore, the present invention can construct a preset re-verification item by pre-storing items that are difficult to classify acoustic features, and when the first recognition model 210 outputs acoustic recognition result information included in the preset re-verification item, the server 100 can select the acoustic sub-data based on which the corresponding acoustic recognition result information is generated as re-verification acoustic sub-data that requires re-verification. That is, a pre-defined re-verification item can be pre-configured based on similar sounds that are difficult to distinguish, and if the output of the first recognition model (i.e., the sound recognition result information) corresponds to the pre-defined re-verification item, the corresponding sound sub-data can be determined as the re-verification sound sub-data.

[0026] According to an embodiment, the step of generating the audio recognition result information may include the step of generating combined re-verification audio sub-data based on the selected re-verification audio sub-data (S312). Specifically, when the reverification audio sub-data is selected, the server 100 may identify additional audio sub-data related before and after the reverification audio sub-data. Also, the server 100 may combine the reverification audio sub-data with the additional audio sub-data to generate combined reverification audio sub-data. Also, in the embodiment, the step of generating the acoustic recognition result information may include processing the combined re-verification acoustic sub-data as an input to the second recognition model 220 to generate the acoustic recognition result information (S313). According to an embodiment, the acoustic recognition model of the present invention may include a second recognition model 220 that receives combined re-verification acoustic sub-data generated through a combination between a plurality of acoustic sub-data as input and provides an output corresponding to the combined re-verification acoustic sub-data. In this case, the second recognition model 220 may be a model in which the length of the section of the audio sub-data used as input is different from that of the first recognition model 210. The second recognition model 220 may be a neural network model having higher performance than the first recognition model 210. For example, the first recognition model 210 may be a neural network model trained to derive an audio recognition result based on audio sub-data of a 1-second section, and the second recognition model 220 may be a neural network model that derives an audio recognition result based on audio sub-data of a 2-second section. That is, the first recognition model 210 and the second recognition model 220 are neural network models having different performances, and the second recognition model 220, which has better performance, is used for re-verification. In the embodiment, the first recognition model 210 may be a neural network model that recognizes audio sub-data of a relatively short section and outputs a result, and the second recognition model 220 may be a neural network model that recognizes audio sub-data of a relatively long section and outputs a result. In the embodiment, the second recognition model 220 may be implemented through a cloud API. Since the second recognition model 220 is implemented through a cloud API, it may be possible to use the second recognition model 220 with relatively high performance even in a low-performance device. As a specific example, referring to FIG. 7, a relatively low-performance, lightweight first recognition model 210 can be utilized to input a plurality of acoustic sub-data (first acoustic sub-data 11, second acoustic sub-data 12, and third acoustic sub-data 13) generated primarily corresponding to acoustic data 10, and output acoustic recognition result information. Here, if it is determined that the reliability of the acoustic recognition result information initially output corresponding to the second acoustic sub-data 12 is somewhat low, the acoustic sub-data corresponding to the acoustic recognition result information (i.e., the second acoustic sub-data) can be selected as re-verification acoustic sub-data that requires re-verification. In this case, the server 100 can identify additional audio subdata generated before and after the second audio subdata 12. For example, the server 100 can identify the third audio subdata 13, which is audio subdata generated at a time adjacent to the second audio subdata 12, as additional audio subdata.

[0027] The server 100 generates combined re-verification audio subdata by combining the re-verification audio subdata with the additional audio subdata. That is, the combined re-verification audio subdata 20 may be generated by combining the second audio subdata 12 and the third audio subdata 13. In this case, the combined re-verification audio sub-data 20 may have a longer length than the audio sub-data input to the initial first recognition model 210 because it is generated through a combination of audio sub-data. That is, when it is determined that the recognition result for the audio sub-data may be inaccurate, the combined re-verification audio sub-data may be generated so that re-verification may be performed on the audio sub-data leading to the inaccurate output. The server 100 processes the combined re-verification audio sub-data 20 as an input to the second recognition model 220 and outputs audio recognition result information. In this case, since the second recognition model is a neural network model that performs recognition on audio sub-data of a longer section than the first recognition model, the second recognition model derives audio recognition results through analysis of audio sub-data (i.e., combined re-verification audio sub-data) of a larger range than the first recognition model, and therefore can generate audio recognition result information with higher accuracy. In other words, the server 100 utilizes a relatively lightweight first recognition model, and performs only re-verification using a second model as necessary, thereby effectively reducing the amount of calculation while improving the reliability of the recognition result. FIG. 8 is a flow chart illustrating an example of a process of correcting audio recognition result information according to a correlation between audio sub-data in accordance with an embodiment of the present invention. According to the embodiment, the step of generating the audio recognition result information may include the step of generating correlation information between the plurality of pieces of audio recognition result information (S321). The correlation information may be information indicating correlation between each piece of audio recognition result information. For example, if the first audio recognition result information generated as a recognition result for the first audio sub-data corresponding to the first section is a "gunshot" and the second audio recognition result information generated as a recognition result for the second audio sub-data corresponding to the second section is a "clap sound", the server 100 may generate correlation information indicating that the first audio recognition result information and the second audio recognition result information are not related to each other. The above-mentioned specific descriptions of each piece of audio recognition result information and the specific descriptions of the correlation information are merely examples, and the present invention is not limited thereto. Also, the step of generating the audio recognition result information may include a step of correcting at least one of the audio recognition result information corresponding to each audio subdata based on the association information (S322). The server 100 may correct at least one of the audio recognition result information based on the association information between the audio recognition result information. The server 100 may determine whether to correct the audio recognition result information based on the association information between each audio recognition information. According to a specific embodiment, the step of correcting at least one piece of audio recognition result information may include, when the first audio recognition result information and the second audio recognition result information are generated within a preset time, correcting at least one of the first audio recognition result and the second audio recognition result based on association information corresponding to the first audio recognition result and the second audio recognition result information. That is, the server 100 can perform post-processing correction on at least a part of the audio recognition result information based on the audio recognition result information output within a certain time. The audio data is divided at a certain time interval and input to a neural network in the form of frames, and audio recognition result information corresponding to each divided data is derived. The server 100 generates association information between the audio recognition results output within a certain time, and can thereby correct a specific audio recognition result information to another audio recognition result according to the association between the generated audio recognition results corresponding to adjacent time points. This configuration is for improving accuracy when generating audio recognition result information based on only audio sub-data. For example, if an audio recognition result of sound A (e.g., gunshot) is output and then an audio recognition result of sound B (e.g., applause) is output within n seconds, the recognition result of sound A can be corrected to sound C (e.g., firecracker sound). That is, by performing correction for at least one of the audio recognition results in consideration of the correlation between the audio recognition results output (or recognized) within a certain time, the recognition accuracy can be improved.

[0028] As a more specific example, when a single frame (i.e., one audio subdata) as shown in FIG. 9A is input to an audio recognition model, audio recognition result information "gunshot" may be generated. This is a result derived by simply analyzing only one audio subdata related to an explosion sound, so high accuracy cannot be guaranteed. For example, in the case of gunshot, there are many sounds with similar acoustic characteristics, such as firecracker sounds, fireworks, or balloon bursting sounds, so there is a risk of some lack of accuracy. As shown in FIG. 9B, based on specific audio recognition result information (i.e., gunshot), audio recognition result information corresponding to frames existing within a certain time period can be identified as voices singing a birthday song, clapping sounds, and sounds of joy. The server 100 can generate association information indicating that the specific audio recognition result (i.e., gunshot) is not related to other audio recognition result information through other audio recognition results output within a certain time period. Accordingly, the specific audio recognition result information is post-processed and corrected to other audio recognition results. That is, the audio recognition result information of gunshot output first can be corrected to other audio recognition result information, "sound of firecrackers exploding," based on association information generated based on association with other audio recognition result information output within a certain time period (e.g., voices singing a birthday song, clapping sounds, sounds of joy). In the embodiment, the server 100 can identify that the first output gunshot is included in a specific item (e.g., an item corresponding to an explosion sound), and select other types of keywords included in the specific item to perform correction on the audio recognition result information. The server 100 can select a keyword (e.g., the sound of a firecracker exploding) that is related to other sound recognition results from among other types of sounds included in a specific item, and perform correction to the initially output sound recognition result information. Accordingly, the initially output sound recognition result information of "gunshot" is corrected to "the sound of a firecracker exploding." That is, the server 100 can improve the accuracy of recognition by performing post-processing correction on at least one of the acoustic recognition result information in consideration of the correlation between the acoustic recognition result information output (or recognized) within a certain time. In other words, the server 100 can improve the accuracy of the acoustic recognition result by identifying specific acoustic recognition result information that is not similar to the acoustic recognition result information output at a recent time and correcting it to other acoustic recognition result information that is related to the other acoustic recognition result information. FIG. 10 is a flowchart illustrating an example of a process for generating audio recognition result information in consideration of the surrounding environment and atmosphere according to an embodiment of the present invention. According to the embodiment, the server 100 may generate mood information corresponding to the audio recognition result information through a mood detection model. The mood detection model may be a neural network model trained to recognize the audio recognition result information and output mood information corresponding to the surrounding situation at each time point. Mood information is prediction information for an atmosphere related to a space in which sound data is acquired, and may include location prediction information and emotion prediction information. The mood sensing model may be a neural network model that grasps a real-time atmosphere based on the audio recognition result information derived through each audio sub-data. The mood sensing model may generate mood information including prediction information on where the current audio data will be acquired and prediction information on the atmosphere where the audio data will be acquired through the audio recognition result information. The mood sensing model may analyze the user's emotions through keyword analysis and voice pitch analysis, and may recognize the location where the dialogue is being performed. In an embodiment, the mood sensing model may include a natural language processing model and a sentiment analysis model. The natural language processing model may be used to understand and process text data (e.g., audio recognition result information), and the sentiment analysis model may be used to detect sentiment contained in a text.

[0029] For example, by detecting that sound recognition result information related to vehicle running noise is continuously acquired, it can be determined that the location where the current sound data is acquired is a "vehicle running". As another example, by detecting that sound recognition result information of gradually higher tones is acquired, it can be determined that the real-time atmosphere is related to the emotion of "anger". The above-mentioned specific descriptions of the locations and emotions are merely examples, and the present invention is not limited thereto. The server 100 can generate audio recognition result information based on the mood information and audio sub-data generated through the mood detection model in order to recognize the situation or atmosphere before and after the input of the audio sub-data and generate audio recognition result information with higher accuracy by taking this into consideration. Referring to FIG. 11, the specific audio sub-data can be processed as an input of an audio recognition model to derive audio recognition result information of "child scream". However, it may be difficult to distinguish whether the child's scream is related to an urgent situation or a child playing screams because he or she is having fun. That is, since not all screams indicate an urgent situation, it is necessary to distinguish whether the scream is related to an urgent situation or occurred in a happy situation. Since the present invention can generate mood information corresponding to the atmosphere around the specific audio sub-data through a mood model, audio recognition result information suitable for a more appropriate situation can be derived. The server 100 can grasp the surrounding atmosphere, i.e., mood information, by utilizing the mood detection model, and generate more appropriate audio recognition result information by utilizing the mood information. For example, as shown in Fig. 12, the server 100 can detect "water splashing", "children's laughter", and "mother's voice calling child" at adjacent points in time, and generate audio recognition result information of "child's scream in a happy situation" corresponding to specific audio sub-data. In a specific embodiment, the step of generating the acoustic recognition result information may include a step of generating association information between the first acoustic recognition result information corresponding to the first acoustic subdata and mood information corresponding to the first acoustic subdata (S331). In the embodiment, the association information may be information indicating a degree of association between mood information and audio recognition result information generated in real time, expressed as a numerical value. For example, if the mood information is related to "feelings of happiness" and the audio recognition result information is related to "gunshots", the server 100 may determine that the degree of association between the mood information and the audio recognition result information is low and generate association information with a 20% association between the mood information and the audio recognition result information. As another example, if the mood information is related to "feelings of anger and fear" and the audio recognition result information is related to "gunshots", the server 100 may determine that the degree of association between the mood information and the audio recognition result information is high and generate association information with a 93% association between the mood information and the audio recognition result information. The above-mentioned specific description of the generation of association information is merely an example, and the present invention is not limited thereto. The step of generating the acoustic recognition result information may include a step of not performing correction on the first acoustic recognition result information if the correlation information is equal to or greater than a preset reference value (S332). The preset reference value is determined in advance by a user and may be a reference for determining whether the acoustic recognition result information output in real time is appropriate. If the correlation information between the acoustic recognition result information and the mood information is equal to or greater than a preset reference value, the currently output acoustic recognition result information may be determined to be appropriate and no additional correction may be performed. The step of generating the audio recognition result information may include a step of performing correction on the first audio recognition result information when the correlation information is less than a preset reference value (S333).When the correlation information between the audio recognition result information and the mood information is less than a preset reference value, the accuracy of the currently output audio recognition result information is reduced, so a separate correction should be performed. More specifically, the server 100 may identify a plurality of keywords having similarity to the first sound recognition result information. For example, if the first sound recognition result information is "gunshot", the server 100 may identify a plurality of keywords such as "firecracker sound", "fireworks sound", "balloon bursting sound", etc. The above-mentioned specific description of the first sound recognition result information and the plurality of keywords is merely illustrative, and the present invention is not limited thereto. In addition, the server 100 can generate multiple relevance sub-information between each of the multiple keywords and the mood information. That is, multiple relevance sub-information can be generated by calculating the degree of relevance between each keyword and the mood information. For example, if the mood information is "happy atmosphere," a first relevance sub-information of 93% can be generated corresponding to the sound of firecrackers, a second relevance sub-information of 82% can be generated corresponding to the sound of fireworks being set off, and a third relevance sub-information of 76% can be generated corresponding to the sound of balloons bursting. The specific numerical values ​​for each relevance sub-information are merely examples, and the present invention is not limited thereto.

[0030] The server 100 may identify maximum relevance sub-information corresponding to a maximum value among the plurality of relevance sub-information, and may perform correction on the first audio recognition result information based on a keyword corresponding to the maximum relevance sub-information. For example, the server 100 may perform correction on the first audio recognition result information based on a keyword (i.e., firecracker sound) corresponding to the maximum relevance sub-information (93%). That is, when the server 100 determines that the specific sound recognition result information does not match the mood information (i.e., when the correlation information between the sound recognition result information and the mood information is less than a preset reference value), the server 100 can perform correction on the specific sound recognition result information using the keyword that is most correlated with the mood information among the keywords similar to the corresponding sound recognition result. The steps of a method or algorithm described in connection with the embodiments of the present invention may be embodied directly in hardware, in a software module executed by hardware, or in a combination thereof. The software module may reside in a Random Access Memory (RAM), a Read Only Memory (ROM), an Erasable Programmable ROM (EPROM), an Electrically Erasable Programmable ROM (EEPROM), a Flash Memory, a hard disk, a removable disk, a CD-ROM, or any other form of computer readable recording medium commonly known in the art to which the present invention pertains. The components of the present invention may be embodied as a program (or application) and stored on a medium for execution in conjunction with a computer, which is hardware. The components of the present invention may be implemented as software programming or software elements, and similarly, embodiments include various algorithms embodied as a combination of data structures, processes, routines, or other programming constructs, and may be embodied in programming or scripting languages ​​such as C, C++, Java, assembler, etc. Functional aspects may be embodied as algorithms executed by one or more processors. Although the embodiments of the present invention have been described above with reference to the attached drawings, those skilled in the art to which the present invention pertains will understand that the present invention can be embodied in other specific forms without changing the technical spirit or essential features of the present invention. Therefore, the embodiments described above should be understood to be illustrative in all respects and not restrictive.

Claims

1. 1. A method performed by one or more processors of a computing device, comprising: acquiring acoustic data; dividing the acoustic data to generate a plurality of acoustic sub-data; and processing the plurality of acoustic sub-data as inputs of an acoustic recognition model to generate acoustic recognition result information corresponding to each acoustic sub-data.

2. The step of generating the plurality of sound subdata includes: Dividing the sound data into a predetermined size unit to generate a plurality of sound sub-data; The acoustic recognition model is 2. The method for providing an acoustic recognition result with improved reliability as claimed in claim 1, further comprising: a first recognition model that receives each of the plurality of acoustic subdata as an input and provides a plurality of outputs corresponding to each acoustic subdata; and a second recognition model that receives combined re-verification acoustic subdata generated through a combination between the plurality of acoustic subdata as an input and provides an output corresponding to the combined re-verification acoustic subdata.

3. The step of generating the audio recognition result information includes: selecting re-verification acoustic sub-data that needs to be re-verified based on acoustic recognition result information output by the first recognition model corresponding to each of the acoustic sub-data; generating combined re-verification audio sub-data based on the selected re-verification audio sub-data; and processing the combined re-verification acoustic sub-data as an input to the second recognition model to generate acoustic recognition result information; The second recognition model is The method for providing an audio recognition result with improved reliability according to claim 2, wherein the method is implemented through a cloud API.

4. The step of selecting the re-verification acoustic sub-data includes: deriving a similarity score between the recognition items associated with the output of the first recognition model; and The method for providing an audio recognition result with improved reliability according to claim 3 , further comprising the step of: selecting re-verification audio sub-data based on the calculated similarity score.

5. The step of selecting the re-verification acoustic sub-data includes: identifying whether acoustic recognition result information associated with the output of the first recognition model is included within a preset revalidation item; and 4. The method for providing an acoustic recognition result with improved reliability as claimed in claim 3, further comprising: selecting, when the acoustic recognition result information is included in the re-verification item, the acoustic sub-data on which the acoustic recognition result information is calculated as re-verification acoustic sub-data that requires re-verification.

6. The first recognition model is a probability value for each recognition item is output, and acoustic recognition result information is generated based on the recognition item corresponding to the maximum probability value among the calculated probability values; The step of selecting the re-verification acoustic sub-data includes:

4. The method for providing an acoustic recognition result with improved reliability as claimed in claim 3, further comprising the step of: selecting the acoustic sub-data on which the acoustic recognition result information is calculated as re-verification acoustic sub-data that requires re-verification when there are a plurality of recognition items that exceed a predetermined critical reference value among the probability values ​​for each recognition item calculated through the first recognition model.

7. The step of generating the audio recognition result information includes: generating correlation information between the acoustic recognition result information; and 2. The method of claim 1, further comprising: performing correction on at least one of the pieces of acoustic recognition result information corresponding to each piece of acoustic subdata based on the correlation information.

8. The step of correcting at least one piece of audio recognition result information comprises:

10. The method of claim 7, wherein, when the first acoustic recognition result information and the second acoustic recognition result information are generated within a predetermined time, at least one of the first acoustic recognition result and the second acoustic recognition result is corrected based on association information corresponding to the first acoustic recognition result and the second acoustic recognition result information.

9. The method comprises: generating mood information corresponding to the audio recognition result information by utilizing a mood sensing model; The mood information includes: Prediction information for an atmosphere related to a space in which sound data is acquired, the prediction information including location prediction information and emotion prediction information; The step of generating the audio recognition result information includes: generating association information between first acoustic recognition result information corresponding to first acoustic sub-data and the mood information corresponding to the first acoustic sub-data; not performing correction on the first audio recognition result information when the correlation information is equal to or greater than a preset reference value; and performing a correction on the first audio recognition result information when the relevance information is less than a preset reference value; The mood sensing model is The method of claim 1, wherein the mood information is a neural network model trained to recognize the audio recognition result information and output the mood information corresponding to the surrounding situation at each time point.

10. The step of correcting the first acoustic recognition result information includes: identifying a plurality of keywords having similarity with the first acoustic recognition result; generating a plurality of association sub-information between each of the plurality of keywords and the mood information; and 10. The method of claim 9, further comprising: identifying maximum relevance sub-information corresponding to a maximum value among the plurality of relevance sub-information; and performing correction on the first acoustic recognition result information based on a keyword corresponding to the maximum relevance sub-information.

11. memory storing one or more instructions; and a processor for executing the one or more instructions stored in the memory; An apparatus, the processor performing the method of claim 1 by executing the one or more instructions.

12. A computer program stored on a computer-readable recording medium so as to be combined with a computer that is hardware and capable of carrying out the method according to claim 1.

Images