Acoustic recognition result providing method, apparatus, and computer program with improved reliability

Abstract

In one embodiment of the present invention for solving the above problem, an acoustic recognition result providing method with improved reliability is disclosed. The method comprises the following steps: acquiring acoustic data; a step of segmenting the acoustic data to generate a plurality of acoustic sub-data; and a step of processing the plurality of acoustic sub-data by an input of an acoustic recognition model to generate acoustic recognition result information corresponding to each of the acoustic sub-data.

Description

Technical Field

[0001] The present invention relates to a method for improving the recognition rate of acoustic data, and more specifically, to the following method, namely, in the process of dividing acoustic data into time unit intervals and performing recognition on each interval, the recognition rate is improved while reducing the amount of calculation, and providing an acoustic recognition result with improved accuracy corresponding to various surrounding conditions. Background Art

[0002] Hearing-impaired people who cannot hear sounds at all or cannot distinguish sounds have difficulty judging situations based on the sounds they hear, so they have many difficulties in their daily lives. They cannot use sound information to identify dangerous situations in indoor and outdoor environments, so they cannot respond immediately. Not only hearing-impaired people, but also pedestrians wearing headphones, the elderly, and other people who have no hearing or limited hearing will also block out the sounds generated around the user. In addition, in situations where it is difficult to detect sounds, such as when the user is sleeping, there is a risk of being in a dangerous situation or having an accident because they cannot recognize the surrounding situation.

[0003] On the other hand, the need to develop technology to detect and identify sound events in such environments is growing. The technology to detect and identify sound events is a technology that can be applied to various fields such as real-life environment scene recognition, dangerous situation recognition, media content recognition, and situation analysis in wired communications, and research is ongoing.

[0004] As a sound event recognition technology, it mainly extracts various feature values ​​such as Mel-frequency cepstral coefficients (MFCC), energy, spectral flux, zero crossing rate, etc. from audio signals, verifies excellent features, and studies classification methods based on Gaussian mixture models or rules. Recently, in order to improve the above methods, machine learning methods based on deep learning are being studied. However, these methods can guarantee the accuracy of acoustic detection under low signal-to-noise ratios, and have the limitation that it is difficult to distinguish between surrounding noise and event sounds.

[0005] That is, in a real-life environment including various surrounding noises, it may be difficult to detect sound events with high reliability. Specifically, in order to detect valid sound events, it is necessary to determine whether a sound event has occurred in the acoustic data acquired in a time series (i.e., continuity), and it is necessary to identify what event category has occurred at the same time, so it may be difficult to ensure high reliability. In addition, when more than two events occur at the same time, it is also necessary to solve the problem of multi-event (polyphonic) recognition, rather than a single event (monophonic), so the recognition rate of sound events may be lower.

[0006] Therefore, there may be a need to provide an improved recognition rate corresponding to acoustic data acquired in a time series in a real-life environment to provide acoustic recognition with improved reliability.

[0007] Prior art literature

[0008] Patent Literature

[0009] Korean authorized patent 10-2014-0143069 Summary of the invention

[0010] Technical issues

[0011] The technical problem to be solved by the present invention is to solve the above-mentioned problem, that is, in the process of dividing acoustic data into time unit intervals and performing recognition on each interval, reduce the amount of calculation while seeking to improve the recognition rate, and provide acoustic recognition results with improved accuracy corresponding to various surrounding conditions.

[0012] The technical problems to be solved by the present invention are not limited to the above-mentioned technical problems, and a person skilled in the art can clearly understand other technical problems not mentioned from the following description.

[0013] Technical Solution

[0014] A method for providing an acoustic recognition result with improved reliability according to various embodiments of the present invention for solving the above technical problems is disclosed. The method may include: a step of acquiring acoustic data; a step of segmenting the acoustic data to generate a plurality of acoustic sub-data; and a step of processing the plurality of acoustic sub-data through input of an acoustic recognition model to generate acoustic recognition result information corresponding to each acoustic sub-data.

[0015] In an alternative embodiment, the step of generating the multiple acoustic sub-data includes: dividing the acoustic data into preset size units to generate multiple acoustic sub-data, and the acoustic recognition model may include: a first recognition model, taking each of the multiple acoustic sub-data as input to provide multiple outputs corresponding to each acoustic sub-data; and a second recognition model, taking combined re-verification acoustic sub-data generated by combining the multiple acoustic sub-data as input, and providing outputs corresponding to the combined re-verification acoustic sub-data.

[0016] In an alternative embodiment, the step of generating the acoustic recognition result information may include: a step of screening out re-verification acoustic sub-data that needs to be re-verified based on the acoustic recognition result information output by the first recognition model corresponding to each acoustic sub-data; a step of generating combined re-verification acoustic sub-data based on the screened re-verification acoustic sub-data; and a step of processing the combined re-verification acoustic sub-data as input to the second recognition model to generate the acoustic recognition result information, wherein the second recognition model is characterized in that it can be implemented through a cloud application programming interface (API).

[0017] In an alternative embodiment, the step of filtering the re-verification acoustic sub-data may include: the step of deriving a similarity score between recognition items related to the output of the first recognition model; and the step of filtering the re-verification acoustic sub-data based on the calculated similarity score.

[0018] In an alternative embodiment, the step of screening and re-verifying the acoustic sub-data may include: a step of identifying whether the acoustic recognition result information related to the output of the first recognition model is included in a preset re-verification item; and when the acoustic recognition result information is included in the re-verification item, the step of screening the acoustic sub-data serving as the basis for calculating the acoustic recognition result information as the re-verification acoustic sub-data that needs to be re-verified.

[0019] In an alternative embodiment, in the first recognition model, the probability value of each recognition item can be output, and acoustic recognition result information can be generated based on the recognition item corresponding to the probability value equivalent to the maximum value among the calculated probability values, and the step of screening the re-verification acoustic sub-data can include: among the probability values ​​of each recognition item calculated by the first recognition model, when there are multiple recognition items that exceed a preset critical reference value, the step of screening the acoustic sub-data that serves as the basis for calculating the acoustic recognition result information as re-verification acoustic sub-data that needs to be re-verified.

[0020] In an alternative embodiment, the step of generating the acoustic recognition result information may include: a step of generating association relationship information between the acoustic recognition result information; and a step of correcting at least one of the acoustic recognition result information corresponding to each acoustic sub-data based on the association relationship information.

[0021] In an alternative embodiment, in the step of correcting the at least one acoustic recognition result information, when the first acoustic recognition result information and the second acoustic recognition result information are generated within a preset time, at least one of the first acoustic recognition result and the second acoustic recognition result is corrected according to the association relationship information corresponding to the first acoustic recognition result and the second acoustic recognition result information.

[0022] In an alternative embodiment, the method may further include: a step of generating emotion information corresponding to the acoustic recognition result information using an emotion detection model, wherein the emotion information may be prediction information for the atmosphere related to the space in which the acoustic data is acquired, and may include place prediction information and emotion prediction information, and the step of generating the acoustic recognition result information may include: a step of generating association information between first acoustic recognition result information corresponding to first acoustic sub-data and the emotion information corresponding to the first acoustic sub-data; a step of not correcting the first acoustic recognition result information when the association information is above a preset reference value; and a step of correcting the first acoustic recognition result information when the association information is less than a preset reference value, and the emotion detection model may be characterized by a neural network model that is learned in a manner of identifying the acoustic recognition result information to output the emotion information corresponding to the surrounding conditions at each time point.

[0023] According to another embodiment of the present invention, an acoustic recognition result providing device with improved reliability is disclosed. The device may include: a memory storing one or more instructions; and a processor executing the one or more instructions stored in the memory, and the processor may execute the acoustic recognition result providing method with improved reliability as described above by executing the one or more instructions.

[0024] According to another embodiment of the present invention, a computer program stored in a computer-readable recording medium is disclosed. The computer program is combined with a computer as hardware to execute the acoustic recognition result providing method with improved reliability as described above.

[0025] Other specific matters of the present invention are included in the detailed description and drawings.

[0026] Effects of the Invention

[0027] According to various embodiments of the present invention, in the process of dividing acoustic data into time unit intervals and performing recognition on each interval, the recognition rate is improved while reducing the amount of calculation, and an acoustic recognition result with improved accuracy corresponding to various surrounding conditions can be provided.

[0028] The effects of the present invention are not limited to the effects set forth above, and those skilled in the art can clearly understand other effects not mentioned from the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 1 is a schematic diagram briefly showing a system for executing a method for providing an acoustic recognition result with improved reliability according to an embodiment of the present invention.

[0030] Figure 2 The present invention is a hardware configuration diagram of a server that executes a method for providing an acoustic recognition result with improved reliability related to an embodiment of the present invention.

[0031] Figure 3 A schematic flow chart showing a method for providing an acoustic recognition result with improved reliability related to an embodiment of the present invention.

[0032] Figure 4 is a schematic diagram for explaining a segmentation process of acoustic data related to an embodiment of the present invention.

[0033] Figure 5 is a flowchart schematically illustrating a process of generating acoustic recognition result information by revalidation using a plurality of artificial intelligence models related to an embodiment of the present invention.

[0034] Figure 6 It is a schematic diagram schematically illustrating a process of deriving an acoustic recognition result from an acoustic recognition model related to an embodiment of the present invention.

[0035] Figure 7 is a schematic diagram of a process of recognizing acoustic data using a first recognition model and a second recognition model related to an embodiment of the present invention.

[0036] Figure 8 is a flowchart schematically illustrating a process of correcting acoustic recognition result information according to an association relationship between acoustic sub-data related to an embodiment of the present invention.

[0037] Fig. 9 It is a schematic diagram for explaining a process of correcting acoustic recognition result information related to an embodiment of the present invention.

[0038] Fig.10 is a flowchart schematically illustrating a process of generating acoustic recognition result information in consideration of surrounding environment and atmosphere related to an embodiment of the present invention.

[0039] Fig.11 1 is a schematic diagram for explaining critical points of acoustic recognition result information derived through a general recognition process related to an embodiment of the present invention.

[0040] Fig.12 It is a schematic diagram schematically illustrating a process of grasping the overall atmosphere through acoustic sub-data corresponding to the time points before and after specific acoustic sub-data related to an embodiment of the present invention, and outputting acoustic recognition result information with further improved accuracy in consideration of the grasped atmosphere. DETAILED DESCRIPTION

[0041] Various embodiments are described below with reference to the accompanying drawings. In this specification, various descriptions are provided to provide an understanding of the present invention. However, it is clear that these embodiments can also be performed without these specific descriptions.

[0042] The terms "component", "module", "system", etc. used in this specification refer to computer-related entities, hardware, firmware, software, a combination of software and hardware, or the execution of software. For example, a component can be a process (procedure), a processor, an object, an execution thread, a program and / or a computer executed on a processor, but is not limited thereto. For example, an application program and a computing device running in a computing device can both be components. More than one component can reside in a processor and / or an execution thread. A component can be localized in a computer. A component can be distributed between more than two computers. In addition, these components can be run from various computer-readable media with various data structures stored therein. Components can, for example, communicate through local and / or remote processing based on signals with more than one data packet (for example, in a local system, a distributed system, through data and / or signals from a component that interacts with another component, and data transmitted through networks such as the Internet and other systems).

[0043] In addition, the meaning of the term "or" is intended to mean an inclusive "or" rather than an exclusive "or". That is, in the absence of special indication or the context is unclear, the meaning of "X utilizes A or B" is intended to be one of the natural inclusive replacements. That is, when X utilizes A; or X utilizes B; or X utilizes A and B, "X utilizes A or B" can be applied to any of these situations. In addition, the term "and / or" used in this specification should be understood to refer to and include all possible combinations of more than one of the listed related items.

[0044] In addition, the terms "include" and / or "comprise" should be understood as the existence of corresponding features and / or constituent elements. However, the terms "include" and / or "comprise" should be understood as not excluding the existence or addition of one or more other features, constituent elements and / or their combinations. In addition, if the text does not specifically clarify or indicate the singular form, in this specification and the scope of protection of the invention, the singular should generally be interpreted as "one or more".

[0045] Those skilled in the art of the present invention will recognize that the various exemplary logic blocks, configurations, modules, circuits, units, logics, and algorithmic steps described in connection with the embodiments disclosed herein may be implemented by electronic hardware, computer software, or a combination thereof. In order to clearly illustrate the interactivity of hardware and software, various exemplary components, blocks, configurations, units, logics, modules, circuits, and steps are described above in terms of their functionality. Whether such functionality is implemented by hardware or software depends on the specific application and design restrictions imposed on the overall system. A skilled person may implement the functionality described in various ways for each specific application. However, the decision to implement these may not be interpreted as departing from the field of the present invention.

[0046] The description of the proposed embodiments is provided so that a person of ordinary skill in the technical field of the present invention can utilize or implement the present invention. Various modifications of these embodiments are apparent to a person of ordinary skill in the technical field of the present invention. The general principles defined herein may be applied to other embodiments without departing from the scope of the present invention. Therefore, the present invention is not limited to the embodiments proposed herein. The present invention should be interpreted in the broadest sense consistent with the principles and novel features proposed herein.

[0047] In this specification, a computer refers to all types of hardware devices including at least one processor, and can be understood to also include software configurations that work in the corresponding hardware devices according to the embodiments. For example, a computer can be understood to include the meaning of user clients and applications driven by smartphones, tablet computers, desktop computers, laptops, and various devices, but is not limited thereto.

[0048] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[0049] The various steps described in this specification are described as being executed by a computer, but the subject of the various steps is not limited thereto, and according to an embodiment, at least a part of the various steps may also be executed in a different device.

[0050] According to various embodiments of the present invention, the method for providing an acoustic recognition result with improved reliability may include: a method for further verifying the recognized acoustic data in order to improve the recognition rate of the acoustic data; a method for correcting at least a portion of the acoustic recognition result based on the acoustic recognition result output within a specified time; or a method for grasping the surrounding environment and atmosphere in real time and considering it together with the acoustic data to derive the acoustic recognition result. In an embodiment, the improvement in the recognition accuracy of the acoustic data may mean the improvement in the recognition accuracy of detecting a specific event from the acoustic data.

[0051] The method of further verifying the acoustic data may mean, for example, using multiple neural network models (e.g., acoustic recognition models) to derive multiple additional acoustic recognition results for verification, thereby seeking to improve the reliability of the acoustic recognition results finally derived. In this case, each of the multiple neural network models may be a neural network model with a different output process. For example, each neural network model may be a model with different length intervals for input acoustic data, one model may be a model for deriving acoustic recognition results based on acoustic data in a 1-second interval, and another model may be a model for deriving acoustic recognition results based on acoustic data in a 5-second interval. However, it is not limited to this, and multiple neural network models may be acoustic recognition models with different performances. For example, a first lightweight recognition model with lower performance can generally be used to perform the first acoustic recognition and output the acoustic recognition result. Among them, when it is determined that the reliability of the acoustic recognition result output for the first time has decreased, the acoustic recognition is performed again by a second recognition model with higher performance than the first recognition model to re-verify the first acoustic recognition result. This is a configuration that uses the first recognition model at ordinary times and uses the second recognition model for re-verification when necessary. It has the advantage of being able to effectively reduce the amount of calculation while improving the reliability of the recognition result.

[0052] In addition, the method of correcting at least a part of the acoustic recognition results based on the acoustic recognition results output within a specified time may mean, for example, post-processing correction of the acoustic recognition results corresponding to the acoustic data. The acoustic data can be segmented at specified time intervals for input, and the present invention outputs the acoustic recognition results corresponding to each segmented data. In this case, a specific acoustic recognition result can be corrected to other acoustic recognition results by grasping the association between the acoustic recognition results output within a specified time. For example, after the acoustic recognition result of sound A (for example, gunshot) is output, if the acoustic recognition result of sound B (for example, applause) is input within n seconds, the recognition result of sound A can be corrected to sound C (for example, firecrackers). That is, taking into account the association between the acoustic recognition results output (or recognized) within a specified time, at least one of the acoustic recognition results is corrected, thereby improving the recognition accuracy.

[0053] In addition, the present invention can grasp the surrounding environment and atmosphere in real time, and consider it together with the acquired acoustic data to derive the acoustic recognition result, thereby improving the recognition accuracy of the acoustic data. At the time point of acquiring the acoustic data, a separate neural network model (e.g., an emotion detection model) that detects the corresponding surrounding environment and atmosphere can be used to recognize the situation before and after the input of the acoustic data, and in the acoustic analysis process, it can be considered together with the acoustic data, thereby improving the reliability of the output acoustic recognition result. For example, for sound A (e.g., the sound of firecrackers), it can be identified as a gunshot in a general acoustic recognition model. For short-term explosions, it is difficult to output acoustic recognition results with high accuracy. The present invention can grasp the atmosphere corresponding to the situation of acquiring the acoustic data based on the acoustic recognition results detected before and after the corresponding sound is detected, and consider it together with the acoustic data to derive the acoustic recognition result. For example, before and after the recognition of the corresponding A sound, it is identified that singing, applause, and cheering are detected, and the atmosphere related to the corresponding A sound is grasped as a celebratory atmosphere (or a party, bright or happy atmosphere), and the corresponding explosion sound is recognized as a "firecracker sound" instead of a "gunshot". That is, the overall atmosphere is grasped based on the acoustic recognition results output at the time points before and after the acoustic data to be recognized, and the acoustic data is recognized in consideration of the grasped atmosphere, so that the output accuracy of the acoustic recognition result can be improved. A more specific description of the method for improving the recognition accuracy of acoustic data will be described in detail below.

[0054] Figure 1 1 is a schematic diagram briefly showing a system for performing a method for providing an acoustic recognition result with improved reliability according to an embodiment of the present invention. Figure 1 As shown, a system for performing a method for providing an acoustic recognition result with improved reliability according to an embodiment of the present invention may include a server 100 for improving the recognition accuracy of acoustic data, a user terminal 200, and an external server 300. Figure 1 The system for performing the method for providing an acoustic recognition result with improved reliability is a system according to an embodiment, and its constituent elements are not limited to Figure 1 The embodiments shown are described below and may be added, changed or deleted as desired.

[0055] In one embodiment, the server 100 for improving the recognition accuracy of acoustic data may determine whether a specific event has occurred based on the acoustic data. Specifically, the server 100 for improving the recognition accuracy of acoustic data may obtain acoustic data related to real life, and determine whether a specific event has occurred by analyzing the acquired acoustic data. In one embodiment, the specific event may be related to the occurrence of security, safety or danger, for example, it may be related to the occurrence of an alarm, a child's cry, glass breaking, a tire blowout, etc. The specific record of the acoustics related to the above-mentioned specific event is only an example, and the present invention is not limited thereto.

[0056] According to an embodiment, the server 100 for improving the recognition accuracy of acoustic data may include any server implemented by an application programming interface (API). For example, the user terminal 200 may obtain acoustic data and transmit it to the server 100 through a cloud application programming interface (Cloud API). For example, the server 100 may obtain acoustic data from the user terminal 200, and may determine that an emergency alarm sound (e.g., an alarm sound) has occurred by analyzing the acoustic data. In an embodiment, the server 100 for improving the recognition accuracy of acoustic data may analyze the acoustic data through an acoustic recognition model (e.g., an artificial intelligence model).

[0057] In one embodiment, the acoustic recognition model (e.g., an artificial intelligence model) is composed of one or more network functions, which may be composed of a collection of interconnected computing units generally referred to as "nodes." These "nodes" may also be referred to as "neurons." One or more network functions include at least one node. The nodes (or neurons) that constitute one or more network functions may be interconnected by one or more "links."

[0058] In an artificial intelligence model, one or more nodes connected by a link can relatively form a relationship between an input node and an output node. The concepts of input nodes and output nodes are relative. For a node, any node in an output node relationship can be in an input node relationship in a relationship with other nodes, and the reverse can be established. As described above, the relationship between an input node and an output node can be generated with a link as the center. One or more output nodes can be connected to an input node through a link, and the reverse can be established.

[0059] In a relationship between an input node and an output node connected by a link, the output node may determine its value based on the data input into the input node. Among them, the nodes connecting the input node and the output node may have weights. The weights may be variable and may be changed by the user or the algorithm in order for the artificial intelligence model to perform the desired function. For example, when more than one input node is interconnected with an output node through each link, the output node may determine the output node value based on the value input to the input node connected to the output node and the weight set in the link corresponding to each input node.

[0060] As described above, in an artificial intelligence model, one or more nodes are interconnected by one or more links, forming a relationship between input nodes and output nodes in the artificial intelligence model. In the artificial intelligence model, the characteristics of the artificial intelligence model can be determined based on the number of nodes and links, the association between nodes and links, and the value of the weight assigned to each link. For example, when there are two artificial intelligence models with the same number of nodes and links and different weight values ​​between the links, the two artificial intelligence models can be identified as different from each other.

[0061] A portion of the nodes that make up the artificial intelligence model may form a layer based on the distance from the initial input node. For example, a set of nodes that are n distances from the initial input node may form n layers. The distance from the initial input node may be defined based on the minimum number of links required to reach the corresponding node from the initial input node. However, this definition of layers is arbitrary for illustration, and within the artificial intelligence model, the number of layers may be defined by methods different from those described above. For example, the layer of a node may also be defined by the distance from the final output node.

[0062] The initial input node may mean one or more nodes in the artificial intelligence model that directly input data without links in the relationship with other nodes. Or it may mean that in the artificial intelligence model network, in the relationship between nodes based on links, there are no nodes that are connected to other input nodes through links. Similarly, the final output node may mean one or more nodes in the artificial intelligence model that do not have output nodes in the relationship with other nodes. In addition, the hidden node may mean the nodes that constitute the artificial intelligence model, other than the initial input node and the final output node. The artificial intelligence model according to an embodiment of the present invention may be an artificial intelligence model in which the nodes in the input layer may be more than the nodes in the hidden layer close to the output layer, and the number of nodes decreases as it proceeds from the input layer to the hidden layer.

[0063] The artificial intelligence model may include more than one hidden layer. The hidden nodes of the hidden layer may take the output of the previous layer and the output of the surrounding hidden nodes as input. The number of hidden nodes in each hidden layer may be the same or different. The number of nodes in the input layer may be determined according to the number of data fields of the input data, and may be the same or different from the number of hidden nodes. The input data input to the input layer may be calculated by the hidden nodes of the hidden layer and may be output by a fully connected layer (FCL) as the output layer.

[0064] In various embodiments, the artificial intelligence model uses a plurality of acoustic data and feature information corresponding to each acoustic data as learning data to perform supervised learning. However, this is not limited thereto, and various learning methods may be applied.

[0065] Among them, supervised learning is usually a method of marking specific data and information related to the specific data to generate learning data, and using it for learning. It refers to a method of marking two data with a causal relationship to generate learning data, and learning through the generated learning data.

[0066] In one embodiment, the server 100 for improving the recognition accuracy of acoustic data may use verification data to determine whether to stop learning when learning of one or more network functions is performed for more than a predetermined epoch. The predetermined epoch may be a part of the entire learning target epoch.

[0067] The verification data may be composed of at least a portion of the labeled learning data. That is, the server 100 for improving the recognition accuracy of acoustic data performs the learning of the artificial intelligence model through the learning data. After the learning of the artificial intelligence model is repeated for more than a predetermined epoch, the verification data can be used to determine whether the learning effect of the artificial intelligence model reaches a predetermined level or above. For example, when the server 100 for improving the recognition accuracy of acoustic data uses 100 learning data to perform learning with a target number of repeated learning times of 10 times, after performing 10 repeated learnings of predetermined epochs, it performs 3 repeated learnings using 10 verification data. When the change in the output of the artificial intelligence model during the 3 repeated learnings is below the predetermined level, it can be determined that further learning is meaningless and the learning is completed.

[0068] That is, the verification data can be used to determine the completion of learning based on whether the learning effect of each epoch in the repeated learning of the artificial intelligence model is above or below the prescribed value. The above-mentioned learning data, the amount of verification data and the number of repetitions are only examples, and the present invention is not limited thereto.

[0069] The server 100 for improving the recognition accuracy of acoustic data uses test data to test the performance of one or more network functions and determines whether to activate one or more network functions, thereby generating an artificial intelligence model. The test data can be used to verify the performance of the artificial intelligence model and can be composed of at least a portion of the learning data. For example, 70% of the learning data can be used for learning the artificial intelligence model (i.e., learning to adjust the weights to output a result value similar to the label), and 30% can be used for test data to verify the performance of the artificial intelligence model. The server 100 for improving the recognition accuracy of acoustic data can input test data into the artificial intelligence model that has completed learning and measure the error, and determine whether to activate the artificial intelligence model based on whether it is above the predetermined performance.

[0070] The server 100 for improving the recognition accuracy of acoustic data can use test data in the learned artificial intelligence model to verify the performance of the learned artificial intelligence model. When the performance of the learned artificial intelligence model is above a predetermined benchmark, the corresponding artificial intelligence model can be activated for use in other applications.

[0071] In addition, the server 100 for improving the recognition accuracy of acoustic data can disable and discard the corresponding artificial intelligence model when the performance of the artificial intelligence model that has completed learning is below a predetermined benchmark. For example, the server 100 for improving the recognition accuracy of acoustic data can judge the performance of an artificial intelligence model generated based on factors such as accuracy, precision, and recall. The above performance evaluation benchmarks are only examples, and the present invention is not limited to this. The server 100 for improving the recognition accuracy of acoustic data can enable each artificial intelligence model to learn independently to generate multiple artificial intelligence models, and by evaluating the performance, only use artificial intelligence models above the specified performance. However, it is not limited to this.

[0072] In this specification, operation model, neural network, network function, neural network can be used with the same meaning (hereinafter described uniformly as neural network). The data structure may include a neural network. And, the data structure including the neural network may be stored in a computer-readable medium. The data structure including the neural network may also include data input to the neural network, weights of the neural network, hyperparameters of the neural network, data obtained from the neural network, activity functions associated with each node or layer of the neural network, and loss functions for neural network learning. The data structure including the neural network may include any constituent elements in the disclosed configuration. That is, the data structure including the neural network may include data input to the neural network, weights of the neural network, hyperparameters of the neural network, data obtained from the neural network, activity functions associated with each node or layer of the neural network, loss functions for training the neural network, etc., or any combination thereof. In addition to the above configuration, the data structure including the neural network may include any other information that determines the characteristics of the neural network. In addition, the data structure may include all forms of data used or generated during the operation of the neural network, and is not limited to the above matters. The computer-readable medium may include a computer-readable recording medium and / or a computer-readable transmission medium. The neural network may be configured by a collection of interconnected computing units that can generally be referred to as nodes. These nodes may also be referred to as neurons. A neural network includes at least one node.

[0073] According to an embodiment of the present invention, the server 100 for improving the recognition accuracy of acoustic data may be a server providing cloud computing services. More specifically, the server 100 for improving the recognition accuracy of acoustic data is a computing based on the Internet, and may be a server providing cloud computing services, which processes information through other computers connected to the Internet instead of the user's computer. The cloud computing service may be a service that stores data on the Internet, does not require the user to install the data or program required by the user on his own computer, can be used anytime and anywhere by accessing the Internet, and can easily share and transfer data stored on the Internet through simple operations and clicks. In addition, the cloud computing service can not only store data in a server on the Internet, but also perform the required work using the functions of the application provided by the network without installing a separate program, and can be a service in which many people share documents and perform work at the same time. In addition, the cloud computing service can be implemented in at least one form of Infrastructure as a Service (IaaS), Platform as a Service (PaaS), Software as a Service (SaaS), a cloud server based on a virtual machine, and a cloud server based on a container. That is, the server 100 for improving the recognition accuracy of acoustic data of the present invention can be implemented in at least one form of the above cloud computing services. The specific description of the above cloud computing services is only an example, and the present invention can also include any platform for building a cloud computing environment.

[0074] In various embodiments, the server 100 for improving the recognition accuracy of acoustic data can be connected to the user terminal 200 via a network, generate and provide an acoustic recognition model for analyzing acoustic data, and can provide information (for example, acoustic recognition result information) obtained by analyzing the acoustic data through the acoustic recognition model to the user terminal.

[0075] The network may refer to a connection structure in which information can be exchanged between various nodes such as multiple terminals and servers. For example, the network includes a local area network (LAN), a wide area network (WAN), the Internet (WWW), wired and wireless data communication networks, telephone networks, wired and wireless television communication networks, etc.

[0076] In addition, the wireless data communication network includes but is not limited to the third generation mobile communication technology (3G), the fourth generation mobile communication technology (4G), the fifth generation mobile communication technology (5G), the third generation partnership project (3GPP), the fifth generation partnership project (5GPP), long term evolution (LTE), world interoperability for microwave access (WIMAX), wireless network (Wi-Fi), Internet, local area network (LAN), wireless local area network (Wireless Local Area Network, Wireless LAN), wide area network (WAN), personal area network (PAN), radio frequency (RF), Bluetooth network, near field communication (NFC) network, satellite broadcasting network, analog broadcasting network, digital multimedia broadcasting (DMB) network, etc.

[0077] In one embodiment, the user terminal 200 can be connected to the server 100 for improving the recognition accuracy of acoustic data via a network, and can provide acoustic data to the server 100 for improving the recognition accuracy of acoustic data. In response to the provided acoustic data, the user terminal 200 can receive acoustic recognition result information about the occurrence of various events (for example, the occurrence of gunshots, alarms, children crying, glass breaking, tire blowouts, etc.).

[0078] The user terminal 200 is a wireless communication device that ensures portability and mobility, and may include but is not limited to navigation, personal communication system (PCS), global system for mobile communications (GSM), personal digital cellular network (PDC), personal handyphone system (PHS), personal digital assistant (PDA), international mobile telecommunication (IMT)-2000, code division multiple access (CDMA)-2000, wideband code division multiple access (W-CDMA), wireless broadband (Wibro) terminal, smart phone (Smartphone), smart pad (Smartpad), tablet PC (Tablet PC) and other portable (Handheld) wireless communication devices of all kinds. For example, the user terminal 200 can be set in a specific area to perform detection related to the specific area. For example, the user terminal 200 is set in a vehicle, and the acoustic data generated by the vehicle during parking or driving can be obtained. For another example, the user terminal 200 may be set corresponding to an area where children are located (eg, a playground or a space within a facility) to detect an emergency situation. The specific location or place of setting the above user terminal is only an example, and the present invention is not limited thereto.

[0079] In one embodiment, the external server 300 can be connected to the server 100 for improving the recognition accuracy of acoustic data via a network, and can provide various information / data required for the server 100 for improving the recognition accuracy of acoustic data to use an artificial intelligence model to analyze acoustic data, or receive result data derived from performing acoustic data analysis using an artificial intelligence model, and store and manage it. For example, the external server 300 can be a storage server additionally provided outside the server 100 for improving the recognition accuracy of acoustic data, but is not limited thereto. Figure 2 , illustrating the hardware configuration of the server 100 for improving the recognition accuracy of acoustic data.

[0080] Figure 2The present invention is related to an embodiment of the present invention and is used to improve the recognition accuracy of acoustic data.

[0081] Reference Figure 2 According to an embodiment of the present invention, a server 100 for improving the recognition accuracy of acoustic data (hereinafter referred to as "server 100") may include one or more processors 110, a memory 120 for loading a computer program 151 executed by the processor 110, a bus 130, a communication interface 140, and a storage device 150 for storing the computer program 151. Figure 2 Only the components related to the embodiments of the present invention are shown in the figure. Therefore, it can be seen that any person skilled in the art can Figure 2 In addition to the components shown, other common components may be included.

[0082] The processor 110 controls the overall operation of each configuration of the server 100. The processor 110 may include a central processing unit (CPU), a microprocessor (MPU), a microcontroller (MCU), a graphics processing unit (GPU), or any form of processor known in the technical field of the present invention.

[0083] The processor 110 may perform data processing of an artificial intelligence model according to an embodiment of the present invention by reading a computer program stored in the memory 120. According to an embodiment of the present invention, the processor 110 may perform operations for learning a neural network. The processor 110 may perform calculations for learning a neural network in deep learning (DL), such as processing of input data for learning, feature extraction from input data, error calculation, weight update of a neural network using backpropagation, and the like.

[0084] In addition, at least one of the central processing unit (CPU), general purpose graphics processing unit (GPGPU) and tensor processing unit (TPU) in the processor 110 can process the learning of network functions. For example, the central processing unit (CPU) and the general purpose graphics processing unit (GPGPU) can process the learning of network functions and the classification of data using network functions together. In addition, in one embodiment of the present invention, the learning of network functions and the classification of data using network functions can be processed by using processors of multiple computing devices together. In addition, the computer program executed by the computing device according to one embodiment of the present invention can be a central processing unit (CPU), a general purpose graphics processing unit (GPGPU) or a tensor processing unit (TPU) executable program.

[0085] In this specification, network function can be used interchangeably with artificial neural network and neural network. In this specification, network function can also include more than one neural network, in which case, the output of network function can be an ensemble of the outputs of more than one neural network.

[0086] The processor 110 may provide an acoustic recognition model according to an embodiment of the present invention by reading a computer program stored in the memory 120. According to an embodiment of the present invention, the processor 110 may perform calculations for learning an acoustic recognition model.

[0087] According to an embodiment of the present invention, the processor 110 may generally process the overall operation of the server 100. The processor 110 may process input or output signals, data, information, etc. or drive an application stored in the memory 120 through components viewed from above, thereby providing appropriate information or functions to a user or a user terminal for processing.

[0088] In addition, the processor 110 may execute operations of at least one application program or program for executing the method according to the embodiment of the present invention, and the server 100 may be provided with more than one processor.

[0089] In various embodiments, the processor 110 may further include a random access memory (RAM, not shown) and a read-only memory (ROM, not shown) for temporarily and / or permanently storing signals (or data) processed inside the processor 110. In addition, the processor 110 may be implemented in the form of a system on chip (SoC) including at least one of a graphics processing unit, a random access memory, and a read-only memory.

[0090] The memory 120 stores various data, instructions and / or information. The memory 120 can load a computer program 151 from the storage device 150 to perform methods / operations according to various embodiments of the present invention. When the computer program 151 is loaded into the memory 120, the processor 110 executes one or more instructions constituting the computer program 151, thereby performing the method / operation. The memory 120 can be implemented as a volatile memory such as a random access memory (RAM), but the technical scope of the present invention is not limited thereto.

[0091] The bus 130 provides a communication function between components of the server 100. The bus 130 can be implemented as various types of buses such as an address bus, a data bus, and a control bus.

[0092] The communication interface 140 supports wired and wireless Internet communication of the server 100. In addition, the communication interface 140 can also support various communication modes other than Internet communication. To this end, the communication interface 140 can include a communication module known in the technical field of the present invention. In some embodiments, the communication interface 140 can also be omitted.

[0093] The storage device 150 may non-temporarily store the computer program 151. When the program for improving the recognition accuracy of acoustic data is executed by the server 100, the storage device 150 may store various information required to provide the program for improving the recognition accuracy of acoustic data.

[0094] The storage device 150 may include a non-volatile memory such as a read-only memory (ROM), an erasable programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), a flash memory, a hard disk, a removable disk, or any form of computer-readable recording medium known in the technical field to which the present invention belongs.

[0095] The computer program 151 may include one or more instructions that cause the processor 110 to perform methods / operations according to various embodiments of the present invention when loaded into the memory 120. That is, the processor 110 may perform the methods / operations according to various embodiments of the present invention by executing the one or more instructions.

[0096] In one embodiment, the computer program 151 may include one or more instructions to execute a method for providing an acoustic recognition result with improved reliability, the method comprising: a step of acquiring acoustic data; a step of segmenting the acoustic data to generate a plurality of acoustic sub-data; and a step of processing the plurality of acoustic sub-data through the input of an acoustic recognition model to generate acoustic recognition result information corresponding to each acoustic sub-data.

[0097] The steps of the method or algorithm described in connection with the embodiments of the present invention may be implemented directly as hardware, or as a software module executed by hardware, or as a combination thereof. The software module may reside in a random access memory (RAM), a read only memory (ROM), an erasable programmable read only memory (EPROM), an electrically erasable programmable read only memory (EEPROM), a flash memory, a hard disk, a removable disk, a CD-ROM, or any form of computer-readable recording medium known in the art to which the present invention belongs.

[0098] The constituent elements of the present invention may be implemented as programs (or applications) and stored in a medium so as to be executed in conjunction with a computer as hardware. The constituent elements of the present invention may be executed by software programming or software factors. Similarly, the embodiments may be implemented in programming or scripting languages ​​such as C, C++, Java, assembler, etc. by including various algorithms implemented by a combination of data structures, programs, routines, or other programming configurations. The functional aspects may be implemented by algorithms executed in one or more processors. Figures 3 to 12 , describing a method for providing an acoustic recognition result with improved reliability performed by the server 100.

[0099] Figure 3 A schematic flow chart of a method for providing an acoustic recognition result with improved reliability in accordance with an embodiment of the present invention. Figure 3 The steps shown may be changed in order, and at least one step may be omitted or added. That is, the following steps are only one embodiment of the present invention, and the scope of protection of the present invention is not limited thereto.

[0100] According to an embodiment of the present invention, a method for providing an acoustic recognition result with improved reliability may include a step S100 of acquiring acoustic data. The server 100 may acquire acoustic data 10. The acoustic data 10 may include information related to acoustics acquired in real life. According to an embodiment of the present invention, the acquisition of acoustic data may be receiving or loading the acoustic data 10 stored in the memory 120. In addition, the acquisition of acoustic data may be based on a wired or wireless communication unit, receiving or loading data from other storage media, other computing devices, or a separate processing module in the same computing device.

[0101] According to an embodiment, the acoustic data 10 can be obtained through a user terminal 200 associated with the user. For example, the user terminal 200 associated with the user may include a smart phone, a smart pad, a tablet PC, and other portable (handheld) wireless communication devices or an electronic device (e.g., a device that can receive acoustic data through a microphone) set in a specific space (e.g., the user's living space), etc.

[0102] According to an embodiment of the present invention, a method for providing an acoustic recognition result with improved reliability may include a step S200 of segmenting acoustic data to generate a plurality of acoustic sub-data.

[0103] In an embodiment, the step of generating a plurality of acoustic sub-data may include the steps of dividing the acoustic data into units of a preset size and generating a plurality of acoustic sub-data.

[0104] The server 100 can divide the acoustic data 10 into a specified time unit to generate a plurality of acoustic sub-data. The plurality of acoustic sub-data can be obtained by dividing the acoustic data as time series information into a plurality of frames based on a specific time unit. Specifically, the server 100 can divide the acoustic data into a predetermined first time unit size to configure a plurality of acoustic sub-data. For acoustics (or sounds), it is time series data that occurs continuously, and therefore, it needs to be divided into specified time intervals and recognition is performed according to each interval.

[0105] For example, when the first acoustic data is acoustic data acquired corresponding to a time of 1 minute, the server 100 may set the first time unit to 1 second to divide the first acoustic data to configure 60 acoustic sub-data (or acoustic subframes). The specific numerical records related to the above-mentioned first time unit and one or more acoustic sub-data are only examples, and the present invention is not limited to this. In an embodiment, in the process of dividing the acoustic data into a plurality of acoustic sub-data, it has the following advantages: the smaller the time unit, the less the calculation amount of the neural network (ie, the acoustic recognition model), thereby improving the performance. When the length of the acoustic sub-data corresponding to the input becomes shorter and the amount of calculation is significantly reduced, the acoustic recognition model can be used as a short interval recognition model and can be used as an Edge SDK. That is, as the acoustic data is divided into a plurality of short intervals, the acoustic recognition model can be driven by an application executable by the server itself.

[0106] In an embodiment, for a specific acoustic, it may be difficult to derive acoustic recognition result information with improved reliability by performing an acoustic recognition model for short interval recognition. For example, for a sound with a short duration, it may be difficult to accurately recognize the corresponding acoustic by only referring to a specific interval.

[0107] More specifically, for example, referring to Figure 4 In part (a), the first acoustic data 10a related to the gunshot with short duration can be segmented in units of 1 second and divided into the first acoustic sub-data 11a to the eighth acoustic sub-data 18a. In this case, since the gunshot generated in the third acoustic sub-data 13a is completely contained in the interval corresponding to the third acoustic sub-data 13a, even if one interval (i.e., the third acoustic sub-data 13a) is considered, accurate recognition can be performed.

[0108] On the other hand, refer to Figure 4 In part (b), the second acoustic data 10b related to the gunshot with a short duration can also be divided into 1 second units and divided into the first acoustic sub-data 11b to the eighth acoustic sub-data 18b. In this case, Figure 4 As shown in part (b) of FIG. 1 , the gunshot may cover the second acoustic sub-data 12b and the third acoustic sub-data 13b. Acoustic data is data acquired in a time series and is segmented by a specified size. As described above, the explosion sound and the echo of the gunshot may exist in different intervals. In this case, when each of the second acoustic sub-data 12b and the third acoustic sub-data 13b is identified by the acoustic recognition model, since the complete gunshot is not included, it may not be accurately identified, and the output accuracy (i.e., the recognition accuracy) may be reduced.

[0109] That is, as the short-duration sound is divided into different intervals, the accuracy of acoustic recognition may decrease. To overcome this problem, the time unit of the sound can be divided into large intervals (for example, 2 seconds), but in this case, the computational complexity of the acoustic recognition model increases, making it impossible to use low-cost equipment and having a low efficiency problem.

[0110] Therefore, the server 100 of the present invention can perform acoustic recognition again in the absence of recognition accuracy by further adding a separate neural network model. That is, by using multiple neural network models, additional acoustic recognition results are derived multiple times to perform re-verification, thereby seeking to improve the reliability of the acoustic recognition results finally derived. In particular, instead of simply using multiple neural network models multiple times, additional verification is performed in response to specific situations where it is predicted that the recognition accuracy will be somewhat insufficient, thereby effectively reducing the amount of calculation while improving the reliability of the recognition results. The configuration instructions for using multiple neural network models to perform re-verification and improve recognition accuracy will refer to Figure 5 , described in detail below.

[0111] Figure 5 is a flowchart schematically illustrating a process of generating acoustic recognition result information by revalidation using multiple artificial intelligence models related to an embodiment of the present invention. Figure 5 The steps shown may be omitted or at least one step may be added. That is, the following steps are only one embodiment of the present invention, and the scope of protection of the present invention is not limited thereto.

[0112] According to one embodiment, the step of generating acoustic recognition result information may include: based on the acoustic recognition result information output by the first recognition model 210 corresponding to each acoustic sub-data, a step S311 of screening out re-verifying acoustic sub-data that needs to be re-verified.

[0113] According to an embodiment, the acoustic recognition model of the present invention may include a first recognition model 210 that takes each of a plurality of acoustic sub-data as an input to provide a plurality of outputs corresponding to each acoustic sub-data.

[0114] According to one embodiment, the server 100 can make the autoencoder learn by unsupervised learning. Specifically, the server 100 can make the dimensionality reduction network function (e.g., encoder) and the dimensionality recovery network function (e.g., decoder) constituting the autoencoder learn so that the output is similar to the output data of the input data. In detail, through the dimensionality reduction network function, only the core feature data (or, feature) of the acoustic data input during the encoding process is learned through the hidden layer, and the remaining information can be lost. In this case, during the decoding process through the dimensionality recovery network function, the output data of the hidden layer can be an approximation of the input data (i.e., acoustic data) rather than a perfect copy. That is, the server 100 adjusts the weights so that the output data and the input data are as similar as possible, thereby enabling the autoencoder to learn.

[0115] An autoencoder may be a type of neural network for outputting output data similar to input data. The autoencoder may include at least one hidden layer, and an odd number of hidden layers may be provided between input / output layers. The number of nodes in each layer is reduced from the number of nodes in the input layer to an intermediate layer called a bottleneck layer (encoding), and then may be reduced and symmetrically expanded from the bottleneck layer to the output layer (symmetrical to the input layer). The number of input layers and output layers may correspond to the number of items of input data remaining after preprocessing of the input data. The autoencoder structure may have a structure in which the number of nodes in the hidden layer included in the encoder decreases as it moves away from the input layer. When the number of nodes in the bottleneck layer (the layer with the fewest nodes between the encoder and the decoder) is too small, a sufficient amount of information may not be transmitted, and therefore may be maintained above a specific number (e.g., more than half of the input layer, etc.).

[0116] The server 100 can use a learning data set containing multiple learning data with object information respectively marked as the input of the learning dimensionality reduction network, match the output feature data of each object with the marked object information and store it. Specifically, the server 100 uses the dimensionality reduction network function to use the first learning data subset marked with the first acoustic identification information (for example, the sound of glass breaking) as the input of the dimensionality reduction network function, and can obtain the feature data of the first object of the learning data contained in the first learning data subset. The obtained feature data can be represented by a vector. In this case, the feature data output corresponding to each of the multiple learning data contained in the first learning data subset is the output of the learning data related to the first sound, and therefore can be located at a relatively close distance in the vector space. The server 100 can match the first acoustic identification information (i.e., the sound of glass breaking) with the feature data related to the first sound represented by the vector and store it.

[0117] For the dimension reduction network function of the learned autoencoder, it is possible to learn to extract the features of the dimension recovery network function well and to restore the features of the input data well.

[0118] For another example, a plurality of learning data included in each of the first learning data subsets marked with the second acoustic identification information (e.g., alarm sound) may be converted into feature data (i.e., features) through a dimensionality reduction network function and displayed on a vector space. In this case, the feature data is output through the learning data related to the second acoustic identification information (i.e., alarm sound), and therefore may be located at a relatively close position on the vector space. In this case, the feature data corresponding to the second acoustic identification information may be displayed on a vector space different from the feature data corresponding to the first acoustic identification information (e.g., glass breaking sound).

[0119] In an embodiment, the server 100 may configure an acoustic recognition model that includes a dimensionality reduction network function in a learned autoencoder. That is, when the acoustic recognition model configured with the dimensionality reduction network function generated by the learning process as described above takes acoustic sub-data as input, the acoustic sub-data may be used to extract feature information (i.e., features) corresponding to the acoustic sub-data by using the operation of the dimensionality reduction network function.

[0120] In this case, the acoustic recognition model can evaluate the similarity of the sound style by comparing the distance in the vector space between the area showing the features corresponding to the acoustic sub-data and the feature data of each object, and can output the acoustic recognition result information corresponding to the acoustic sub-data based on the similarity evaluation. In one embodiment, the acoustic recognition result information may include multiple recognition items and a probability value corresponding to each of the multiple recognition items.

[0121] Specifically, the acoustic recognition model (or the first recognition model) can calculate the acoustic sub-data by using a dimensionality reduction network function to output feature information (i.e., features). In this case, the acoustic recognition model can output a plurality of recognition items corresponding to the acoustic sub-data and a probability value corresponding to each recognition item based on the feature information corresponding to the acoustic sub-data output and the position between the feature data of each acoustic identification information pre-recorded in the vector space by learning.

[0122] Multiple recognition items are used to identify which items the sound is related to, for example, they may include gunshots, firecrackers, screams, tire explosions, alarms, dog barking, rain, etc., but are not limited to these. These multiple recognition items can be generated based on acoustic identification information that is closer to the position of the feature information corresponding to the acoustic sub-data output in the vector space. For example, the acoustic recognition model can configure multiple recognition items by matching the acoustic identification information adjacent to the first feature information corresponding to the first acoustic sub-data output. The specific records related to the above-mentioned multiple recognition items are only examples, and the present invention is not limited to this.

[0123] The probability value corresponding to each recognition item may be information related to the accuracy rate predicted for each recognition item. For example, the acoustic recognition model may configure multiple recognition item information by matching the acoustic identification information at a position close to the first feature information output corresponding to the first acoustic sub-data. In this case, the closer the position between the first feature information and the feature information corresponding to each acoustic identification information, the higher the calculated probability value, and the farther the position between the first feature information and the feature information corresponding to each acoustic identification information, the lower the calculated probability value.

[0124] For example, Figure 6 As shown, corresponding to the first acoustic sub-data, the acoustic recognition model (or the first recognition model) can output multiple recognition items 310, namely, associated with "alarm sound", "screaming sound", "glass breaking sound" and "other sounds". In addition, corresponding to each recognition item 310, the acoustic recognition model can output a probability value 320, namely, "1", "95", "3" and "2". That is, corresponding to the first acoustic sub-data, the acoustic recognition model can output a probability value 320, namely, the probability associated with the alarm sound is "1", the probability associated with the screaming sound is "95", the probability associated with the glass breaking sound is "3", and the probability associated with the other sounds is "2". The recording of each specific numerical value of the above recognition items and probability values ​​is only an example, and the present invention is not limited thereto.

[0125] That is, the server 100 can output a recognition item corresponding to each of the multiple acoustic sub-data generated based on the acoustic data and a probability value corresponding to each recognition item through the acoustic recognition model. For example, the acoustic recognition model can output a first recognition item and a probability value corresponding to each of the first recognition items corresponding to the first acoustic sub-data, and output a second recognition item and a probability value corresponding to each of the second recognition items corresponding to the second acoustic sub-data.

[0126] According to an embodiment, the first recognition model 210 may be characterized in that a probability value of each recognition item is calculated, and acoustic recognition result information is generated based on the recognition item corresponding to the probability value equivalent to the maximum value among the calculated probability values. Figure 6 , through the recognition item corresponding to the maximum probability value of 95%, the acoustic recognition result information that the recognition result of the acoustic sub-data is scream is generated.

[0127] Furthermore, in an embodiment, the first recognition model 210 may also be characterized by calculating a probability value for each recognition item, and generating acoustic recognition result information based on whether the calculated probability value exceeds a preset critical probability value in each recognition item.

[0128] According to various embodiments, the acoustic recognition model of the present invention may be set in multiples. The acoustic recognition model may include a first recognition model 210 and a second recognition model 220. In this case, the second recognition model 220 may be a model for which the length of the interval of the acoustic sub-data input is different from that of the first recognition model 210. The second recognition model 220 may be a neural network model with higher performance than the first recognition model 210. For example, the first recognition model 210 may be a neural network model that is learned to derive an acoustic recognition result based on acoustic sub-data of a 1-second interval, and the second recognition model 220 may be a neural network model that derives an acoustic recognition result based on acoustic sub-data of a 2-second interval. That is, the first recognition model 210 and the second recognition model 220 are neural network models with different performances, and in re-verification, the second recognition model 220 with better performance will be used. In an embodiment, the first recognition model 210 may be a neural network model that recognizes acoustic sub-data of a shorter interval to output a result, and the second recognition model 220 may be a neural network model that recognizes acoustic sub-data of a longer interval to output a result.

[0129] The first recognition model 210 has the advantages of less computation and less computational power consumption because it analyzes acoustics in a short interval. On the contrary, the second recognition model 220 has the disadvantage of being difficult to install and use as an application on a low-cost device because it analyzes acoustics in a longer interval, and has the advantage of high output accuracy because the recognized interval is longer.

[0130] In various embodiments, the server 100 may determine whether to use another neural network model (e.g., a second recognition model) for re-verification based on the output result corresponding to the first recognition model 210, i.e., the acoustic recognition result information. This is to use an additional neural network model when the lightweight first recognition model 210 is usually used but the output accuracy is insufficient, thereby reducing the amount of calculation and the computing power used.

[0131] The server 100 can screen out the re-verified acoustic sub-data that need to be re-verified based on the acoustic recognition result information output by the first recognition model 210 corresponding to each acoustic sub-data. The server 100 can perform re-verification corresponding to whether the acoustic recognition result information is associated with a pre-classified item (e.g., an item that needs to be re-verified) or when it is determined that the accuracy of the acoustic recognition result information is ambiguous, and can screen out the acoustic sub-data that become the object of re-verification execution.

[0132] In a specific embodiment, the server 100 may calculate the similarity scores between the calculated recognition items through the first recognition model 210, and filter the re-verification acoustic sub-data based on the calculated similarity scores.

[0133] For example, the first recognition model may correspond to specific acoustic sub-data to output recognition items of "gunshot", "balloon explosion" and "fireworks", and the probability values ​​of each recognition item are calculated to be 85%, 90% and 94%. In this case, the acoustic recognition result information can be generated based on the fireworks with the highest probability value. On the other hand, the server 100 can calculate the similarity scores between the recognition items output by the first recognition model 210. In an embodiment, the similarity scores between the recognition items can be performed based on the similarity evaluation between the sounds corresponding to each item. For example, the representative acoustic sub-data corresponding to each of the recognition items is input into the dimension reduction network function, so that the embedding corresponding to each recognition item can be obtained, and the similarity between the recognition items can be evaluated by the embedding similarity. For another example, the cosine similarity that measures the similarity by calculating the cosine angle between the embedded word vectors can also be used to evaluate the similarity between the recognition items. For another example, in order to judge the semantic text similarity, a model based on Siamese network and transformer can be used. The specific description of the similarity evaluation method between the above keywords is only an example, and the present invention is not limited to this. In an additional embodiment, Jaccard similarity may be used to measure the similarity between keywords or words.

[0134] A similarity score is derived based on the similarity evaluation between the recognition items output by the first recognition model 210. The higher the similarity between the recognition items, the higher the calculated similarity score, and the lower the similarity between the recognition items, the lower the calculated similarity score. According to an embodiment, the higher the similarity between the recognition items, the lower the output accuracy of the first recognition model 210 is likely to be.

[0135] For example, when the recognition items output by the first recognition model 210 corresponding to the A acoustic sub-data are "screams", "collisions" and "raining sounds", the acoustic similarities between the corresponding items are small, so the calculated similarity score is low. That is, the more different the candidate recognition items for result prediction are, the more obviously different the features of each item are, thus ensuring the reliability of the output of the first recognition model. On the contrary, when the recognition items output by the first recognition model 210 corresponding to the B acoustic sub-data are "gunshots", "balloon explosions" and "fireworks", the acoustic similarities between the corresponding items are high, so the calculated similarity score is high. That is, the candidate recognition items for result prediction are very similar, so the difficulty of dividing the features of each item is high, thereby reducing the reliability of the output of the first recognition model.

[0136] In other words, when the output recognition items have completely dissimilar acoustic feature factors (for example, screams, shattering sounds, and rain sounds), the final output recognition result is likely to be accurate (as the acoustic features of each recognition item are completely different, the probability of the neural network providing accurate output is high). When the output recognition items have high similarity (for example, gunshots, firecrackers, and fireworks), the final output recognition result may be somewhat inaccurate.

[0137] That is, the server 100 can determine whether the acoustic recognition result information is reliable by calculating the similarity scores of the recognition items output by the first recognition model 210. When the similarity scores between the recognition items are low (i.e., below the specified reference value), the acoustic recognition result information is determined to be reliable, and the acoustic sub-data is not re-verified separately. When the similarity scores between the recognition items are high (i.e., exceeding the specified reference value), the reliability of the acoustic recognition result information is determined to be low, and the acoustic sub-data can be screened as re-verified acoustic sub-data that needs to be re-verified.

[0138] In an embodiment, re-validation of the acoustic sub-data may mean deriving acoustic recognition result information again using the second recognition model 220 .

[0139] In addition, when there are multiple recognition items in the probability value of each recognition item calculated by the first recognition model 210 that exceed the preset critical reference value, the server 100 can filter the acoustic sub-data serving as the basis for calculating the acoustic recognition result information as re-verification acoustic sub-data that needs to be re-verified.

[0140] For example, the first recognition model may output recognition items of "gunshot", "balloon explosion" and "fireworks" corresponding to specific acoustic sub-data, and calculate the probability values ​​of each recognition item as 85%, 90% and 94%. In this case, the acoustic recognition result information is generated based on the fireworks with the highest probability value. However, all other recognition items exceed the preset critical reference values ​​corresponding to each recognition item (for example, 82%, 85% and 88%). This is because the first recognition model 210 judges that the probability of other recognition items is also high, which may mean that the reliability of the acoustic recognition result information will be reduced to some extent. For example, when the probability values ​​of recognition items a, b, and c are 97%, 6% and 3% respectively, only the probability value of recognition item a exceeds the preset critical reference value (for example, 80%), so the reliability of the acoustic recognition result information is high.

[0141] In other words, the fewer the number of identification items whose probability values ​​exceed the critical reference values ​​corresponding to each identification item, the greater the possibility that the recognition result outputted will be accurate (for example, when there is one identification item whose probability value exceeds the critical reference value, the reliability of the recognition result may be the highest); the more the number of identification items whose probability values ​​exceed the critical reference values ​​corresponding to each identification item, the greater the possibility that the recognition result outputted will be inaccurate.

[0142] That is, the server 100 can identify whether there are multiple recognition items exceeding the preset critical reference value among the probability values ​​of each recognition item calculated by the first recognition model 210, so as to determine whether the acoustic recognition result information is reliable. When there is one recognition item whose probability value exceeds the critical reference value, the acoustic recognition result information is determined to be reliable, and no additional re-verification of the acoustic sub-data is performed. When there are multiple recognition items (i.e., more than two) whose probability values ​​exceed the critical reference value, the acoustic recognition result information is determined to be low, and the acoustic sub-data can be screened as re-verified acoustic sub-data that needs to be re-verified.

[0143] In addition, in an embodiment, the step of screening the re-verification acoustic sub-data may include: a step of identifying whether the acoustic recognition result information related to the output of the first recognition model 210 is included in a preset re-verification item; and when the acoustic recognition result information is included in the re-verification item, the step of screening the acoustic sub-data that serves as the basis for calculating the acoustic recognition result information as the re-verification acoustic sub-data that needs to be re-verified.

[0144] The preset re-verification items are pre-defined combinations of items because the output accuracy is expected to decrease. For example, for explosion sounds, the acoustic features of gunshots, balloon explosions, firecrackers, fireworks, etc. are similar and therefore difficult to distinguish. When one of the sounds is used to generate acoustic recognition result information, it is difficult to ensure sufficient reliability.

[0145] Therefore, the present invention can pre-store items with difficult-to-distinguish acoustic features to construct preset re-verification items. When the first recognition model 210 outputs the acoustic recognition result information included in the preset re-verification items, the server 100 can filter the acoustic sub-data that serves as the basis for generating the acoustic recognition result information as re-verification acoustic sub-data that needs to be re-verified.

[0146] That is, a preset re-verification item can be pre-configured based on similar sounds that are difficult to distinguish, and when the output of the first recognition model (i.e., the acoustic recognition result information) corresponds to the preset re-verification item, the acoustic sub-data can be determined as the re-verification acoustic sub-data.

[0147] According to an embodiment, the step of generating acoustic recognition result information may include a step S312 of generating combined re-verification acoustic sub-data based on the screened re-verification acoustic sub-data.

[0148] Specifically, when screening the reverification acoustic sub-data, the server 100 may identify the context-related additional acoustic sub-data based on the reverification acoustic sub-data. In addition, the server 100 generates combined reverification acoustic sub-data by combining the reverification acoustic sub-data and the additional acoustic sub-data.

[0149] In addition, in an embodiment, the step of generating acoustic recognition result information may include a step S313 of processing the combined re-verification acoustic sub-data as input to the second recognition model 220 to generate the acoustic recognition result information.

[0150] According to an embodiment, the acoustic recognition model of the present invention may include a second recognition model 220 that takes combined re-verification acoustic sub-data generated by combination between a plurality of acoustic sub-data as input to provide an output corresponding to the combined re-verification acoustic sub-data.

[0151] In this case, the second recognition model 220 may be a model for which the length of the interval of the input acoustic sub-data is different from that of the first recognition model 210. The second recognition model 220 may be a neural network model with higher performance than the first recognition model 210. For example, the first recognition model 210 may be a neural network model that is learned to derive acoustic recognition results based on acoustic sub-data of an interval of 1 second, and the second recognition model 220 may be a neural network model that derives acoustic recognition results based on acoustic sub-data of an interval of 2 seconds. That is, the first recognition model 210 and the second recognition model 220 are neural network models with different performances, and in re-verification, the second recognition model 220 with better performance will be used. In an embodiment, the first recognition model 210 may be a neural network model that recognizes acoustic sub-data of a shorter interval to output a result, and the second recognition model 220 may be a neural network model that recognizes acoustic sub-data of a longer interval to output a result.

[0152] In an embodiment, the second recognition model 220 is characterized in that it can be implemented through a cloud application programming interface (API). As the second recognition model 220 is implemented through a cloud application programming interface (API), the second recognition model 220 with higher performance can also be used on a low-performance device.

[0153] For example, refer to Figure 7 First, a first lightweight recognition model 210 with lower performance can be used to take each of the multiple acoustic sub-data (first acoustic sub-data 11, second acoustic sub-data 12 and third acoustic sub-data 13) generated for the first time corresponding to the acoustic data 10 as input to output acoustic recognition result information.

[0154] Among them, when it is determined that the reliability of the acoustic recognition result information corresponding to the second acoustic sub-data 12 output for the first time has decreased, the acoustic sub-data corresponding to the acoustic recognition result information (ie, the second acoustic sub-data) can be screened as re-verified acoustic sub-data that needs to be re-verified.

[0155] In this case, the server 100 may identify additional acoustic sub-data generated before and after the second acoustic sub-data 12. For example, the third acoustic sub-data 13, which is acoustic sub-data generated at a time point adjacent to the second acoustic sub-data 12, may be identified as the additional acoustic sub-data.

[0156] The server 100 generates the combined re-verification acoustic sub-data by combining the re-verification acoustic sub-data and the additional acoustic sub-data. That is, through the combination of the second acoustic sub-data 12 and the third acoustic sub-data 13, the combined re-verification acoustic sub-data 20 can be generated.

[0157] In this case, since the combined re-verification acoustic sub-data 20 is generated by combining the acoustic sub-data, it may have a longer length than the acoustic sub-data initially input in the first recognition model 210. That is, when it is determined that the recognition result for the acoustic sub-data is inaccurate, the combined re-verification acoustic sub-data is generated, so that re-verification of the acoustic sub-data that derives the inaccurate output can be performed.

[0158] The server 100 may process the combined re-verified acoustic sub-data 20 as input to the second recognition model 220, and output acoustic recognition result information. In this case, the second recognition model is a neural network model that recognizes acoustic sub-data of a longer interval than the first recognition model, so the acoustic recognition result is derived by analyzing the acoustic sub-data of a larger range than the first recognition model (i.e., the combined re-verified acoustic sub-data), thereby generating acoustic recognition result information with a higher accuracy rate.

[0159] That is, the server 100 uses the lighter first recognition model and uses the second model when necessary so as to perform only re-verification, thereby providing the effect of effectively reducing the amount of calculation while improving the reliability of the recognition result.

[0160] Figure 8 is a flowchart schematically illustrating a process of correcting acoustic recognition result information according to an association relationship between acoustic sub-data related to an embodiment of the present invention.

[0161] According to an embodiment, the step of generating acoustic recognition result information may include: step S321 of generating association relationship information between multiple acoustic recognition result information. The association relationship information may be information indicating the correlation between each acoustic recognition result information. For example, when the first acoustic recognition result information generated by the recognition result of the first acoustic sub-data corresponding to the first interval is "gunshot", and the second acoustic recognition result information generated by the recognition result of the second acoustic sub-data corresponding to the second interval is "applause", the server 100 may generate association relationship information in which the first acoustic recognition result information and the second acoustic recognition result information are not correlated. The specific records of each acoustic recognition result information and the specific records of the association relationship information mentioned above are only examples, and the present invention is not limited thereto.

[0162] In addition, the step of generating acoustic recognition result information may include: based on the association relationship information, a step S322 of correcting at least one acoustic recognition result information in the acoustic recognition result information corresponding to each acoustic sub-data. The server 100 may correct at least one acoustic recognition result information based on the association relationship information between the acoustic recognition result information. The server 100 may determine whether to correct the acoustic recognition result information based on the association relationship information between the respective acoustic recognition information.

[0163] According to a specific embodiment, the step of correcting at least one acoustic recognition result information may be characterized by, when the first acoustic recognition result information and the second acoustic recognition result information are generated within a preset time, at least one of the first acoustic recognition result and the second acoustic recognition result is corrected based on the association relationship information corresponding to the first acoustic recognition result and the second acoustic recognition result information.

[0164] That is, the server 100 can perform post-processing correction on at least a portion of the acoustic recognition result information based on the acoustic recognition result information output within a specified time. The acoustic data is segmented at specified time intervals and input into the neural network in the form of frames, and the acoustic recognition result information is derived corresponding to each segmented data. The server 100 generates association relationship information between the acoustic recognition results output within a specified time, through which specific acoustic recognition result information can be corrected to other acoustic recognition results based on the association relationship between the generated acoustic recognition results corresponding to adjacent time points.

[0165] This configuration is to further improve the accuracy compared to the case where the acoustic recognition result information is generated based solely on the acoustic sub-data. For example, when the acoustic recognition result of sound A (e.g., gunshot) is output, and the acoustic recognition result of sound B (e.g., applause) is output within n seconds, the recognition result of sound A can be corrected to sound C (e.g., firecrackers). That is, considering the correlation between the acoustic recognition results output (or recognized) within a specified time, at least one of the acoustic recognition results is corrected, thereby improving the recognition accuracy.

[0166] More specifically, for example, Fig. 9 As shown in part (a) of FIG. 1 , when a single frame (i.e., one acoustic sub-data) is input as an acoustic recognition model, acoustic recognition result information of "gunshot" can be generated. This is a result derived by simply analyzing one acoustic sub-data related to explosion sound, so high accuracy cannot be guaranteed. For example, for gunshot, there are many sounds with similar acoustic features such as firecrackers, fireworks, or balloon explosions, so there is a fear that some lack accuracy.

[0167] like Fig. 9As shown in part (b), based on the specific acoustic recognition result information (i.e., gunshot), the acoustic recognition result information corresponding to the frame existing within the specified time can be identified as the sound of singing Happy Birthday, applause, and cheers. The server 100 can generate association relationship information that has no correlation between the specific acoustic recognition result (i.e., gunshot) and other acoustic recognition result information through other acoustic recognition results output within the specified time. Therefore, the specific acoustic recognition result information is post-processed and corrected to other acoustic recognition results. That is, based on the association relationship information generated based on the correlation with other acoustic recognition result information (e.g., the sound of singing Happy Birthday, applause, and cheers) output within the specified time, the acoustic recognition result information of the gunshot (gunshot) initially output can be corrected to the "sound of setting off firecrackers" as other acoustic recognition result information. In an embodiment, the server 100 can identify that the gunshot (gunshot) initially output is contained in a specific item (e.g., an item corresponding to an explosion sound), and filter other types of keywords contained in the specific item to correct the acoustic recognition result information. The server 100 can filter out keywords that are relevant to other acoustic recognition results (e.g., the sound of setting off firecrackers) from other types of sounds included in the specific item to correct the acoustic recognition result information that was initially output. Therefore, the acoustic recognition result information of the sound of a gunshot that was initially output is corrected to the sound of setting off firecrackers.

[0168] That is, the server 100 can perform post-processing correction on at least one of the acoustic recognition result information by considering the correlation between the acoustic recognition result information output (or recognized) within a specified time, thereby improving the recognition accuracy. In other words, specific acoustic recognition result information that does not have similarity with the acoustic recognition result information output at a nearby time point is identified and corrected to other acoustic recognition result information that has correlation with other acoustic recognition result information, thereby improving the accuracy of the acoustic recognition result.

[0169] Fig.10 is a flowchart schematically illustrating a process of generating acoustic recognition result information in consideration of surrounding environment and atmosphere related to an embodiment of the present invention.

[0170] According to an embodiment, the server 100 may generate emotion information corresponding to the acoustic recognition result information through an emotion detection model. The emotion detection model may be a neural network model that learns to recognize the acoustic recognition result information to output emotion information corresponding to the surrounding situation at each time point.

[0171] The emotion information is prediction information for the atmosphere related to the space where the acoustic data is acquired, and may include place prediction information and emotion prediction information.

[0172] The emotion detection model may be a neural network model that grasps the real-time atmosphere based on the acoustic recognition result information derived from each acoustic sub-data. The emotion detection model may generate emotion information including prediction information for where the acoustic data is currently acquired and prediction information for the atmosphere of acquiring the acoustic data through the acoustic recognition result information. The emotion detection model may analyze the user's emotions by analyzing keywords and the pitch of the voice, and identify the place where the conversation is conducted. In an embodiment, the emotion detection model may include a natural language processing model and a sentiment analysis model. The natural language processing model may be used to understand and process text data (e.g., acoustic recognition result information), and the sentiment analysis model may be used to detect the emotions contained in a sentence.

[0173] For example, by detecting the continuous acquisition of acoustic recognition result information related to vehicle driving noise, it can be determined that the location where the current acoustic data is acquired is a "moving vehicle". For another example, by detecting the acquisition of acoustic recognition result information with a gradually higher pitch, it can be determined that the real-time atmosphere is related to the emotion of "anger". The specific records of the above locations and emotions are only examples, and the present invention is not limited thereto.

[0174] The server 100 may generate acoustic recognition result information based on the emotion information and acoustic sub-data generated by the emotion detection model, in order to recognize the situation or atmosphere before and after the input of the acoustic sub-data, and generate acoustic recognition result information with higher accuracy in consideration of this.

[0175] Reference Fig.11 , specific acoustic sub-data can be processed by inputting an acoustic recognition model to derive acoustic recognition result information of "child scream". However, it may be difficult to distinguish whether the child's scream is a scream in a critical situation or a scream of a child playing because of happiness. That is, not all screams mean critical situations, so it is necessary to distinguish whether the scream is related to a critical situation or occurs in a happy situation. The present invention can generate emotional information corresponding to the atmosphere of the surrounding time point of specific acoustic sub-data through an emotional model, so that acoustic recognition result information that is more suitable for the situation can be derived.

[0176] The server 100 can use the emotion detection model to grasp the atmosphere before and after, that is, the emotion information, and use the model to generate more appropriate acoustic recognition result information. Fig.12 As shown, the server 100 detects "the sound of splashing in the water", "the laughter of children" and "the voice of the mother calling her child" at adjacent time points, and generates acoustic recognition result information of "the scream of a child in a happy state" corresponding to specific acoustic sub-data.

[0177] In a specific embodiment, the step of generating acoustic recognition result information may include: step S331 of generating association information between the first acoustic recognition result information corresponding to the first acoustic sub-data and the emotion information corresponding to the first acoustic sub-data.

[0178] In an embodiment, the association information may be information that quantifies the degree of association between the emotion information generated in real time and the acoustic recognition result information. For example, when the emotion information is related to "happy emotions" and the acoustic recognition result information is related to "gunshots", the server 100 may determine that the degree of association between the emotion information and the acoustic recognition result information is low, and may generate association information with a degree of association of 20% between the emotion information and the acoustic recognition result information. For another example, when the emotion information is related to "anger and terror emotions" and the acoustic recognition result information is related to "gunshots", the server 100 may determine that the degree of association between the emotion information and the acoustic recognition result information is high, and may generate association information with a degree of association of 93% between the emotion information and the acoustic recognition result information. The specific record of the generation of the above-mentioned association information is only an example, and the present invention is not limited thereto.

[0179] The step of generating acoustic recognition result information may include: when the association information is above a preset reference value, the first acoustic recognition result information is not corrected step S332. The preset reference value is predetermined by the user and can be used as a criterion for judging whether the acoustic recognition result information output in real time is appropriate. When the association information between the acoustic recognition result information and the emotion information is above the preset reference value, the acoustic recognition result information currently output is judged to be appropriate, and therefore a separate correction may not be performed.

[0180] The step of generating acoustic recognition result information may include: when the association information is less than a preset reference value, a step S333 of correcting the first acoustic recognition result information. When the association information between the acoustic recognition result information and the emotion information is less than the preset reference value, the accuracy of the currently output acoustic recognition result information is reduced, and therefore a separate correction is required.

[0181] More specifically, the server 100 may identify multiple keywords that are similar to the first acoustic recognition result information. For example, when the first acoustic recognition result information is "gunshot", the server 100 may use "firecracker sound", "firework sound", "balloon explosion sound" and the like as multiple keywords for identification. The above-mentioned specific description of the first acoustic recognition result information and multiple keywords is only an example, and the present invention is not limited thereto.

[0182] In addition, the server 100 may generate a plurality of relevance sub-information between each of the plurality of keywords and the emotional information. That is, the plurality of relevance sub-information may be generated by calculating the degree of relevance between each keyword and the emotional information. For example, when the emotional information is "happy atmosphere", 93% of the first relevance sub-information may be generated corresponding to the sound of firecrackers, 82% of the second relevance sub-information may be generated corresponding to the sound of fireworks, and 76% of the third relevance sub-information may be generated corresponding to the sound of balloon explosion. The specific numerical values ​​of each relevance sub-information are recorded only as examples, and the present invention is not limited thereto.

[0183] The server 100 may correct the first acoustic recognition result information based on the keyword corresponding to the maximum relevance sub-information by identifying the maximum relevance sub-information equivalent to the maximum value among the plurality of relevance sub-information. For example, the server 100 may correct the first acoustic recognition result information based on the keyword (i.e., firecrackers) corresponding to the maximum relevance sub-information (93%).

[0184] That is, when the server 100 determines that specific acoustic recognition result information is inconsistent with the emotional information (that is, when the correlation information between the acoustic recognition result information and the emotional information is less than a preset reference value), the specific acoustic recognition result information can be corrected by using the keyword that is most correlated with the emotional information among the keywords similar to the acoustic recognition result.

[0185] The steps of the method or algorithm described in connection with the embodiments of the present invention may be implemented directly as hardware, or as a software module executed by hardware, or as a combination thereof. The software module may reside in a random access memory (RAM), a read only memory (ROM), an erasable programmable read only memory (EPROM), an electrically erasable programmable read only memory (EEPROM), a flash memory, a hard disk, a removable disk, a CD-ROM, or any form of computer-readable recording medium known in the art to which the present invention belongs.

[0186] The constituent elements of the present invention may be implemented as programs (or applications) and stored in a medium so as to be executed in combination with a computer as hardware. The constituent elements of the present invention may be executed by software programming or software factors. Similarly, the embodiments may be implemented as programming or scripting languages ​​such as C, C++, Java, assemblers, etc. by including various algorithms implemented by a combination of data structures, programs, routines, or other programming configurations. Functional aspects may be implemented by algorithms executed in more than one processor.

[0187] The embodiments of the present invention are described above with reference to the accompanying drawings, but it is understood by those skilled in the art that the present invention can be implemented in other specific forms without changing its technical ideas or essential features. Therefore, it should be understood that the above embodiments are exemplary in all aspects and not restrictive.

Claims

1. A method for providing acoustic recognition results with improved reliability, the method being executed on one or more processors of a computing device, wherein: include: Steps to obtain acoustic data; The step of segmenting the acoustic data to generate a plurality of acoustic sub-data; as well as The step of processing the plurality of acoustic sub-data by inputting an acoustic recognition model to generate acoustic recognition result information corresponding to each acoustic sub-data.

2. The method for providing acoustic recognition results with improved reliability according to claim 1, wherein: The step of generating the plurality of acoustic sub-data comprises: The step of dividing the acoustic data into units of a preset size to generate a plurality of acoustic sub-data, The acoustic recognition model includes: a first recognition model that takes each of the plurality of acoustic sub-data as input to provide a plurality of outputs corresponding to each acoustic sub-data; and The second recognition model takes the combined re-verification acoustic sub-data generated by the combination between the plurality of acoustic sub-data as input, and provides an output corresponding to the combined re-verification acoustic sub-data.

3. The method for providing acoustic recognition results with improved reliability according to claim 2, wherein: The steps of generating the acoustic recognition result information include: A step of re-verifying acoustic sub-data by screening out acoustic sub-data that need to be re-verified based on the acoustic recognition result information output by the first recognition model corresponding to each acoustic sub-data; A step of generating combined re-verification acoustic sub-data based on the screened re-verification acoustic sub-data; and a step of processing the combined re-verified acoustic sub-data as input to the second recognition model to generate acoustic recognition result information, The second recognition model is implemented through a cloud application programming interface.

4. The method for providing acoustic recognition results with improved reliability according to claim 3, wherein: The steps of screening the re-verification acoustic sub-data include: the step of deriving similarity scores between recognition items associated with outputs of said first recognition model; and The step of screening out re-verifying acoustic sub-data according to the calculated similarity score.

5. The method for providing acoustic recognition results with improved reliability according to claim 3, wherein: The steps of screening the re-verification acoustic sub-data include: A step of identifying whether the acoustic recognition result information related to the output of the first recognition model is included in a preset re-verification item; and When the acoustic recognition result information is included in the re-verification item, the acoustic sub-data serving as a basis for calculating the acoustic recognition result information is screened as re-verification acoustic sub-data that needs to be re-verified.

6. The method for providing acoustic recognition results with improved reliability according to claim 3, wherein: In the first recognition model, the probability value of each recognition item is output, and acoustic recognition result information is generated based on the recognition item corresponding to the probability value equivalent to the maximum value among the calculated probability values. The steps of screening the re-verification acoustic sub-data include: Among the probability values ​​of each recognition item calculated by the first recognition model, when there are multiple recognition items exceeding a preset critical reference value, the acoustic sub-data serving as the basis for calculating the acoustic recognition result information is screened as re-verification acoustic sub-data that needs to be re-verified.

7. The method for providing acoustic recognition results with improved reliability according to claim 1, wherein: The steps of generating the acoustic recognition result information include: A step of generating association relationship information between the acoustic recognition result information; and A step of correcting at least one of the acoustic recognition result information corresponding to each acoustic sub-data according to the association relationship information.

8. The method for providing acoustic recognition results with improved reliability according to claim 7, wherein: In the step of correcting the at least one acoustic recognition result information, when the first acoustic recognition result information and the second acoustic recognition result information are generated within a preset time, at least one of the first acoustic recognition result and the second acoustic recognition result is corrected according to the association relationship information corresponding to the first acoustic recognition result and the second acoustic recognition result information.

9. The method for providing acoustic recognition results with improved reliability according to claim 1, wherein: The method further comprises: using an emotion detection model to generate emotion information corresponding to the acoustic recognition result information, The emotional information is prediction information about the atmosphere related to the space where the acoustic data is acquired, including location prediction information and emotional prediction information. The steps of generating the acoustic recognition result information include: A step of generating association information between first acoustic recognition result information corresponding to first acoustic sub-data and the emotion information corresponding to the first acoustic sub-data; When the associated information is greater than a preset reference value, the first acoustic recognition result information is not corrected; and When the associated information is less than a preset reference value, a step of correcting the first acoustic recognition result information, The emotion detection model is a neural network model that is learned in a manner of outputting the emotion information corresponding to the surrounding situation at each time point by recognizing the acoustic recognition result information.

10. The method for providing acoustic recognition results with improved reliability according to claim 9, wherein: The step of correcting the first acoustic recognition result information includes: a step of identifying a plurality of keywords having similarities with the first acoustic recognition result; a step of generating a plurality of association sub-information between each of the plurality of keywords and the emotion information; and A step of identifying maximum relevance sub-information corresponding to a maximum value among the plurality of relevance sub-information, and correcting the first acoustic recognition result information based on a keyword corresponding to the maximum relevance sub-information.

11. A device, in, include: A memory storing one or more instructions; as well as a processor, executing the one or more instructions stored in the memory, The processor performs the method of claim 1 by executing the one or more instructions.

12. A computer program stored in a computer-readable recording medium, and configured to execute the method according to claim 1 by being combined with a computer as hardware.

Images