Information processing method, information processing device, sound registration system, and program

The information processing method enhances sound registration by allowing user interaction and visual analysis of sound features, addressing inefficiencies in existing sound extraction methods by improving the accuracy of sound feature extraction for machine status estimation.

WO2024237132A1PCT designated stage expired Publication Date: 2024-11-21PANASONIC INTELLECTUAL PROPERTY MANAGEMENT CO LTD
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
PCT/JP2024/016957
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-05-16
Filing Date
2024-05-07
Publication Date
2024-11-21

AI Technical Summary

Technical Problem

Existing methods for obtaining appropriate operating sounds for machine status estimation in facilities like manufacturing factories and distribution warehouses are inefficient due to the complexity of sound extraction and difficulty in accurately distinguishing target machine sounds from background noises, especially when sounds vary slightly during repeated actions.

Method used

An information processing method and system that allows users to interactively select and register sound features by analyzing temporal changes in sound characteristics, using video and spectrogram data to visually identify target sound occurrences and exclude non-target sounds, thereby improving the accuracy of sound feature extraction.

Benefits of technology

Enables more precise registration of suitable sound features for machine status estimation by allowing user interaction and visual confirmation of sound-related information, simplifying the process and improving the accuracy of sound feature extraction.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure JP2024016957_21112024_PF_FP_ABST
    Figure JP2024016957_21112024_PF_FP_ABST
Patent Text Reader

Abstract

This information processing method is for assisting the registration of a sound, and includes the steps of: acquiring sound-related information including a sound, and representing at least one of a change over time in the occurrence state of the sound and a change over time in a feature of the sound; calculating a feature amount vector of the sound included in the sound-related information; creating and outputting first information including at least the sound-related information; detecting a selection in which a part of a time region of the sound-related information corresponding to the display of the output first information is set as a registration target range; and creating, for feature amount vectors, second information including relationship information indicating the relationship between a target vector, which is a feature amount vector corresponding to the registration target range, and a non-target vector, which is a feature amount vector not corresponding to the registration target range, and outputting the second information.
Need to check novelty before this filing date? Find Prior Art

Description

Information processing method, information processing device, sound registration system, and program

[0001] The present disclosure relates to an information processing method, an information processing device and program for registering a sound, and a sound registration system.

[0002] Conventionally, there has been a demand for managing the operational status of machines in facilities that utilize machines for work, such as manufacturing plants and logistics warehouses. Such operational status management can be achieved using the operating sounds of the machines. Meanwhile, to estimate the operational status of a machine from its operating sounds, it is necessary to obtain appropriate operational sounds and associate them with the operational status (an example of registering the sounds). As a means for obtaining appropriate operational sounds, for example, a sound extraction process, as disclosed in Patent Literature 1, can be used, which removes noise other than that of the machine from input sounds, including sounds emitted by the machine, and selectively extracts the target machine sounds.

[0003] In Patent Document 1, sound is collected from multiple positions, and the feature of the target sound is identified based on the feature of the sound and the distance of the sound from the sound collector, and the feature of the sound is extracted.

[0004] Japanese Patent Application Laid-Open No. 2021-71586

[0005] However, there is room for improvement in using the sound extraction process disclosed in Patent Document 1 as a means for obtaining appropriate operation sounds.

[0006] Therefore, an object of the present disclosure is to provide an information processing method and the like that can obtain a more appropriate sound to be used for registration.

[0007] An information processing method according to one aspect of the present disclosure is an information processing method executed by a computer for assisting in the registration of a sound as a reference sound in a sound discrimination device that discriminates sounds by learning the reference sound, the information processing method including the sound and representing at least one of a time change in the occurrence situation of the sound or a time change in the characteristics of the sound; a step of calculating a feature vector of the sound included in the sound-related information; a step of creating and outputting first information including at least the sound-related information; a step of detecting a selection of a portion of the time domain of the sound-related information as a registration target range in response to the output first information being displayed; and a step of creating and outputting second information including relationship information indicating, for each of the feature vectors, a relationship between a target vector, which is the feature vector that falls within the registration target range, and a non-target vector, which is the feature vector that does not fall within the registration target range.

[0008] Furthermore, an information processing device according to one aspect of the present disclosure is an information processing device for assisting in the registration of a sound as a reference sound in a sound discrimination device that discriminates sounds by learning the reference sound, and includes: an acquisition unit that acquires sound-related information that includes the sound and represents at least one of a time change in the occurrence situation of the sound or a time change in the characteristics of the sound; a calculation unit that calculates a feature vector of the sound included in the sound-related information; a first output unit that creates and outputs first information that includes at least the sound-related information; a detection unit that detects selection of a portion of the time domain of the sound-related information as a registration target range in response to display of the output first information; and a second output unit that creates and outputs second information that includes relationship information that indicates the relationship between, for each of the feature vectors, a target vector that is the feature vector that falls within the registration target range and a non-target vector that is the feature vector that does not fall within the registration target range.

[0009] a first output unit that creates and outputs first information including at least the sound-related information; a detection unit that detects selection of a portion of a time domain of the sound-related information as a registration target range in response to display of the output first information; a second output unit that creates and outputs second information including relationship information indicating a relationship between a target vector that is the feature vector that falls within the registration target range and a non-target vector that is the feature vector that does not fall within the registration target range; and a registration unit that registers sounds that fall within the registration target range from the sounds included in the sound-related information as sounds to be registered in the sound discrimination device by outputting the sounds separately from sounds that do not fall within the registration target range.

[0010] Furthermore, one aspect of the present disclosure can also be realized as a program for causing a computer to execute the information processing method described above.

[0011] These comprehensive or specific aspects may be realized as a system, an apparatus, a method, an integrated circuit, a computer program, or a non-transitory recording medium such as a computer-readable CD-ROM, or may be realized as any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium.

[0012] According to the present disclosure, it is possible to obtain a more appropriate sound.

[0013] FIG. 1 is a schematic diagram showing a use case of a sound registration system according to an embodiment. FIG. 2 is a block diagram showing the functional configuration of a sound registration system according to an embodiment. FIG. 3 is a block diagram showing the functional configuration of a calculation unit of an information processing device according to an embodiment. FIG. 4A is a flowchart showing an example of the operation of the sound registration system according to an embodiment. FIG. 4B is a flowchart showing an example of the operation of the sound registration system according to an embodiment. FIG. 5 is a diagram showing an example of a screen displayed on a terminal device in the sound registration system according to an embodiment. FIG. 6A is a diagram showing a portion of an example of a screen displayed on a terminal device in the sound registration system according to an embodiment. FIG. 6B is a diagram showing a portion of an example of a screen displayed on a terminal device in the sound registration system according to an embodiment. FIG. 7 is a diagram showing a portion of an example of a screen displayed on a terminal device in a sound registration system according to another example of an embodiment.

[0014] (Knowledge that forms the basis of the disclosure) In the past, there has been a demand for managing the operational status of machines in facilities that utilize machines for work, such as manufacturing plants and logistics warehouses. Such operational status management involves, for example, estimating the corresponding operational status using the waveform of the power supplied from the machine's power source, log information such as the program that controls each part of the machine, and physical quantities detected by an external sensor. In particular, using physical quantities detected by an external sensor is useful because it enables estimation of the operational status by retrofitting the external sensor even if the machine itself does not have a configuration for estimating the operational status. A known technique for detecting physical quantities detected by such external sensors is to collect the operating sound of a machine using a sound collector. Specifically, by distinguishing between the operating sound collected by the sound collector when the machine performs a certain operation and the operating sound collected by the sound collector when the machine performs a different operation, it is possible to estimate whether the machine is currently performing a certain operation or a different operation.

[0015] On the other hand, to estimate the operating status of a machine from its operating sound as described above, it is necessary to obtain appropriate operating sounds and associate them with the operating status (an example of registering the sounds). As a means for obtaining appropriate operating sounds, for example, a sound extraction process, as disclosed in Patent Document 1, can be used, which removes noise other than that of the machine from input sounds, including sounds emitted by the machine, and selectively extracts the target machine sound. In Patent Document 1, sounds are collected from multiple positions, and the target sound feature is identified based on the sound feature and the distance of the sound from a sound collector, and the sound feature is extracted. However, such a sound extraction process requires collecting sounds from multiple positions, which is cumbersome. Furthermore, there is a risk that accurate sound feature extraction will not be possible due to slight differences in the operating sound that occur with repeated repetitions of the same operation, or that it will be difficult to set a threshold value for the difference to allow for such differences in operating sound.

[0016] Therefore, the present disclosure provides a system configured to allow a user and the system to interactively input and output information to obtain appropriate operation sounds (hereinafter sometimes simply referred to as sounds), and to allow the user to specify a specific appropriate operation sound from a time series of consecutive sounds, as well as an information processing method executed in the system. Furthermore, as the pre-extraction sounds, including the operation sounds, sounds contained in, for example, a video recording of the machine's operating status are used. As a result, while viewing sound-related information such as a video corresponding to the sound, the user can easily extract portions where the target operation is definitely occurring or exclude portions where the target operation is clearly not occurring from the extraction, based on visual information. This has the advantage of facilitating the user's input of information into the system. Examples of using video image information in addition will also be described as other embodiments.

[0017] (Summary of the Disclosure) The summary of the disclosure will be listed below.

[0018] An information processing method according to a first aspect of the present disclosure is an information processing method executed by a computer for assisting in the registration of a sound as a reference sound in a sound discrimination device that discriminates sounds by learning the reference sound, and includes the steps of: acquiring sound-related information that includes the sound and represents at least one of a time change in the occurrence situation of the sound or a time change in the characteristics of the sound; calculating a feature vector of the sound included in the sound-related information; creating and outputting first information that includes at least the sound-related information; detecting a selection of a portion of the time domain of the sound-related information as a registration target range in response to the output first information being displayed; and creating and outputting second information that includes relationship information that indicates, for each feature vector, a relationship between a target vector, which is a feature vector that falls within the registration target range, and a non-target vector, which is a feature vector that does not fall within the registration target range.

[0019] In this information processing method, a user can select a registration target range in response to display of first display information including sound-related information. Feature vectors of sounds included in the sound-related information are calculated, and relationship information indicating the relationship between target vectors, which are feature vectors that fall within the registration target range, and non-target vectors, which are feature vectors that do not fall within the registration target range, is further displayed. Therefore, the user can determine whether the selected registration target range is appropriate based on whether the relationship between the target vectors and the non-target vectors is appropriate. Therefore, it is possible to obtain sounds suitable for registration from the registration target range that has been determined to be appropriate through such user-interactive information input / output.

[0020] An information processing method according to a second aspect of the present disclosure is the information processing method according to the first aspect, wherein the relationship information includes two-dimensional data in which the similarity between each of the feature vectors is expressed as a two-dimensional distance between each of the feature vectors, and in the two-dimensional data, feature vectors corresponding to target vectors and feature vectors corresponding to non-target vectors can be distinguished from each other.

[0021] This allows the user to determine whether the selected registration target range is appropriate by using two-dimensional data that can distinguish between feature vectors corresponding to target vectors and feature vectors corresponding to non-target vectors, and in which the similarity between each feature vector is expressed as the two-dimensional distance between each feature vector.

[0022] Furthermore, an information processing method according to a third aspect of the present disclosure is the information processing method described in the first or second aspect, wherein the relationship information includes separation degree information indicating the degree of separation of a set made up of feature vectors corresponding to target vectors from a set made up of feature vectors corresponding to non-target vectors.

[0023] This allows the user to determine whether the selected registration target range is appropriate by using separation degree information that indicates the degree of separation of a set made up of feature vectors corresponding to target vectors from a set made up of feature vectors corresponding to non-target vectors.

[0024] Furthermore, an information processing method according to a fourth aspect of the present disclosure is an information processing method according to any one of the first to third aspects, in which the feature vectors are a plurality of vectors corresponding to unit sounds obtained by dividing sounds included in the sound-related information into unit times.

[0025] This allows the user to determine whether the selected registration target range is appropriate based on whether the relationship between target vectors and non-target vectors is appropriate for feature vectors, which are multiple vectors corresponding to each unit sound obtained by dividing the sound included in the sound-related information into unit times.

[0026] An information processing method according to a fifth aspect of the present disclosure is an information processing method according to any one of the first to fourth aspects, wherein in the step of calculating a feature vector, a first-dimensional vector having feature quantities with a first number of dimensions is calculated from sounds included in the sound-related information, and the calculated first-dimensional vector is converted to calculate a feature vector having feature quantities with a second number of dimensions, which is fewer than the first number of dimensions.

[0027] According to this, a first-dimensional vector having features of a first dimension is calculated from the sound included in the sound-related information, and the calculated first-dimensional vector is converted to calculate a feature vector having features of a second dimension, which is a dimension smaller than the first dimension.The user can then determine whether the selected registration target range is appropriate based on whether the relationship between the target vector and the non-target vector is appropriate for such feature vector.

[0028] An information processing method according to a sixth aspect of the present disclosure is the information processing method according to any one of the first to fifth aspects, further comprising the step of creating, for each feature vector, second information including relationship information indicating the relationship between a target vector, which is a feature vector that falls within the changed registration target range, and a non-target vector, which is a feature vector that does not fall within the changed registration target range, and outputting the second information.

[0029] In this way, the user can change the registration target range in response to the display of the second display information. Then, the second display information including relationship information indicating the relationship between the changed target vector and the changed non-target vector is created and displayed again, allowing the user to determine whether the changed registration target range is appropriate.

[0030] Furthermore, an information processing method according to a seventh aspect of the present disclosure is an information processing method according to the second aspect or any one of the third to sixth aspects that cite the second aspect, further including a step of generating two-dimensional data as relationship information prior to the step of creating and outputting first information, wherein the step of creating and outputting the first information creates and outputs first information that includes sound-related information and two-dimensional data.

[0031] This allows the first display information to also include two-dimensional data.

[0032] An information processing method according to an eighth aspect of the present disclosure is the information processing method according to the second or seventh aspect, wherein, in two-dimensional data, feature vectors corresponding to target vectors and feature vectors corresponding to non-target vectors can be distinguished from each other by differences in display shape.

[0033] According to this, the feature vectors corresponding to the target vectors and the feature vectors corresponding to the non-target vectors can be distinguished from each other by the difference in the display shape.

[0034] An information processing method according to a ninth aspect of the present disclosure is the information processing method according to the second, seventh, or eighth aspect, wherein, in two-dimensional data, feature vectors corresponding to target vectors and feature vectors corresponding to non-target vectors can be distinguished from each other by difference in display color.

[0035] According to this, the feature vectors corresponding to the target vectors and the feature vectors corresponding to the non-target vectors can be distinguished from each other by the difference in display color.

[0036] In addition, an information processing method according to a tenth aspect of the present disclosure is an information processing method described in any one of the first to ninth aspects, and further includes a step of separating and outputting, among sounds included in the sound-related information, sounds that fall within the registration target range from sounds that do not fall within the registration target range, and registering the sounds that fall within the registration target range as sounds for registration.

[0037] According to this, among the sounds included in the sound-related information, sounds that fall within the range to be registered can be used as sounds for registration, and can be output separately from sounds that do not fall within the range to be registered, and used for registration.

[0038] Furthermore, an information processing device according to an eleventh aspect of the present disclosure is an information processing device for assisting in the registration of a sound as a reference sound in a sound discrimination device that discriminates sounds by learning the reference sound, and includes: an acquisition unit that acquires sound-related information that includes the sound and represents at least one of a time change in the occurrence situation of the sound or a time change in the characteristics of the sound; a calculation unit that calculates a feature vector of the sound included in the sound-related information; a first output unit that creates and outputs first information that includes at least the sound-related information; a detection unit that detects selection of a part of the time domain of the sound-related information as a registration target range in response to display of the output first information; and a second output unit that creates and outputs second information that includes relationship information that indicates, for each feature vector, a relationship between a target vector that is a feature vector that falls within the registration target range and a non-target vector that is a feature vector that does not fall within the registration target range.

[0039] This can achieve the same effect as the above-described information processing method.

[0040] a first output unit that creates and outputs first information including at least the sound-related information; a detection unit that detects selection of a portion of the time domain of the sound-related information as a registration target range in response to display of the output first information; a second output unit that creates and outputs second information including relationship information that indicates, for each feature vector, a relationship between a target vector that is a feature vector that falls within the registration target range and a non-target vector that is a feature vector that does not fall within the registration target range; and a registration unit that registers sounds that fall within the registration target range from among the sounds included in the sound-related information as sounds for registration, by outputting the sounds separately from sounds that do not fall within the registration target range.

[0041] This makes it possible to achieve the same effect as the above-described information processing method and to register the sound in the sound discrimination device.

[0042] A program according to a thirteenth aspect of the present disclosure is a program for causing a computer to execute the information processing method according to any one of the first to tenth aspects.

[0043] This makes it possible to achieve the same effect as the above-described information processing method using a computer.

[0044] Furthermore, these comprehensive or specific aspects may be realized as a system, an apparatus, a method, an integrated circuit, a computer program, or a non-transitory recording medium such as a computer-readable CD-ROM, or may be realized as any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium.

[0045] Hereinafter, embodiments will be described in detail with reference to the drawings. Note that the embodiments described below are all comprehensive or specific examples. The numerical values, shapes, materials, components, component placement and connection configurations, steps, and step order shown in the following embodiments are merely examples and are not intended to limit the present disclosure. Furthermore, among the components in the following embodiments, components that are not recited in independent claims will be described as optional components. Note that each figure is a schematic diagram and is not necessarily an exact illustration. Furthermore, in each figure, substantially identical components are assigned the same reference numerals, and duplicated descriptions may be omitted or simplified.

[0046] In the following description, elements may be assigned ordinal numbers such as first, second, and third. These ordinal numbers are assigned to elements to distinguish them from one another and do not necessarily correspond to a meaningful order. These ordinal numbers may be rearranged, newly added, or removed as appropriate.

[0047] (Embodiment) [Overview] First, an overview of a sound registration system according to an embodiment will be described. FIG. 1 is a schematic diagram illustrating a use example of the sound registration system according to an embodiment. FIG. 1 shows a user performing an operation to register a sound in the sound registration system. In the following description, an example will be described in which a video, which is an example of a time change in the sound generation situation, is used as sound-related information related to input to and output from the information processing device 110. However, sound-related information other than a video may be used for the input to and output from the information processing device 110. For example, in addition to a video, sound-related information may be a spectrogram (which represents frequency characteristics with frequency on the vertical axis, time on the horizontal axis, and intensity as shades of gray, etc.), which represents the time change in the characteristics of the sound itself, or power (sound intensity). If the target sound is loud, the waveform of the sound wave itself may be used. The sound-related information may be any information that can simultaneously indicate at least information (such as the situation in which the sound is generated, frequency characteristics, power, or waveform) that can estimate the sound being generated at each time point for two or more time points. Such sound-related information allows the user to identify the time point at which the desired sound is thought to be generated from among two or more time points. In other words, the information used for the sound-related information is required to enable a bird's-eye view (across multiple time points) of the time domain at which the desired sound is generated. Furthermore, the sound-related information is preferably information that can be recognized visually so that two or more time points can be recognized simultaneously. Note that, when sound-related information other than video is used for input to the information processing device 110, for example, a conversion process from audio or video may be performed on an application of the terminal device 300 (see FIG. 2 described later), and the sound-related information may be generated and transmitted to the information processing device 110.

[0048] As shown in FIG. 1 , in the sound registration system according to the present embodiment, a user uses a terminal device 300 (see FIG. 2 , described later) to capture video of a machine operating. The captured video is then transmitted to an information processing device 110 (see FIG. 2 , described later), and sounds associated with specific actions are extracted from the video (separated from other sounds) through interactive information input and output between the user and the information processing device 110. The resulting separated sounds are then registered in a sound discrimination device 120 (see FIG. 2 , described later), which can then use the registered sounds as a reference sound and compare them to determine whether the sounds are identical to the reference sound. The sounds registered in the sound discrimination device 120 are determined through the input and output of information between the user and the information processing device 110. Furthermore, during this information input and output, the user can appropriately evaluate and determine and register sounds based on information relating to sound features, described later. This allows the sound registration system to easily and appropriately obtain sounds for registration.

[0049] [Configuration] A specific configuration of the sound registration system will be described below with reference to Fig. 2 and subsequent figures. Fig. 2 is a block diagram showing the functional configuration of the sound registration system according to the embodiment. Fig. 3 is a block diagram showing the functional configuration of a calculation unit of an information processing device according to the embodiment. As shown in Fig. 2, the sound registration system is realized as a server device 100. The server device 100 is a computer configured to be able to communicate with the terminal device 300, and is realized as an edge server or a cloud server.

[0050] The terminal device 300 is another information processing device used for inputting and outputting information between the information processing device 110 in the server device 100 and a user. Examples of the terminal device 300 include a smartphone, a tablet device, and a PC. The terminal device 300 includes an imaging unit 301, a display unit 302, and a sound collection unit 303. While the terminal device 300 will be described as a single device including all of the imaging unit 301, the display unit 302, and the sound collection unit 303, the terminal device 300 may be configured as a device group including three or more devices, such as a device including only the imaging unit 301 among the imaging unit 301, the display unit 302, and the sound collection unit 303, a device including only the display unit 302 among the imaging unit 301, the display unit 302, and the sound collection unit 303, and a device including only the sound collection unit 303 among the imaging unit 301, the display unit 302, and the sound collection unit 303.

[0051] The image capturing unit 301 is composed of an image sensor that captures images and a communication interface for communicating with the outside. The image capturing unit 301 captures images continuously in time series, and is linked to the sound collecting unit 303, so that it can generate a video that includes sounds collected by the sound collecting unit 303.

[0052] The display unit 302 is configured with a display for displaying images, a processing circuit for generating images to be displayed on the display, and a communication interface for communicating with the outside. The display unit 302 is used to present information to the user for inputting and outputting information between the information processing device 110 and the user, and displays first display information (an example of first information) and second display information (an example of second information) to be presented to the user as images.

[0053] The sound collection unit 303 is configured with a microphone that collects sound and a communication interface for communicating with the outside. The sound collection unit 303 is linked to the image capture unit 301, and can generate a video including consecutive images in chronological order that are captured by the image capture unit 301.

[0054] The terminal device 300 can communicate with the server device 100 via a network 150. The network 150 here may be a wide-area communication network such as the Internet, a local communication network such as an intranet within a facility, or infrared or 2.4 GHz band short-range wireless communication. There are no particular limitations on the communication method used in the network 150. The image capturing unit 301, the display unit 302, and the sound collecting unit 303 can each communicate with the server device 100 via a communication interface and the network 150 to send and receive information.

[0055] The server device 100 includes an information processing device 110 and a sound discrimination device 120. A part of the configuration of the server device 100 can be said to implement a sound registration system.

[0056] The information processing device 110 is realized by executing a predetermined program using a processor and a memory. The information processing device 110 includes, as functional parts that are realized by executing the program, an acquisition unit 111, a calculation unit 112, a first output unit 113, a detection unit 114, a second output unit 115, and a registration unit 116.

[0057] The acquisition unit 111 is a functional part that acquires the video generated by the terminal device 300. The acquisition unit 111 acquires the video and separates the video into images and sounds. In other words, the acquisition unit 111 extracts sounds from the acquired video. The extracted sounds are further divided into unit sounds of unit time that are used in subsequent processing. Note that the unit time may be set appropriately depending on the subsequent processing; for example, in this example, the sound is divided into unit sounds of 2 seconds each. Even when divided into unit sounds in this way, each unit sound is linked so that it is possible to identify which part of the original video each unit sound corresponds to.

[0058] In this example, when dividing the sound into unit sounds, the sound is divided every two seconds, but two consecutive unit sounds may or may not have at least a partial overlap. In the former case, of two consecutive unit sounds, the leading first unit sound is shifted, for example, by several hundred milliseconds from the following second unit sound, resulting in an overlap of one second or more. Therefore, the first and second unit sounds have a sound difference of several hundred milliseconds before and after, in that the second unit sound does not have the leading several hundred milliseconds of the first unit sound, and the first unit sound does not have the trailing several hundred milliseconds of the second unit sound, but they share a sound of one second or more in between. In the latter case, of two consecutive unit sounds, the leading first unit sound is composed only of sounds that are completely different in time from the following second unit sound.

[0059] The calculation unit 112 is a functional part that calculates the feature quantities of sounds included in a moving image as vectors (feature quantity vectors). As shown in Fig. 3, the functional parts of the calculation unit 112 can be subdivided into a first-dimensional data creation unit 1121, a feature quantity creation unit 1122, a two-dimensional data creation unit 1123, and a separability information creation unit 1124.

[0060] When calculating a feature vector from sounds included in a video, the calculation unit 112 first calculates a vector having features of a first dimension (first-dimensional vector) using the first-dimensional data creation unit 1121. The first-dimensional data creation unit 1121 has a trained model, such as OpenL3, that receives sound and outputs a first-dimensional vector corresponding to the sound, consisting of features of 512 dimensions (i.e., the first dimension is 512). The first-dimensional data creation unit 1121 inputs unit sounds into this trained model and calculates the first-dimensional vector by outputting a first-dimensional vector for each unit sound. The above is just an example, and the first-dimensional data creation unit 1121 may be configured in any way as long as it can calculate a vector having features of a first dimension corresponding to the processing from sound.

[0061] However, such first-dimensional vectors, such as 512 dimensions, cannot be drawn, making it difficult for users to visually understand the relationships between vectors using graphs, etc., and therefore must be converted into user-understandable information. In this example, the feature creation unit 1122 performs vector conversion to calculate a feature vector having features with a second dimension, which is a dimension (lower order) than the first dimension, such as 512 dimensions. For this vector conversion, for example, t-Distributed Stochastic Neighbor Embedding (t-SNE) can be used. As a result, the second dimension of the feature vector can be set to 2 or 3 while maintaining the similarity between vectors as the distance between the vectors as much as possible, allowing for graphing. In other words, the user can visually understand the feature vector as a graph. Note that any conversion may be performed for this vector conversion as long as it can reduce the number of dimensions below the first dimension. In the following description, the first-dimensional vector is converted into a two-dimensional vector by the feature creation unit 1122.

[0062] The calculation unit 112 uses the feature vectors converted into two-dimensional vectors to create relationship information indicating the relationships between the feature vectors corresponding to each unit sound. Specifically, as the relationship information, the two-dimensional data creation unit 1123 creates two-dimensional data, and the separability information creation unit 1124 creates separability information. The two-dimensional data creation unit 1123 and the separability information creation unit 1124 are examples of a relationship information creation unit.

[0063] The two-dimensional data created by the two-dimensional data creation unit 1123 is represented in a two-dimensional graph as the proximity of the vectors to each other while minimizing the similarity between the original first-dimension numeric vectors. In other words, the closer the distance between vectors in the two-dimensional data, the higher the similarity between the first-dimension numeric vectors. In this embodiment, the sounds to be registered are determined and registered while information is input and output between the information processing device 110 and the user. During this information input and output, target vectors, which are feature vectors for each unit sound that fall within a registration target range tentatively determined by the user as suitable for registration at a certain point in time, and non-target vectors, which are feature vectors for each unit sound that do not fall within the registration target range, may be set. In the above two-dimensional data, feature vectors corresponding to target vectors and feature vectors corresponding to non-target vectors are graphed so that they can be distinguished from each other. However, if the user has not set a tentative registration target range, no feature vectors corresponding to target vectors will exist, and a graph of only feature vectors corresponding to non-target vectors will be created as two-dimensional data. Even with such two-dimensional data, it can be said that it is possible to distinguish between feature vectors corresponding to target vectors and feature vectors corresponding to non-target vectors, simply because there are no feature vectors corresponding to target vectors.

[0064] The separability information created by the separability information creation unit 1124 is information including a degree of separation indicating the degree of separation of a set composed of feature vectors corresponding to a target vector from a set composed of feature vectors corresponding to a non-target vector. Among the feature vectors calculated by the feature creation unit 1122, the set of feature vectors corresponding to the target vector is classified into one cluster, and the feature vectors corresponding to the non-target vectors are classified into another cluster. The separability information creation unit 1124 calculates the degree of separation based on the inter-cluster distance between these clusters. The inter-cluster distance can be calculated, for example, by calculating the distance between feature vectors for all combinations when one feature vector is selected from each of these two clusters, and averaging the distances. Note that the above calculation of the separability is merely an example. Any calculation may be used to calculate the degree of separation of a set composed of feature vectors corresponding to a target vector from a set composed of feature vectors corresponding to a non-target vector, i.e., the degree of separation between the two sets, as long as it can be quantified.

[0065] 2, the first output unit 113 is a functional part that creates and outputs first display information for display to the user, which includes video and two-dimensional data. Specific examples of the first display information will be described later.

[0066] The detection unit 114 is a functional part that detects the selection of the registration target range from the user when the first display information is displayed to the user. In other words, the detection unit 114 receives an input from the user to select the registration target range after the first display information is displayed to the user.

[0067] The second output unit 115 is a functional part that creates and outputs second display information for display to a user, which includes video, two-dimensional data, and separability information. Specific examples of the second display information will be described later.

[0068] The registration unit 116 is a functional part that registers sounds contained in a video that fall within the registration target range as registration sounds and outputs them separately from sounds that do not fall within the registration target range in the sound discrimination device 120.

[0069] The sound discrimination device 120 is a device that discriminates whether a sound is similar to the reference sound or different from the reference sound by comparing it with the reference sound, and is realized by executing a predetermined program using a processor and memory. The sound discrimination device 120 also includes a storage unit 121 realized by a semiconductor memory or the like. The storage unit 121 stores the sound output by the registration unit 116. In this way, the sound is registered.

[0070] The registration unit 116 may output the sound in a format corresponding to the form of sound discrimination in the sound discrimination device 120. The registration unit 116 may output the sound as is and register it in the storage unit 121 as described above, or may, for example, acquire from the first-dimensional data creation unit 1121 a vector having first-dimensional features corresponding to sounds falling within the registration target range, output this as a sound to be registered, and register it in the storage unit 121. Alternatively, the registration unit 116 may acquire from the feature creation unit 1122 a feature vector having second-dimensional features corresponding to sounds falling within the registration target range, output this as a sound to be registered, and register it in the storage unit 121. In this way, sound registration is a broad concept that includes not only the registration of the sound itself, but also the registration of information about the sound, such as the analysis results of the sound.

[0071] The sound discrimination device 120 includes a functional part for discriminating sounds (not shown), which is implemented by a processor and a memory executing a predetermined program. Depending on the configuration of this functional part for discriminating sounds, a method for using registered sounds as reference sounds is appropriately set. For example, the sound discrimination device 120 is configured such that the first-dimensional vectors of each registered sound are linked and registered. The sound discrimination device 120 calculates the first-dimensional vector of a target sound to be discriminated, and compares the calculation result with the registered first-dimensional vector to determine whether the target sound is similar to the reference sound or a different sound from the reference sound. Alternatively, the sound discrimination device 120 includes a clustering model trained using the registered sounds. The sound discrimination device 120 inputs the target sound into the clustering model, which outputs whether the target sound is clustered in the same cluster as the reference sound, and can determine whether the target sound is similar to the reference sound or a different sound from the reference sound based on the output result.

[0072] [Operation] Details of the sound registration operation in this embodiment will be described below with reference to FIGS. 2 and 3 as well as FIGS. 4A to 7. FIGS. 4A and 4B are flowcharts illustrating an example of the operation of the sound registration system according to this embodiment. In FIGS. 4A and 4B, the components of the information processing device 110 other than the calculation unit 112, i.e., the acquisition unit 111, the first output unit 113, the detection unit 114, the second output unit 115, and the registration unit 116, are collectively illustrated as input / output units. Also, in FIGS. 4A and 4B, arrows (arrows extending horizontally on the page) between components indicate that information selected, generated, or output in the immediately preceding step is being transmitted to the indicated component. Note that FIG. 4B illustrates operations subsequent to steps S110 and S111 in FIG. 4A. However, in FIG. 4B, the first dimension data creation unit and feature creation unit, which are not involved in the subsequent operations, are omitted from illustration, and instead, a sound discrimination device involved in the subsequent operations is illustrated.

[0073] As shown in FIG. 4A , a user uses the terminal device 300, more specifically, the imaging unit 301 and the sound collection unit 303, to capture video of a machine operating, including the operating sound, performing an operation of the machine that is to be used as a reference sound. For example, after capturing several videos, the user selects a video that was successfully captured (step S101) and transmits it to the information processing device 110. In the information processing device 110, the acquisition unit 111 acquires the video, extracts audio from the video, and divides it into unit sounds (step S102). The division into unit sounds may involve dividing the video into unit times and then extracting only the audio, or may involve extracting audio from the video and then dividing the audio into unit times. The acquisition unit 111 transmits the unit sounds to the calculation unit 112. In the calculation unit 112, the first-dimensional data creation unit 1121 calculates a vector having a first-dimensional feature (first-dimensional vector) (step S103).

[0074] The first-dimensional data creation unit 1121 transmits the calculated first-dimensional vector to the feature creation unit 1122. The feature creation unit 1122 performs vector transformation to reduce the number of dimensions of the first-dimensional vector, thereby calculating a feature vector (step S104). The feature creation unit 1122 transmits the calculated feature vector to the two-dimensional data creation unit 1123 and the separability information creation unit 1124. The two-dimensional data creation unit 1123 creates two-dimensional data (step S105). The two-dimensional data creation unit 1123 transmits the created two-dimensional data to the first output unit 113. The first output unit 113 creates first display information including the acquired video and the created two-dimensional data (step S106). The first output unit 113 outputs the created first display information and transmits it to the terminal device 300. The display unit 302 displays the received first display information (step S107). Here, Fig. 5 is a diagram showing an example of a screen displayed on a terminal device in a sound registration system according to an embodiment. Fig. 5 displays second display information, which will be described later, but this second display information is almost identical to the first display information, and therefore the first display information will be described using Fig. 5. Specifically, the second display information differs from the first display information in that it includes separation degree information and a registration button. In other words, except for the separation degree information and the registration button in Fig. 5, the first display information is similar to the second display information shown in Fig. 5. However, the first display information only needs to include at least a moving image, and does not necessarily need to include the two-dimensional data shown in the figure.

[0075] As shown in Figure 5, the first display information displayed on the display of the display unit 302 includes a video timeline in which video thumbnails are arranged in chronological order, a preview of the video at a current position that can be set to any position on this video timeline, and two-dimensional data.

[0076] In addition to the current position, the video timeline allows the setting of a provisional registration target range (target range). Anything outside the target range is automatically set as a non-target range. As the target range, if the start and end points are determined on the video timeline, the section between the start and end points is automatically set. Multiple sets of start and end points may also be set. In this case, the section between the closest combination of start and end points (limited to this order) is automatically set as the target range. Note that the user can play the video to set the target range, and the video being played can be checked in the video preview. Furthermore, while the video is playing, the current position automatically moves to the position corresponding to the frame in the video being played.

[0077] Here, in the two-dimensional data, before the target range is set, there are no feature vectors corresponding to the target range, so unlike the example in the figure, all feature vectors are plotted as white dots. In other words, the first display information does not show the correspondence between the video and the feature vectors in the two-dimensional data. However, even with such two-dimensional data, peculiar sounds (for example, the sound of a barcode being read) are plotted at a position separate from other feature vectors, which may be useful in determining the target range and is expected to be effective.

[0078] Using this feature, if the first display information displayed on the display unit 302 includes two-dimensional data, the two-dimensional data may be used to set the registration target range. For example, the user may be able to individually select or deselect any plot of a feature vector on the two-dimensional data. As an example, a range may be selected on the two-dimensional data, and all selected plots may be set as the registration target range. Range selection refers to selecting all plots within a range using a rectangle, circle, or any shape drawn with the mouse cursor, etc. Because the feature vector is linked to a vector having features of the first dimension and the sound before analysis, even when the registration target range is set using two-dimensional data, it is possible to register information about the sound itself or the analysis results of the sound.

[0079] 4A , a user operation to set a registration target range in response to the display of the first display information, i.e., to select a portion of the video, is performed (step S108). The terminal device 300 is equipped with input devices such as a touch panel and a mouse for receiving the selection. The selection of the portion of the video as the selected registration target range is transmitted to the information processing device 110 and detected by the detection unit 114 (step S109).

[0080] The detection unit 114 transmits the detected registration target range to the two-dimensional data creation unit 1123 and the separability information creation unit 1124. The two-dimensional data creation unit 1123 creates two-dimensional data that graphs the feature vectors corresponding to the registration target range, i.e., the target vector, in a manner that allows the feature vectors corresponding to the non-target vectors to be distinguished from each other, by coloring the feature vectors corresponding to the target vector or by coloring the feature vectors corresponding to the non-target vector (step S110). In other words, in step S110, the two-dimensional data creation unit 1123 updates the two-dimensional data. Meanwhile, the separability information creation unit 1124 calculates the degree of separation of a set composed of feature vectors corresponding to the target vector from a set composed of feature vectors corresponding to the non-target vector, and creates separability information (step S111).

[0081] Referring to FIG. 4B , the two-dimensional data creation unit 1123 then transmits the created two-dimensional data to the second output unit 115, and the separability information creation unit 1124 transmits the created separability information to the second output unit 115. The second output unit 115 creates second display information including the video, the two-dimensional data, and the separability information (step S112). The second output unit 115 outputs the created second display information and transmits it to the terminal device 300. The display unit 302 displays the received second display information (step S113). As a result, as shown in FIG. 5 , the second display information including the separability information is displayed. As shown in FIG. 5 , several feature vectors are colored (indicated by hatching) in the two-dimensional data, and it can be seen that they correspond to the registration target range, i.e., the feature vectors correspond to the target vector. Furthermore, predetermined criteria are set for the separability information, and the user can determine whether the separability is sufficient based on the criteria. In the figure, the degree of separation increases from the 9 o'clock direction to the 12 o'clock direction and from the 12 o'clock direction to the 3 o'clock direction of the semicircle (showing good separation). Here, the 11 o'clock direction is indicated, and the degree of separation can be said to be about 30%. Note that the second display information only needs to include at least one of the relationship information, i.e., at least one of the two-dimensional data and the degree of separation information. Reducing the amount of information in the second display information not only reduces the required communication bandwidth, but also has the advantage of being displayed on a small terminal device 300 such as a smartphone, which can only display a limited amount of information at one time.

[0082] 4B , a user operation to change the range to be registered in response to the display of the second display information, i.e., to change the selected portion of the video (step S114), is performed. The change in the selection of the portion of the video to be registered is transmitted to the information processing device 110 and detected by the detection unit 114 (step S115). Note that the change in selection here may also be performed by selecting a range on the two-dimensional data.

[0083] The detection unit 114 transmits the detected registration target range to the two-dimensional data creation unit 1123 and the separability information creation unit 1124. The two-dimensional data creation unit 1123 creates two-dimensional data corresponding to the changed registration target range, i.e., by coloring the feature vectors corresponding to the changed target vectors or by coloring the feature vectors corresponding to the changed non-target vectors, in which the feature vectors corresponding to the changed target vectors and the feature vectors corresponding to the changed non-target vectors are graphed in a manner that allows them to be distinguished from each other (step S116). In other words, in step S116, the two-dimensional data creation unit 1123 updates the two-dimensional data again. Meanwhile, the separability information creation unit 1124 calculates the degree of separation between a set formed by feature vectors corresponding to the changed target vectors and a set formed by feature vectors corresponding to the changed non-target vectors, and creates separability information (step S117). In other words, in step S117, the separability information generating unit 1124 updates the separability information.

[0084] Next, the 2D data creation unit 1123 transmits the created 2D data to the second output unit 115, and the separability information creation unit 1124 transmits the created separability information to the second output unit 115. The second output unit 115 creates second display information including the video, the 2D data, and the separability information (step S112 is performed again). The second output unit 115 outputs the created second display information and transmits it to the terminal device 300. The display unit 302 displays the received second display information (step S113 is performed again). Thereafter, the operations of displaying the second display information to the user, changing the registration target range, creating the changed second display information, and displaying it again are repeated. In other words, while inputting and outputting information between the information processing device 110 and the user, the registration target range can be optimized until at least one of the 2D data and the separability information is in an appropriate state. FIG. 6A is a diagram partially illustrating an example of a screen displayed on a terminal device in a sound registration system according to an embodiment. FIG. 6B is a diagram partially illustrating an example of a screen displayed on a terminal device in a sound registration system according to an embodiment. Fig. 6A shows only a portion of the information in Fig. 5, and Fig. 6B shows the same information after the registration target range has been changed several times. As shown in Fig. 6A and Fig. 6B (a), the registration target range has been changed so that the start point and the end point are closer to each other. As shown in Fig. 6B (b), in the two-dimensional data, the feature vectors corresponding to the target vectors have been consolidated into a portion. As shown in Fig. 6C (c), the degree of separation, which was initially about 30%, has increased to about 70%.

[0085] 4B , when the user determines that the registration target range has been sufficiently optimized, the user presses the registration button to confirm the registration target range, i.e., to confirm the selected portion of the video (step S117). Information that the registration button has been pressed is then transmitted to the information processing device 110. The registration unit 116 selects, from among the sounds included in the video, sounds that fall within the confirmed registration target range as sounds for registration, and separates them from sounds that do not fall within the registration target range (step S118). The registration unit 116 outputs the separated sounds and transmits them to the sound discrimination device 120. The sound discrimination device 120 registers the sounds by storing the transmitted sounds in the storage unit 121 (step S119).

[0086] Other Embodiments Although the embodiments have been described above, the present disclosure is not limited to the above-described embodiments.

[0087] For example, in the above embodiment, the feature vectors corresponding to the target vectors and the feature vectors corresponding to the non-target vectors are colored to display the feature vectors corresponding to the target vectors and the feature vectors corresponding to the non-target vectors in a graph so that they can be distinguished from each other by different display colors. However, the feature vectors may also be distinguished by different display shapes. FIG. 7 is a diagram partially illustrating an example of a screen displayed on a terminal device in a sound registration system according to another embodiment. FIG. 7 shows two-dimensional data according to another example in the same situation as FIG. 6B . As shown in FIG. 7 , the feature vectors corresponding to the target vectors are plotted as triangles, and the feature vectors corresponding to the non-target vectors are plotted as circles, so that the feature vectors corresponding to the target vectors and the feature vectors corresponding to the non-target vectors can be distinguished from each other by different display shapes. Furthermore, different display shapes and different display colors may be combined.

[0088] Although the above description focuses on calculating a feature vector using mainly sound information from a video, this is not limiting. Since OpenL3, used to calculate the first-dimensional vector, is trained using both sound and image inputs, the first-dimensional vector may be calculated using both sound information and image information from the video. In this case, a first-dimensional vector of, for example, 512 dimensions may be obtained from the sound information, and a first-dimensional vector of, for example, 512 dimensions may be obtained from the image information. Similar operations can be performed by concatenating these first-dimensional vectors and subjecting the resulting 1024-dimensional concatenated first-dimensional vector to subsequent processing.

[0089] Furthermore, a model such as OpenL3 may not be used when calculating a first-dimensional vector from image information. For example, the image information may include brightness value information at each two-dimensional point, and this information may be used to calculate the first-dimensional vector. As an example, when a machine is photographed with a fixed camera to obtain a video, observing a specific pixel area in the image information included in the video reveals differences in brightness values ​​indicating whether a material, product, or the like is placed in that pixel area as the machine operates. This clearly shows that brightness values ​​are related to the operating state of the machine, so it is effective to use this information in combination with sound information as described above. Note that, depending on the type of machine, the first-dimensional vector may be calculated using only image information instead of sound information, and processing similar to that described in the above embodiment may be performed.

[0090] In the above-described embodiment, the processing performed by a specific processing unit may be performed by another processing unit. The order of multiple processing operations may be changed, or multiple processing operations may be performed in parallel.

[0091] In the above-described embodiments, each component may be realized by executing a software program suitable for that component, or by a program execution unit such as a CPU or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory.

[0092] Furthermore, each component may be realized by hardware. For example, each component may be a circuit (or integrated circuit). These circuits may form a single circuit as a whole, or each may be a separate circuit. Furthermore, each of these circuits may be a general-purpose circuit or a dedicated circuit.

[0093] Furthermore, the general or specific aspects of the present disclosure may be realized as an apparatus, a method, an integrated circuit, a computer program, or a computer-readable recording medium such as a CD-ROM, etc. Furthermore, the general or specific aspects of the present disclosure may be realized as any combination of an apparatus, a method, an integrated circuit, a computer program, and a recording medium.

[0094] For example, the present disclosure may be realized as an information processing method executed by a computer, or as a program for causing a computer to execute the information processing method. The present disclosure may also be realized as a computer-readable non-transitory recording medium on which such a program is recorded.

[0095] In addition, this disclosure also includes forms obtained by applying various modifications to each embodiment that a person skilled in the art would think of, or forms realized by arbitrarily combining the components and functions of each embodiment within the scope that does not deviate from the intent of this disclosure.

[0096] The present disclosure is useful in reproducing sound, for example, by allowing a user to perceive stereoscopic sound.

[0097] REFERENCE SIGNS LIST 100 Server device 110 Information processing device 111 Acquisition unit 112 Calculation unit 113 First output unit 114 Detection unit 115 Second output unit 116 Registration unit 120 Sound discrimination device 121 Storage unit 150 Network 300 Terminal device 301 Imaging unit 302 Display unit 303 Sound collection unit 1121 First dimension number data creation unit 1122 Feature amount creation unit 1123 Two-dimensional data creation unit 1124 Separability information creation unit

Claims

1. An information processing method executed by a computer for assisting in the registration of a sound as a reference sound in a sound discrimination device that discriminates sounds by learning reference sounds, comprising the steps of: acquiring sound-related information including a sound and representing at least one of a time change in the occurrence status of the sound or a time change in a characteristic of the sound; calculating a feature vector of the sound included in the sound-related information; creating and outputting first information including at least the sound-related information; detecting a selection of a part of the time domain of the sound-related information as a registration target range in response to the outputted first information being displayed; and creating and outputting second information including relationship information for each of the feature vectors that indicates a relationship between a target vector, which is the feature vector that falls within the registration target range, and a non-target vector, which is the feature vector that does not fall within the registration target range.

2. The information processing method of claim 1, wherein the relationship information includes two-dimensional data in which the similarity between each of the feature vectors is expressed as a two-dimensional distance between each of the feature vectors, and in the two-dimensional data, the feature vectors corresponding to the target vectors and the feature vectors corresponding to the non-target vectors are distinguishable from each other.

3. The information processing method according to claim 1 or 2, wherein the relationship information includes separation degree information indicating the degree of separation of a set composed of the feature vectors corresponding to the target vector from a set composed of the feature vectors corresponding to the non-target vector.

4. The information processing method according to claim 1, wherein the feature vector is a plurality of vectors corresponding to unit sounds obtained by dividing the sounds included in the sound-related information into unit times.

5. The information processing method of claim 1, wherein in the step of calculating the feature vector, a first-dimensional vector having features of a first dimension is calculated from the sounds included in the sound-related information, and the calculated first-dimensional vector is converted to calculate the feature vector having features of a second dimension, which is less than the first dimension.

6. The information processing method of claim 1, further comprising: a step of detecting a change in the registration target range in response to the display of the output second information; and a step of creating and outputting the second information including relationship information for each of the feature vectors indicating a relationship between a target vector, which is the feature vector that falls within the changed registration target range, and a non-target vector, which is the feature vector that does not fall within the changed registration target range.

7. The information processing method of claim 2, further comprising a step of generating the two-dimensional data as the relationship information prior to the step of creating and outputting the first information, wherein in the step of creating and outputting the first information, the first information including the sound-related information and the two-dimensional data is created and output.

8. The information processing method according to claim 2 or 7, wherein, in the two-dimensional data, the feature vectors corresponding to the target vectors and the feature vectors corresponding to the non-target vectors can be distinguished from each other by a difference in display shape.

9. The information processing method according to claim 2 or 7, wherein, in the two-dimensional data, the feature vectors corresponding to the target vectors and the feature vectors corresponding to the non-target vectors are distinguishable from each other by difference in display color.

10. The information processing method of claim 1, further comprising a step of separating and outputting sounds included in the sound-related information that fall within the registration target range from sounds that do not fall within the registration target range, and registering the sounds that fall within the registration target range as sounds for registration.

11. An information processing device for assisting in the registration of a sound as a reference sound in a sound discrimination device that discriminates sounds by learning reference sounds, comprising: an acquisition unit that acquires sound-related information including a sound and representing at least one of a time change in the occurrence status of the sound or a time change in a characteristic of the sound; a calculation unit that calculates a feature vector of the sound included in the sound-related information; a first output unit that creates and outputs first information including at least the sound-related information; a detection unit that detects a selection of a part of a time domain of the sound-related information as a registration target range in response to the outputted first information being displayed; and a second output unit that creates and outputs second information including relationship information indicating a relationship between a target vector, which is the feature vector that falls within the registration target range, and a non-target vector, which is the feature vector that does not fall within the registration target range, for each of the feature vectors.

12. A sound registration system comprising: a sound discrimination device that discriminates sounds by learning reference sounds; and an information processing device for assisting in the registration of sounds as the reference sounds, wherein the information processing device comprises: an acquisition unit that acquires sound-related information including sounds and that represents at least one of a time change in the occurrence status of the sound or a time change in a feature of the sound; a calculation unit that calculates a feature vector of the sound included in the sound related information; a first output unit that creates and outputs first information including at least the sound related information; a detection unit that detects a selection of a part of a time domain of the sound related information as a registration target range in response to the outputted first information being displayed; a second output unit that creates and outputs second information including relationship information for each of the feature vectors that indicates a relationship between a target vector that is the feature vector that falls within the registration target range and a non-target vector that is the feature vector that does not fall within the registration target range; and a registration unit that registers sounds included in the sound related information that fall within the registration target range as sounds for registration, by separating them from sounds that do not fall within the registration target range and outputting them in the sound discrimination device.

13. A program for causing a computer to execute the information processing method according to claim 1.

Citation Information

Patent Citations

  • Sound identification system, sound identification device, sound identification method, and program

    JP2013225248A

  • Sound information setting method and sound information setting device

    JP2018005171A

  • Sound source separation evaluation device and sound source separation device

    JP2020012976A