Sound diagnosis system, sound diagnosis method, and sound diagnosis program

Abstract

To provide a technology capable of easily narrowing down a generation source of abnormal noise from a plurality of components.SOLUTION: A sound diagnostic system 100 includes a collection section for collecting sound information related to the sound emitted from a device 130, a detection section for detecting a first abnormal sound having periodicity from the sound information, a first selection section for selecting one or more components corresponding to the generation period of the first abnormal sound from among a plurality of components included in the device 130, an acquisition section for acquiring operation information indicating the operation periods of each of the plurality of components, and a second selection section for selecting one or more candidate causes of the first abnormal sound from the one or more components on the basis of the operation information and the sound information.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to a sound diagnostic system, and more particularly to a sound diagnostic system using sound pressure. [Background technology]

[0002] When a device such as a home appliance used at home or an image forming device used in an office breaks down, the device may emit an abnormal sound. A serviceman can identify the broken part from the abnormal sound and replace the broken part. However, when a device equipped with multiple motors breaks down, the serviceman may not be able to identify which part of the device is generating the abnormal sound. In this case, the serviceman cannot identify which part should be repaired or replaced. Therefore, there is a need for a sound diagnosis technology that can easily identify the part generating the abnormal sound in the device.

[0003] Regarding sound diagnosis, for example, JP 2020-071436 A (Patent Document 1) discloses "a source identification device and an image forming apparatus capable of identifying the source of abnormal sound." The device "includes a memory and a processor. The memory stores information related to the drive frequency of the drive mechanism. The processor detects abnormal sounds from the sounds of multiple drive mechanisms, identifies the occurrence interval of the detected abnormal sounds, and identifies the drive mechanism that is the source of the abnormal sound based on the occurrence interval of the identified abnormal sound and the information stored in the memory" (see [Abstract]).

[0004] Japanese Patent Laid-Open Publication No. 2007-114272 (Patent Document 2) discloses an "image forming apparatus that quickly diagnoses operating parts that generate abnormal noises." The apparatus "has a plurality of operating parts for forming an image on a recording medium, and an abnormal noise diagnosis operation mode that controls the operation of almost all of the operating parts to be driven singly or in succession in multiples" (see paragraph

[0008] ). [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2020-071436 A [Patent Document 2] JP 2007-114272 A Summary of the Invention [Problem to be solved by the invention]

[0006] The device disclosed in Patent Document 1 identifies the drive mechanism that is the source of the abnormal noise based on the interval at which the abnormal noise occurs. Therefore, if there are multiple parts that operate at the same interval, multiple parts may be selected as the source of the abnormal noise. Therefore, the user needs to narrow down the multiple selected parts.

[0007] The device disclosed in Patent Document 2 has an abnormal sound diagnosis operation mode that controls almost all of the multiple operating parts to be driven singly or in sequence, thereby making it possible to narrow down the source of the abnormal sound. However, the user needs to identify the operating part that is the source of the abnormal sound while checking the spectrogram of the sound when each operating part is driven. Therefore, when there are many driving parts, it takes time and effort to identify the operating part that is the source of the abnormal sound.

[0008] The present disclosure has been made in consideration of the above-described background, and an object of one aspect is to provide a technology that makes it possible to easily narrow down the source of abnormal noise from among multiple components. [Means for solving the problem]

[0009] According to one embodiment, there is provided a sound diagnostic system including a collection unit that collects sound information related to sounds emitted by an apparatus, a detection unit that detects a periodic first abnormal sound from the sound information, a first selection unit that selects one or more components corresponding to the occurrence period of the first abnormal sound from among a plurality of components included in the apparatus, an acquisition unit that acquires operation information indicating the operation period of each of the plurality of components, and a second selection unit that selects one or more candidate causes of the first abnormal sound from among the one or more components based on the operation information and the sound information.

[0010] In one aspect, the second selection unit selects the first part as one of the one or more candidate causes in response to at least one of the multiple occurrence timings of the first abnormal sound falling within an operating period of a first part of the one or more parts.

[0011] In one aspect, the second selection unit classifies, from among the one or more candidate causes, candidate causes in which at least one of a plurality of occurrence timings of the first abnormal sound does not belong to an operating period, into a first group.

[0012] In one aspect, the second selection unit selects the first part as one of the one or more candidate causes in response to all of the multiple occurrence timings of the first abnormal sound falling within an operating period of the first part among the one or more parts.

[0013] In one aspect, the second selection unit classifies, from among the one or more candidate causes, candidate causes whose operating period includes a normal period in which the first abnormal sound does not occur, into a second group. The normal period is longer than the occurrence cycle of the first abnormal sound.

[0014] In one aspect, the acquisition unit acquires the motion information from the device. In one aspect, the operation information defines an operation period based on a start timing of a specific process by the device. The device emits a specific sound when starting the specific process. The detection unit detects the occurrence timing of the specific sound and the occurrence timing of the first abnormal sound from the sound information. The second selection unit determines whether each of the multiple occurrence timings of the first abnormal sound belongs to the operation period based on a time difference between the occurrence timing of the specific sound and each of the multiple occurrence timings of the first abnormal sound.

[0015] In one aspect, the specific process is started in response to pressing a button included in the device, and the specific sound is emitted in response to pressing the button.

[0016] In one aspect, the plurality of components includes a plurality of motors for performing a specific process, and the specific sound is a driving sound of a motor that is driven first in the specific process among the plurality of motors.

[0017] In one aspect, the detection unit detects a non-periodic second abnormal sound from the sound information. The multiple components include a group of components that may generate the non-periodic abnormal sound. The second selection unit selects the second component as a candidate cause of the second abnormal sound in response to the occurrence timing of the second abnormal sound falling within an operating period of a second component of the group of components.

[0018] In one aspect, the device is an image forming device. According to another embodiment, there is provided a sound diagnosis method for an apparatus, the sound diagnosis method including: one or more processors collecting sound information regarding a sound emitted by the apparatus, the one or more processors detecting a periodic first abnormal sound from the sound information, the one or more processors selecting, from a plurality of components included in the apparatus, one or more components corresponding to the occurrence period of the first abnormal sound, the one or more processors acquiring operation information indicating an operation period of each of the plurality of components, and the one or more processors selecting, from the one or more components, one or more candidate causes of the first abnormal sound based on the operation information and the sound information.

[0019] According to yet another embodiment, there is provided a sound diagnosis program for an apparatus. The sound diagnosis program causes a computer to execute the following steps: collect sound information on sounds emitted by the apparatus, detect a periodic first abnormal sound from the sound information, select one or more components corresponding to the occurrence period of the first abnormal sound from among a plurality of components included in the apparatus, obtain operation information indicating the operation period of each of the plurality of components, and select one or more candidate causes of the first abnormal sound from among the one or more components based on the operation information and the sound information. Effect of the Invention

[0020] According to an embodiment, the sound diagnostic system can easily narrow down the source of the abnormal noise from among multiple components.

[0021] The above and other objects, features, aspects and advantages of the present disclosure will become apparent from the following detailed description of the disclosure taken in conjunction with the accompanying drawings. [Brief description of the drawings]

[0022] [Figure 1] 1 is a diagram illustrating an example of the configuration of a sound diagnostic system 100 to which the technology of the present disclosure can be applied. [Diagram 2] FIG. 1 is a diagram showing an example of a configuration of a sound diagnosis system 100 according to the present embodiment. [Diagram 3] 4 is a diagram showing an example of a procedure of diagnostic processing executed by the sound diagnostic system 100. FIG. [Figure 4] FIG. 4 is a diagram showing an example of a spectrogram 400. [Diagram 5] FIG. 11 is a diagram illustrating an example of an analysis process of an acquired spectrogram. [Figure 6] FIG. 6 is a diagram showing an example of data 600 of the sum of sound pressures of each frequency over time. [Figure 7] FIG. 7 is a diagram showing an example of peak value information 700. [Figure 8] FIG. 1 is a diagram conceptually illustrating a spectrogram. [Figure 9]FIG. 13 is a diagram showing an example of peak prominence of the total sound pressure. [Figure 10] FIG. 13 is a diagram showing an example of a procedure of clustering (classification) by the peak classification unit 208. [Figure 11] FIG. 13 is a diagram illustrating an example of a clustering result of peak timings. [Figure 12] 12 is a diagram showing an example of a first screen 1200 output by an output unit 224. FIG. [Figure 13] 13 is a diagram showing an example of a second screen 1300 output by the output unit 224. FIG. [Figure 14] 13 is a diagram showing a continuation of the second screen 1300 output by the output unit 224. FIG. [Figure 15] FIG. 2 is a diagram showing an example of a periodic table 214. [Figure 16] FIG. 13 is a diagram showing a first example of a set of clustered peak timings. [Figure 17] FIG. 17 is a diagram showing an example of a calculation result 1700 of occurrence periods of clustered peak timings. [Figure 18] FIG. 13 is a diagram showing a second example of a set of clustered peak timings. [Figure 19] FIG. 19 is a diagram showing an example of a calculation result 1900 of occurrence periods of clustered peak timings. [Figure 20] FIG. 2 is a diagram showing an example of operation information 232. [Figure 21] FIG. 13 is a diagram illustrating an example of peak timing for each cluster. [Figure 22] FIG. 1 illustrates a method for grouping one or more candidate causes. [Figure 23] 23 is a diagram showing an example of a third screen 2300 output by the output unit 224. FIG. [Figure 24] FIG. 4 is a diagram showing a first example of a procedure of a subroutine process of step S380 shown in FIG. [Diagram 25] FIG. 4 is a diagram showing a second example of the procedure of the subroutine process of step S380 shown in FIG. [Figure 26]It is a diagram showing a first example of the procedure of the subroutine of step S2407 shown in FIG. 24. [Figure 27] It is a diagram showing a second example of the procedure of the subroutine of step S2407 shown in FIGS. 24 and 25. [Figure 28] It is a diagram showing a third example of the procedure of the subroutine of step S2407 shown in FIGS. 24 and 25. [Figure 29] It is a block diagram showing the hardware configuration of the computer 2900 that functions as the server 110 or the diagnostic device 120.

Embodiments for Carrying Out the Invention

[0023] Hereinafter, embodiments of the technical idea according to the present disclosure will be described with reference to the drawings. In the following description, the same parts are denoted by the same reference numerals. Their names and functions are also the same. Therefore, detailed descriptions thereof will not be repeated. Also, each embodiment and each modification example may be selectively combined as appropriate.

[0024] <A. Application Example> FIG. 1 is a diagram showing an example of the configuration of a sound diagnosis system 100 to which the technology of the present disclosure is applicable. With reference to FIG. 1, the outline of the sound diagnosis system 100 to which the technology of the present disclosure is applicable and the problems in sound diagnosis will be described. At the same time, the terms used in this specification will be described.

[0025] (a. Configuration of a System to which the Technology of the Present Disclosure is Applicable) The sound diagnosis system 100 includes, as an example, a diagnostic device 120 and a server 110. According to an embodiment, the diagnostic device 120 may have the functions of the server 110. The device 130 is the object of sound diagnosis. The device 130 is any device having a plurality of drive parts. The device 130 includes office equipment such as an image forming device, home appliances, in-factory equipment, research equipment, and the like.

[0026] The diagnostic device 120 collects sound information related to sounds emitted by the device 130 and displays the analysis results. A serviceman carries the diagnostic device 120 to perform maintenance, repair, etc. of a device to be maintained that is used by an end user. The diagnostic device 120 acquires at least abnormal sounds emitted by the device to be maintained and displays the analysis results of the acquired abnormal sounds. The analysis results of the abnormal sounds include information indicating one or more candidate causes of the abnormal sounds selected from among multiple components included in the device 130.

[0027] According to an embodiment, the diagnostic device 120 transmits the sampled sound information to the server 110. Then, the diagnostic device 120 receives an analysis result from the server 110. Furthermore, the diagnostic device 120 displays the analysis result on a display. The analysis result includes at least information on one or more candidate causes of the abnormal sound. The one or more candidate causes are parts among a plurality of driving parts that are estimated to be generating the abnormal sound. The diagnostic device 120 and the server 110 are connected to each other via a wireless or wired network so as to be able to transmit and receive data to and from each other.

[0028] According to another embodiment, the diagnostic device 120 analyzes the sampled sound information. Then, the diagnostic device 120 displays the analysis result on a display. The analysis result includes at least information on one or more possible causes of the abnormal sound. In this case, the sound diagnostic system 100 does not need to include the server 110.

[0029] According to another embodiment, the diagnostic device 120 is a tablet, a smartphone, a personal computer, smart glasses, or any other information processing device. The diagnostic device 120 may include a built-in normal microphone or a directional microphone. The diagnostic device 120 may be configured to be connectable to a normal microphone or a directional microphone.

[0030] The server 110 receives sound information from the diagnostic device 120 via a network. The server 110 analyzes the sound information and returns the analysis result to the diagnostic device 120. The analysis result includes information on one or more possible causes of the abnormal sound. Details of the procedure for analyzing the sound information will be described later. According to an embodiment, the server 110 can be realized as one or more devices, a virtual machine built on a cloud environment, or a combination of these.

[0031] The device 130 is a device to be diagnosed. The device 130 includes a plurality of moving parts. The sounds emitted by each moving part may have different frequencies. The diagnosis device 120 uses the frequencies of the sounds emitted by each moving part to identify the source of each sound.

[0032] (b. Problems that may occur in sound diagnosis) As described above, the device 130 includes a plurality of driving parts. For example, an image forming device used in an office or the like includes a large number of driving parts. Therefore, even if the diagnostic device 120 collects abnormal sounds, it is not easy to identify which driving part is generating the abnormal sound. To solve this problem, the sound diagnostic system 100 has a function of picking up a plurality of sounds generated by the device 130 and classifying each sound. In addition, the sound diagnostic system 100 has a function of identifying the driving part generating each sound from each classified sound.

[0033] (c. Terms Used in This Specification) Next, the terms used to describe the technology of the present disclosure will be explained.

[0034] In this specification, the term "system" includes a configuration of one or more devices, a server, etc. The system also includes a virtual machine or a container built in a cloud environment, or a configuration of at least a part of these.

[0035] In this specification, the term "device" refers to any information processing device such as a personal computer, a workstation, a server device, a tablet, or a smartphone. The device may also be a combination of these.

[0036] According to one embodiment, the sound diagnosis system 100 may be connected to input / output devices such as a display and a keyboard and used by a user. According to another embodiment, the sound diagnosis system 100 may provide various functions to a user as a service or a web application via a network. In this case, the user may use the functions of the sound diagnosis system 100 via a browser or client software installed on his / her terminal.

[0037] In this specification, the "device to be diagnosed" includes a device installed anywhere, such as a home, an office, a factory, a school, or a research laboratory. As an example, the device to be diagnosed is any device, such as a refrigerator, an air conditioner, an image forming device, a manufacturing device or a robot installed in a factory. The device to be diagnosed includes multiple driving parts. In this specification, the device to be diagnosed may also be simply referred to as a device.

[0038] In this specification, a "part (driving part)" is a part that is included in the device to be diagnosed. The part may be a motor, or a combination of a motor and a power transmission part. The part may also be a combination of a motor, a power transmission part, and a part beyond that. Alternatively, the part may be a part that is driven by a motor. Alternatively, the part may be a clutch or a solenoid.

[0039] In this specification, "sound and sound information" refers to sounds emitted by the device to be diagnosed. The sound and sound information may also be data picked up by the diagnostic device 120 using a microphone, i.e., sampled sound data. The sampled sound data is digital data represented at any resolution.

[0040] "Sound pressure" as used in this specification is the variation from atmospheric pressure due to sound. Sound pressure is defined as the effective value of the maximum amplitude.

[0041] "Overall sound pressure (overall sound intensity)" as used in this specification is the sum or average value of the sound pressures of each frequency component at a certain timing in the spectrogram. The spectrogram includes a plurality of frequency components for each timing. For example, assume that frequency components X, Y, and Z are included at timing A of the spectrogram. In this case, the overall sound pressure (overall sound intensity) at timing A is calculated by the formula "Overall sound pressure at timing A = Sound pressure of frequency component X + Sound pressure of frequency component Y + Sound pressure of frequency component Z". Alternatively, the overall sound pressure (overall sound intensity) at timing A may be the average value of the sound pressures of frequency components X, Y, and Z.

[0042] "(Peak timing of overall sound pressure)" as used in this specification is the timing that has a sound pressure higher than a certain level with respect to the sound pressures at the surrounding timings in the graph of continuous overall sound pressure. The peak timing may also be simply referred to as a peak. Also, the sound (overall sound pressure) at a certain peak timing may be referred to as a peak.

[0043] "Operation information" as used in this specification indicates the operation period of each of a plurality of drive components. Typically, the operation information indicates the start timing and end timing of the period during which each drive component is operating. Alternatively, the operation information may indicate the start timing and end timing of the period during which each drive component is stopped.

[0044] <B. Device Configuration> FIG. 2 is a diagram showing an example of the configuration of the sound diagnosis system 100 according to the present embodiment. Some or all of the configurations shown in FIG. 2 may be realized as hardware. Also, some of the configurations shown in FIG. 2 may be realized as software. In this case, the sound diagnosis system 100 includes one or more processors (not shown) for executing each software, one or more memories (not shown), and one or more storages (not shown).

[0045] The sound diagnosis system 100 includes a collection unit 203 and a detection unit 211. The collection unit 203 includes a microphone 202 and a sound collection unit 204. The detection unit 211 includes a total sound pressure calculation unit 205, a peak detection unit 206, a peak classification unit 208, and a period calculation unit 210. Furthermore, the sound diagnosis system 100 includes a storage unit 212, a first selection unit 222, a second selection unit 223, an output unit 224, and an acquisition unit 230. The storage unit 212 stores a periodic table 214, model parameters 216, a UI (User Interface) program 218, and operation information 232.

[0046] According to an embodiment, the diagnostic device 120 includes a collection unit 203 and an output unit 224. The server 110 includes a detection unit 211, a storage unit 212, a first selection unit 222, a second selection unit 223, and an acquisition unit 230.

[0047] According to another embodiment, the diagnostic device 120 may also have the functionality of the server 110, in addition to the collection unit 203 and the output unit 224. In this case, the diagnostic device 120 includes a detection unit 211, a storage unit 212, a first selection unit 222, a second selection unit 223, and an acquisition unit 230.

[0048] The microphone 202 and the sound collection unit 204 constitute a collection unit 203 that collects sound information regarding sounds emitted by the device.

[0049] The microphone 202 converts sound into an electrical signal and outputs the electrical signal to the sound collector 204. The device 130 includes multiple sound sources (driving components). Therefore, the microphone 202 picks up a composite sound emitted by the multiple sound sources.

[0050] The sound collection unit 204 converts the electrical signal output from the microphone 202 into a digital signal to generate sound information (sampling result). The sound collection unit 204 converts an analog signal into sound information, which is a digital signal, at an arbitrary resolution. The sound collection unit 204 outputs the generated sound information (sampling result).

[0051] The total sound pressure calculation unit 205, the peak detection unit 206, the peak classification unit 208, and the period calculation unit 210 constitute a detection unit 211 that detects abnormal sounds from sound information. The detection unit 211 detects periodic abnormal sounds (hereinafter referred to as "periodic abnormal sounds"). The periodic abnormal sounds correspond to "first abnormal sounds" in the present disclosure. Furthermore, the detection unit 211 may detect non-periodic abnormal sounds (hereinafter referred to as "non-periodic abnormal sounds"). The non-periodic abnormal sounds correspond to "second abnormal sounds" in the present disclosure.

[0052] The total sound pressure calculation unit 205 calculates the total sound pressure of the sounds of each frequency included in each timing of the sampling result. More specifically, the total sound pressure calculation unit 205 calculates the sum or average value of the sound pressure of each frequency at each timing of the sampling result. The total sound pressure calculation unit 205 performs a short-time Fourier transform (STFT) on the sound information (sound data). The total sound pressure calculation unit 205 obtains a frequency spectrum waveform (spectrogram) by performing STFT on the sound information. The total sound pressure calculation unit 205 calculates the sum of the values ​​of each frequency at each timing from the spectrogram to obtain the sound pressure at a certain timing. Alternatively, the total sound pressure calculation unit 205 calculates the average value of each frequency at each timing from the spectrogram to obtain the sound pressure at a certain timing.

[0053] The peak detection unit 206 determines the peak timing of the sound pressure. More specifically, the peak detection unit 206 extracts, as the peak timing, a timing at which the difference between the sound pressure at each timing and the adjacent sound pressure is equal to or greater than a preset value. Alternatively, the peak detection unit 206 may extract the peak timing based on a prominence value (prominence) of the sound pressure. Details of the peak detection unit 206 will be described later with reference to Figs. 4 to 9. According to an embodiment, the peak detection unit 206 may also have the function of the total sound pressure calculation unit 205.

[0054] The peak classification unit 208 classifies the extracted peak timings into a predetermined number of patterns (number of clusters). In order to classify each peak timing, the peak classification unit 208 uses the frequency and peak prominence value of the peak timing as feature quantities. The peak classification unit 208 may classify the sound pressure of each peak timing based on machine learning. As an example, the peak classification unit 208 uses k-means clustering for unsupervised learning. Details of the classification of the sound pressure of each peak timing will be described later with reference to FIG. 10 and FIG. 11.

[0055] The period calculation unit 210 calculates the interval between the peak timings for each category (cluster) of the sound pressure of the classified peak timings. As one example, the period calculation unit 210 calculates the period by finding the most frequent value of the interval between the peak timings. The interval between the peak timings is the interval between the occurrence timings of the respective peak sounds. As another example, the period calculation unit 210 calculates the period using an average value or a median value.

[0056] When the intervals between the peak timings classified into a certain cluster are periodic, the peak timings classified into the certain cluster correspond to the occurrence timings of a periodic abnormal sound.When the intervals between the peak timings classified into a certain cluster are non-periodic, the peak timings classified into the certain cluster correspond to the occurrence timings of a non-periodic abnormal sound.

[0057] The first selection unit 222 selects one or more parts corresponding to the occurrence period of the periodic abnormal sound from among a plurality of driving parts included in the device 130. The one or more selected parts are candidates for the cause of the periodic abnormal sound. More specifically, the first selection unit 222 compares each period calculated in each cluster with the periodic table 214. The first selection unit 222 selects a part that emits a sound whose period is close to that of the sound in each cluster through the comparison. Details of the processing by the first selection unit 222 will be described later with reference to Figs. 12 to 14.

[0058] The acquiring unit 230 acquires operation information 232 indicating an operation period of each of the multiple driving components. Typically, the acquiring unit 230 acquires the operation information 232 from the device 130. An example of the operation information 232 will be described later with reference to FIG. 20.

[0059] The second selection unit 223 selects one or more candidate causes of the periodic abnormal noise from among the one or more parts selected by the first selection unit 222, based on the operation information 232 and the sound information. In other words, the second selection unit 223 narrows down the candidates for the source of the periodic abnormal noise.

[0060] Furthermore, the second selection section 223 may select one or more candidate causes of the non-periodic abnormal noise based on the motion information 232 and the sound information. Details of the processing by the second selection section 223 will be described later with reference to Figs. 21 to 28.

[0061] The storage unit 212 stores data necessary for performing each process of the flow of FIG. 3 and data of the UI provided to the user. The storage unit 212 stores a periodic table 214. The periodic table 214 includes periodic data of sounds emitted by each driving part. The periodic table 214 may also include periodic data of clustered sound information. The storage unit 212 stores model parameters 216. The model parameters 216 include parameters of a machine learning model used for clustering. The storage unit 212 stores a UI program 218. The UI program 218 is data or a program of the UI provided to the user. The UI program 218 includes UI design data, an event program linked to each UI component, and the like. Furthermore, the storage unit 212 stores operation information 232.

[0062] The output unit 224 outputs the analysis result of the sound information and one or more candidate causes of the abnormal sound from among the multiple driving parts. As an example, the output unit 224 displays the screens shown in Fig. 12 to Fig. 14 on the display of the diagnostic device 120. Alternatively, the output unit 224 may transmit display data to another smartphone 226, a PC (Personal Computer) 228, or any other device.

[0063] <C. Procedure for Acoustic Diagnosis> FIG. 3 is a diagram showing an example of the procedure of the diagnostic process executed by the acoustic diagnosis system 100. Referring to FIG. 3, the outline of the diagnostic process of the acoustic diagnosis system 100 will be described. For the details of each process in FIG. 3, reference will be made to FIGS. 4 to 28 and described later. According to an embodiment, the processor of the acoustic diagnosis system 100 may read a program for performing the process of FIG. 3 from the storage into the memory and execute the program. In other aspects, part or all of the process may also be realized as a combination of circuit elements configured to execute the process.

[0064] In step S310, the diagnostic device 120 picks up the sound of the device 130 to be diagnosed with a microphone (microphone 202). More specifically, the microphone 202 of the diagnostic device 120 picks up the sound emitted by the device 130 in response to the operation of a user (for example, a service technician). After the user starts the operation of the microphone 202, the user presses the start button of the device 130. Thereby, the device 130 starts a specific process described later. Also, the sound collection unit 204 converts the sound (analog signal) into sound information (digital data) obtained by sampling.

[0065] In step S320, the diagnostic device 120 transmits the sound information to the server 110. In an embodiment, when the diagnostic device 120 also has the function of the server 110, the acoustic diagnosis system 100 may not execute the process of step S320.

[0066] In step S330, the server 110 performs STFT on the acquired sound information. In step S340, the server 110 obtains a spectrogram as a result of the STFT. As an example, the server 110 obtains a spectrogram 400 as shown in FIG. 4.

[0067] In step S350, server 110 calculates the sum (total sound pressure) of the sound pressures of each frequency for each time (timing) in the spectrogram. Alternatively, server 110 calculates the average value of the sound pressures of each frequency for each time (timing) in the spectrogram. As an example, server 110 acquires data 600 as shown in Fig. 6. Data 600 includes information indicating the sum or average value of the sound pressures of each frequency for each time (timing) in the spectrogram 400.

[0068] In step S360, server 110 extracts peak timing of the total sound pressure from data 600. A peak timing is a timing in the graph of total sound pressure where the total sound pressure is greater than the timings before and after the peak timing. In other words, it is a portion that is greatly convex in the graph of total sound pressure. As an example, server 110 obtains peak value information 700 of the peak timing as shown in FIG. 7.

[0069] In step S365, server 110 (detection unit 211) detects the occurrence timing of a specific sound emitted when device 130 starts a specific process. The specific process is a process that is started in response to, for example, pressing a start button included in device 130. Device 130 emits a specific sound when starting a specific process. Server 110 detects the occurrence timing of the specific sound based on the characteristics of the specific sound. The characteristics of the specific sound include, for example, a frequency or a time length. Server 110 stores the characteristics of the specific sound in advance. Server 110 extracts a sound that is close to the characteristics of the specific sound from the sound information, and detects the occurrence timing of the extracted sound.

[0070] In step S370, the server 110 clusters each peak timing included in the peak value information 700 based on the frequency and / or sound pressure of each peak timing.

[0071] In step S380, the server 110 selects one or more cause candidates of abnormal noise from among a plurality of drive components based on the analysis result, the periodic table 214, and the operation information 232. The server 110 transmits the analysis result and one or more cause candidates of abnormal noise to the diagnostic device 120.

[0072] In step S390, the diagnostic device 120 displays the received analysis result and one or more cause candidates on a display (not shown). According to an embodiment, the diagnostic device 120 may transfer the analysis result to another device (such as a PC or a smartphone).

[0073] That is, the sound diagnosis system 100 acquires the sound emitted by the device 130 by the sound collection unit 204. At this time, the sound collection unit 204 can convert the analog signal of the sound into a digital signal (sampling result). The sound diagnosis system 100 analyzes the sampling result of the sound by the period calculation unit 210, and calculates the overall sound pressure of the sound at each frequency included in each timing of the sampling result (see FIG. 4). The sound diagnosis system 100 extracts a plurality of peak timings from the overall sound pressure by the peak detection unit 206 (see FIG. 7). The sound diagnosis system 100 clusters a plurality of peak timings by the peak classification unit 208 (see FIG. 11). The sound diagnosis system 100 calculates the occurrence period of the periodic abnormal noise based on the frequency and / or each overall sound pressure by the period calculation unit 210 (see FIG. 11). The sound diagnosis system 100 selects one or more cause candidates of abnormal noise from among a plurality of drive components by the first selection unit 222 and the second selection unit 223 (see FIGS. 15 to 17, FIGS. 21 to 28). The sound diagnosis system 100 displays the selected one or more cause candidates by the output unit 224 (see FIGS. 13, 14, and 23).

[0074] <D. Details of Each Procedure> Next, with reference to FIGS. 4 to 28, the details of each process of the flow in FIG. 3 will be described. More specifically, with reference to FIGS. 4 to 28, the processes from step S330 to S390 will be described in detail.

[0075] FIG. 4 is a diagram showing an example of a spectrogram 400. The spectrogram 400 shows an example of sound information (sound data) acquired in steps S330 and S340. The sound diagnosis system 100 acquires the spectrogram 400 by performing STFT on the sound information of the device 130. The sound information includes multiple frequency components. That is, the sound information is a collection of multiple sounds having different frequencies. Hereinafter, "frequency or frequency component" may indicate a sound of a certain frequency or sound data of a certain frequency. The spectrogram 400 shows the time change of the sound pressure distribution of each frequency component. In other words, the spectrogram 400 includes information on various sounds emitted by each part of the device 130.

[0076] As described below, the sound diagnostic system 100 calculates the sum or average value of the sound pressure of each frequency from the spectrogram 400. In this way, the sound diagnostic system 100 calculates the overall sound pressure (sum or average value) of the sounds of each frequency included in each timing of the sampling result.

[0077] FIG. 5 is a diagram showing an example of an analysis process of an acquired spectrogram. The process of FIG. 5 corresponds to the process of step S350. Conceptual diagram 510 conceptualizes spectrogram 400. The vertical axis of conceptual diagram 510 indicates frequency. The horizontal axis of conceptual diagram 510 indicates time. Numbers in the graph indicate sound pressure for each frequency at a certain timing. Using conceptual diagram 510 as an example, the sound pressure of a frequency of 45 Hz at timing a is 2. Similarly, the sound pressure of a frequency of 10 Hz at timing a is 1.

[0078] Now, let us focus on timing a. In column 511 for timing a, from the bottom, 1, blank, blank, 2, 2, 2, 2 are displayed. This indicates that at timing a, the sound pressure of 10 Hz is 1, no sounds of 15, 20, and 25 Hz are generated, and the sound pressure of 30, 35, 40, and 45 Hz is 2.

[0079] The server 110 calculates the total sound pressure at the timing a. Specifically, the server 110 adds up the sound pressures of the frequency components at the timing a. As described above, the total sound pressure (total sound intensity) is the sum or average value of the sound pressures of the frequency components at a certain timing of the spectrogram. Taking the conceptual diagram 510 as an example, the server 110 calculates the total sound pressure "9" at the timing a by calculating "1+2+2+2+2=9". According to an embodiment, the server 110 may calculate the average value of the sound pressure at the timing a. Similarly, the server 110 calculates the total sound pressure at each timing (timings b, c, d, etc.). The server 110 acquires the data 600 by performing such calculations.

[0080] According to an embodiment, the server 110 may multiply the sound pressure of a frequently occurring frequency component (e.g., 1000 Hz or higher) by a coefficient. The server 110 may calculate the occurrence frequency of each frequency from previously acquired sound information. In other words, the server 110 (total sound pressure calculation unit 205) may adjust the added value of each frequency sound according to the occurrence frequency of each frequency sound.

[0081] According to another embodiment, the server 110 may sum up the sound pressure for each frequency at a certain timing, excluding frequencies generated from parts other than the part being maintained. For example, if the part being maintained does not generate frequency components below 500 Hz, the frequency components below 500 Hz are considered to be simply noise. Therefore, the server 110 sums up the sound pressure for each frequency at a certain timing, excluding frequency components below 500 Hz.

[0082] As described above, the server 110 obtains the sound pressure from the spectrogram, not the amplitude. This allows the server 110 to execute processing that takes into account the importance of the frequency band (weighting according to the frequency of occurrence of the frequency component). In addition, since sound pressure peaks tend to appear more clearly than amplitude peaks, the server 110 can more easily execute peak timing detection processing.

[0083] Graph 520 shows the relative value of the total sound pressure for each timing (relative total sound pressure). For example, the sound intensity at timing a is 9. The sound intensity at timing a is 8 higher on average than the sound intensities at the previous and following timings. Therefore, the value on the vertical axis at timing a in graph 520 is 8.

[0084] FIG. 6 is a diagram showing an example of data 600 of the sum of sound pressures of each frequency for each time period. Data 600 shows an example of data obtained by executing the process of step S350. As described above, server 110 obtains the sound pressure of each frequency for each timing of spectrogram 400. Then, server 110 calculates the sum of the sound pressures of each frequency (total sound pressure) for each timing of spectrogram 400. In this way, server 110 obtains data 600. Data 600 shows continuous values ​​of the total sound pressure.

[0085] FIG. 7 is a diagram showing an example of peak value information 700. Peak value information 700 shows an example of data obtained by executing the process of step S350. Server 110 (peak detection unit 206) compares the total sound pressure at a certain timing with the total sound pressure at an adjacent timing. Assume that the total sound pressure at a certain timing is higher than the total sound pressure at the adjacent timing by a predetermined threshold or more. In this case, server 110 (peak detection unit 206) determines that certain timing to be a peak timing. Server 110 (peak detection unit 206) acquires the sound intensity at that time as a peak value. An "x" in peak value information 700 indicates a peak value. Hereinafter, the value of the peak timing may be referred to as a "peak value or peak sound pressure."

[0086] Using conceptual diagram 510 as an example, a method for determining peak timing will be described in more detail. The sound intensity at timing a is 9. The sound intensity at the timing immediately before timing a is 1. The sound intensity at the timing immediately after timing a is 1. Server 110 (peak detection unit 206) determines the difference between the sound intensity at timing a and the sound intensities before and after timing a. The difference is "9-1=8", which is a relative value to the timings before and after timing a.

[0087] Assume that the predetermined threshold is 0. In this case, the sound intensity "9" at the timing a exceeds the predetermined threshold "0". Therefore, the server 110 (peak detection unit 206) determines that the timing a is a peak timing. The server 110 (peak detection unit 206) then determines that the sound intensity at the timing a is a peak value or a peak sound pressure. From the above explanation, it can be said that the graph 520 shows the sound intensity selected as a peak. The sound diagnosis system 100 calculates the total sound pressure at a certain timing among the timings by calculation. The sound diagnosis system 100 also calculates the total sound pressures of timings before and after the certain timing. The sound diagnosis system 100 then compares the difference between the total sound pressure at the certain timing and the total sound pressures of the timings before and after it. The sound diagnosis system 100 determines whether the certain timing is a peak timing based on the difference. In other words, the sound diagnosis system 100 extracts a certain timing as a peak timing based on the fact that the difference is equal to or greater than a predetermined threshold (above a certain level).

[0088] Fig. 8 is a diagram conceptually showing a spectrogram. With reference to Fig. 8, the reason for calculating the relative sound pressure (relative sound intensity) as in Fig. 7 in order to detect peak timing will be described.

[0089] For example, suppose that a certain part of the device 130 has broken down and is making an abnormal sound. In this case, the broken part periodically makes a momentary abnormal sound. On the other hand, the broken part also makes a normal driving sound. Therefore, the broken part may make only an abnormal sound at one time and make both an abnormal sound and a normal sound at another time.

[0090] The faulty part emits only an abnormal sound 810 (sound pressure 8) at a certain timing 801. On the other hand, the faulty part emits an abnormal sound 810 (sound pressure 8) and a normal sound 820 (sound pressure 9) at another timing 802. In such a case, the server 110 calculates the total sound pressure at each of the timings 801 and 802 as described with reference to Figs. 5 and 6. In this case, the total sound pressure at timing 801 is "8". Furthermore, the total sound pressure at timing 802 is "17 (8+9)".

[0091] An abnormal sound occurs at both times 801 and 802. Therefore, it is desirable for the server 110 to classify the sounds at times 801 and 802 into the same cluster in the clustering process described below. However, the total sound pressure at times 801 and 802 is significantly different from each other. Therefore, there is a possibility that the server 110 cannot classify the sounds at times 801 and 802 into the same cluster.

[0092] Therefore, the server 110 utilizes the characteristic that parts emit normal sounds for a certain period of time. For example, a broken part emits abnormal sounds and normal sounds at a certain timing. In this case, it is highly likely that the broken part emits only normal sounds before and after that. In the example of FIG. 8, normal sounds 820 are generated continuously. The server 110 compares the total sound pressure at the timing when the abnormal sounds and normal sounds are generated with the sound pressure before and after that (total sound pressure at the timing when only normal sounds are generated). Then, the server 110 can calculate the difference between the total sound pressure at a certain timing and the sound pressure before and after that, that is, the relative sound pressure. In the example of FIG. 8, the relative sound pressure "8" is calculated at the timing 802. In this way, the server 110 can extract only the abnormal sounds from the combination of abnormal sounds and normal sounds. In other words, the server 110 can extract only the abnormal sound components by comparing the total sound pressure at a certain timing with the total sound pressure before and after that timing.

[0093] 9 is a diagram showing an example of the prominence of the peak of the total sound pressure. The server 110 may detect the peak timing of the total sound pressure based on the prominence value (prominence) of the peak. Prominence is a method of calculating the prominence of each peak based on the relative position of a peak to other peaks.

[0094] A specific method for calculating prominence is as follows. First, a certain peak (peak timing) is focused on. Next, the horizontal line of the peak is extended left and right until it intersects with a higher peak or reaches the left and right ends of the signal (graph ends). Next, a minimum value signal is found on the right and left sides of the section to which the horizontal line is extended. Next, the larger of the minimum value of the right section and the minimum value of the left section is set as the reference value. Next, a peak greater than the reference level is set as the prominence value.

[0095] Calculations using prominence measure the degree of prominence from the surroundings (the height of the peak relative to the surroundings) even if the peak timing has a larger value that includes environmental sounds. Therefore, calculations using prominence can detect peaks using values ​​similar to peak timings that do not include environmental sounds. Therefore, in the clustering described below, peaks with the same prominence value are more likely to be classified into the same category.

[0096] According to an embodiment, the server 110 may not perform peak detection for a certain period of time after detecting a peak in either of the procedures of Fig. 6 and Fig. 7. This allows the server 110 to handle cases where peaks occur within a very short time due to noise.

[0097] 10 is a diagram showing an example of a procedure of clustering (classification) by the peak classification unit 208. The peak classification unit 208 classifies the extracted peaks into a predetermined number of clusters (groups). For example, if the predetermined number of clusters is four, the peak classification unit 208 classifies the extracted peaks into four clusters. The peak classification unit 208 uses the frequency and peak prominence value of the peak timing as feature quantities when classifying each peak.

[0098] It has been experimentally proven that the peak prominence value of a target part is not easily affected by the surrounding environment. Therefore, classification using both frequency and peak prominence value as features is more accurate than classification using only frequency as a feature. In classification using both frequency and peak prominence value as features, peak points with close peak prominence values ​​are likely to be classified into the same cluster even if they have different frequencies.

[0099] The peak classification unit 208 may perform classification processing using machine learning. In this case, the peak classification unit 208 assigns each of the multiple peak timings to one of the clusters by clustering using machine learning. In the example of Fig. 10, the peak classification unit 208 uses k-means clustering of unsupervised learning. According to an embodiment, the peak classification unit 208 may use a non-hierarchical method other than k-means, or a hierarchical method.

[0100] The k-means clustering will be described with reference to FIG. 10 as an example. In practice, the peak classification unit 208 performs multi-dimensional clustering combining frequency and peak prominence values. For simplicity, two-dimensional k-means clustering will be described here. In the following example, the peak classification unit 208 classifies eight data into two clusters.

[0101] In step S1010, clustering data is provided to the peak classification unit 208. Eight pieces of data (peak timings) are provided to the peak classification unit 208.

[0102] In step S1020, the peak classification unit 208 randomly clusters the given data. The peak classification unit 208 classifies four pieces of data into cluster 0, and the remaining four pieces of data into cluster 1. Next, the peak classification unit 208 calculates the center of gravity of each of clusters 0 and 1 from the points of each cluster.

[0103] In step S1030, the peak classification unit 208 re-clusters each data. More specifically, the peak classification unit 208 calculates the distance between each centroid and each point obtained in step S1020. Next, the peak classification unit 208 reclassifies each point into the cluster of the centroid to which the distance is closest.

[0104] In step S1040, the peak classification unit 208 calculates the center of gravity of the cluster again from each point. If the peak classification unit 208 determines that the position of the center of gravity has converged, the process ends. If not, the peak classification unit 208 redoes the clustering of each point until the position of the center of gravity has converged.

[0105] According to one embodiment, the peak classifier 208 may accept the number of clusters (number of groups) from a user. In this case, the peak classifier 208 classifies each point into the number of clusters specified by the user. According to another embodiment, the peak classifier 208 may automatically adjust the number of clusters using a sum of squared error (SSE) index within a cluster using the elbow method.

[0106] Furthermore, according to another embodiment, the peak classification unit 208 may use, as the clustering method, a classification method using non-parametric estimation such as kmeans++ which is an application of kmeans, or kernel density estimation.

[0107] According to another embodiment, the peak classification unit 208 may use only the frequency at each peak timing as the feature amount. Alternatively, the peak classification unit 208 may use only the relative height (relative sound pressure) or prominence of the peak as the feature amount.

[0108] According to another embodiment, the peak classification unit 208 may reclassify each point using prominence as a post-processing step after clustering is performed. For example, the peak classification unit 208 may reclassify points that are classified into different clusters but have similar prominence values ​​into the same cluster.

[0109] FIG. 11 is a diagram showing an example of a clustering result of peak timings. In the example of FIG. 11, the peak classification unit 208 determines the optimal number of clusters by the elbow method, and clusters each peak timing by kmeans. Referring to FIG. 11, it can be seen that each peak timing is classified into four clusters 0 to 3. In the explanation so far, it has been explained that each peak timing is classified into each cluster. This can also be said that the sound that causes each peak timing is classified into each cluster. Therefore, hereafter, in order to indicate the classified data, it may be called "peak timing for each cluster", "peak for each cluster", "sound for each cluster", etc.

[0110] Next, a description will be given of the processing performed by the period calculation section 210. The period calculation section 210 calculates the interval between peaks for each cluster of classified peak timings.

[0111] The period calculation unit 210 calculates the variation in the interval between peak timings for each cluster. The variation in the interval between peak timings being equal to or greater than a certain level means that the peak timings are not occurring at equal intervals. A sound classified into a cluster in which the variation in the interval between peak timings is equal to or greater than a certain level is highly likely not a sound generated by a rotational driving part. Therefore, the period calculation unit 210 may exclude a cluster in which the variation in the interval between peak timings is equal to or greater than a certain level from the extraction targets. Alternatively, the period calculation unit 210 may identify a sound classified into a cluster in which the variation in the interval between peak timings is equal to or greater than a certain level as a non-periodic abnormal sound. A non-periodic abnormal sound may be generated by a driving part such as a clutch or a solenoid.

[0112] The sound classified into the cluster in which the variation in the interval between the peak timings is less than a certain level may be generated by a rotationally driven part such as a motor or a part driven by a motor. Therefore, the period calculation unit 210 identifies the sound classified into the cluster in which the variation in the interval between the peak timings is less than a certain level as a periodic abnormal sound. The period calculation unit 210 may calculate the occurrence period of the periodic abnormal sound classified into each cluster, for example, by obtaining the most frequent value of the interval between the peak timings. As another example, the period calculation unit 210 may calculate the occurrence period of the periodic abnormal sound classified into each cluster based on the average value or median value of the interval between the peak timings. That is, the period calculation unit 210 calculates the occurrence period of the periodic abnormal sound for each cluster based on the interval between adjacent peak timings in the same cluster. The occurrence period of the periodic abnormal sound for each cluster indicates the occurrence period of each peak timing included in the cluster.

[0113] An actual cluster may include peak timings caused by noise or acoustic noise. Therefore, the period calculation unit 210 may assume in advance the distribution of the intervals between peak timings when calculating the period (statistical value) of each cluster. In this case, the period calculation unit 210 may regard a value that is a certain distance away from the interval between peak timings assumed in advance as an outlier, and may exclude the outlier from the period calculation. In other words, the period calculation unit 210 may calculate the period excluding peak timings that are estimated to be erroneous detections or misclassifications.

[0114] The first selection unit 222 compares the period of the periodic abnormal sound for each cluster with the period for each part in the periodic table 214. The first selection unit 222 searches for parts that emit sounds with periods close to the period of the periodic abnormal sound for each cluster. For example, if the period of the periodic abnormal sound of cluster "0" is close to the period of the sound of part X, the first selection unit 222 selects part X as a candidate for the source of the periodic abnormal sound of cluster "0". According to an embodiment, the first selection unit 222 may select multiple parts as candidates for the source of the periodic abnormal sound of a certain cluster. For example, if the period of the periodic abnormal sound of cluster "0" is close to the periods of the sounds of parts X and Y, the first selection unit 222 may select parts X and Y as candidates for the source of the periodic abnormal sound of cluster "0".

[0115] As an example, the first selection unit 222 may determine a part p that is a candidate for the source of the periodic abnormal noise by the following formulas 101 to 104. According to an embodiment, the first selection unit 222 may calculate the following formula 101 by excluding clusters having periods larger or smaller than a certain value. This is because there is a high possibility that sounds with periods larger than or smaller than a certain value are not sounds emitted by driving parts.

[0116]

number

[0117] 12 is a diagram showing an example of a first screen 1200 output by the output unit 224. According to one embodiment, the first screen 1200 is displayed on a display of the diagnostic device 120. According to another embodiment, the output unit 224 transmits data of the first screen 1200 to another device (such as a smartphone 226 or a PC 228). The other device displays the data of the first screen 1200 on a display. The first screen 1200 is generated by the UI program 218.

[0118] The first screen 1200 is a screen before sound diagnosis is performed. The first screen 1200 includes parameter settings required for analysis, buttons for recording and analysis execution, and the like. A user (such as a serviceman) can obtain the analysis results by selecting a sound file (collected sound information) on the first screen 1200 and pressing the analysis button. When the analysis button is pressed, the first screen 1200 transitions to a second screen 1300.

[0119] 13 is a diagram showing an example of a second screen 1300 output by the output unit 224. According to one embodiment, the second screen 1300 is displayed on a display of the diagnostic device 120. According to another embodiment, the output unit 224 transmits data of the second screen 1300 to another device (such as a smartphone 226 or a PC 228). The other device displays the data of the second screen 1300 on a display. The second screen 1300 is generated by the UI program 218.

[0120] The second screen 1300 is a screen showing the analysis results of the sound file (collected sound information). The second screen 1300 displays the period of each cluster and the name of the part that emits a sound with a period close to the period of the periodic abnormal sound of each cluster. If there are multiple parts that emit sounds with periods close to the period of the periodic abnormal sound of each cluster, the second screen 1300 may display the names of multiple parts.

[0121] FIG. 14 is a diagram showing a continuation of the second screen 1300 output by the output unit 224. The diagnostic device 120 or other device may successively display the screen of FIG. 13 and the screen of FIG. 14 by accepting a scroll operation from the user. According to an embodiment, the screen of FIG. 13 and the screen of FIG. 14 may be displayed separately. In this case, the diagnostic device 120 or other device may repeatedly display the screen of FIG. 13 and the screen of FIG. 14 by accepting a screen or tab switching operation from the user. The second screen 1300 may include any information, such as a spectrogram, in addition to the cluster classification result and the names of parts that are candidates for the source of the abnormal noise.

[0122] <Example of noise removal and selection of one or more components by the first selection unit 222> Next, with reference to FIGS. 15 to 19, a method for removing noise from the sounds within each cluster and a selection procedure for one or more components by the first selection unit 222 will be described in detail.

[0123] FIG. 15 is a diagram showing an example of the periodic table 214. The periodic table 214 includes periodic information for each driving component included in the apparatus 130. According to an embodiment, the periodic table 214 may be represented as a table in a relational database. Also, the periodic table 214 may be represented in any other data format such as JSON (JavaScript (registered trademark) Object Notation). The first selection unit 222 collates the periodic table 214 with the calculation result of the occurrence period of the clustered peak timings as shown in FIGS. 17 and 19.

[0124] FIG. 16 is a diagram showing a first example of a set of clustered peak timings. A, B, C, D, E, and F in FIG. 16 are peak timings classified into the same cluster. The period calculation unit 210 calculates the periods between the peak timings A, B, C, D, E, and F. By this calculation, the calculation result 1700 shown in FIG. 17 is obtained.

[0125] FIG. 17 is a diagram showing an example of the calculation result 1700 of the occurrence period of the clustered peak timings. The period calculation unit 210 calculates the interval between adjacent peak timings. For example, the period calculation unit 210 calculates the interval "1.01 seconds" between the peak timings A and B. Next, the period calculation unit 210 calculates the average value "1.008 seconds" of the calculated intervals between adjacent peak timings.

[0126] The first selection unit 222 compares the average value of the interval between peak timings in the cluster, "1.008 seconds," with the sound periods of each part in the periodic table 214. It is unlikely that the average value of the interval between peak timings in the cluster and the sound periods of each part in the periodic table 214 will completely match. Therefore, according to an embodiment, the first selection unit 222 selects a part that emits a sound with a period closest to the average value of the interval between peak timings in the cluster. In the examples of FIGS. 12 and 17, the first selection unit 222 selects "fixing gear 1," which emits a sound with a period of "1.12," which is closest to the average value of "1.008 seconds."

[0127] According to another embodiment, the first selection unit 222 may select one or more parts that emit a sound with a period that is closest to the average value of the interval between peak timings in the cluster by a certain amount or more. In the examples of Fig. 12 and Fig. 17, the first selection unit 222 selects "fixing gear 1" that emits a sound with a period of "1.12" that is close to the average value of "1.008 seconds". The first selection unit 222 also selects "fixing belt" that emits a sound with a period of "1.02" that is close to the average value of "1.008 seconds".

[0128] FIG. 18 is a diagram showing a second example of a set of clustered peak timings. With reference to FIG. 18, a procedure for removing unrelated sounds (noise) when the cluster of sounds of a certain part contains the noise will be described. A, B, C, D, E, F, and G in FIG. 18 are peak timings classified into the same cluster. In the example of FIG. 18, D is a noise component. The period calculation unit 210 calculates the periods between the peak timings A, B, C, D, E, F, and G. This calculation results in a calculation result 1900 shown in FIG. 19.

[0129] FIG. 19 is a diagram showing an example of a calculation result 1900 of the occurrence period of clustered peak timings. The period calculation section 210 calculates the interval between adjacent peak timings. According to the calculation result 1900, the interval between the sections C to D and the interval between the sections D to E are clearly different from the intervals between the other sections. In this way, the period calculation section 210 can remove the section that is deviated from the value of the other sections by a certain amount or more as a noise component. Alternatively, the period calculation section 210 can remove the section that is deviated from the average value of the other sections by a certain amount or more as a noise component. In this case, the period calculation section 210 may ignore D and adopt the section from C to E as valid data. Alternatively, the period calculation section 210 may treat the section from C to E, which includes D, as invalid data.

[0130] The period calculation unit 210 may obtain the accuracy of the detection time from the periodic table 214. For example, according to the periodic table 214, the difference between the fixing belt and the fixing gear is about 10%. Therefore, the period calculation unit 210 sets 10% as the exclusion threshold. The period calculation unit 210 can remove, as noise, the peak timing included in a section that deviates from the average interval of each section by a threshold value (for example, 10%) or more.

[0131] According to an embodiment, the threshold value may be determined by a user or may be a fixed value. Moreover, periods greater than "1.12 seconds" in the periodic table 214 do not exist as periods of sounds emitted by parts. Periods smaller than "0.42 seconds" in the periodic table 214 do not exist as periods of sounds emitted by parts. Therefore, the period calculation unit 210 can remove such sounds (peak sounds) with periods that are not emitted by parts as noise. That is, when extracting multiple peak timings from the total sound pressure, the sound diagnosis system 100 can exclude sounds of frequencies that are generated from parts other than the monitored parts from the extraction target. In other words, the sound diagnosis system 100 can exclude sounds of frequencies that are not likely to be emitted by the monitored parts from the extraction target.

[0132] Next, the period calculation unit 210 calculates the average value of the intervals between adjacent peak timings calculated in a state where the noise component D is removed. Since the procedure for identifying the component that is the source of the abnormal noise is the same as the procedure described with reference to FIG. 17, the description of this procedure will not be repeated.

[0133] <Example of operation information 232> FIG. 20 is a diagram showing an example of the operation information 232. The operation information 232 is acquired from the device 130. The operation information 232 indicates the operation periods of the respective drive components included in the device 130.

[0134] According to an embodiment, the operation information 232 defines the operation periods of the respective drive components with reference to the start timing of a specific process by the device 130 (hereinafter referred to as the “reference timing”). Typically, each of the start timing and the end timing of the operation period is represented by the elapsed time from the reference timing. The device 130 includes a counter for measuring time. The device 130 resets the counter according to the start timing of the specific process. The device 130 records the measured values of the counter at each of the start timing and the end timing of the operation period of each drive component. The device 130 generates the operation information 232 according to the recording result.

[0135] The device 130 identifies the start timing and the end timing of the operation period based on the operation states of the respective drive components. The operation states of the respective drive components include an on state indicating that the component is operating and an off state indicating that the component is stopped. The start timing of the operation period corresponds to the start timing of the on state (in other words, the end timing of the off state). The end timing of the operation period corresponds to the start timing of the off state (in other words, the end timing of the on state).

[0136] A plurality of drive components start operating at various timings after a specific process is started. Each drive component may repeat an on state and an off state. For example, the operation information 232 shown in FIG. 20 shows three operation periods for the component "first motor". The first operation period is the period from the start timing "T1" to the end timing "T11". The second operation period is the period from the start timing "T21" to the end timing "T31". The third operation period starts from the start timing "T41". The operation information 232 shown in FIG. 20 does not show the end timing of the third operation period of the component "first motor". This means that the third on state of the component "first motor" is currently ongoing.

[0137] Similarly, the operation information 232 shows two operation periods for the component "second motor". The operation information 232 shows two operation periods for the component "third motor". The operation information 232 shows one operation period for the component "fourth motor". The operation information 232 shows three or more operation periods for the component "first clutch". The operation information 232 shows two operation periods for the component "second clutch". The operation information 232 shows one operation period for the component "first solenoid".

[0138] The operation information 232 does not show the operation period for the component "fifth motor". This means that the component "fifth motor" has not operated at all since the apparatus 130 started the specific process until now.

[0139] <Example of selection of one or more cause candidates by the second selection unit 223> The second selection unit 223 selects one or more cause candidates for the periodic abnormal noise from among the one or more components selected by the first selection unit 222. The second selection unit 223 selects one or more cause candidates based on the operation information 232 and the sound information. In other words, the second selection unit 223 narrows down the cause candidates for the periodic abnormal noise for each cluster.

[0140] The second selection unit 223 compares the peak timing for each cluster with the operation period of one or more parts selected by the first selection unit 222. The peak timing is the occurrence timing of periodic abnormal noise.

[0141] The second selection unit 223 aligns the time between the operation information 232 and the sound information based on the specific sound emitted when the device 130 starts a specific process. The second selection unit 223 defines the peak timing for each cluster as the elapsed time from the occurrence timing of the specific sound. The occurrence timing of the specific sound is detected by the detection unit 211. This allows the second selection unit 223 to determine whether the peak timing for each cluster belongs to the operation period of each part. In other words, the second selection unit 223 calculates the time difference between the occurrence timing of the specific sound and the peak timing for each cluster. The second selection unit 223 determines whether the peak timing for each cluster belongs to the operation period of each part based on the time difference.

[0142] According to one embodiment, the specific sound is a beep sound emitted in response to pressing a start button for starting a specific process. According to another embodiment, the multiple driving components included in the device 130 include multiple motors for performing the specific process. The specific sound is the driving sound of the motor that is driven first in the specific process among the multiple motors.

[0143] FIG. 21 is a diagram showing an example of peak timing for each cluster. FIG. 21 shows multiple peak timings for cluster "A." Each peak timing is expressed as the time elapsed from the occurrence timing of a specific sound. In the example shown in FIG. 21, the first peak timing is "t1." The second peak timing is "t2," which is the occurrence cycle Δta of the periodic abnormal sound that has elapsed from timing "t1." The third peak timing is "t3," which is the occurrence cycle Δta that has elapsed from timing t2.

[0144] According to an embodiment, the second selection unit 223 selects the first part as a candidate cause in response to at least one of the multiple peak timings of a certain cluster belonging to the operation period of the first part. In this manner, the second selection unit 223 selects one or more candidate causes of periodic abnormal noise classified into a certain cluster.

[0145] Furthermore, the second selection unit 223 may classify one or more candidate cause into a plurality of groups. According to an embodiment, the second selection unit 223 may classify one or more candidate cause into any of first to third groups. The first group is a group of candidate cause in which at least one of the plurality of peak timings does not belong to an operating period. In other words, the operating period of the candidate cause belonging to the first group includes some of the plurality of peak timings but does not include other parts.

[0146] The second group is a group of candidate cause candidates whose operating period includes a normal period in which no periodic abnormal noise occurs. The normal period in which no periodic abnormal noise occurs is a period in which no peak timing exists. The normal period is longer than the occurrence period of the periodic abnormal noise. In other words, the operating period of the candidate cause candidates belonging to the second group includes at least a portion of the multiple peak timings, but also includes a normal period in which no periodic abnormal noise occurs.

[0147] The third group is a group of cause candidates that are not included in the first group or the second group. In other words, the operating periods of the cause candidates that belong to the third group include all of the multiple peak timings, and do not include a normal period in which no periodic abnormal noise occurs.

[0148] According to another embodiment, the second selection unit 223 classifies one or more cause candidates into the above-mentioned first group and fourth group. The fourth group is a group of cause candidates not included in the first group.

[0149] FIG. 22 is a diagram showing a method of grouping one or more cause candidates. FIG. 22 shows five peak timings "t1", "t2", "t3", "t4", and "t5" of a certain cluster. The five peak timings occur at equal intervals. The cycle of the five peak timings is "Δta". The operation period of the first motor is from start timing "T1" to end timing "T11". The operation period of the second motor is from start timing "T2" to end timing "T12". The operation period of the third motor is from start timing "T3" to end timing "T13".

[0150] All five peak timings belong to the operation period of the first motor and the operation period of the second motor. Peak timings "t2", "t3", and "t4" belong to the operation period of the third motor. Therefore, the second selection unit 223 selects the first to third motors as candidate causes of the periodic abnormal noise.

[0151] The peak timings "t1" and "t5" do not belong to the operation period of the third motor, so the second selection unit 223 classifies the third motor into the first group.

[0152] The operation period of the first motor includes a normal period TA during which no periodic abnormal noise occurs. The normal period TA is longer than the occurrence period Δta of the periodic abnormal noise. Therefore, the second selection unit 223 classifies the first motor into the second group.

[0153] The operating period of the second motor does not include a normal period in which no periodic abnormal noise occurs. Also, as described above, all of the five peak timings belong to the operating period of the second motor. Therefore, the second selection unit 223 classifies the second motor into the third group.

[0154] According to another embodiment, the second selection unit 223 selects the first component as a cause candidate in response to all of the multiple peak timings of a certain cluster belonging to the operation period of the first component. In this case, the second selection unit 223 may classify one or more cause candidates into the above-mentioned second group and fifth group. The fifth group is a group of cause candidates not included in the second group.

[0155] As described above, when the interval between the peak timings classified into a certain cluster is non-periodic, the peak timings classified into the certain cluster correspond to a non-periodic abnormal sound. The non-periodic abnormal sound may be generated by a driving part such as a clutch or a solenoid. The second selection unit 223 may store in advance a group of parts that may generate a non-periodic abnormal sound. The second selection unit 223 may select one or more candidate causes of the non-periodic abnormal sound from the group of parts. The second selection unit 223 selects the second part as a candidate cause of the non-periodic abnormal sound in response to the peak timing belonging to an operating period of the second part of the group of parts. In this manner, the second selection unit 223 selects one or more candidate causes of the non-periodic abnormal sound classified into a certain cluster.

[0156] 23 is a diagram showing an example of a third screen 2300 output by the output unit 224. According to one embodiment, the third screen 2300 is displayed on a display of the diagnostic device 120. According to another embodiment, the output unit 224 transmits data of the third screen 2300 to another device (such as a smartphone 226 or a PC 228). The other device displays the data of the third screen 2300 on a display. The third screen 2300 is generated by the UI program 218.

[0157] The third screen 2300 includes a list of one or more candidate causes selected by the second selection unit 223. In the example shown in FIG. 23, the third screen 2300 displays the parts "XX roller" and "YY gear" as candidate causes of the periodic abnormal noise. This allows the user to identify the part that may be causing the periodic abnormal noise. As a result, the user can give priority to investigating the one or more displayed candidate cause parts.

[0158] The third screen 2300 may display the cause candidates classified into the first group among one or more cause candidates separately from the cause candidates classified into other groups. In the example shown in FIG. 23, the third screen 2300 shows the component "YY gear" as a cause candidate classified into the first group. The components of the cause candidates classified into the first group are the components that had stopped when a part of the periodic abnormal noise occurred. Therefore, the user can recognize the cause candidates classified into the first group as cause candidates with a relatively low possibility of generating the periodic abnormal noise.

[0159] Furthermore, the third screen 2300 may display the cause candidates classified into the second group among one or more cause candidates separately from the cause candidates classified into other groups. The components of the cause candidates classified into the second group are the components that had operated even during the normal period when the periodic abnormal noise did not occur. Therefore, the user can recognize the cause candidates classified into the second group as cause candidates with a relatively low possibility of generating the periodic abnormal noise.

[0160] <H. First Example of the Subroutine of Step S380> FIG. 24 is a diagram showing a first example of the processing procedure of the subroutine of step S380 shown in FIG. 3. The procedure shown in FIG. 24 is carried out for each cluster. Hereinafter, the cluster in which the procedure shown in FIG. 24 is carried out is referred to as the "target cluster".

[0161] In step S2400, the server 110 (acquisition unit 230) acquires the operation information 232 (see FIG. 20) from the apparatus 130.

[0162] In step S2401, the server 110 (detection unit 211) determines whether the peak timing of the target cluster is periodic. Typically, the server 110 determines whether the variation in the interval of the peak timing is equal to or greater than a certain value. The server 110 determines that the peak timing is periodic when the variation in the interval of the peak timing is less than a certain value.

[0163] If the peak timing is not periodic (NO in step S2410), the server 110 performs step S2402. In step S2402, the server 110 (detection unit 211) determines that the peak timing of the target cluster corresponds to a non-periodic abnormal sound. The server 110 (second selection unit 223) selects one or more candidate causes of the non-periodic abnormal sound from among the group of parts that may emit the non-periodic abnormal sound. Specifically, the server 110 defines the peak timing of the target cluster using the elapsed time from the occurrence timing of the specific sound. In this way, the server 110 time-aligns the peak timing with the operation information 232. Then, the server 110 selects the second part as a candidate cause of the non-periodic abnormal sound in accordance with the fact that the peak timing belongs to the operation period of the second part of the group of parts. After step S2401, the process of step S380 for the target cluster ends.

[0164] If the peak timings are periodic (YES in step S2401), server 110 performs step S2403. In step S2403, server 110 (first selection unit 222) may calculate the periodicity of the peak timings from the occurrence timings of each peak timing. For example, server 110 may compare the sound data (sound pressure, period, etc.) for each part pre-stored in storage unit 212 with the analysis results of each cluster. As a result, server 110 may select one or more parts that may be the source of the periodic abnormal sound classified into the target cluster.

[0165] In step S2404, server 110 (second selection unit 223) sets all of variables N, NN, and n to "1." Variable N represents an ordinal number assigned to each of the one or more parts selected in step S2403. Variable NN represents an ordinal number of an operating period. Variable n represents an ordinal number of a peak timing.

[0166] In step S2405, the server 110 defines the peak timing of the target cluster using the elapsed time from the occurrence timing of the specific sound. In this way, the server 110 time-aligns the peak timing with the operation information 232. Then, the server 110 determines whether the n-th peak timing belongs to the NN-th operation period of the N-th part.

[0167] If the n-th peak timing belongs to the NN-th operating period of the N-th part (YES in step S2405), the server 110 performs step S2406. In step S2406, the server 110 selects the N-th part as a cause candidate.

[0168] In step S2407, the server 110 classifies the Nth part into one of two or more groups. After step S2407, the server 110 moves the process to step S2412.

[0169] If the n-th peak timing does not belong to the NN-th operating period of the N-th part (NO in step S2405), server 110 performs step S2408. In step S2408, server 110 determines whether or not there is a remaining operating period.

[0170] If there is a remaining operating period (YES in step S2408), the server 110 performs step S2409. In step S2409, the server 110 adds "1" to the variable NN. After step S2409, the server 110 returns the process to step S2405.

[0171] If there is no remaining operating period (NO in step S2408), the server 110 performs step S2410. In step S2410, the server 110 determines whether there is a remaining peak timing.

[0172] If there are remaining peak timings (YES in step S2410), the server 110 performs step S2411. In step S2411, the server 110 adds "1" to the variable n. After step S2411, the server 110 returns the process to step S2405.

[0173] If there are no remaining peak timings (NO in step S2410), the server 110 performs step S2412. In step S2412, the server 110 determines whether there are remaining parts.

[0174] If there are remaining parts (YES in step S2412), the server 110 performs step S2413. In step S2413, the server 110 adds "1" to the variable N. Further, the server 110 sets the variables NN and n to "1". After step S2413, the server 110 returns the process to step S2405.

[0175] If there are no remaining parts (NO in step S2412), the process of step S380 for the target cluster ends.

[0176] According to the procedure shown in FIG. 24, the server 110 selects the first part as a candidate cause in response to at least one of the plurality of peak timings of the target cluster belonging to the operation period of the first part.

[0177] <I. Second Example of the Subroutine of Step S380> FIG. 25 is a diagram showing a second example of the procedure of the process of the subroutine of step S380 shown in FIG. 3. The procedure shown in FIG. 25 is performed for each cluster. Hereinafter, the cluster for which the procedure shown in FIG. 25 is performed is referred to as the "target cluster".

[0178] The procedure shown in FIG. 25 is different from the procedure shown in FIG. 24 in the following points. · If YES in step S2405, the server 110 performs step S2410 instead of step S2406. · If NO in step S2410, the server 110 sequentially performs steps S2406 and S2407. · If NO in step S2408, the server 110 performs step S2412 instead of step S2410.

[0179] According to the procedure shown in FIG. 25, the server 110 selects the first component as a candidate cause in response to all of the plurality of peak timings of the target cluster belonging to the operation period of the first component.

[0180] <First Example of Subroutine of Step S2407> FIG. 26 is a diagram showing a first example of the processing procedure of the subroutine of step S2407 shown in FIG. 24. The procedure shown in FIG. 26 is performed for the component selected as a candidate cause in step S2407. Hereinafter, the component selected as a candidate cause in step S2407 is referred to as the "target component".

[0181] In step S2600, the server 110 (second selection unit 223) sets the variable n to "1".

[0182] In step S2601, the server 110 determines whether there is an operation period including the n-th peak timing.

[0183] If there is no operation period including the n-th peak timing (NO in step S2601), the server 110 performs step S2602. In step S2602, the server 110 classifies the target component into the first group. After step S2602, the server 110 ends step S2407.

[0184] If there is an operation period including the n-th peak timing (YES in step S2601), the server 110 performs step S2603. In step S2603, the server 110 determines whether there is any remaining peak timing.

[0185] If there is any remaining peak timing (YES in step S2603), the server 110 performs step S2604. In step S2604, the server 110 adds "1" to the variable n. After step S2604, the server 110 returns the process to step S2601.

[0186] If there is no remaining peak timing (NO in step S2603), the server 110 ends step S2408. In this case, the target part is classified into the fourth group.

[0187] <K. Second Example of the Subroutine of Step S2407> FIG. 27 is a diagram showing a second example of the processing procedure of the subroutine of step S2407 shown in FIGS. 24 and 25.

[0188] In step S2700, the server 110 (the second selection unit 223) sets the variable NN to "1".

[0189] In step S2701, the server 110 determines whether a normal period longer than the generation period of the periodic abnormal noise is included in the NN-th operation period. The normal period is a period in which there is no peak timing.

[0190] If the normal period is included in the NN-th operation period (YES in step S2701), the server 110 performs step S2702. In step S2702, the server 110 classifies the target part into the second group. After step S2702, the server 110 ends step S2407.

[0191] If the normal period is not included in the NN-th operation period (NO in step S2701), the server 110 performs step S2703. In step S2703, the server 110 determines whether there is a remaining operation period.

[0192] If there is a remaining operation period (YES in step S2703), the server 110 performs step S2704. In step S2704, the server 110 adds "1" to the variable NN. After step S2704, the server 110 returns the process to step S2701.

[0193] If there is no remaining operation period (NO in step S2703), the server 110 ends step S2408. In this case, the target part is classified into the fifth group.

[0194] <L. Third Example of the Subroutine of Step S2407> FIG. 28 is a diagram showing a third example of the processing procedure of the subroutine of step S2407 shown in FIGS. 24 and 25. The third example corresponds to a combination of the first example shown in FIG. 26 and the second example shown in FIG. 27.

[0195] The server 110 performs steps S2600 to S2604. If NO in step S2603, or after step S2602, the server 110 performs steps S2701 to S2704.

[0196] Thus, if at least one peak timing does not belong to the operation period, the target part is classified into the first group. If the normal period is included in any of the operation periods, the target part is classified into the second group. Parts that do not belong to the first group or the second group are classified into the third group.

[0197] <Hardware Configuration> FIG. 29 is a block diagram showing the hardware configuration of the computer 2900 that functions as the server 110 or the diagnostic device 120.

[0198] The computer 2900 includes, as its main components, a processor 1, a RAM (Random Access Memory) 2, a storage device 3, a communication interface 4, an input device 5, and a display 6. The components are mutually connected by a data bus.

[0199] The processor 1 includes, for example, a CPU (Central Processing Unit) and an MPU (Micro Processing Unit), and executes a program 7 stored in a storage device 3 .

[0200] The RAM 2 volatilely stores data generated by the execution of the program 7 by the processor 1, data input via the input device 5, data received from the outside via the communication interface 4, and the like.

[0201] The storage device 3 stores data in a non-volatile manner and is realized by a hard disk device, a solid state drive (SSD) device, or the like.

[0202] The input device 5 includes, for example, a touch panel, a keyboard, a mouse, etc. The display 6 is, for example, a liquid crystal display or an organic EL (Electro Luminescence) display.

[0203] The processing in the computer 2900 is realized by cooperation of each piece of hardware and a program 7 executed by the processor 1. The program 7 is stored in the storage device 3 in advance. Alternatively, the program 7 may be stored in a CD-ROM (Compact Disc-Read Only Memory) or other recording medium and distributed. Alternatively, the program 7 may be provided as a downloadable application program by an information provider connected to the Internet. Such a program 7 is read from the recording medium by an optical disk drive (not shown) or other reading device, or downloaded via the communication interface 4, and then temporarily stored in the storage device 3.

[0204] Each component constituting the computer 2900 shown in Fig. 29 is a general one. Therefore, one of the essential parts of the technical idea according to the present disclosure can be said to be the program 7 stored in the RAM 2, the storage device 3, a CD-ROM or other recording medium, or the program 7 downloadable via a network. The recording medium may include a non-transitory (non-volatile) computer-readable data recording medium. Note that the operation of each piece of hardware of the computer 2900 is well known, and therefore detailed description will not be repeated.

[0205] The recording medium is not limited to a CD-ROM, a FD (Flexible Disk), or a hard disk, but may also be a medium that carries a program in a fixed manner, such as a magnetic tape, a cassette tape, an optical disk (MO (Magnetic Optical Disc) / MD (Mini Disc) / DVD (Digital Versatile Disc)), an IC (Integrated Circuit) card (including a memory card), an optical card, a mask ROM, an EPROM (Electronically Programmable Read-Only Memory), an EEPROM (Electronically Erasable Programmable Read-Only Memory), a flash ROM, or other semiconductor memory.

[0206] The program 7 includes not only a program that can be directly executed by the processor 1, but also a source program format program, a compressed program, an encrypted program, and the like.

[0207] <M.バリエーション> In the present specification, the diagnostic device 120 is described as having the microphone 202, but the application of the technology of the present disclosure is not limited to this. According to an embodiment, the device 130 to be diagnosed may have a microphone. In this case, the device 130 transmits sound information to the diagnostic device 120 or the server 110.

[0208] The sound information may be analog sound data. In this case, the detection unit 211 may have an encoding function for the analog sound data. The sound information may be digital sound data. As an example, the collection unit 203 may encode the sound acquired by the microphone 202. The sound information including the multiple frequencies emitted by the device 130 may include a sampling result (sampling information) of the recorded sound. The predetermined period for extracting information regarding the multiple sound pressures may be a period of any length that is preset in the sound diagnosis system 100. For example, the predetermined period may include a period of any length, such as several milliseconds, several seconds, or several minutes. The information regarding the sound pressure may include information such as the total sound pressure of the sound of each frequency, the relative sound pressure of the sound of each frequency, and the peak timing included in the total sound pressure.

[0209] In one aspect, the detection unit 211 may analyze the periodicity of the occurrence of the sound information based on information on a plurality of sound pressures at each timing. As an example, the detection unit 211 may analyze the periodicity of the occurrence of the peak timings included in the sound information. The detection unit 211 may also perform clustering in advance on the peak timings included in the sound information.

[0210] Furthermore, the first selection unit 222 may estimate candidates for components that are sources of sound information, based on information about a plurality of sound pressures at each timing detected by the detection unit 211. As an example, the first selection unit 222 may identify a component that generates a sound with an onset cycle similar to that of certain sound information. The onset cycle of the certain sound information here may be the onset cycle of a peak timing.

[0211] In one aspect, calculating the total sound pressure of the sounds of each frequency included in each timing of the sound information may include calculating the sum or average value of the sound pressure of each frequency at each timing. In another aspect, the total sound pressure calculation unit 205 may adjust the added value of the sounds of each frequency according to the occurrence frequency of the sounds of each frequency.

[0212] In one aspect, extracting a plurality of peak timings from the total sound pressure may include excluding sounds of frequencies generated from components other than the monitored components from the extracted timings. In another aspect, extracting a plurality of peak timings from the total sound pressure may include extracting each peak timing based on a difference between a total sound pressure calculated by calculation at a certain timing and a total sound pressure calculated by calculations before and after the certain timing.

[0213] In one aspect, peak detection section 206 may determine a certain timing as a peak timing when the difference is equal to or greater than a predetermined threshold value. In another aspect, peak detection section 206 may stop detecting peak timings for a certain period after detecting the peak timing.

[0214] In one aspect, the peak classification unit 208 may assign each of the multiple peak timings to one of the clusters by clustering using machine learning. In another aspect, the peak classification unit 208 may automatically determine the number of clusters.

[0215] In a certain aspect, the period calculation unit 210 may calculate the period for each cluster based on the interval between adjacent peak timings within the same cluster. In another aspect, the period calculation unit 210 calculates the period excluding peak timings estimated to be false detections or misclassifications.

[0216] In a certain aspect, the apparatus 130 may be an image forming apparatus including a plurality of drive sources. In this case, the plurality of drive sources may drive any one of a fixing belt, a paper discharge roller, a paper feed roller, and a fixing gear.

[0217] In this specification, the timing may include not only an instantaneous point in time series but also an arbitrary length of time (period). As an example, the auscultation diagnosis system 100 may analyze sound information segmented every 1 μs. In this case, a certain timing means a period of 1 μs. Thus, the timing may be an arbitrary period such as 1 μs or 1 ms. The sound pressure within a certain period (for example, a period of 1 μs) may be, for example, the average value or the median value of the sound pressure within that period.

[0218] As described above, the auscultation diagnosis system 100 according to the technology of the present disclosure acquires sound information including a plurality of frequencies emitted by the apparatus 130. Further, the auscultation diagnosis system 100 extracts information regarding a plurality of sound pressures corresponding to a plurality of frequencies included in each specific timing within a predetermined period from the acquired sound information. Furthermore, the auscultation diagnosis system 100 analyzes the information regarding the plurality of sound pressures at each extracted timing and outputs the analysis result. By doing so, even when the sound information obtained from the apparatus 130 to be diagnosed includes a plurality of frequencies, the auscultation diagnosis system 100 can appropriately analyze the sound information.

[0219] <N. Postscript> As described above, the present embodiment includes the following disclosure.

[0220] [Configuration 1] A collection unit that collects sound information regarding the sound emitted by the apparatus, A detection unit that detects a periodic first abnormal sound from the sound information, a first selection unit that selects, from among a plurality of components included in the device, one or more components that correspond to the occurrence period of the first abnormal noise; an acquisition unit that acquires operation information indicating an operation period of each of the plurality of components; a second selection unit that selects one or more candidate causes of the first abnormal sound from among the one or more components based on the operation information and the sound information.

[0221] [Configuration 2] the second selection unit selects the first part as one of the one or more cause candidate parts in response to at least one of a plurality of occurrence timings of the first abnormal sound belonging to the operation period of a first part of the one or more parts.

[0222] [Configuration 3] 3. The sound diagnosis system according to configuration 2, wherein the second selection unit classifies, from among the one or more candidate causes, a candidate cause in which at least one of the plurality of occurrence timings of the first abnormal sound does not belong to the operating period, into a first group.

[0223] [Configuration 4] the second selection unit selects the first part as one of the one or more cause candidates in response to all of a plurality of occurrence timings of the first abnormal sound belonging to the operation period of a first part of the one or more parts.

[0224] [Configuration 5] 5. The sound diagnosis system according to any one of configurations 2 to 4, wherein the second selection unit classifies, from the one or more candidate cause, a candidate cause in which a normal period in which the first abnormal sound does not occur is included in the operation period, into a second group, and the normal period is longer than an occurrence cycle of the first abnormal sound.

[0225] [Configuration 6] 6. The sound diagnosis system according to any one of configurations 1 to 5, further comprising an output unit that outputs a processing result by the second selection unit.

[0226] [Configuration 7] 7. The sound diagnosis system according to any one of configurations 1 to 6, wherein the acquisition unit acquires the motion information from the device.

[0227] [Configuration 8] the operation information defines the operation period based on a start timing of a specific process by the device; The device emits a specific sound when starting the specific process; The detection unit detects an occurrence timing of the specific sound and an occurrence timing of the first abnormal sound from the sound information, 6. The sound diagnosis system according to any one of configurations 2 to 5, wherein the second selection unit determines whether or not each of the plurality of occurrence timings of the first abnormal sound belongs to the operation period, based on a time difference between the occurrence timing of the specific sound and each of the plurality of occurrence timings of the first abnormal sound.

[0228] [Configuration 9] the specific process is started in response to pressing a button included in the device, 9. The sound diagnosis system according to claim 8, wherein the specific sound is emitted in response to pressing of the button.

[0229] [Configuration 10] the plurality of components include a plurality of motors for performing the specific process; 9. The sound diagnosis system according to claim 8, wherein the specific sound is a driving sound of a motor that is driven first in the specific process among the plurality of motors.

[0230] [Configuration 11] The detection unit detects a non-periodic second abnormal sound from the sound information, the plurality of components includes a group of components that may generate abnormal noise non-periodically, 11. The sound diagnosis system according to any one of configurations 1 to 10, wherein the second selection unit selects the second part as a candidate cause of the second abnormal sound in accordance with a timing of occurrence of the second abnormal sound falling within the operation period of a second part of the group of parts.

[0231] [Configuration 12] 12. The sound diagnostic system according to any one of configurations 1 to 11, wherein the device is an image forming device.

[0232] [Configuration 13] one or more processors collecting sound information relating to sounds emitted by the device; detecting a periodic first abnormal sound from the sound information by the one or more processors; the one or more processors selecting, from among a plurality of components included in the device, one or more components corresponding to an occurrence period of the first abnormal noise; obtaining, by the one or more processors, operational information indicative of an operational period of each of the plurality of components; and selecting, by the one or more processors, one or more candidate causes of the first abnormal sound from among the one or more components based on the operation information and the sound information.

[0233] [Configuration 14] Collecting sound information relating to sounds emitted by the device; Detecting a periodic first abnormal sound from the sound information; selecting, from among a plurality of parts included in the device, one or more parts corresponding to an occurrence period of the first abnormal noise; acquiring operation information indicating an operation period of each of the plurality of components; and selecting one or more candidate causes of the first abnormal noise from among the one or more components based on the operation information and the sound information.

[0234] The embodiments disclosed herein should be considered to be illustrative and not restrictive in all respects. The scope of the present disclosure is indicated by the claims, not by the above description, and is intended to include all modifications within the meaning and scope of the claims. In addition, the disclosure contents described in the embodiments and each modified example are intended to be implemented, as far as possible, either alone or in combination. [Explanation of symbols]

[0235] 1 processor, 2 RAM, 3 storage device, 4 communication interface, 5 input device, 6 display, 7 program, 100 sound diagnosis system, 110 server, 120 diagnosis device, 130 device, 202 microphone, 203 collection unit, 204 sound collection unit, 205 total sound pressure calculation unit, 206 peak detection unit, 208 peak classification unit, 210 period calculation unit, 211 detection unit, 212 storage unit, 214 periodic table, 216 model parameters, 218 UI program, 222 first selection unit, 223 second selection unit, 224 output unit, 226 smartphone, 230 acquisition unit, 232 operation information, 400 spectrogram, 510 conceptual diagram, 520 graph, 600 data, 700 peak value information, 810 abnormal sound, 820 normal sound, 1200 First screen, 1300 Second screen, 1700, 1900 Calculation results, 2300 Third screen, 2900 Computer.

Claims

1. A collection unit for collecting sound information relating to sounds emitted by the device; a detection unit that detects a periodic first abnormal sound from the sound information; a first selection unit that selects, from among a plurality of components included in the device, one or more components that correspond to a generation period of the first abnormal noise; an acquisition unit that acquires operation information indicating an operation period of each of the plurality of components; a second selection unit that selects one or more candidate causes of the first abnormal sound from among the one or more components based on the operation information and the sound information.

2. 2. The sound diagnosis system according to claim 1, wherein the second selection unit selects the first part as one of the one or more cause candidate points in response to at least one of a plurality of occurrence timings of the first abnormal sound belonging to the operation period of a first part of the one or more parts.

3. 3. The sound diagnosis system according to claim 2, wherein the second selection unit classifies, from among the one or more candidate causes, a candidate cause in which at least one of the plurality of occurrence timings of the first abnormal sound does not belong to the operating period, into a first group.

4. 2. The sound diagnosis system according to claim 1, wherein the second selection unit selects the first part as one of the one or more cause candidates in accordance with all of a plurality of occurrence timings of the first abnormal sound belonging to the operation period of a first part of the one or more parts.

5. 5. The sound diagnosis system according to claim 2, wherein the second selection unit classifies, from the one or more candidate cause candidates, candidate cause candidates in which a normal period in which the first abnormal sound does not occur is included in the operation period into a second group, and the normal period is longer than an occurrence cycle of the first abnormal sound.

6. The sound diagnosis system according to claim 1 , further comprising an output unit that outputs a result of the processing by the second selection unit.

7. The sound diagnosis system according to claim 1 , wherein the acquisition unit acquires the motion information from the device.

8. the operation information defines the operation period based on a start timing of a specific process by the device; The device emits a specific sound when starting the specific process; The detection unit detects occurrence timing of the specific sound and occurrence timing of the first abnormal sound from the sound information, 5. The sound diagnosis system according to claim 2, wherein the second selection unit determines whether or not each of the plurality of occurrence timings of the first abnormal sound belongs to the operation period, based on a time difference between the occurrence timing of the specific sound and each of the plurality of occurrence timings of the first abnormal sound.

9. the specific process is started in response to pressing a button included in the device, The sound diagnosis system according to claim 8 , wherein the specific sound is emitted in response to pressing of the button.

10. the plurality of components include a plurality of motors for performing the specific process; The sound diagnosis system according to claim 8 , wherein the specific sound is a driving sound of a motor that is driven first in the specific process among the plurality of motors.

11. The detection unit detects a non-periodic second abnormal sound from the sound information, the plurality of components includes a group of components that may generate abnormal noise non-periodically, 2. The sound diagnosis system according to claim 1, wherein the second selection unit selects the second part as a candidate cause of the second abnormal sound in accordance with a timing of occurrence of the second abnormal sound falling within the operation period of a second part of the group of parts.

12. The sound diagnostic system according to claim 1 , wherein the device is an image forming device.

13. one or more processors collecting sound information relating to sounds emitted by the device; detecting a periodic first abnormal sound from the sound information by the one or more processors; The one or more processors select, from among a plurality of parts included in the device, one or more parts corresponding to an occurrence period of the first abnormal noise; obtaining, by the one or more processors, operational information indicative of an operational period of each of the plurality of components; and selecting, by the one or more processors, one or more candidate causes of the first abnormal sound from among the one or more components based on the operation information and the sound information.

14. Collecting sound information relating to sounds emitted by the device; Detecting a periodic first abnormal sound from the sound information; selecting, from among a plurality of components included in the device, one or more components corresponding to an occurrence period of the first abnormal noise; acquiring operation information indicating an operation period of each of the plurality of components; and selecting one or more candidate causes of the first abnormal noise from among the one or more components based on the operation information and the sound information.

Images