Signal processing device, method, program and elevator monitoring device

Abstract

To enable improving accuracy to extract an operation sound of an observation object.SOLUTION: A signal processing device according to the present embodiment comprises: an acquisition unit; a calculation unit; and a determination unit. The acquisition unit acquires an environmental sound in an environment in which an operation sound of an observation object can be collected. The calculation unit calculates a chromatic characteristic and a variation of a frequency spectrum intensity which are associated with the environmental sound. The determination unit compares between a degree of the chromatic characteristic and the variation, and a threshold value, and determines whether the operation sound is included in the environmental sound.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] FIELD Embodiments of the present invention relate to a signal processing device, a method, a program, and an elevator monitoring device. [Background technology]

[0002] Various methods have been proposed for detecting abnormalities in equipment from environmental sounds, including the operating sounds of the equipment. However, it is difficult to extract the desired mechanical sound from environmental sounds acquired in an environment where similar mechanical sounds exist. One method extracts the desired mechanical sound from the periodicity or spectral stationarity of the operating sound of a motor or other device. However, this method is not applicable when detecting a non-periodic mechanical sound as the desired sound. Furthermore, methods that detect spectral stationarity can erroneously detect stationary noise or noise at specific frequencies. Additionally, there are methods for automatically extracting features using trained models such as deep neural networks, but the use of trained models requires a large amount of processing power and requires computational resources. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2012-168345 Summary of the Invention [Problem to be solved by the invention]

[0004] The present disclosure has been made to solve the above-mentioned problems, and aims to provide a signal processing device, method, program, and elevator monitoring device that can improve the accuracy of extracting the operating sound of an object to be observed. [Means for solving the problem]

[0005] The signal processing device according to this embodiment includes an acquisition unit, a calculation unit, and a determination unit. The acquisition unit acquires environmental sounds in an environment in which operation sounds of an observation target can be collected. The calculation unit calculates coloration and variability in frequency spectrum intensity of the environmental sounds. The determination unit compares the degree of coloration and variability with a threshold value to determine whether the environmental sounds include the operation sounds. [Brief explanation of the drawings]

[0006] [Figure 1] FIG. 1 is a block diagram showing a signal processing device according to a first embodiment. [Figure 2] FIG. 10 is a diagram showing an example of calculation of chromaticity and frequency spectrum intensity variability. [Figure 3A] FIG. 4 is a diagram showing a first example of a determination process performed by a determination unit. [Figure 3B] FIG. 4 is a diagram showing a first example of a determination process performed by a determination unit. [Figure 4A] FIG. 10 is a diagram showing a second example of the determination process of the determination unit. [Figure 4B] FIG. 10 is a diagram showing a second example of the determination process of the determination unit. [Figure 5] FIG. 4 is a block diagram showing a first modified example of the signal processing device according to the first embodiment. [Figure 6] 4 is a flowchart showing the operation of the signal processing device according to the first embodiment. [Figure 7] 10 is a flowchart showing another example of the process of determining operation sounds in the signal processing device. [Figure 8] FIG. 10 is a block diagram showing a second modified example of the signal processing device according to the first embodiment. [Figure 9] FIG. 10 is a block diagram showing a third modified example of the signal processing device according to the first embodiment. [Figure 10] FIG. 10 is a block diagram showing a fourth modified example of the signal processing device according to the first embodiment. [Figure 11] FIG. 10 is a block diagram showing a fifth modified example of the signal processing device according to the first embodiment. [Figure 12] FIG. 10 is a block diagram showing a signal processing system according to a second embodiment. [Figure 13]FIG. 10 is a conceptual diagram of an elevator monitoring device including a signal processing device according to a third embodiment. [Figure 14] FIG. 1 is a diagram showing an example of the hardware configuration of a signal processing device according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0007] The signal processing device, method, program, and elevator monitoring device according to the present embodiment will be described in detail below with reference to the drawings. In the following embodiments, parts with the same reference numerals perform similar operations, and redundant descriptions will be omitted as appropriate.

[0008] (First embodiment) A signal processing device according to the first embodiment will be described with reference to the block diagram of FIG. The signal processing device 10 according to the first embodiment includes an acquisition unit 101, a coloredness calculation unit 102, a frequency variation calculation unit 103, and a determination unit 104. The coloredness calculation unit 102 and the frequency variation calculation unit 103 are collectively referred to simply as a calculation unit.

[0009] The acquisition unit 101 acquires environmental sounds collected in an environment where operation sounds of an observation target can be collected. The acquisition unit 101 may acquire data of the environmental sounds from an external source, or may connect a microphone to the signal processing device 10 as the acquisition unit 101 and acquire the environmental sounds directly using the microphone.

[0010] The coloration calculation unit 102 receives the environmental sound from the acquisition unit 101 and calculates the coloration of the environmental sound. When a frequency spectrum of a sound is generated, coloration indicates a state in which the intensity of the spectrum is not constant (flat) but is biased, or the degree of bias. Here, coloration indicates the strength of the periodicity of signals with single and multiple frequencies. For example, coloration is observed when the spectral intensity of a specific frequency or its harmonic frequencies is strong, as in the case of human voice.

[0011] The frequency variation calculation unit 103 receives the environmental sound from the acquisition unit 101 and calculates the variation in the intensity of the frequency spectrum of the environmental sound (also called frequency spectrum intensity), that is, the variance of the frequency spectrum intensity in the frequency direction. Note that this is not limited to variance, and standard deviation may also be used. The frequency spectrum intensity is, for example, a power spectrum, an energy spectrum, or a power spectrum density. In order to reduce the amount of processing, the coloredness calculation unit 102 and the frequency variation calculation unit 103 may downsample the acquired environmental sound, for example, from 48 kHz to 8 kHz, before performing their respective calculation processes. Alternatively, the acquisition unit 101 may downsample the environmental sound, and then output the downsampled environmental sound to the coloredness calculation unit 102 and the frequency variation calculation unit 103 at the subsequent stages.

[0012] The determination unit 104 receives the coloredness from the coloredness calculation unit 102 and the degree of variation calculated from the frequency variation calculation unit 103. The determination unit 104 compares the coloredness and the degree of variation with a threshold, determines whether or not the environmental sound includes the operation sound of the observation target, and generates a determination result. Specifically, if the environmental sound is colored and the variation is small, the determination unit 104 determines that the environmental sound includes the operation sound of the observation target.

[0013] Next, an example of calculating the coloration and the variation in frequency spectrum intensity will be described with reference to FIG. Figure 2 shows an example of the frequency spectrum of sampling data related to environmental sounds, where the sampling data is arranged in chronological order. The frequency spectrum can be generated, for example, by performing FFT (Fast Fourier Transform) on the sampling data of the environmental sounds.

[0014] Coloredness is calculated as the strength of periodicity by calculating the kth-order autocorrelation coefficient (k is an integer equal to or greater than 1, resulting in a non-zeroth-order autocorrelation coefficient). The kth-order is the lag number L, which indicates the number of samples to be shifted when performing autocorrelation processing on environmental sound sampling data. In other words, it indicates which sampling data the autocorrelation coefficient is calculated between: if k=1, it is between adjacent sampling data; if k=2, it is between every other sampling data; and if k=3, it is between every third sampling data. In the example of Figure 2, k=3, i.e., the autocorrelation coefficient is calculated between every third sampling data. Visually, if the spectral intensity of the sampling data shown in Figure 2 has peaks at multiple frequencies when viewed on the frequency axis and is not flat, it is determined to be colored. Note that partial autocorrelation coefficients may be calculated instead of the kth-order autocorrelation coefficient. Furthermore, autocorrelation coefficients calculated over a predetermined time span (frames) may be shifted by a certain time span (shift length) and then calculated multiple times, and the fluctuations in the time direction may be used to evaluate coloredness. Although it depends on the object being observed, for human voices, a small k such as k=1 can result in significant coloration, so it is preferable to use a slightly larger value such as k=3.

[0015] On the other hand, the variation of the frequency spectrum intensity is calculated by calculating the variance of the frequency spectrum intensity. The variation can be visually determined from the distribution D of the maximum and minimum peaks of the frequency spectrum intensity of the sampling data shown in Figure 2. The variation of the frequency spectrum intensity may be calculated based on the center of gravity of the frequency spectrum intensity. Alternatively, the center of gravity of the spectrum intensity within a certain frequency range may be calculated. Calculating the center of gravity of the spectrum intensity makes it possible to express the bias in the distribution of the spectrum intensity. Alternatively, the variation or center of gravity of the frequency spectrum intensity may be calculated using a logarithmic power spectrum.

[0016] Next, a first example of the determination process of the determination unit 104 will be described with reference to FIGS. 3A and 3B. FIG. 3A shows the third-order autocorrelation coefficients of the sampled data for each frame, where one frame is the sampled data of the predetermined time width shown in FIG. 2. The vertical axis represents the autocorrelation coefficients, and the horizontal axis represents the frame index in the time direction. The determination unit 104 uses the autocorrelation coefficients as the degree of coloration. In FIG. 3A, most of the values ​​of the autocorrelation coefficients for each frame exceed the threshold value TH1.

[0017] 3B shows an example where the variation in frequency spectrum intensity of the same sampling data as in FIG. 3A is calculated using the variance of the logarithmic power spectrum converted to dB for each frame. The vertical axis represents the variance value, and the horizontal axis represents the frame index in the time direction. The determination unit 104 uses the variance of the logarithmic power spectrum as the degree of variation. In FIG. 3B, the variance value for each frame is equal to or less than the threshold value TH2.

[0018] 3A and 3B, the determination unit 104 counts the number of frames for which the autocorrelation coefficient is equal to or greater than a threshold TH1 and the variance value is equal to or less than a threshold TH2, and if the ratio of the number of such frames to all frames is equal to or greater than a predetermined value, it determines that the environmental sound contains the operation sound of the object to be observed. In other words, the determination unit 104 determines that the environmental sound contains the operation sound of the object to be observed if the environmental sound is colored and has small variations in frequency spectrum intensity. The reason for determining the percentage of all frames is that the analysis time span for determination is set to a time span longer than the frame, so results are obtained for multiple frames, and the results for these multiple frames are integrated to determine the final determination result.

[0019] Next, a second example of the determination process of the determination unit 104 will be described with reference to FIGS. 4A and 4B. 4A and 4B are diagrams showing the same autocorrelation coefficients and variances of frequency spectrum intensities as FIGS. 3A and 3B, respectively, except that the sampling data is different.

[0020] In Fig. 4A, most of the autocorrelation coefficients for each frame are less than the threshold value TH1. Therefore, the determining unit 104 can determine that the environmental sound of the sampling data shown in Fig. 4A is not colored.

[0021] On the other hand, in Fig. 4B, there are many values ​​in which the variance value for each frame is greater than the threshold value TH2. Therefore, the determining unit 104 can determine that the environmental sound of the sampling data shown in Fig. 4B has large frequency variations.

[0022] 4A and 4B, the determination unit 104 can determine that the environmental sound does not include the operation sound of the observation target because the environmental sound is not colored and has large frequency variation. Note that the determination unit 104 may determine that the environmental sound does not include the operation sound of the observation target if at least one of the conditions of having color and having small frequency variation is not met.

[0023] Next, a first modified example of the signal processing device 10 according to the first embodiment will be described with reference to the block diagram of FIG. The signal processing device 10 according to the first modification includes an acquisition unit 101, a coloration calculation unit 102, a frequency variation calculation unit 103, a judgment unit 104, a section extraction unit 105, a status monitoring unit 106, a storage unit 107, and an alarm transmission unit 108.

[0024] The section extraction unit 105 receives environmental sound from the outside and the determination result from the determination unit 104. The section extraction unit 105 extracts partial environmental sound, which is a section of the environmental sound that has been determined by the determination unit 104 to include operation sound of the observation target, from the environmental sound. Note that when the environmental sound is collected by microphones on multiple channels that are time-synchronized, the environmental sound collected by the microphone on one channel may be used for determination, and the environmental sound collected by the microphone on another channel may be used for section extraction and status monitoring.

[0025] The state monitoring unit 106 receives the partial environmental sound from the section extraction unit 105, monitors the state of the observation target based on the partial environmental sound, calculates the degree of abnormality, determines that the observation target is normal if the degree of abnormality is less than a predetermined threshold, or determines that the observation target is abnormal if the degree of abnormality is equal to or greater than the threshold, and generates status information including the determination result of whether the observation target is normal or abnormal and the time when the degree of abnormality was determined. For example, the state monitoring unit 106 determines the state of the observation target using a trained model. For example, the state monitoring unit 106 determines the state of the observation target using a deep neural network such as an autoencoder, as described in JP 2021-33842 A.

[0026] The storage unit 107 receives the partial environmental sound and status information (time, degree of abnormality, and result of determination of normality / abnormality) from the status monitoring unit 106 and stores them as log information. The alarm issuing unit 108 receives status information from the status monitoring unit 106 and issues an alert or the like to the outside.

[0027] Next, the operation of the first modified example of the signal processing device 10 shown in FIG. 5 will be described with reference to the flowchart of FIG.

[0028] In step SA1, the acquisition unit 101 acquires sampling data of the environmental sound. In step SA2, the coloredness calculation unit 102 calculates the autocorrelation coefficient of the environmental sound, for example, by the method described above with reference to FIG. In step SA3, the decision unit 104 compares the autocorrelation coefficient with a threshold value (TH1).

[0029] In step SA4, the frequency variation calculation unit 103 calculates the variation in frequency spectrum intensity of the environmental sound, for example, by the method described above with reference to FIG. In step SA5, the determination unit 104 compares the variation in frequency spectrum intensity with a threshold value (TH2). Note that the determination unit 104 may set two threshold values ​​and determine whether the value of the variation is within the range between the two threshold values. This makes it possible to determine whether an operation sound of a specific frequency is included in the environmental sound.

[0030] In step SA6, the determination unit 104 determines whether the operation sound determination conditions are met. Specifically, it determines whether the operation sound is colored and the variation in frequency spectrum intensity is small. If the operation sound is colored and the variation in frequency spectrum intensity is small, it is determined that the environmental sound includes the operation sound of the observation target, and the process proceeds to step SA7. On the other hand, if the operation sound is not colored or the variation in frequency spectrum intensity is large, it is determined that the environmental sound does not include the operation sound of the observation target, and the process returns to step SA1 and repeats the same process.

[0031] In step SA7, the section extracting unit 105 extracts the partial environmental sound. In step SA8, the state monitoring unit 106 monitors the observation target based on the partial environmental sound. In step SA9, the state monitoring unit 106 determines whether an abnormality has occurred in the observation target. For example, if the trained model used by the state monitoring unit 106 is an autoencoder, the state monitoring unit 106 inputs a partial environmental sound to the trained model, and if the difference between the output from the trained model and the input partial environmental sound is equal to or greater than a threshold, it can determine that the observation target is abnormal. If an abnormality has occurred in the observation target, the process proceeds to step SA11, and if no abnormality has occurred in the observation target, the process proceeds to step SA10.

[0032] In step SA10, the storage unit 107 associates the partial environmental sound with status information indicating whether an abnormality has occurred, and stores the result as log information. After that, the process returns to step SA1, and the same processing is repeated.

[0033] In step SA11, the alarm unit 108 issues an external alarm that an abnormality has occurred in the observation target. The alarm unit may, for example, output an alert sound, output information about the abnormality as a synthesized voice from a speaker, or display information about the abnormality on a monitoring display. In addition to the processing of step SA11, status information indicating the occurrence of the abnormality and the partial environmental sound may be stored in the storage unit 107 as log information.

[0034] The processing of steps SA2 to SA5 is not limited to the above-mentioned order, and steps SA2 and SA4 may be processed in parallel, or the processing of steps SA3 and SA5 may be executed in parallel by the determination unit 104.

[0035] Next, another example of the process of determining operation sounds in the signal processing device 10 will be described with reference to the flowchart of FIG. The calculation of coloredness in steps SA2 and SA3 is the same as in Fig. 6, and therefore will not be described here. Note that as a result of the comparison between the autocorrelation coefficient and the threshold value (TH1) in step SA3, for example, 1 may be output as the comparison result if the autocorrelation coefficient is equal to or greater than the threshold value TH1, and 0 may be output as the comparison result if the autocorrelation coefficient is less than the threshold value TH1.

[0036] In step SB1, the frequency variation calculation unit 103 calculates the variance of the frequency spectrum intensity of the environmental sound. In step SB2, the decision unit 104 compares the variance of the frequency spectrum intensity with a threshold (TH2). For example, if the variance is less than the threshold TH2, 1 is output as the comparison result, and if the variance is equal to or greater than the threshold TH2, 0 is output.

[0037] In step SB3, the frequency variation calculation unit 103 calculates the center of gravity of the frequency spectrum intensity of the environmental sound. In step SB4, the determination unit 104 compares the center of gravity of the frequency spectrum intensity with a threshold (for convenience of explanation, this is assumed to be TH3). For example, if the center of gravity is less than the threshold TH3, 1 is output as the comparison result, and if it is greater than or equal to the threshold TH3, 0 is output. In steps SB2 and S4, the comparison result may be whether or not the variance or center of gravity value of the frequency spectrum intensity is within a predetermined range (not less than the first threshold and not more than the second threshold).

[0038] In step SB5, the determination unit 104 performs a logical operation on the three comparison results (binary values ​​of 0 or 1) determined in steps SA3, SB2, and SB4, respectively. For example, the logical AND of the three comparison results is calculated as the logical operation result.

[0039] In step SB6, the determination unit 104 may compare the logical operation result with a threshold value to determine whether or not the environmental sound includes operation sounds. For example, if the logical operation result is greater than the threshold value, it can be said that each of the three comparison results satisfies the condition, and it can be determined that the environmental sound includes operation sounds.

[0040] While FIG. 7 illustrates an example in which three comparison results are used in the operation sound determination process, the present invention is not limited to this example and logical operations may be performed using four or more comparison results. For example, the variance of the power spectrum, the variance of the logarithmic power spectrum, the partial autocorrelation coefficient, etc. may be calculated and compared with a threshold value, and the comparison results may be added as elements of the above-mentioned logical operations. Also, in FIG. 7, the three feature amounts calculated in steps SA2, SB1, and SB3 are individually compared with threshold values ​​in steps SA3, SB2, and SB4, but this is not limited to this example. For example, the determination unit 104 may set a predetermined coefficient (weight) for each of the three feature amounts, calculate a linear combination (weighted sum) value, compare the linearly combined value with a predetermined threshold, and determine that the environmental sound includes operation sound if the linearly combined value is greater than the threshold value.

[0041] Next, a second modified example of the signal processing device 10 according to the first embodiment will be described with reference to the block diagram of FIG. The signal processing device 10 shown in FIG. 8 includes a degradation estimation unit 109 in addition to the configuration shown in FIG.

[0042] The deterioration estimation unit 109 acquires log information (time, abnormality level, and normal / abnormality determination result) from the storage unit 107 and estimates the deterioration state of the observed object based on the log information. For example, as described in JP 2021-135780 A, a deterioration curve is calculated from the abnormality level calculated using a deep neural network such as an autoencoder and the corresponding time, and the degree of deterioration is estimated from a sudden increase in the abnormality level or the difference with a deterioration curve based on a standard operating time known in advance, thereby estimating the remaining lifespan. For example, if the observed object is a motor, if the intensity of the high-frequency components of the motor's operating sound in the partial environmental sound where no abnormality is occurring gradually increases over time as the operation progresses, it can be estimated that the abnormality level is gradually increasing and that the motor is beginning to deteriorate. In this way, by referring to the partial environmental sound including the operating sound of the object being observed and the log information based on the status information, it is possible to estimate how the operating sound is changing, and therefore the deterioration of the object being observed can be estimated.

[0043] Next, a third modified example of the signal processing device 10 according to the first embodiment will be described with reference to the block diagram of FIG. The signal processing device 10 shown in FIG. 9 includes an SNR calculation unit 110 in addition to the configuration shown in FIG. The SNR calculation unit 110 receives environmental sounds from the outside and the determination result from the determination unit 104, and calculates the signal-to-noise ratio (SNR). This makes it possible to calculate the ratio of the operating sounds of the object being observed to the environmental sounds. For example, if the object being observed has malfunctioned and is making a loud abnormal noise, it can be determined that the operating sounds are louder than when the object being observed is not malfunctioning, and this can be used as an index for determining abnormalities, etc.

[0044] Next, a fourth modified example of the signal processing device 10 according to the first embodiment will be described with reference to the block diagram of FIG. The signal processing device 10 shown in FIG. 10 includes a suppression unit 111 in addition to the configuration shown in FIG. The suppression unit 111 receives environmental sounds from the outside and the determination result from the determination unit 104, suppresses environmental sounds other than the operation sounds of the observation target, and extracts the operation sounds of the observation target as desired sounds. Alternatively, the suppression unit 111 may suppress the desired sounds and extract background sounds by removing the desired sounds from the environmental sounds. For example, the suppression unit 111 performs a short-time Fourier transform on the environmental sound signal, calculates a suppression gain using the spectral subtraction method, Wiener filter method, maximum likelihood estimation method, or the like using the components of the operation sounds based on the determination result, multiplies the environmental sound signal by the suppression gain, and then performs an inverse Fourier transform to extract the desired sound. Although not shown, the state monitoring unit 106 shown in FIG. 5 may perform similar state monitoring based on the desired sound output from the suppression unit 111 instead of the partial environmental sound.

[0045] Next, a fifth modified example of the signal processing device 10 according to the first embodiment will be described with reference to the block diagram of FIG. The signal processing device 10 shown in FIG. 11 has the same configuration as that shown in FIG. 5, but differs in that the acquisition unit 101 further acquires state information of the observation target.

[0046] The state information of the observed object includes, for example, on / off information of the operation of the observed object, stage information indicating the intensity or level of the operation, and position information of the observed object. For example, if the observed object is a motor, the on / off information indicates whether the motor is running or stopped, and the stage information is information regarding the number of rotations or rotation frequency of the motor. The position information is information indicating the position of the observed object relative to the means for acquiring environmental sound.

[0047] By acquiring state information of the observation target in this way, the determination unit 104 may perform determination processing only on environmental sounds when the observation target is in operation. This improves the accuracy of the segment extraction processing by the section extraction unit 105.

[0048] Furthermore, the determination unit 104 can change the determination threshold based on the operation information of the observation target. For example, if the operation information indicates that the observation target is operating (on), a situation in which a particular frequency becomes high is conceivable, and therefore the determination unit 104 sets a higher threshold for determining whether the object is colored. On the other hand, if the operation information indicates that the observation target is stationary (off), the determination unit 104 sets a lower threshold for determining whether the object is colored.

[0049] According to the first embodiment described above, the coloration and frequency spectrum intensity variations of environmental sounds are calculated, and whether or not the environmental sounds contain the operation sounds of the observation target is determined by threshold judgment. As a result, even if it is desired to extract an operation sound with properties similar to ambient noise from environmental sounds as a desired sound, noise has a strong white characteristic, so by evaluating the coloration and frequency spectrum intensity variations, the accuracy of extracting the operation sounds of the observation target can be improved. This enables highly accurate judgment with a small amount of processing, and can be installed in small edge devices with limited computing resources.

[0050] (Second embodiment) In the second embodiment, it is assumed that a trained model is used for state monitoring, and the trained model is generated and updated by sending partial environmental sounds to a server.

[0051] A signal processing system according to the second embodiment will be described with reference to the block diagram of FIG. The signal processing system shown in Fig. 12 includes a signal processing device 10 and a server 20. The signal processing device 10 is similar to the signal processing device 10 shown in Fig. 5. The server 20 includes a signal accumulation unit 21, a model training unit 22, and a model storage unit 23.

[0052] The signal storage unit 21 receives the partial environmental sound from the section extraction unit 105 of the signal processing device 10 and stores the partial environmental sound.

[0053] The model training unit 22 receives the partial environmental sound from the signal accumulation unit 21, and trains a machine learning model with the task of anomaly detection using the partial environmental sound as input, thereby generating a trained model. The machine learning model performs unsupervised learning to distinguish between normal data and abnormal data, using a neural network such as an autoencoder or a variational autoencoder. Note that the training method for the machine learning model can be performed within a general machine learning framework with the task of anomaly detection, and therefore will not be described here. It should be noted that the present invention is not limited to a neural network, and a support vector machine, logistic regression, or the like may also be used.

[0054] The model storage unit 23 receives the trained model from the model training unit 22 and stores it. The state monitoring unit 106 of the signal processing device 10 receives the trained model from the model storage unit 23, and if it is the first time, it retains the generated trained model, and if it is the second or subsequent time that the trained model is acquired, it updates it to the newly acquired trained model.

[0055] According to the second embodiment described above, a machine learning model is trained in the server using partial environmental sounds including operation sounds of the observed object, and a trained model is generated. The state monitoring unit uses the generated or updated trained model to determine the state of the observed object, i.e., monitor the observed object. As a result, even if there is a change in the environmental sound, an appropriate trained model can be generated from the partial environmental sounds, and the monitoring accuracy of the observed object can be maintained and improved.

[0056] (Third embodiment) In the third embodiment, an example in which the signal processing device 10 is used for elevator monitoring will be shown as a specific example of use. An elevator monitoring device including a signal processing device 10 according to the third embodiment will be described with reference to the conceptual diagram of FIG. The elevator monitoring device 50 shown in FIG. 13 includes the signal processing device 10, an elevator control panel 30, and one or more elevator cars 40. The elevator control panel 30 includes a car management unit 31 . The car management unit 31 controls the floor movement of the elevator car 40, the opening and closing of the doors, the state of the ventilation fan, and the like.

[0057] The elevator car 40 is provided with a car control unit 41, a ventilation fan 42, and a sound collection unit 43. The car control unit 41 receives instructions from the elevator control panel 30 and controls the elevator car 40 in accordance with the instructions. Specifically, it controls the opening and closing of the doors of the elevator car 40, the direction of travel of the elevator car 40, the destination floor (hall) and speed, and the on / off of the ventilation fan.

[0058] A ventilation fan 42 is installed to exhaust the air inside the elevator car 40 to the outside. The sound collection unit 43 is, for example, a microphone installed in the elevator car 40, and collects environmental sounds around the elevator car 40. Examples of collected environmental sounds include the operating sounds of the elevator car 40 when it ascends or descends, the operating sounds of the ventilation fan 42, the operating sounds of the doors opening and closing, people talking inside the car, people talking at the landing side when the doors are open, music broadcast over the public address system, and other ambient noise from the surrounding environment, such as noise from construction work on nearby buildings, and the operating sounds of adjacent elevators. These include sounds caused by people getting on and off the elevator car 40, footsteps as people move inside the car, sounds caused by people carrying luggage such as shopping carts and getting on and off the car, and sounds caused by people or luggage colliding with the doors when they open or close. When using a single microphone, the sound collection unit 43 is preferably placed on the ceiling door side outside the elevator car 40. Sound collection unit 43 may be composed of multiple microphones, in which case multiple microphones are preferably placed on elevator car 40, one on the door side of the ceiling outside elevator car 40 and one on the door side near the ceiling inside the car. By placing them in this manner, it is possible to accurately collect the operating sounds of elevator car 40 to be observed (the sounds when the car goes up and down, and the sounds when the doors open and close when the car is stopped).

[0059] In this case, elevator monitoring device 50 sets elevator car 40 and ventilation fan 42 as objects to be observed. That is, in this case, for elevator maintenance, the object to be observed is the mechanism that moves the car up and down when the car is moving up and down, the mechanism that opens and closes the doors when the car is stopped, and the ventilation fan mechanism, which operates both when the car is moving up and down and when the car is stopped. Note that the observation objects are not limited to the operating sounds of elevator car 40 and ventilation fan 42, but may also include sound sources in the elevator operating environment, such as speakers that output public address systems inside the doors or elevator car 40.

[0060] The signal processing device 10 receives status information of the observation target from the elevator control panel 30 and environmental sounds acquired by the sound collection unit 43, and determines whether the environmental sounds include the operation sounds of the observation target, i.e., the operation sounds of the elevator car 40 and the ventilation fan 42. More specifically, the signal processing device 10 obtains information from the control unit 41 indicating whether the car 40 is ascending or descending or stopped, and determines whether the environmental sounds include the operation sounds of the car ascending or descending and the ventilation fan when the car is ascending or descending, and the operation sounds of the doors opening and closing and the ventilation fan when the car is stopped. As mentioned above, various types of environmental sounds include sounds other than the desired operation sounds (disturbances). The operation of the signal processing device 10 is similar to that of the above-described embodiment. The operation sounds of the car and the ventilation fan are characterized by high coloration and small frequency variation. In a closed space such as an elevator, sound reverberates, increasing diffusion, resulting in greater frequency variation than sounds of a specific frequency. Human voices have a moderate degree of coloration and a moderate degree of frequency variation; steady noise has a low degree of coloration and a high degree of frequency variation; noise at specific frequencies has a high degree of coloration and very low degree of frequency variation; noise like music has a low degree of coloration and a moderate degree of frequency variation; and noise caused by car vibration or attack noise has a high degree of coloration and a high degree of frequency variation. Using both coloration and frequency variation is effective in determining whether a sound includes the operating sound of the object of observation based on these sound characteristics. In other words, it is effective in distinguishing between sounds caused by vibration and mechanical sounds of the object of monitoring. The signal processing device 10 included in the elevator monitoring device 50 may be installed in the elevator car 40 or incorporated into the elevator control panel 30. Alternatively, the signal processing device 10 may be included in an external server if the environmental sounds acquired by the sound collection unit 43 can be transmitted wirelessly. As in the first embodiment, in order to reduce the amount of processing, the environmental sound acquired by the sound collection unit 43 may be downsampled, for example, from 48 kHz to 8 kHz, before the calculation process for coloredness and frequency variation is performed.

[0061] According to the third embodiment described above, it is determined whether the environmental sounds collected by the sound collection unit installed in the elevator car include the operation sounds of the elevator car and the ventilation fan, which are the targets of observation. This makes it possible to properly extract the operation sounds of the targets of observation, i.e., the operation sounds of the elevator car and the ventilation fan, from the environmental sounds, making it easy to determine whether there are any abnormalities in operation and achieving efficient elevator monitoring.

[0062] Next, an example of the hardware configuration of the signal processing device 10 according to the above embodiment is shown in the block diagram of FIG. The signal processing device 10 includes a CPU (Central Processing Unit) 141, a RAM (Random Access Memory) 142, a ROM (Read Only Memory) 143, a storage 144, a display device 145, an input device 146, and a communication device 147, each of which is connected by a bus.

[0063] The CPU 141 is a processor that executes arithmetic processing, control processing, etc. in accordance with a program. The CPU 141 uses a predetermined area of ​​the RAM 142 as a working area and executes the processing of each part of the signal processing device 10 described above in cooperation with programs stored in the ROM 143, the storage 144, etc.

[0064] The RAM 142 is a memory such as an SDRAM (Synchronous Dynamic Random Access Memory), and functions as a work area for the CPU 141. The ROM 143 is a memory that stores programs and various types of information in a non-rewritable manner.

[0065] The storage 144 is a device that writes and reads data to and from a magnetic recording medium such as a hard disk drive (HDD), a semiconductor storage medium such as a flash memory, a magnetically recordable storage medium such as a HDD, an optically recordable storage medium, etc. The storage 144 writes and reads data to and from the storage medium in accordance with control from the CPU 141.

[0066] The display device 145 is a display device such as an LCD (Liquid Crystal Display), etc. The display device 145 displays various information based on a display signal from the CPU 141.

[0067] The input device 146 is an input device such as a mouse, a keyboard, etc. The input device 146 receives information input by a user as an instruction signal, and outputs the instruction signal to the CPU 141.

[0068] The communication device 147 communicates with external devices via a network under the control of the CPU 141 .

[0069] The instructions shown in the processing procedures described in the above-described embodiments can be executed based on a software program. A general-purpose computer system can store this program in advance and, by loading this program, achieve effects similar to those achieved by the control operation of the signal processing device described above. The instructions described in the above-described embodiments can be recorded as a computer-executable program on a magnetic disk (such as a flexible disk or hard disk), an optical disk (such as a CD-ROM, CD-R, CD-RW, DVD-ROM, DVD±R, DVD±RW, or Blu-ray (registered trademark) Disc), a semiconductor memory, or a similar recording medium. The recording medium may take any storage format as long as it is readable by a computer or embedded system. A computer can load the program from the recording medium and execute the instructions described in the program on a CPU based on the program, thereby achieving operations similar to those of the signal processing device described in the above-described embodiments. Of course, the computer may acquire or load the program via a network. In addition, an OS (operating system), database management software, network middleware, etc. running on a computer may execute some of the processes required to realize this embodiment based on instructions from a program installed on the computer or embedded system from a recording medium. Furthermore, the recording medium in this embodiment is not limited to a medium independent of a computer or an embedded system, but also includes a recording medium that stores or temporarily stores a program downloaded via a LAN, the Internet, or the like. Furthermore, the number of recording media is not limited to one, and cases where the processing in this embodiment is executed from multiple media are also included in the recording media in this embodiment, and the media may have any configuration.

[0070] The computer or embedded system in this embodiment is for executing each process in this embodiment based on a program stored on a recording medium, and may be configured as either a device consisting of a single device such as a personal computer or a microcomputer, or a system in which multiple devices are connected to a network. Furthermore, the computer in this embodiment is not limited to a personal computer, but also includes an arithmetic processing unit, a microcomputer, etc. included in information processing equipment, and is a general term for equipment or devices that can realize the functions in this embodiment by a program.

[0071] Although several embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be embodied in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope and spirit of the invention, and are also included in the scope of the invention and its equivalents as defined in the claims. [Explanation of symbols]

[0072] 10 Signal processing device, 20 Server, 21 Signal accumulation unit, 22 Model training unit, 23 Model storage unit, 30 Elevator control panel, 31 Management unit, 41 Control unit, 42 Ventilation fan, 43 Sound collection unit, 50 Elevator monitoring device, 101 Acquisition unit, 102 Coloredness calculation unit, 103 Calculation unit, 104 Judgment unit, 105 Section extraction unit, 106 Status monitoring unit, 107 Storage unit, 108 Alarm generation unit, 109 Deterioration estimation unit, 110 SNR calculation unit, 111 Suppression unit, 144 Storage, 145 Display unit, 146 Input device, 147 Communication device

Claims

1. an acquisition unit that acquires environmental sounds in an environment where operation sounds of an observation target can be collected; a calculation unit that calculates the coloration and the variation in frequency spectrum intensity of the environmental sound; a determination unit that compares the degree of coloration and the degree of variation with a threshold value and determines whether the environmental sound includes the operation sound; A signal processing device comprising:

2. The signal processing device according to claim 1 , wherein the determination unit determines that the environmental sound includes the operation sound when the environmental sound has the colored property and the variation is small.

3. The signal processing device according to claim 2 , further comprising: a cutting unit that cuts out, from the environmental sound, a partial environmental sound that is determined by the determining unit to include the operation sound.

4. The signal processing device according to claim 3 , further comprising a monitoring unit that monitors a state of the observation target based on the partial environmental sound.

5. The signal processing device according to claim 4 , wherein the monitoring unit determines whether the observation target is normal or abnormal from the operation sound included in the partial environmental sound.

6. The signal processing device according to claim 1 , further comprising an SNR calculation unit that calculates a signal-to-noise ratio of the partial environmental sound that is determined by the determination unit to include the operation sound.

7. The signal processing device according to claim 1 , further comprising a suppression unit that generates a desired sound by suppressing the partial environmental sound determined by the determination unit to include the operation sound, except for the operation sound.

8. The signal processing device according to claim 7 , further comprising a monitoring unit that monitors a state of the observation target based on the desired sound.

9. The signal processing device according to claim 8 , wherein the monitoring unit determines whether the observation target is normal or abnormal based on the desired sound.

10. The signal processing device according to claim 1 , wherein the calculation unit calculates the coloredness by calculating an autocorrelation coefficient of the environmental sound.

11. The signal processing device according to claim 1 , wherein the calculation unit calculates the variation based on at least one of a variance and a center of gravity of the frequency spectrum intensity of the environmental sound.

12. The signal processing device according to claim 1 , wherein the determination unit determines whether the environmental sound includes the operation sound based on at least one of on / off information of the operation of the object of observation while the environmental sound is being acquired and position information of the object of observation.

13. Acquire environmental sounds in an environment where the operating sounds of the object to be observed can be collected, Calculating the coloration and the variability of the frequency spectrum intensity of the environmental sound; comparing the degree of coloration and the degree of variation with a threshold value to determine whether the environmental sound includes the operation sound; Signal processing methods.

14. Computer, an acquisition means for acquiring environmental sounds in an environment in which operation sounds of an observation target can be collected; a calculation means for calculating the coloration and the variation in frequency spectrum intensity of the environmental sound; a determination means for comparing the degree of coloration and the degree of variation with a threshold value and determining whether the environmental sound includes the operation sound; A signal processing program to function as a

15. one or more sound collection units that are installed in the elevator car and that acquire environmental sounds in an environment where the operation sounds of the object to be observed can be collected; a calculation unit that calculates the coloration and the variation in frequency spectrum intensity of the environmental sound; a determination unit that compares the degree of coloration and the degree of variation with a threshold value and determines whether the environmental sound includes the operation sound; An elevator monitoring device comprising:

Images