State detection method, state detection system, and program

Abstract

The present disclosure relates to a state detection method, a state detection system, and a program that make it possible to more reliably detect the state of a mobile object. In this state detection method, the state of a mobile object or an acceleration sensor is detected on the basis of a difference between an actually measured value of noise inputted to a microphone mounted in the mobile object and an estimated value of noise estimated on the basis of a sensor value obtained from an acceleration sensor mounted in the mobile object. The technology according to the present disclosure can be applied to, for example, a road noise canceling system.

Description

Condition detection method, condition detection system, and program

[0001] The present disclosure relates to a status detection method, a status detection system, and a program, and more particularly to a status detection method, a status detection system, and a program that enable more reliable detection of the status of a moving object.

[0002] When performing feedforward road noise canceling using an acceleration sensor, changes in the acceleration sensor or the environment inside and outside the vehicle can change the transmission system of vibrations transmitted from the noise source to the acceleration sensor and vibrations transmitted from the noise source to a listening position such as the ear. In this case, the noise cancellation effect can be reduced, and in some cases, the noise at the ear can be increased by the noise cancellation signal.

[0003] Unlike feedback-type road noise canceling, which does not have an error microphone at ear position, feedforward-type road noise canceling does not diverge due to unexpected sudden inputs when reproducing the noise cancellation signal. However, feedforward-type road noise canceling cannot detect changes in the transfer characteristics even if an abnormality occurs in the vibration transfer path from the noise source to the acceleration sensor or from the noise source to the ear position.

[0004] In response to this, there is a technology that estimates the vibration detected by an acceleration sensor based on test vibrations, and determines whether there is an abnormality in the acceleration sensor based on the difference between the estimated vibration value and the actual measured value actually detected by the acceleration sensor (see, for example, Patent Document 1).

[0005] JP 2009-107475 A

[0006] The technology disclosed in Patent Document 1 is intended to detect abnormalities in an acceleration sensor (vibration detection means) that detects vibrations in order to reduce noise, and cannot be used to detect abnormalities in the transmission path that occur in the environment in which the user is using the device.

[0007] The present disclosure has been made in consideration of such circumstances, and aims to enable more reliable detection of the state of a moving body.

[0008] The state detection method disclosed herein is a state detection method that detects the state of a moving body or an acceleration sensor based on the difference between an actual measured value of noise input to a microphone mounted on the moving body and an estimated value of the noise estimated based on a sensor value obtained from an acceleration sensor mounted on the moving body.

[0009] The state detection system disclosed herein is a state detection system that includes a microphone mounted on a moving body, an acceleration sensor mounted on the moving body, and a state detection unit that detects the state of the moving body or the acceleration sensor based on the difference between the actual measured value of noise input to the microphone and an estimated value of the noise estimated based on the sensor value obtained from the acceleration sensor.

[0010] The program disclosed herein is a program for causing a computer to execute a process for detecting the state of a moving body or an acceleration sensor based on the difference between an actual measured value of noise input to a microphone mounted on the moving body and an estimated value of the noise estimated based on a sensor value obtained from an acceleration sensor mounted on the moving body.

[0011] In the present disclosure, the state of the moving body or the acceleration sensor is detected based on the difference between the actual measured value of noise input to a microphone mounted on the moving body and the estimated value of the noise estimated based on the sensor value obtained from the acceleration sensor mounted on the moving body.

[0012] 1 is a diagram illustrating an example configuration of a state detection system to which the technology according to the present disclosure is applied. FIG. 2 is a flowchart illustrating the flow of state detection processing. FIG. 3 is a diagram illustrating an example configuration of an anomaly detection system when generating an estimation filter. FIG. 4 is a diagram illustrating an example configuration of an anomaly detection system when estimating noise. FIG. 5 is a flowchart illustrating the flow of an anomaly detection processing. FIG. 6 is a diagram illustrating an example waveform of an actual measured value and an estimated value of noise in a normal state. FIG. 7 is a diagram illustrating an example level difference between an actual measured value and an estimated value of noise. FIG. 8 is a diagram illustrating an example correlation value between an actual measured value and an estimated value of noise. FIG. 9 is a diagram illustrating another example configuration of an anomaly detection system when estimating noise. FIG. 10 is a diagram illustrating an example configuration of a driving environment detection system when generating an estimation filter. FIG. 11 is a diagram illustrating an example configuration of a driving environment detection system when estimating a driving environment. FIG. 12 is a flowchart illustrating the flow of driving environment detection processing. FIG. 13 is a diagram illustrating an example configuration of a state detection system using a trained model. FIG. 14 is a flowchart illustrating the flow of state detection processing. FIG. 15 is a block diagram illustrating an example configuration of computer hardware.

[0013] Modes for carrying out the present disclosure (hereinafter referred to as embodiments) will be described below in the following order.

[0014] 1. Problems with the prior art and an overview of the technology according to the present disclosure 2. A condition detection system to which the technology according to the present disclosure is applied and its operation 3. First embodiment (application to an anomaly detection system) 4. Second embodiment (application to a driving environment detection system) 5. Third embodiment (condition detection using a trained model) 6. Application example 7. Example of computer hardware configuration

[0015] 1. Problems with the Prior Art and Overview of the Technology Relating to the Present Disclosure> (Problems with the Prior Art) When performing feedforward road noise canceling using an acceleration sensor, changes in the acceleration sensor or the environment inside and outside the vehicle body can change the transmission system of vibrations transmitted from the noise source to the acceleration sensor and vibrations transmitted from the noise source to a listening position such as the ear. In this case, the noise cancellation effect can be reduced, and in some cases, the noise at the ear can be increased by the noise cancellation signal.

[0016] Unlike feedback-type road noise canceling, which does not have an error microphone at ear position, feedforward-type road noise canceling does not diverge due to unexpected sudden inputs when reproducing the noise cancellation signal. However, feedforward-type road noise canceling cannot detect changes in the transfer characteristics even if an abnormality occurs in the vibration transfer path from the noise source to the acceleration sensor or from the noise source to the ear position.

[0017] In response to this, there is a technology that estimates the vibration detected by an acceleration sensor based on test vibrations, and determines whether there is an abnormality in the acceleration sensor based on the difference between the estimated vibration value and the actual measured value actually detected by the acceleration sensor.

[0018] This technology is intended to detect abnormalities in acceleration sensors (vibration detection means) that detect vibrations in order to reduce noise, and cannot be used to detect abnormalities in the transmission path that occur in the environment in which the user is using the device.

[0019] Japanese Patent Laid-Open Publication No. 2021-86105 discloses an active noise reduction device that detects an abnormality in a reference signal source when the signal level of a reference signal output from the reference signal source, which is correlated with noise in the space inside the vehicle cabin, is below a predetermined threshold. However, since the range of sound pressure levels of driving noise is wide, it is difficult to set an appropriate threshold, and it is not possible to detect small changes such as those caused by changes in a specific band or aging.

[0020] International Publication No. 2017 / 135012 discloses an active noise and vibration control device that limits an increase in a cancellation output when it is determined that the autocorrelation value of a target signal, such as an error signal, a reference signal, or a base signal, is below an autocorrelation threshold. This limits the increase in the cancellation output (canceling sound or vibration) when the influence of a disturbance that has entered the error signal, the reference signal, or the base signal is significant, thereby preventing divergence of control and the occurrence of unintended cancellation outputs. However, while this method can respond to disturbances and the like, it cannot respond to changes in the transmission path, such as changes over time.

[0021] (Summary of Technology Relating to the Present Disclosure) In the technology relating to the present disclosure, the difference between the actual noise signal input to a microphone and the estimated noise signal estimated based on the acceleration value is used as a feature for detecting a change in the transmission path in an in-vehicle road noise canceling (RNC) due to an abnormality in the wheel (tire), distortion of the axle or vehicle body, etc. Specifically, the level difference and correlation value (coherence) between the actual noise signal and the estimated noise signal are used as the feature.

[0022] In the technology disclosed herein, by using the difference between the measured noise signal and the estimated signal rather than the level of the measured noise signal input to the microphone, it is possible to limit the estimated band, etc., enabling more accurate anomaly detection.

[0023] 2. Condition Detection System to which the Technology According to the Present Disclosure is Applied and Its Operation> FIG. 1 is a diagram illustrating an example of the configuration of a condition detection system to which the technology according to the present disclosure is applied.

[0024] The vehicle 1 shown in FIG. 1 includes a noise canceling system that performs road noise canceling (RNC) and outputs a noise canceling signal that cancels noise generated by the vehicle's movement at the ear position of a passenger PA.

[0025] Specifically, the noise cancellation unit 3 convolves a noise cancellation filter with the sensor value (acceleration value) obtained from one or more acceleration sensors 2 installed in the vehicle 1, thereby generating a noise cancellation signal that is in phase opposite to the road noise at the ear position and outputting it from the speaker SP.

[0026] The vehicle 1 shown in FIG. 1 also includes a state detection system consisting of a microphone 11 (hereinafter referred to as microphone 11) installed on the ceiling or the like of the passenger compartment of the vehicle 1, a noise estimation unit 12, and a state detection unit 13.

[0027] The state detection system detects the state of the vehicle 1 based on the difference between an actual measurement value of noise input to a microphone 11 installed in the cabin of the vehicle 1 and an estimated value of noise estimated based on a sensor value obtained from an acceleration sensor 2 used for noise cancellation in the cabin. The microphone 11 is not limited to being installed in the cabin of the vehicle 1, but may be installed in any space inside the vehicle 1, such as an engine room or a trunk, or may be mounted on the body of the vehicle 1.

[0028] The flow of the state detection process executed by the state detection system included in the vehicle 1 will be described with reference to the flowchart of FIG.

[0029] In step S11, the microphone 11 measures the noise inside the vehicle 1. The obtained measured noise values ​​are sequentially supplied to the state detection unit 13.

[0030] In step S12, the noise estimation unit 12 estimates noise based on sensor values ​​obtained from one or more acceleration sensors 2. Specifically, the noise estimation unit 12 estimates noise at the ear position of the passenger PA using the sensor values ​​(acceleration values) obtained from the acceleration sensors 2 and an estimation filter for noise estimation.

[0031] The estimation filter is generated in advance based on the actual measurement value of noise input to the microphone 11 while the vehicle 1 is traveling and the sensor value obtained from the acceleration sensor 2. The estimation filter is, for example, a feedforward FIR (Finite Impulse Response) filter.

[0032] The noise estimation values ​​obtained by the noise estimation unit 12 are sequentially supplied to the state detection unit 13 .

[0033] In step S13, the state detection unit 13 calculates the difference between the actual measurement value of noise from the microphone 11 and the estimated value of noise from the noise estimation unit 12. Specifically, the state detection unit 13 calculates the level difference between the actual measurement value and the estimated value as the difference between the actual measurement value and the estimated value of noise. The state detection unit 13 may also calculate a correlation value between the actual measurement value and the estimated value as the difference between the actual measurement value and the estimated value of noise.

[0034] Then, in step S14, the state detection unit 13 detects the state of the vehicle 1 based on the calculated difference between the actual measurement value and the estimated value of the noise.

[0035] For example, the state detection unit 13 detects an abnormality in the vehicle 1 or the acceleration sensor 2 as the state of the vehicle 1. When an abnormality in the vehicle 1 or the acceleration sensor 2 is detected, the noise cancellation unit 3 stops or limits the output of the noise canceling signal via the speaker SP.

[0036] Furthermore, the state detection unit 13 may detect the driving environment of the vehicle 1 as the state of the vehicle 1. In this case, the noise cancellation unit 3 can perform noise cancellation using a noise cancellation filter suitable for the detected driving environment.

[0037] According to the above configuration and processing, by using the difference between the actual measured value of the noise input to the microphone and the estimated value of the noise estimated using an estimation filter generated in advance, even if the transmission path of the vibration from the noise source gradually changes, the change can be reliably detected, and ultimately the state of the vehicle can be detected more reliably.

[0038] Hereinafter, an embodiment of a condition detection system to which the technology according to the present disclosure is applied will be described.

[0039] 3. First Embodiment (Application to Anomaly Detection System) First, as a first embodiment of the present disclosure, an anomaly detection system that detects an anomaly in a vehicle 1 or a sensor 2 will be described.

[0040] (System Configuration in a State Where RNC is Not Executed) FIG. 3 is a diagram showing an example of the configuration of an anomaly detection system when generating an estimation filter.

[0041] The anomaly detection system 100 shown in Fig. 3 has a system configuration in which an estimation filter is pre-designed in the simplest manner in the normal driving state of the vehicle 1. Specifically, the anomaly detection system 100 shown in Fig. 3 has a system configuration in which an estimation filter is designed in a state in which the RNC is not running.

[0042] 3 is configured to include a plurality of sensors 2-1, 2-2, 2-3, ..., 2-N and a microphone 11, as well as an estimation filter generation unit 111 and an estimation filter holding unit 112. The estimation filter generation unit 111 and the estimation filter holding unit 112 may be realized in a single on-board electronic device mounted on the vehicle 1, or may be realized in separate on-board electronic devices.

[0043] The sensors 2-1 to 2-N are configured as acceleration sensors installed in the vehicle 1. The sensors 2-1 to 2-N are configured as acceleration sensors used for road noise canceling (RNC) in the passenger compartment of the vehicle 1. In this case, the sensors 2-1 to 2-N are attached near the wheels of the vehicle 1. Hereinafter, when there is no need to distinguish between the sensors 2-1 to 2-N, they will simply be referred to as sensors 2.

[0044] As described with reference to FIG. 1, the microphone 11 is a microphone that is installed on the ceiling or the like of the passenger compartment of the vehicle 1 .

[0045] The estimation filter generation unit 111 generates an estimation filter (e.g., an FIR filter) for estimating the noise input to the microphone 11 based on the actual measurement value of the noise input to the microphone 11 while the vehicle 1 is traveling and the sensor values ​​obtained from each sensor 2. Here, it is assumed that the estimation filter is generated in a state where the RNC is not running.

[0046] The estimation filter holding unit 112 stores (holds) the estimation filter generated by the estimation filter generation unit 111. The estimation filter generated and held in advance in this manner is used to estimate noise input to the microphone 11 while the vehicle 1 is traveling thereafter.

[0047] FIG. 4 is a diagram illustrating an example of the configuration of an anomaly detection system when estimating noise.

[0048] The anomaly detection system 100 shown in FIG. 4 is configured to include a noise estimation unit 121, a state detection unit 122, and a noise cancellation unit 123 in addition to the sensor 2, the microphone 11, and the estimation filter holding unit 112.

[0049] The noise estimation unit 121 estimates noise based on the sensor values ​​obtained from each sensor 2. Specifically, the noise estimation unit 121 estimates noise input to the microphone 11 using the sensor values ​​(acceleration values) obtained from each sensor 2 and the estimation filter held in the estimation filter holding unit 112. The noise estimation values ​​obtained by the noise estimation unit 121 are sequentially supplied to the state detection unit 122.

[0050] If the difference between the actual measured value of the noise input to the microphone 11 and the estimated value of the noise from the noise estimation unit 121 exceeds a threshold value, the state detection unit 122 detects an abnormality in the vehicle 1 or one of the sensors 2, and outputs an abnormality detection signal indicating that an abnormality has been detected to the noise cancellation unit 123.

[0051] The noise cancellation unit 123 executes RNC by generating a noise cancellation signal in accordance with the sensor values ​​from each sensor 2 and outputting the signal via the speaker SP while the vehicle 1 is traveling. Furthermore, when the status detection unit 122 detects an abnormality in the vehicle 1 or any of the sensors 2, that is, when an abnormality detection signal is output from the status detection unit 122, the noise cancellation unit 123 stops or limits the output of the noise cancellation signal via the speaker SP.

[0052] The flow of the process of detecting an abnormality in the vehicle 1 or the sensor 2 by the abnormality detection system 100 will be described with reference to the flowchart in Fig. 5. The process in Fig. 5 is executed while the vehicle 1 is traveling and noise cancellation is being performed by the noise cancellation unit 123.

[0053] In step S111, the microphone 11 measures the noise inside the vehicle 1. The obtained measured noise values ​​are sequentially supplied to the state detection unit 122.

[0054] In step S112, the noise estimation unit 121 estimates noise based on the sensor values ​​obtained from the sensors 2. The obtained noise estimation values ​​are sequentially supplied to the state detection unit 122.

[0055] In step S113, the state detection unit 122 calculates the difference between the actual measurement value of noise from the microphone 11 and the estimated value of noise from the noise estimation unit 121. Here, the state detection unit 122 calculates, as the difference between the actual measurement value and the estimated value of noise, the average level difference between the actual measurement value and the estimated value over a certain period of time, or coherence as the correlation value between the actual measurement value and the estimated value.

[0056] In step S114, the state detection unit 122 determines whether the calculated difference exceeds a threshold value.

[0057] The threshold value here can be set for each frequency band or for each state of the vehicle 1. The threshold value may be a preset value, a value calculated from design drawings or the like, or a statistically set value, such as a value set based on statistical analysis of time-series data of differences under normal conditions.

[0058] The statistically set value is the confidence interval of the probability distribution, such as a 3δ interval, calculated based on the level difference or correlation value (coherence) between the actual measured value and the estimated value of noise during normal driving of vehicle 1, which is recorded in advance.

[0059] The threshold value may also be set based on a difference obtained in advance when an increase in sound occurs at the ear position due to the noise cancellation signal.

[0060] If it is determined that the difference does not exceed the threshold, the process returns to step S111, and the calculation of the difference between the actual noise value and the estimated noise value is repeated. On the other hand, if it is determined that the difference exceeds the threshold, for example, if the difference for each band exceeds one of the thresholds set for each band, the process proceeds to step S115, and the state detection unit 122 generates an abnormality detection signal, determining that an abnormality has been detected in the vehicle 1 or any of the sensors 2, and outputs the abnormality detection signal to the noise cancellation unit 123.

[0061] In step S116, the noise cancellation unit 123 stops or limits the output of the noise cancellation signal via the speaker SP. That is, muting is performed on the noise cancellation signal output from the speaker SP, or attenuation is performed on the signal level of the noise cancellation signal.

[0062] If the difference between the actual noise and the estimated noise becomes large, it means that a malfunction has occurred in the sensor 2 or that the physical transmission path of the noise from the sensor 2 to the microphone 11 has changed. In other words, it can be said that the transmission path from the sensor 2 to the listening position, such as the ear position, has also changed, which may worsen the effect of noise canceling at the listening position. In this case, it is desirable to detect an abnormality and stop outputting the noise cancellation signal thereafter.

[0063] For example, when the vehicle 1 is traveling at a low speed, the noise level is low to the user, and the system may be set not to reproduce the noise cancellation signal. Even in such a case, by detecting an abnormality based on the difference between the actual noise and the estimated noise, the noise cancellation signal can be prevented from being output even when the vehicle 1 changes from a low speed to a normal traveling state and the noise cancellation signal is reproduced.

[0064] FIG. 6 is a diagram showing an example of waveforms of actually measured and estimated noise values ​​under normal conditions.

[0065] 6, the waveform of the actual measured noise value is shown by a solid line, and the waveform of the estimated noise value is shown by a dashed line. For ease of understanding, in this example, the waveforms are shown after applying a 100-200 Hz band-pass filter to each of the actual measured noise value and the estimated noise value.

[0066] As shown in FIG. 6, under normal conditions, the waveforms of the actually measured and estimated noise values ​​are similar to each other, indicating that noise estimation is performed with high accuracy.

[0067] FIG. 7 is a diagram showing an example of the level difference between the actually measured noise value and the estimated noise value.

[0068] In FIG. 7, for each 1 / 3 octave band in the range of 100 to 160 Hz, the actual noise level when the noise cancellation signal is not being reproduced (left), the estimated noise level that is correctly estimated (center), and the estimated noise level when an abnormality occurs in sensor 2 or the like (right) are shown as bar graphs.

[0069] As shown in Figure 7, under normal conditions, the level difference between the actual noise and the estimated noise is small, indicating that noise estimation is performed with high accuracy. On the other hand, under abnormal conditions, the level difference between the actual noise and the estimated noise becomes large, particularly at 160 Hz, with a level difference of approximately 7.5 dB, deteriorating the accuracy of noise estimation. Here, if the threshold value is set to, for example, a level difference of twice the amplitude of the waveforms of the actual noise and the estimated noise, i.e., a level difference of 6 dB, then in the example of Figure 7, it will be determined that an abnormality has been detected.

[0070] As described above, the level difference between the actual measured value and the estimated value of noise during normal driving of the vehicle 1 can be recorded in advance, and the 3δ section of the normal distribution for each band of the level difference created based on the recorded level difference can be set as the threshold for each band. Note that, since noise tends to vary greatly in high frequencies, it is preferable to create a normal distribution of the level of driving noise during normal driving of the vehicle 1, and set the threshold for each band according to the degree of variation in each band.

[0071] Furthermore, the threshold value may be changed as appropriate based on the acquired vehicle speed information, by acquiring vehicle speed information of the vehicle 1. In particular, when traveling at high speeds, not only road noise but also wind noise increases, so by acquiring vehicle speed information from an in-vehicle infotainment (IVI) or the like and relaxing the threshold value (increasing the level difference) when traveling at high speeds, more accurate abnormality detection becomes possible.

[0072] FIG. 8 is a diagram showing an example of a correlation value between an actual measurement value and an estimated value of noise.

[0073] In FIG. 8, the correlation value between actual noise and normally estimated noise (left) and the correlation value between actual noise and estimated noise in an abnormal state (right) are shown in bar graphs for each 1 / 3 octave band in the range of 100 to 160 Hz.

[0074] As shown in FIG. 8 , under normal conditions, all correlation values ​​are 0.9 or higher, indicating that noise estimation is performed with high accuracy. On the other hand, under abnormal conditions, correlation values ​​generally decrease, particularly at 160 Hz, falling below 0.4, deteriorating the accuracy of noise estimation. Here, the threshold value may be set to, for example, 0.7, a value considered to have a strong correlation. Similarly to the level difference, correlation values ​​between actual noise measurements and estimated noise values ​​during normal driving of the vehicle 1 may be recorded in advance, and the 3δ interval of a normal distribution for each band of correlation values ​​created based on these may be set as the threshold for each band. In particular, since the accuracy of noise estimation tends to deteriorate and the correlation tends to decrease as the frequency band increases, it is desirable to set the correlation value threshold lower as the frequency band increases.

[0075] According to the above configuration and processing, by using the difference between the actual measured value of the noise input to the microphone and the estimated value of the noise estimated using an estimation filter generated in advance, even if the transmission path of the vibration from the noise source gradually changes, the change can be reliably detected, and ultimately, abnormalities in the vehicle or sensor due to aging or the like can be more reliably detected.

[0076] (System configuration when RNC is running) Normally, when the driving noise is small, such as when the vehicle 1 is driving at a low speed, the output of the noise cancellation signal may be stopped. On the other hand, when the vehicle 1 is driving at a normal speed, the noise cancellation signal is output from the speaker SP, so that the microphone 11 installed in the vehicle cabin picks up not only noise generated by vibrations of the vehicle 1 but also the noise cancellation signal from the listening position.

[0077] FIG. 9 is a diagram showing an example of the configuration of an anomaly detection system when generating an estimation filter while the RNC is running.

[0078] 9 , while the noise cancellation unit 123 is executing RNC, the estimation filter generation unit 111 generates an estimation filter for estimating a signal in which a noise cancellation signal is added to the road noise input to the microphone 11. Instead of the road noise input to the microphone 11, a noise signal generated by simulation may be used.

[0079] Using the estimation filter generated in this manner, in anomaly detection system 100 during noise estimation described with reference to Fig. 4, noise estimation unit 121 estimates the mixed noise (driving noise + noise cancellation signal input near the ceiling) input to microphone 11. Then, as in the anomaly detection process when RNC is not running (Fig. 5), if the accuracy of noise estimation deteriorates and an anomaly is detected, output of the noise cancellation signal is stopped or limited.

[0080] (System Configuration in a State Where a Music Signal is Being Played) In addition to road noise, music may also be played using the speaker SP inside the vehicle 1. In this case, the music signal may be input to the microphone 11 in addition to the road noise, which may result in a deterioration in the accuracy of noise estimation.

[0081] FIG. 10 is a diagram showing an example of the configuration of an anomaly detection system when estimating noise while a music signal is being played back.

[0082] The anomaly detection system 100 shown in FIG. 10 is configured to further include a music signal filtering unit 131 in addition to the same configuration as the anomaly detection system 100 shown in FIG.

[0083] The music signal filtering unit 131 generates a pseudo audio signal by convolving a filter that simulates a transfer function from the speaker SP to the microphone 11 with a previously acquired audio signal.

[0084] With this configuration, even when a mixed signal containing a mixture of driving noise and an audio signal is input to the microphone 11, the state detection unit 122 receives a signal obtained by subtracting the pseudo audio signal from the mixed signal input to the microphone 11. This makes it possible to properly detect an abnormality without reducing the accuracy of noise estimation, even when a music signal is being played inside the passenger compartment of the vehicle 1.

[0085] 4. Second Embodiment (Application to a Driving Environment Detection System) The technology according to the present disclosure can be applied not only to a configuration for detecting an abnormality in the vehicle 1 or the sensor 2, but also to a configuration for detecting the driving environment of the vehicle 1 and selecting (switching) a more appropriate noise cancellation filter (hereinafter referred to as an NC filter).

[0086] In noise estimation, the estimation accuracy is highest in a driving environment that is the same as or similar to the driving environment, such as the road surface, when the estimation filter was generated. For example, if estimation filters a, b, c, etc. are generated and stored for environments A, B, C, etc., noise estimation accuracy by estimation filter c will be highest when driving in environment C. In other words, applying each estimation filter to noise input to microphone 11 in a certain environment and selecting the estimation filter with the highest estimation accuracy can be said to be equivalent to detecting the environment in which the vehicle is driving.

[0087] This can also be applied to a method for selecting an optimal filter as an NC filter. For example, if the road surface or road surface condition differs significantly between the driving environment when the NC filter is generated and the driving environment during actual driving, the noise canceling effect may be reduced. To address this, driving data is acquired for each driving environment, such as a different road surface, and an NC filter is generated. Then, if it is possible to detect the driving environment based on the noise input to the microphone 11 and the estimated filter and select the optimal NC filter depending on the driving environment, it is possible to prevent the noise canceling effect from deteriorating.

[0088] Therefore, as a second embodiment of the present disclosure, a driving environment detection system that detects the driving environment of the vehicle 1 will be described.

[0089] FIG. 11 is a diagram illustrating an example of the configuration of a driving environment detection system when generating an estimation filter.

[0090] 11 is configured to include, in addition to a plurality of sensors 2 and microphones 11, an estimation filter generation unit 211, an NC filter generation unit 212, a linking processing unit 213, an estimation filter holding unit 214, and an NC filter holding unit 215. The estimation filter generation unit 211 to the NC filter holding unit 215 may be realized in a single on-board electronic device mounted on the vehicle 1, or may be realized in separate on-board electronic devices.

[0091] The estimation filter generation unit 211 generates an estimation filter for estimating the noise input to the microphone 11 based on the actual measured value of the noise input to the microphone 11 while the vehicle 1 is traveling and the sensor values ​​obtained from each sensor 2.

[0092] The NC filter generation unit 212 generates an NC filter for canceling out noise at the ear position based on the sensor values ​​obtained from each sensor 2 while the vehicle 1 is traveling.

[0093] The generation of the estimation filter by the estimation filter generation unit 211 and the generation of the NC filter by the NC filter generation unit 212 are performed for each driving environment, such as for each different road surface.

[0094] The linking processing unit 213 links the estimation filters generated by the estimation filter generating unit 211 and the NC filters generated by the NC filter generating unit 212 for each driving environment in which they were generated.

[0095] The estimation filter storage unit 214 stores (stores) the estimation filter generated for each driving environment. The estimation filter generated and stored in advance in this manner is used to estimate noise input to the microphone 11 while the vehicle 1 is subsequently driving.

[0096] The NC filter storage unit 215 stores (stores) the estimation filter generated for each driving environment. The NC filter generated and stored in advance in this manner is used for noise cancellation when the vehicle 1 is subsequently driving.

[0097] FIG. 12 is a diagram showing an example of the configuration of a driving environment detection system when estimating a driving environment.

[0098] The driving environment detection system 200 shown in Figure 12 is configured to include a sensor 2, a microphone 11, an estimation filter holding unit 214, and an NC filter holding unit 215, as well as a state detection unit 221, an NC filter acquisition unit 222, and a noise cancellation unit 223.

[0099] The state detection unit 221 estimates noise based on sensor values ​​obtained from each sensor 2. Specifically, the state detection unit 221 estimates noise input to the microphone 11 using the sensor values ​​(acceleration values) obtained from each sensor 2 and each of a plurality of estimation filters (for each driving environment) held in the estimation filter holding unit 214. Then, the state detection unit 221 detects the driving environment of the vehicle 1 based on the difference between the actual measurement value of the noise input to the microphone 11 and the plurality of estimated values ​​of noise from the state detection unit 221.

[0100] The NC filter acquisition unit 222 acquires the NC filter that is linked to the driving environment detected by the state detection unit 221 from among the NC filters held in the NC filter holding unit 215, and supplies it to the noise cancellation unit 223.

[0101] The noise cancellation unit 223 executes RNC to which the NC filter supplied from the NC filter acquisition unit 222 (generated in the running environment detected by the state detection unit 221) is applied.

[0102] The flow of the process of detecting the driving environment of the vehicle 1 by the driving environment detection system 200 will be described with reference to the flowchart of Fig. 13. The process of Fig. 13 is executed while the vehicle 1 is driving.

[0103] In step S211, the microphone 11 measures the noise inside the vehicle 1. The obtained measured noise values ​​are sequentially supplied to the state detection unit 221.

[0104] In step S212, the state detection unit 221 estimates noise based on the sensor values ​​obtained from each sensor 2 and a plurality of estimation filters (for each driving environment) held in the estimation filter holding unit 214.

[0105] In step S213, the state detection unit 221 calculates the difference between the actual measurement value of noise from the microphone 11 and the multiple estimated values ​​(for each estimation filter) estimated by the state detection unit 221. Here, the state detection unit 221 may calculate an average level difference between the actual measurement value and the estimated value over a certain period of time as the difference between the actual measurement value and the estimated value of noise, or may calculate a correlation value (coherence) between the actual measurement value and the estimated value.

[0106] In step S214, the state detection unit 221 detects the current driving environment of the vehicle 1 based on the estimation filter with the highest estimation accuracy of the noise estimation value. Specifically, the state detection unit 221 determines the estimation filter associated with the estimation value used to calculate the smallest difference between the calculated actual noise value and the estimated noise value as the estimation filter with the highest estimation accuracy, and detects the driving environment associated with that estimation filter as the current driving environment of the vehicle 1.

[0107] In step S215, the NC filter acquisition unit 222 acquires an NC filter generated in the driving environment detected by the state detection unit 221 from multiple NC filters (for each driving environment) stored in the NC filter storage unit 215.

[0108] Then, in step S216 , the noise canceling unit 223 performs noise canceling using the NC filter acquired by the NC filter acquisition unit 222 .

[0109] With the above configuration and processing, the driving environment can be detected by using the difference between the actual measurement value of noise input to the microphone and the estimated value of noise estimated using a pre-generated estimation filter. As a result, it is possible to switch to an optimal NC filter depending on the driving environment, and it is possible to prevent a deterioration in the noise canceling effect.

[0110] 5. Third Embodiment (State Detection Using Trained Model) FIG. 14 is a diagram illustrating a configuration example of a state detection system according to a third embodiment of the present disclosure.

[0111] In the condition detection system 300 shown in FIG. 14, the same components as those of the condition detection system included in the vehicle 1 in FIG. 1 are denoted by the same reference numerals, and the description thereof will be omitted as appropriate.

[0112] That is, the condition detection system 300 in FIG. 14 differs from the condition detection system in FIG. 1 in that it newly includes an estimation filter generation unit 311, an estimation filter holding unit 312, and a learning unit 313.

[0113] The estimation filter generation unit 311 is realized, for example, in a cloud server, etc. The estimation filter generation unit 311 generates an estimation filter for noise estimation based on the sensor values ​​obtained from the sensor 2 and the data (actual measured values) collected by the microphone 11, which are uploaded to the cloud server as learning data.

[0114] The estimation filter storage unit 312 may be realized in an on-board electronic device mounted on the vehicle 1, and stores (stores) the estimation filter generated by the estimation filter generation unit 311 and downloaded from a server on the cloud.

[0115] The learning unit 313 is also realized, for example, on a server on the cloud. The learning unit 313 generates a trained model for detecting the state of the vehicle 1 or the sensor 2 by learning the difference between the actual measured value and the estimated value according to the state of the vehicle 1 or the sensor 2. For example, the learning unit 313 generates an environment estimation model for detecting (estimating) the driving environment of the vehicle 1 as the state of the vehicle 1. The actual measured value and the estimated value according to the state may also be uploaded to the server on the cloud as training data.

[0116] The generated trained model is used by the state detection unit 13 to detect the state of the vehicle 1 or the sensor 2.

[0117] The flow of the state detection process of the vehicle 1 or the sensor 2 by the state detection system 300 will be described with reference to the flowchart of FIG.

[0118] The processing in steps S311 to S313 in the flowchart of FIG. 15 is the same as the processing in steps S11 to S13 in the flowchart of FIG. 2, and therefore a description thereof will be omitted.

[0119] That is, in step S314, the state detection unit 13 receives as input the difference between the actual measured value of noise from the microphone 11 and the estimated value of noise from the noise estimation unit 12, and detects the state of the vehicle 1 or the sensor 2 by using the trained model generated by the learning unit 313. For example, the state detection unit 13 receives as input the difference between the actual measured value and the estimated value of noise, and uses an environment estimation model to estimate the driving environment of the vehicle 1. Because the environment estimation model has learned the difference between the actual measured value and the estimated value according to the driving environment, it is possible to detect, for example, driving environments such as a very strong wind blowing or driving on a rough road when the difference is significantly large.

[0120] With the above configuration and processing, it becomes possible to update the threshold value used for anomaly detection, for example, based on feedback information after an anomaly is detected (whether or not a malfunction actually occurred, what kind of anomaly it was, etc.) Also, by generating an estimation filter for noise estimation and an NC filter associated therewith based on data uploaded from a vehicle that is determined to be in a normal state where no anomaly is detected, it becomes possible to update the filter coefficients of the estimation filter and the NC filter.

[0121] 6. Application Examples In the above, an embodiment has been described in which the technology according to the present disclosure is applied to a vehicle (automobile) on which two or more sensors are installed, but the technology can also be applied to other moving bodies including trains, ships, aircraft, and even drones.

[0122] In this case, a status detection system incorporating the technology disclosed herein can detect the status of the mobile object or the acceleration sensor based on the difference between the actual measured value of noise input to a microphone mounted on the mobile object and the estimated value of noise estimated based on the sensor value obtained from the acceleration sensor mounted on the mobile object. For example, the status of the mobile object or the acceleration sensor can be detected by detecting an abnormality in the mobile object or the acceleration sensor, or by detecting the moving environment of the mobile object. The microphone or the acceleration sensor may be installed at any location on the mobile object.

[0123] 7. Example of Computer Hardware Configuration The above-described series of processes can be executed by hardware or software. When the series of processes is executed by software, the program constituting the software is installed from a program recording medium into a computer incorporated in dedicated hardware, a general-purpose personal computer, or the like.

[0124] 16 is a block diagram showing an example of the hardware configuration of a computer that executes the above-described series of processes using a program. At least a part of the condition detection system (FIG. 1), anomaly detection system 100, driving environment detection system 200, and condition detection system 300 included in vehicle 1 is configured by, for example, a computer 500 having a configuration similar to that shown in FIG.

[0125] A CPU (Central Processing Unit) 501 , a ROM (Read Only Memory) 502 , and a RAM (Random Access Memory) 503 are interconnected by a bus 504 .

[0126] An input / output interface 505 is also connected to the bus 504. An input unit 506 including a keyboard, a mouse, etc., and an output unit 507 including a display, a speaker, etc. are connected to the input / output interface 505. Also connected to the input / output interface 505 are a storage unit 508 including a hard disk, a nonvolatile memory, etc., a communication unit 509 including a network interface, etc., and a drive 510 that drives removable media 511.

[0127] In the computer 500 configured as described above, the CPU 501 performs the above-described series of processes by, for example, loading a program stored in the memory unit 508 into the RAM 503 via the input / output interface 505 and the bus 504 and executing it.

[0128] The program executed by the CPU 501 is installed in the storage unit 508 by being recorded on, for example, a removable medium 511 or provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital broadcasting.

[0129] The program executed by computer 500 may be a program that processes in chronological order according to the order described in this specification, or may be a program that processes in parallel or at the required timing, such as when called.

[0130] In this specification, a system refers to a collection of multiple components (devices, modules (components), etc.), regardless of whether all of the components are contained in the same housing. Therefore, multiple devices housed in separate housings and connected via a network, and a single device housed in a single housing with multiple modules, are both systems.

[0131] The effects described in this specification are merely examples and are not limiting, and other effects may also be present.

[0132] The embodiments of the present disclosure are not limited to the above-described embodiments, and various modifications are possible within the scope of the gist of the present disclosure.

[0133] For example, the embodiment of the present disclosure can be configured as a cloud computing system in which a single function is shared and processed collaboratively by multiple devices via a network.

[0134] Furthermore, each step described in the above flowchart can be executed by one device, or can be shared and executed by a plurality of devices.

[0135] Furthermore, when one step includes multiple processes, the multiple processes included in that one step can be executed by one device or can be shared and executed by multiple devices.

[0136] The effects described in this specification are merely examples and are not limiting, and other effects may also be present.

[0137] Furthermore, the technology according to the present disclosure may have the following configurations. (1) A condition detection method for detecting a condition of a moving object or an acceleration sensor based on a difference between an actual measurement value of noise input to a microphone mounted on the moving object and an estimated value of the noise estimated based on a sensor value obtained from an acceleration sensor mounted on the moving object. (2) The condition detection method according to (1), in which the estimated value is estimated based on an estimation filter generated based on the actual measurement value of the noise input to the microphone while the moving object is moving and the sensor value obtained from the acceleration sensor. (3) The condition detection method according to (2), in which the estimation filter is a feedforward FIR (Finite Impulse Response) filter. (4) The condition detection method according to (2) or (3), in which the difference is a level difference between the actual measurement value and the estimated value, or a correlation value between the actual measurement value and the estimated value. (5) The condition detection method according to (4), wherein the moving body is a vehicle in which the microphone is installed inside the vehicle cabin and one or more acceleration sensors used for noise cancellation inside the vehicle cabin are installed. (6) The condition detection method according to (5), wherein an abnormality in the vehicle or the acceleration sensor is detected when the difference exceeds a threshold, and when an abnormality is detected, output of a noise cancellation signal is stopped or limited. (7) The condition detection method according to (6), wherein an abnormality in the vehicle or the acceleration sensor is detected when the difference exceeds one of the thresholds set for each band. (8) The condition detection method according to (6) or (7), wherein the threshold is a preset value. (9) The condition detection method according to (6) or (7), wherein the threshold is a value set based on statistical analysis of time-series data of the difference under normal conditions. (10) The condition detection method according to (5), wherein the traveling environment of the vehicle is detected based on the difference between the actual measured value of the noise and a plurality of the estimated values ​​estimated based on a plurality of the estimation filters generated for each environment. (11) The state detection method according to (10), further comprising detecting an environment in which the estimation filter with the highest estimation accuracy of the estimated value was generated as the traveling environment of the vehicle.(12) The state detection method according to (11), in which noise canceling is performed by applying the noise canceling filter generated for the detected traveling environment from among a plurality of noise canceling filters generated for each environment. (13) The state detection method according to any one of (1) to (12), in which a trained model that detects the state of the moving body or the acceleration sensor is generated by learning the difference according to the state of the moving body. (14) The state detection method according to (13), in which the state of the moving body or the acceleration sensor is detected by using the trained model with the calculated difference as input. (15) A state detection system comprising: a microphone mounted on a moving body; an acceleration sensor mounted on the moving body; and a state detection unit that detects the state of the moving body or the acceleration sensor based on the difference between an actual measurement value of noise input to the microphone and an estimated value of the noise estimated based on a sensor value obtained from the acceleration sensor. (16) A program for causing a computer to execute a process of detecting the state of a moving object or an acceleration sensor based on the difference between an actual measurement value of noise input to a microphone mounted on the moving object and an estimated value of the noise estimated based on a sensor value obtained from an acceleration sensor mounted on the moving object.

[0138] REFERENCE SIGNS LIST 1 Vehicle, 2, 2-1 to 2-N Sensors, 11 Microphone, 12 Noise estimation unit, 13 State detection unit, 100 Abnormality detection system, 111 Estimation filter generation unit, 112 Estimation filter holding unit, 121 Noise estimation unit, 122 State detection unit, 123 Noise cancellation unit, 131 Music signal filtering unit, 200 Driving environment detection system, 211 Estimation filter generation unit, 212 NC filter generation unit, 213 Linking processing unit, 214 Estimation filter holding unit, 215 NC filter holding unit, 221 State detection unit, 222 NC filter acquisition unit, 223 Noise cancellation unit, 311 Estimation filter generation unit, 312 Estimation filter holding unit, 313 Learning unit

Claims

1. A state detection method for detecting the state of a moving object or an acceleration sensor based on the difference between an actual measurement value of noise input to a microphone mounted on the moving object and an estimated value of the noise estimated based on a sensor value obtained from an acceleration sensor mounted on the moving object.

2. The state detection method according to claim 1, wherein the estimated value is estimated based on an estimation filter generated based on the actual measured value of the noise input to the microphone while the moving object is moving and the sensor value obtained from the acceleration sensor.

3. The condition detection method according to claim 2, wherein the estimation filter is a feedforward type FIR (Finite Impulse Response) filter.

4. The condition detection method according to claim 2, wherein the difference is a level difference between the actual measurement value and the estimated value, or a correlation value between the actual measurement value and the estimated value.

5. The state detection method according to claim 4, wherein the moving body is a vehicle in which the microphone is installed inside the vehicle cabin and one or more acceleration sensors used for noise canceling inside the vehicle cabin are installed.

6. The condition detection method according to claim 5, further comprising detecting an abnormality in the vehicle or the acceleration sensor when the difference exceeds a threshold value, and stopping or limiting output of the noise cancellation signal when an abnormality is detected.

7. The condition detection method according to claim 6, wherein an abnormality in the vehicle or the acceleration sensor is detected when the difference for each band exceeds the threshold value set for each band.

8. The condition detection method according to claim 6, wherein the threshold value is a preset value.

9. The condition detection method according to claim 6, wherein the threshold value is a value set based on statistical analysis of time-series data of the difference under normal conditions.

10. A state detection method according to claim 5, wherein the driving environment of the vehicle is detected based on the difference between the actual measured value of the noise and a plurality of estimated values estimated based on a plurality of estimation filters generated for each environment.

11. The state detection method according to claim 10, wherein an environment in which the estimation filter with the highest estimation accuracy of the estimated value was generated is detected as the traveling environment of the vehicle.

12. The state detection method according to claim 11, wherein the noise canceling is performed by applying the noise cancellation filter generated in the detected driving environment from among a plurality of noise cancellation filters generated for each environment.

13. The state detection method according to claim 1, wherein a trained model for detecting the state of the moving object or the acceleration sensor is generated by learning the difference according to the state of the moving object or the acceleration sensor.

14. The state detection method according to claim 13, wherein the calculated difference is used as an input and the trained model is used to detect the state of the moving object or the acceleration sensor.

15. A status detection system comprising: a microphone mounted on a moving object; an acceleration sensor mounted on the moving object; and a status detection unit that detects the status of the moving object or the acceleration sensor based on the difference between an actual measured value of noise input to the microphone and an estimated value of the noise estimated based on a sensor value obtained from the acceleration sensor.

16. A program for causing a computer to execute a process for detecting the state of a moving object or an acceleration sensor based on the difference between the actual measured value of noise input to a microphone mounted on the moving object and the estimated value of the noise estimated based on the sensor value obtained from an acceleration sensor mounted on the moving object.

Images