How to learn mapping

Abstract

To generate a mapping where the accuracy of estimating the cause of the sound generated hardly varies, even when the sound detected by a microphone includes background noise.SOLUTION: The object to be trained for mapping is a learning model that accepts a sound signal as an input variable and outputs a variable that indicates the cause of the sound generated in a vehicle. The learning method includes: a signal correction process (step S95) of superimposing a noise signal on a sound signal so as to correct the sound signal; and an update process (steps S97-S101) of updating the mapping by machine learning that adopts the sound signal corrected by the signal correction process as training data and the cause of the sound generated that is paired with the sound signal as teaching data.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] The present invention relates to a method for learning a mapping defined by mapping data. [Background technology]

[0002] There is a known method for estimating the cause of a sound by using a map that has been subjected to machine learning. In this method, when a sound signal, which is a signal related to a sound detected by a microphone, is input as an input variable to the map, an output variable is output from the map. Then, the cause of the sound is estimated based on the output variable.

[0003] Patent Document 1 discloses a method for removing noise from a sound signal related to a sound detected by a microphone. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2007-199017 A Summary of the Invention [Problem to be solved by the invention]

[0005] Consider a case where a microphone is installed in the cabin of a moving vehicle and the cause of a sound detected by the microphone is estimated. In this case, various noises are superimposed on the sound signal related to the sound detected by the microphone. Therefore, even if a noise removal process is performed on the sound signal by the method disclosed in the above Patent Document 1, it is difficult to stably remove the noise from the sound signal. Therefore, when the sound signal after the noise removal process is input as an input variable to the above mapping, the accuracy of the estimation of the cause of the sound is likely to vary. [Means for solving the problem]

[0006] The mapping learning method for solving the above problem is a method used to estimate a cause of sound generation in a vehicle based on a sound signal, which is a signal related to sound detected by a microphone. The mapping is a learning model that outputs a variable indicating a cause of sound generation in the vehicle when the sound signal is input as an input variable. The learning method causes a learning device to execute a signal correction process for correcting the sound signal by superimposing a noise signal on the sound signal, and an update process for updating the mapping by machine learning using the sound signal corrected by the signal correction process as training data and the cause of sound generation paired with the sound signal as teacher data.

[0007] Consider a case where a sound signal related to a sound detected by a microphone is input to a mapping, and the cause of a sound is estimated based on the factor variables output from the mapping. In this case, the sound detected by the microphone may contain background noise. In other words, a sound signal including a background noise component may be input to the mapping.

[0008] In this regard, in the above-mentioned learning method, the mapping is updated by performing machine learning using a sound signal on which a noise signal is intentionally superimposed as training data, that is, the sound signal used as training data also contains a signal corresponding to the above-mentioned background noise component.

[0009] Therefore, when estimating the cause of a sound based on the output variables of a mapping that has been subjected to such machine learning when a sound signal related to a sound detected by a microphone is input, the estimation accuracy of the cause of the sound is less likely to vary due to the influence of background noise detected by the microphone. Therefore, the above learning method can generate a mapping in which the estimation accuracy of the cause of the sound is less likely to vary even if the sound detected by the microphone contains background noise. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a block diagram showing a configuration of a system according to an embodiment. [Diagram 2]In FIG. 2, (A) is a flowchart showing the flow of a series of processes executed by a vehicle control device, and (B) is a flowchart showing the flow of a series of processes executed by a mobile terminal. [Diagram 3] FIG. 3 is a diagram showing an example of a sound signal related to a sound detected by a microphone of a mobile terminal. [Figure 4] FIG. 4 is a flowchart showing the flow of a series of processes executed by the center control device. [Diagram 5] FIG. 5 is a block diagram showing the configuration of a learning device that applies machine learning to a mapping. [Figure 6] FIG. 6 is a flow chart illustrating one embodiment of a training method. [Figure 7] FIG. 7 is a diagram showing a map for determining the amplitude of a noise signal. [Figure 8] In FIG. 8, (A) is a diagram showing the sound signal before correction, (B) is a diagram showing the noise signal, and (C) is a diagram showing the corrected sound signal. [Figure 9] FIG. 9 is a flowchart showing the flow of processing for verifying whether or not the result of estimating the cause of abnormal noise using the mapping is correct. [Figure 10] FIG. 10 is a diagram showing a modified example of the map for determining the amplitude of the noise signal. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, one embodiment of the mapping learning method will be described with reference to FIGS. FIG. 1 illustrates a system including a vehicle 10, a mobile terminal 30 carried by a passenger of the vehicle 10, and a data analysis center 60 installed outside the vehicle 10.

[0012] <Vehicle> The vehicle 10 includes a detection system 11 , a communication device 13 , and a vehicle control device 15 . The detection system 11 has N sensors 111, 112, 113, ..., 11N, where "N" is an integer equal to or greater than 4. The multiple sensors 111-11N output signals according to detection results to the vehicle control device 15. The multiple sensors 111-11N include sensors that detect vehicle state quantities such as vehicle speed and acceleration, and sensors that detect occupant operation quantities such as accelerator operation quantity and brake operation quantity. The multiple sensors 111-11N may also include sensors that detect the operating state of the drive device of the vehicle 10, such as an engine or an electric motor, and sensors that detect the temperature of the coolant or the oil temperature.

[0013] The communication device 13 communicates with the mobile terminal 30 brought into the passenger compartment of the vehicle 10. The communication device 13 outputs information received from the mobile terminal 30 to the vehicle control device 15, and transmits information output from the vehicle control device 15 to the mobile terminal 30.

[0014] The vehicle control device 15 controls the vehicle 10 based on output signals from multiple sensors 111 to 11N. That is, the vehicle control device 15 controls the running speed, acceleration, yaw rate, etc. of the vehicle 10 by operating the drive device, braking device, steering device, etc. of the vehicle 10.

[0015] The vehicle control device 15 has a CPU 16, a first storage device 17, and a second storage device 18. The first storage device 17 stores various control programs executed by the CPU 16. The first storage device 17 also stores vehicle model information, which is information related to the vehicle model and grade of the vehicle 10. The second storage device 18 stores the calculation results of the CPU 16, etc.

[0016] <Mobile device> The mobile terminal 30 is, for example, a smartphone or a tablet terminal. The mobile terminal 30 includes a touch panel 31, a display screen 33, a microphone 35, a communication device 37, and a terminal control device 39. The touch panel 31 is a user interface disposed so as to overlap the display screen 33. When the mobile terminal 30 is brought into the vehicle cabin, the microphone 35 can detect sounds transmitted to the vehicle cabin.

[0017] The communication device 37 has a function of communicating with the vehicle 10 when it is located inside the cabin of the vehicle 10. The communication device 37 outputs information received from the vehicle control device 15 to the terminal control device 39, and transmits information output by the terminal control device 39 to the vehicle control device 15.

[0018] The communication device 37 also has a function of communicating with other mobile terminals 30 and the data analysis center 60 via the global network 100. The communication device 37 outputs information received from other mobile terminals 30 or the data analysis center 60 to the terminal control device 39, and transmits information output by the terminal control device 39 to the other mobile terminals 30 or the data analysis center 60.

[0019] The terminal control device 39 has a CPU 41, a first storage device 42, and a second storage device 43. The first storage device 42 stores various control programs executed by the CPU 41. The second storage device 43 stores the calculation results of the CPU 41, etc.

[0020] <Data Analysis Center> The data analysis center 60 estimates the cause of the sound detected by the microphone 35. It is assumed that there are M causes of abnormal noise in the vehicle 10, where "M" is an integer equal to or greater than 2. In this case, the data analysis center 60 selects one cause from the M cause candidates.

[0021] The data analysis center 60 includes a communication device 61 and a center control device 63 . The communication device 61 has a function of communicating with a plurality of mobile terminals 30 via the global network 100. The communication device 61 outputs information received from the mobile terminals 30 to the center control device 63, and transmits information output by the center control device 63 to the mobile terminals 30.

[0022] The center control device 63 has a CPU 64, a first storage device 65, and a second storage device 66. The first storage device 65 stores various control programs executed by the CPU 64.

[0023] The second storage device 66 stores mapping data 71 that defines a mapping that has been subjected to machine learning. The mapping is a trained model that, when an input variable is input, outputs a variable that identifies a cause of sound generation in the vehicle 10. An example of the mapping is a function approximator. For example, the mapping is a fully connected forward propagation type neural network with one intermediate layer.

[0024] The output variable y of the mapping will be explained. As described above, there are M candidate causes of abnormal noise in the vehicle 10. Therefore, when the input variables are input to the mapping, M output variables y(1), y(2), ..., y(M) are output from the mapping. When the actual cause of occurrence is taken as the real cause, the output variable y(1) is a value indicating the probability that the first candidate cause of occurrence among the M candidate causes of occurrence is a real cause. The output variable y(2) is a value indicating the probability that the second candidate cause of occurrence among the M candidate causes of occurrence is a real cause. The output variable y(M) is a value indicating the probability that the Mth candidate cause of occurrence among the M candidate causes of occurrence is a real cause.

[0025] The second storage device 66 stores factor identification data 72. The factor identification data 72 is data for estimating the cause of sound generation in the vehicle 10 based on the output variable y of the mapping. The factor identification data 72 stores M candidates for generation factors. Of the M candidates for generation factors, the candidate for the first generation factor corresponds to the output variable y(1). Of the M candidates, the candidate for the second generation factor corresponds to the output variable y(2). Of the M candidates, the candidate for the Mth generation factor corresponds to the output variable y(M).

[0026] <How to identify the cause of abnormal noise> The method for identifying the cause of occurrence will be described with reference to Fig. 2 to Fig. 4. Fig. 2(A) illustrates the flow of processing executed by the CPU 16 of the vehicle control device 15. The CPU 16 executes a control program stored in the first storage device 17, thereby repeatedly executing the series of processing shown in Fig. 2(A).

[0027] 2A, in step S11, the CPU 16 determines whether or not synchronization with the mobile terminal 30 has been established. If the CPU 16 determines that synchronization has been established (S11: YES), the CPU 16 shifts the process to step S13. On the other hand, if the CPU 16 determines that synchronization has not been established (S11: NO), the CPU 16 temporarily ends the process.

[0028] In step S13, the CPU 16 determines whether or not the vehicle model information of the vehicle 10 has been transmitted to the mobile terminal 30. If the vehicle model information has been transmitted to the mobile terminal 30 (S13: YES), the CPU 16 shifts the process to step S17. On the other hand, if the CPU 16 has not yet transmitted the vehicle model information to the mobile terminal 30 (S13: NO), the CPU 16 shifts the process to step S15. In step S15, the CPU 16 causes the communication device 13 to transmit the vehicle model information to the mobile terminal 30. After that, the CPU 16 shifts the process to step S17.

[0029] In step S17, the CPU 16 acquires state variables of the vehicle 10. Specifically, the CPU 16 acquires detection values ​​of the various sensors 111 to 11N and processed values ​​obtained by processing the detection values ​​as state variables. For example, the CPU 16 acquires the traveling speed SPD of the vehicle 10, the acceleration G of the vehicle 10, the engine speed NE, and the engine torque Trq.

[0030] In step S19, the CPU 16 causes the communication device 13 to transmit the acquired state variables of the vehicle 10 to the mobile terminal 30. After that, the CPU 16 temporarily ends a series of processes. Fig. 2(B) illustrates a flow of processing executed by the CPU 41 of the terminal control device 39. The CPU 41 executes a control program stored in the first storage device 42, whereby the series of processing shown in Fig. 2(B) is repeatedly executed.

[0031] 2B, in step S31, the CPU 41 determines whether or not synchronization with the vehicle control device 15 has been established. If the CPU 41 determines that synchronization has been established (S31: YES), the CPU 41 shifts the process to step S33. On the other hand, if the CPU 41 determines that synchronization has not been established (S31: NO), the CPU 41 temporarily ends the process.

[0032] In step S33, the CPU 41 acquires the vehicle type information transmitted from the vehicle control device 15. In step S35, the CPU 41 starts recording by the microphone 35. In step S37, the CPU 41 starts acquiring the state variables of the vehicle 10 transmitted from the vehicle control device 15.

[0033] In step S39, the CPU 41 determines whether or not there is a signal indicating that an occupant of the vehicle 10 has sensed an abnormal noise generated in the vehicle 10. For example, if the occupant performs a predetermined operation on the mobile terminal 30, it is determined that there is a signal. Conversely, if the occupant does not perform the predetermined operation on the mobile terminal 30, it is determined that there is no signal. If the CPU 41 determines that there is a signal (S39: YES), it shifts the process to step S41. On the other hand, if the CPU 41 determines that there is no signal (S39: NO), it repeatedly executes the determination of step S39 until it determines that there is a signal.

[0034] Here, Fig. 3 illustrates an example of an abnormal sound generated in the vehicle 10. When an abnormal sound such as that shown in Fig. 3 occurs, the occupant of the vehicle 10 may feel uncomfortable by the abnormal sound. In such a case, the occupant may perform a predetermined operation on the mobile terminal 30. Note that the abnormal sound detected by the microphone 35 is superimposed on background noise as shown by the dashed line in Fig. 3. The background noise here includes road noise, wind noise, and the like.

[0035] Returning to FIG. 2B, in step S41, the CPU 41 starts storing a sound signal, which is a signal related to a sound detected by the microphone 35, and a state variable of the vehicle 10 acquired from the vehicle control device 15. At this time, the CPU 41 associates the sound signal with the state variable and stores them in the second storage device 43. In step S43, the CPU 41 determines whether or not a predetermined time has elapsed since the time when it was determined that the signal was present. If the elapsed time from the time has not reached the predetermined time, the CPU 41 determines that the predetermined time has not elapsed (S43: NO) and returns the process to step S41. That is, the CPU 41 continues the process of storing the sound signal and the state variable in the second storage device 43. On the other hand, if the elapsed time from the time has reached the predetermined time, the CPU 41 determines that the predetermined time has elapsed (S43: YES) and moves the process to step S45.

[0036] In step S45, the CPU 41 executes a transmission process. That is, in the transmission process, the CPU 41 causes the communication device 37 to transmit the time series data of the sound signal and the time series data of the state variables of the vehicle 10 stored in the second storage device 43 to the data analysis center 60. In addition, in the transmission process, the CPU 41 causes the communication device 37 to transmit the vehicle type information acquired in step S33 to the data analysis center 60. Then, when the CPU 41 completes the transmission, it temporarily ends the series of processes.

[0037] Fig. 4 shows the flow of processing executed by the CPU 64 of the center control device 63. The CPU 64 executes a control program stored in the first storage device 65, whereby the series of processing shown in Fig. 4 is repeatedly executed.

[0038] In step S61 of the series of processes, the CPU 64 determines whether or not the communication device 61 has received the data transmitted by the mobile terminal 30 to the data analysis center 60 in step S45 above. If the communication device 61 has received the data (S61: YES), the CPU 64 moves the process to step S63. On the other hand, if the communication device 61 has not received the data (S61: NO), the CPU 64 temporarily ends the series of processes.

[0039] In step S63, the CPU 64 acquires the vehicle type information of the vehicle 10 received by the communication device 61. In step S65, the CPU 64 acquires the time series data of the sound signal and the time series data of the state variables of the vehicle 10 received by the communication device 61.

[0040] In step S67, the CPU 64 inputs the time series data of the sound signal acquired in step S65 and the time series data of the state variables of the vehicle 10 to the mapping as input variables x. Then, in step S69, the CPU 64 acquires the output variable y output from the mapping.

[0041] In step S71, the CPU 64 estimates the cause of the sound sensed by the microphone 35 based on the output variable y acquired in step S69. Specifically, the CPU 64 selects the output variable with the largest value from the M output variables y(1), y(2), ..., y(M). Then, the CPU 64 uses the cause identification data 72 to estimate a candidate cause of the sound according to the selected output variable as an actual candidate.

[0042] In step S73, the CPU 64 causes the communication device 61 to transmit information relating to the estimated cause of the sound to the mobile terminal 30. After that, the CPU 64 temporarily ends a series of processes. When the CPU 41 of the terminal control device 39 acquires the information on the cause of the sound transmitted from the data analysis center 60, the CPU 41 notifies the occupant of the cause of the sound indicated by the information. For example, the CPU 41 displays the cause on the display screen 33.

[0043] <How to learn mapping> Referring to FIG. 5, a learning device 80 that applies machine learning to the mapping will be described. A sound signal, which is a signal related to a sound detected by microphone 35A, is input to learning device 80. A detection signal is also input to learning device 80 from learning detection system 11A. The sensor that constitutes learning detection system 11A is the same as the sensor that constitutes detection system 11 of vehicle 10.

[0044] The learning device 80 includes a CPU 81, a first storage device 82, and a second storage device 83. The first storage device 82 stores a control program executed by the CPU 81. The second storage device 83 stores mapping data 71a that defines a mapping, and factor identification data 72. The mapping defined by the mapping data 71a is a learning model for which machine learning has not been completed.

[0045] Prior to machine learning of the mapping, the learning device 80 acquires L data pairs each consisting of training data and teacher data. The training data includes time series data of a sound signal related to a sound detected by the microphone 35A and time series data of a state variable of the vehicle 10. The teacher data includes a cause of the abnormal sound detected by the microphone 35A.

[0046] An example of a sound signal related to a sound detected by the microphone 35A is shown in Fig. 8(A). As shown in Fig. 8(A), background noise is not superimposed on the sound signal related to the sound detected by the microphone 35A. In other words, the sound is detected by the microphone 35A under conditions in which the microphone 35A does not detect background noise.

[0047] A flow of processing executed by the learning device 80 to apply machine learning to a mapping will be described with reference to Fig. 6. That is, the mapping learning method in this embodiment includes causing the CPU 81 of the learning device 80 to execute a series of processing shown in Fig. 6. Note that the series of processing is repeatedly executed by the CPU 81 executing a control program stored in the first storage device 82.

[0048] 6, in step S91 of the series of processes, the CPU 81 acquires time series data of a sound signal and time series data of a state variable that constitute one data pair out of L data pairs prepared in advance. In step S93, the CPU 81 acquires the cause of the abnormal sound that is paired with the time series data of the sound signal and the time series data of the state variable acquired in step S91.

[0049] In step S95, the CPU 81 corrects the time-series data of the sound signal acquired in step S91. That is, the CPU 81 corrects the sound signal by superimposing a noise signal on the sound signal. Therefore, step S95 corresponds to a "signal correction process." The signal correction process will be described later.

[0050] When the CPU 81 completes the correction of the sound signal, the process proceeds to step S97. In step S97, the CPU 81 inputs the time series data of the sound signal corrected in step S95 and the time series data of the state variable acquired in step S91 to the mapping as the input variable x. Then, in step S99, the CPU 81 acquires the output variable y output from the mapping.

[0051] In step S101, the CPU 81 updates the mapping using the output variable y acquired in step S99 and the abnormal sound generation factor acquired in step S93. Here, the abnormal sound generation factor acquired as the teacher data in step S93 is referred to as a "learning factor". For example, the CPU 81 selects an output variable with the largest value from among M output variables y(1), y(2), ..., y(M). The CPU 81 selects a candidate for the generation factor according to the selected output variable by using the factor identification data 72. If the selected candidate differs from the learning factor, the CPU 81 adjusts various variables in the function approximator of the mapping so that the output variable corresponding to the learning factor among the output variables y(1) to y(M) becomes large. For example, if the learning factor is a candidate for the first generation factor, the CPU 81 adjusts various variables in the function approximator of the mapping so that the output variable y(1) becomes the largest among the output variables y(1) to y(M). In this embodiment, steps S97 to S101 correspond to an "update process" that updates the mapping by machine learning using the sound signal corrected by the signal correction process as training data and the cause of the abnormal sound paired with the sound signal as teacher data.

[0052] After updating the mapping, the CPU 81 temporarily ends the series of processes. In this embodiment, the CPU 81 changes the data pair used to update the mapping and executes the series of processes again. That is, the CPU 81 executes the series of processes L times.

[0053] The signal correction process will be described with reference to FIGS. Fig. 7 shows an example of a map MP that determines the amplitude of a noise signal to be superimposed on a sound signal. Fig. 8(A) shows a raw sound signal, which is a sound signal before correction. Fig. 8(B) shows an example of a noise signal to be superimposed on a raw sound signal. Fig. 8(C) shows a corrected sound signal, which is a corrected sound signal.

[0054] The CPU 81 uses the map MP to set the amplitude of the noise signal. The CPU 81 changes the amplitude of the noise signal depending on the frequency. Specifically, the CPU 81 makes the amplitude of the noise signal at a frequency equal to or lower than the specified frequency FqA larger than the amplitude of the noise signal at a frequency higher than the specified frequency FqA. In this embodiment, the frequency range equal to or lower than the specified frequency FqA corresponds to the "first frequency range", and the frequency range higher than the specified frequency FqA corresponds to the "second frequency range". In this case, the amplitude of the noise superimposed on the first frequency range of the sound signal is larger than the amplitude of the noise superimposed on the second frequency range of the sound signal.

[0055] Furthermore, the CPU 81 superimposes a plurality of noises with different amplitudes on the portion of the sound signal in the first frequency range. The amplitude of the noise signal in the first frequency range set in the map MP is set as a "first amplitude AM1." At this time, as shown in Fig. 8(B), the CPU 81 changes the amplitude of the noise signal within a range not exceeding the first amplitude AM1. For example, the CPU 81 randomly changes the amplitude of the noise signal so that the amplitude of the noise signal in the first period differs from the amplitude of the noise signal in the second period that is the period following the first period.

[0056] Similarly, the CPU 81 also superimposes a plurality of noises with different amplitudes on the portion of the sound signal in the second frequency range. The amplitude of the noise signal in the second frequency range set in the map MP is set as the "second amplitude AM2." At this time, as shown in FIG. 8(B), the CPU 81 changes the amplitude of the noise signal within a range not exceeding the second amplitude AM2. For example, the CPU 81 randomly changes the amplitude of the noise signal so that the amplitude of the noise signal in the third period differs from the amplitude of the noise signal in the fourth period, which is the period following the third period.

[0057] However, the CPU 81 generates the noise signal so that the average value of the amplitudes of multiple noises superimposed on the first frequency range portion of the sound signal is greater than the average value of the amplitudes of noises superimposed on the second frequency range portion of the sound signal.

[0058] Then, the CPU 81 generates a corrected sound signal as shown in FIG. 8(C) by superimposing the noise signal shown in FIG. 8(B) on the raw sound signal shown in FIG. 8(A). If the peak value of the raw sound signal is relatively small, the peak value of the actual abnormal sound may be buried in the noise signal when the noise signal is superimposed on the raw sound signal. Therefore, it is preferable that the CPU 81 corrects the amplitude of the noise signal according to the magnitude of the peak value of the raw sound signal. This makes it possible to prevent the peak value of the actual abnormal sound from being buried in the noise signal.

[0059] <Method of evaluating the mapping> A method for evaluating a mapping that has been subjected to machine learning using the above learning method will be described with reference to FIG.

[0060] In the first step S121, the sound indicated by the corrected sound signal, which is the sound signal corrected in step S95 above, is played. In the next step S123, it is determined whether or not the worker can confirm the played sound. If the worker can confirm the played sound (S123: YES), the process proceeds to step S125. On the other hand, if the worker cannot confirm the played sound (S123: NO), the process returns to step S121. In other words, the sound is repeatedly played until the worker can confirm the sound.

[0061] In step S125, the sound components corresponding to the cause of the abnormal sound estimated based on the output variable y output from the mapping are deleted from the raw sound signal. The sound signal generated in this way is called a "deleted sound signal."

[0062] In step S127, the sound indicated by the deleted sound signal is reproduced. In step S129, the worker is prompted to determine whether the abnormal sound has disappeared from the reproduced sound. If the worker determines that the abnormal sound has disappeared from the reproduced sound (S129: YES), the process proceeds to step S131. On the other hand, if the worker determines that the abnormal sound has not disappeared from the reproduced sound (S129: NO), the process proceeds to step S133.

[0063] In step S131, the cause of the abnormal noise is determined. In this case, the estimation result of the cause of the abnormal noise is correct. In step S133, the cause of the abnormal noise is semi-determined. In this case, the estimation result of the cause of the abnormal noise is incorrect. Therefore, it is preferable to modify the map MP shown in FIG. 7 and then perform the machine learning of the mapping described with reference to FIG. 6 again. The modification of the map MP here includes changing the specified frequency FqA and changing the magnitude of at least one of the first amplitude AM1 and the second amplitude AM2.

[0064] <Actions and Effects of the Present Embodiment> (1) The microphone 35 arranged in the passenger compartment of the vehicle 10 detects background noise as well as abnormal noise generated in the vehicle 10. Therefore, when estimating the cause of an abnormal sound generated in the vehicle 10, a sound signal including a background noise component is input to the mapping.

[0065] Therefore, in this embodiment, a noise signal is intentionally superimposed on a raw sound signal, which is a sound signal related to a sound detected by the microphone 35A, to generate a corrected sound signal as shown in Fig. 8(C). During machine learning of the mapping, the corrected sound signal is input to the mapping as an input variable x. That is, the sound signal used as training data contains a signal component equivalent to the above-mentioned background noise component.

[0066] Therefore, when estimating the cause of a sound based on the output variable y of a mapping that has been subjected to such machine learning when a sound signal related to a sound detected by the microphone 35 is input, the estimation accuracy of the cause of the sound is less likely to vary due to the influence of background noise. Therefore, a mapping can be generated in which the estimation accuracy of the cause of the sound is less likely to vary even if the sound detected by the microphone 35 contains background noise.

[0067] (2) The present inventors have found that, as shown in FIG. 3, the sound pressure level of background noise detected by the microphone 35 tends to be larger in the lower frequency range. Therefore, in the present embodiment, the amplitude of the low frequency range portion of the noise signal superimposed on the raw sound signal is made larger than the amplitude of the high frequency range portion. Therefore, the frequency characteristics of the sound signal corrected by the signal correction process can be made closer to the frequency characteristics of the sound signal related to the sound detected by the microphone 35 arranged in the passenger compartment of the vehicle 10. Therefore, by updating the mapping using the corrected sound signal, which is the sound signal corrected by the signal correction process, it is possible to generate a mapping in which the estimation accuracy of the cause of the sound is less likely to vary.

[0068] (3) The central control device 63 stores mapping data 71 that defines the mapping generated by the learning method described above. When a sound signal related to a sound detected by the microphone 35 is input as an input variable x to this mapping, the cause of the abnormal sound is estimated based on the output variable y of the mapping. Even if the sound signal input to the mapping as the input variable x contains background noise components, the cause of the abnormal sound can be estimated with high accuracy.

[0069] <Example of change> The above embodiment can be modified as follows: The above embodiment and the following modifications can be combined with each other to the extent that no technical contradiction occurs.

[0070] In the signal correction process, the amplitude of the noise signal may be set by using a map different from the map MP shown in Fig. 7. For example, in the signal correction process, the CPU 81 may set the amplitude of the noise signal using the map MP1 shown by the solid line in Fig. 10, or may set the amplitude of the noise signal using the map MP2 shown by the dashed line in Fig. 10. In this case, the CPU 81 can generate a noise signal whose amplitude gradually decreases as the frequency increases. Even in this case, it is preferable to make the amplitude of the noise signal superimposed on the part of the sound signal in the first frequency range larger than the amplitude of the noise signal superimposed on the part of the sound signal in the second frequency range.

[0071] Also, for example, the CPU 81 may set the amplitude of the noise signal using the map MP3 indicated by the dashed line in Fig. 10. In this case, the CPU 81 can generate a noise signal whose amplitude is the same regardless of the high or low frequency.

[0072] Also, for example, the CPU 81 may set the amplitude of the noise signal using a map MP4 indicated by a two-dot chain line in FIG. In the above embodiment, multiple noises with different amplitudes are superimposed on the portion of the sound signal in the first frequency range, but this is not limited to the above. For example, the amplitude of the noise signal superimposed on the portion of the sound signal in the first frequency range may be constant.

[0073] In the above embodiment, multiple noises with different amplitudes are superimposed on the portion of the sound signal in the second frequency range, but this is not limited to the above. For example, the amplitude of the noise signal superimposed on the portion of the sound signal in the second frequency range may be constant.

[0074] The neural network is not limited to a feedforward network with one hidden layer. For example, the neural network may be a network with two or more hidden layers, or may be a convolutional neural network or a recurrent neural network.

[0075] The trained model by machine learning does not have to be a neural network. For example, a support vector machine may be used as the trained model. The learning device 80 is not limited to having a CPU and a ROM and executing software processing. That is, the learning device 80 may have any one of the following configurations (a) to (c).

[0076] (a) The learning device 80 includes one or more processors that execute various processes according to a computer program. The processor includes a CPU and memory such as RAM and ROM. The memory stores program codes or instructions that are configured to cause the CPU to execute processes. Memory, i.e., computer-readable media, includes any available media that can be accessed by a general-purpose or special-purpose computer.

[0077] (b) The learning device 80 is equipped with one or more dedicated hardware circuits that execute various processes. Examples of dedicated hardware circuits include application specific integrated circuits, i.e., ASIC or FPGA. Note that ASIC is an abbreviation for "Application Specific Integrated Circuit" and FPGA is an abbreviation for "Field Programmable Gate Array."

[0078] (c) The learning device 80 includes a processor that executes some of the various processes in accordance with a computer program, and a dedicated hardware circuit that executes the remaining processes among the various processes.

[0079] <Technical philosophy> Next, the technical ideas that can be understood from the above-described embodiments and modified examples will be described as supplementary notes.

[0080] [Appendix 1] A method for learning a mapping used to estimate a cause of a sound generated in a vehicle based on a sound signal related to a sound detected by a microphone, comprising: the mapping is a learning model that outputs a variable indicating a cause of sound generation in the vehicle when the sound signal is input as an input variable, a signal correction process for correcting the sound signal by superimposing a noise signal on the sound signal; a learning method for a mapping, the learning device executing an update process for updating the mapping by machine learning using the sound signal corrected by the signal correction process as training data and the cause of sound generation paired with the sound signal as teacher data.

[0081] [Appendix 2] The method for learning a mapping described in Appendix 1, wherein the signal correction process is a process of correcting the sound signal so that an amplitude of a noise signal superimposed on a portion of the sound signal in a first frequency range is different from an amplitude of a noise signal superimposed on a portion of the sound signal in a second frequency range.

[0082] [Appendix 3] The first frequency range is a range having a lower frequency than the second frequency range, The mapping learning method according to claim 2, wherein the signal correction process is a process of correcting the sound signal so that an amplitude of noise superimposed on a portion of the sound signal in the first frequency range is greater than an amplitude of noise superimposed on a portion of the sound signal in the second frequency range.

[0083] [Appendix 4] The signal correction process is a process of superimposing a plurality of noises having different amplitudes on the portion of the sound signal in the first frequency range, The mapping learning method according to claim 2 or 3, wherein an average value of amplitudes of a plurality of noises superimposed on the portion of the sound signal in the first frequency range is greater than an average value of amplitudes of noises superimposed on the portion of the sound signal in the second frequency range.

[0084] [Appendix 5] The method for learning a mapping according to any one of Appendices 2 to 4, wherein the signal correction process is a process of superimposing a plurality of noises having different amplitudes on the portion of the sound signal in the second frequency range. [Explanation of symbols]

[0085] 10…Vehicle 15...Vehicle control device 30. Mobile terminal 35,35A…Microphone 39...Terminal control device 60…Data Analysis Center 63...Central control device 80...Learning device 81…CPU 83…Second storage device

Claims

1. A method for learning a mapping used to estimate a cause of a sound generated in a vehicle based on a sound signal related to a sound detected by a microphone, comprising: the mapping is a learning model that outputs a variable indicating a cause of sound generation in the vehicle when the sound signal is input as an input variable, a signal correction process for correcting the sound signal by superimposing a noise signal on the sound signal; an update process for updating the mapping by machine learning using the sound signal corrected by the signal correction process as training data and a generation factor of the sound paired with the sound signal as teacher data; and The signal correction process is a process of correcting the sound signal such that the amplitude of noise superimposed on a portion of the sound signal in a first frequency range is greater than the amplitude of noise superimposed on a portion of the sound signal in a second frequency range that is higher in frequency than the first frequency range. How to learn mappings.

2. the signal correction process is a process of superimposing a plurality of noises having different amplitudes on a portion of the sound signal in the first frequency range, An average value of amplitudes of a plurality of noises superimposed on the portion of the sound signal in the first frequency range is greater than an average value of amplitudes of noises superimposed on the portion of the sound signal in the second frequency range. The method for learning a mapping according to claim 1 .

3. The signal correction process is a process of superimposing a plurality of noises having different amplitudes on the portion of the sound signal in the second frequency range. The method for learning a mapping according to claim 2.

Images