Abnormal Sound Diagnostic System

Abstract

To improve the accuracy of diagnosing an allophone by reducing the influence of sounds other than the allophone in the diagnosis of an allophone.SOLUTION: The allophone diagnosis system according to the present disclosure is a diagnosis device for diagnosing an allophone generated in a vehicle on the basis of data of sound generated in the vehicle and diagnosis information regarding the allophone generated in the vehicle, and includes: an operation processing unit for acquiring a spectrogram which shows the relations among a time, a frequency, and a sound pressure from the sound data; a voice recognition unit for extracting a voice production time zone as a time zone including human voice from the sound data by sound recognition; a display unit for displaying the spectrogram; and a display control unit for displaying the spectrogram in the display unit so that the voice production time extracted by the voice recognition unit is visually distinguished from a time zone which does not include human voice.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to an abnormal sound diagnosis system that diagnoses abnormal sounds generated in a vehicle. [Background technology]

[0002] Conventionally, there is known a sound and vibration analyzer that performs analysis by taking in data on sound or vibration generated with the rotation of rotating bodies and data on the rotation speed of selected rotating bodies when the power transmission mechanism of a vehicle having multiple rotating bodies is in operation (see, for example, Patent Document 1). This sound and vibration analyzer performs frequency analysis on the sound or vibration data, and calculates an order according to the specifications of the rotating body from the frequency-analyzed sound or vibration data. Furthermore, in order to facilitate consistency with a sensory test, the sound and vibration analyzer displays the sound pressure level calculated from the sound or vibration data in a color-coded manner according to the magnitude, and in correspondence with the order and vehicle speed on a display unit. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2005-98984 A Summary of the Invention [Problem to be solved by the invention]

[0004] Here, sound data acquired while the vehicle is running or while a reproduction test is being performed in which the vehicle is run (operated) on a roadway or on a test bench to reproduce abnormal sounds includes data on sounds other than the abnormal sounds that should be acquired. Therefore, in order to improve the accuracy of abnormal sound diagnosis, it is necessary to reduce the influence of sounds other than the abnormal sound in the abnormal sound diagnosis.

[0005] Therefore, a primary object of the present disclosure is to further improve the accuracy of abnormal sound diagnosis by reducing the influence of sounds other than the abnormal sound in the diagnosis of the abnormal sound. [Means for solving the problem]

[0006] The abnormal sound diagnosis system disclosed herein is an abnormal sound diagnosis system including a diagnostic device that diagnoses abnormal sounds generated in a vehicle based on data of sounds emitted from the vehicle and medical interview information regarding the abnormal sound generated in the vehicle, and includes an arithmetic processing unit that obtains a spectrogram indicating the relationship between time, frequency, and sound pressure from the sound data, a voice recognition unit that extracts speaking time periods that are time periods that include human voices from the sound data using voice recognition, a display unit that displays the spectrogram, and a display control unit that displays the spectrogram on the display unit so that the speaking time periods extracted by the voice recognition unit are visually distinguished from time periods that do not include human voices.

[0007] In the abnormal sound diagnosis system of the present disclosure, the calculation processing unit obtains a spectrogram showing the relationship between time, frequency, and sound pressure from data of a sound emitted from a vehicle, and the voice recognition unit extracts a time period of speech including human voice from the sound data. Here, the frequency range of human voice is relatively wide, and the range corresponding to the time period of speech of the spectrogram contains a lot of noise, so it is not suitable as a diagnostic range of a diagnostic device. In light of this, in the abnormal sound diagnosis system of the present disclosure, the spectrogram is displayed on the display unit so that the time period of speech including human voice and the time period not including human voice are visually distinguished. This makes it possible to effectively prevent the user of the abnormal sound diagnosis system from selecting the range corresponding to the time period of speech of the spectrogram as the diagnostic range of the diagnostic device. As a result, it is possible to reduce the influence of human voice, which is a sound other than the abnormal sound, in the diagnosis of the abnormal sound, and to further improve the diagnostic accuracy of the abnormal sound.

[0008] The display control unit may make a range of the spectrogram corresponding to the speaking time period unselectable on the display unit.

[0009] This makes it possible to exclude the range of the spectrogram corresponding to the speaking time period from the diagnostic range of the diagnostic device.

[0010] Furthermore, the display control unit may be configured to, when the two speaking time periods are extracted by the voice recognition unit, display on the display unit a range between the two speaking time periods in the spectrogram as a diagnostic range of the diagnostic device.

[0011] When acquiring sound data for diagnosing an abnormal sound, human voices (conversations) in the vehicle cabin are often recorded immediately after the vehicle starts running (operating). In addition, if an abnormal sound occurs while acquiring sound data, a human voice reacting to the abnormal sound may be recorded immediately after the abnormal sound occurs. That is, when there are two speech time periods including human voices extracted by the voice recognition unit, the timing of the abnormal sound occurring is likely to be included in the range between the two speech time periods in the spectrogram. Therefore, if the range between the two speech time periods in the spectrogram is displayed in advance on the display unit as the diagnostic range of the diagnostic device, the diagnostic device can diagnose the diagnostic range after obtaining confirmation from the user of the abnormal sound diagnosis system, thereby obtaining a more accurate diagnostic result. In addition, in this aspect, it is also possible to determine the diagnostic range of the diagnostic device by using the voice of the operator as a trigger, by the operator speaking before and after the occurrence of the abnormal sound.

[0012] In addition, the display control unit may, when one or more of the vocalization time periods are extracted by the voice recognition unit, cause the display unit to display an instruction to select a diagnostic range of the diagnostic device from a range of the spectrogram corresponding to a time period other than the vocalization time period.

[0013] That is, when the speech recognition unit extracts one or three or more speaking time periods, if the diagnostic range of the diagnostic device is determined on the system side based on the speaking time periods, the diagnostic range may be too wide or may not include the timing of the occurrence of the abnormal sound. Therefore, in such a case, it is preferable to display an instruction on the display unit for selecting the diagnostic range of the diagnostic device from a range corresponding to a time period other than the speaking time period of the spectrogram. This makes it possible to allow the user of the abnormal sound diagnosis system to select the diagnostic range of the spectrogram, or to determine the diagnostic range on the system side based on the interview information, etc., according to the intention of the user of the abnormal sound diagnosis system.

[0014] Furthermore, the abnormal sound diagnosis system may include the display unit, a sound acquisition unit that acquires the sound data, a medical interview information acquisition unit that acquires the medical interview information, the arithmetic processing unit, the voice recognition unit, and the display control unit, as well as a portable terminal that communicates with the diagnostic device to exchange information.

[0015] This makes it possible to easily obtain sound data, reduces the load on the mobile terminal, and enables more accurate diagnostic results to be obtained from the diagnostic device.

[0016] In addition, the diagnostic device may include a diagnostic unit constructed by machine learning to diagnose the cause of an abnormal sound based on given information, and the voice recognition unit may be constructed by machine learning to extract speaking time periods including human voices from sound data. [Brief description of the drawings]

[0017] [Figure 1] 1 is a schematic configuration diagram showing an abnormal sound diagnosis system according to the present disclosure. [Diagram 2] FIG. 13 is an explanatory diagram illustrating an example of an input screen for medical interview information. [Diagram 3] 4 is a flowchart showing a series of processes executed in a terminal constituting the abnormal sound diagnosis system of the present disclosure. [Figure 4]FIG. 2 is an explanatory diagram showing an example of a spectrogram acquired by a terminal constituting the abnormal sound diagnosis system of the present disclosure. [Diagram 5] FIG. 2 is an explanatory diagram showing an example of a spectrogram displayed on a display unit of a terminal constituting the abnormal sound diagnosis system of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0018] Next, an embodiment of the present disclosure will be described with reference to the drawings.

[0019] 1 is a schematic diagram showing an abnormal sound diagnosis system 1 according to the present disclosure. The abnormal sound diagnosis system 1 shown in the figure is for diagnosing the cause of abnormal sound generated in a vehicle V, such as a vehicle equipped with only an engine as a power generation source, a hybrid vehicle (HEV, PHEV), or an electric vehicle (BEV, FCEV), and includes a mobile terminal 10 and a server 20 as a diagnostic device capable of exchanging information with the mobile terminal 10 through communication.

[0020] The mobile terminal 10 is used by a worker (user of the abnormal sound diagnosis system 1) at a vehicle dealership, a maintenance shop, or the like when responding to a user (owner) of the vehicle V in which an abnormal sound has occurred, or when performing a reproduction test in which the vehicle V is driven (operated) on a roadway or a test bench to reproduce the abnormal sound. In this embodiment, the mobile terminal 10 is a smartphone including a SoC, a ROM, a RAM, an auxiliary storage device (flash memory) M, a display unit 11, a communication module 12, a microphone (not shown), and the like. An abnormal sound diagnosis support application (program) is also installed in the mobile terminal 10. As shown in FIG. 1, the mobile terminal 10 includes a medical interview information acquisition unit 13, a sound acquisition unit 14, a vehicle state acquisition unit 15, a calculation processing unit 16, a voice recognition unit 17, a diagnosis range setting unit 18, and a display control unit 19, which are each constructed by cooperation between the abnormal sound diagnosis support application (software) and hardware such as the SoC of the mobile terminal 10.

[0021] The display unit 11 of the mobile terminal 10 includes a touch panel type liquid crystal panel or an organic EL panel. The communication module 12 exchanges various information with the electronic control device of the vehicle V via short-distance wireless communication or a cable (dongle), and can exchange various information with the server 20 via a network such as the Internet. The medical interview information acquisition unit 13 acquires information indicating the state of the vehicle V at the time of the occurrence of the abnormal sound (hereinafter referred to as "medical interview information") provided by the user of the vehicle V via the display unit 11. The sound acquisition unit 14 acquires time axis data of the sound (sound pressure) when the reproduction test is performed by the operator. The vehicle state acquisition unit 15 acquires information indicating the state of the vehicle V (hereinafter referred to as "vehicle state data") in synchronization with the acquisition of the time axis data of the sound by the sound acquisition unit 14 when the reproduction test is performed. The vehicle state data includes a plurality of physical quantities (vehicle speed, engine speed, etc.) corresponding to the items of the medical interview information.

[0022] The calculation processing unit 16 performs STFT (Short-Time Fourier Transform) on the time axis data of the sound acquired by the sound acquisition unit 14 to acquire a spectrogram (acoustic spectrogram) showing the relationship between time, frequency, and sound pressure. The voice recognition unit 17 extracts a time period including a human voice (hereinafter referred to as "vocalization time period") from the time axis data of the sound acquired by the sound acquisition unit 14 by voice recognition. In this embodiment, the voice recognition unit 17 includes a neural network (recurrent neural network (RNN) or LSTM (Long Short Term Memory)) constructed by machine learning so as to extract the vocalization time period from the time axis data of the sound. In addition, as a technology for constructing the voice recognition unit 17, for example, one described in JP 2018-40904 A can also be adopted. A diagnostic range setting unit 18 sets a range (hereinafter referred to as a "diagnostic range") to be diagnosed (analyzed) by the server 20 among the spectrograms acquired by the arithmetic processing unit 16 based on the extraction result of the voice recognition unit 17, the operation of the display unit 11 by the operator, etc. A display control unit 19 controls the display unit 11.

[0023] 2 shows an input screen (medical questionnaire) for inputting medical information displayed on the display unit 11 of the mobile terminal 10 and an example of input. As shown in a part of FIG. 2, the medical information includes vehicle identification information, order, date and time of occurrence, frequency of occurrence, location of the abnormal sound, type of sound, physical quantities that change while the vehicle V is traveling such as vehicle speed, driving state of the vehicle V, warm-up effect in an engine-equipped vehicle, selection items selected by the driver while driving the vehicle V, driving environment information of the vehicle V, and the like, and is input by the operator or the user of the vehicle V.

[0024] The vehicle identification information is information for identifying the vehicle V, such as the vehicle model and the vehicle identification number (chassis number). The order is a detailed content of the occurrence state of the abnormal sound provided by the user of the vehicle V. The occurrence frequency is selected from a drop-down list including multiple options such as always, several times / day, once / day, etc. The location of the abnormal sound is selected from a drop-down list including multiple vehicle parts such as the engine, power train, body, brakes, doors, and interior. The type of sound is selected by the worker from a drop-down list including multiple onomatopoeia (e.g., rattle, rattle, etc.) corresponding to any abnormal sound generated by the vehicle V, and the type of sound is selected by the worker from a drop-down list including multiple onomatopoeia (e.g., rattle, rattle, etc.) corresponding to any abnormal sound generated by the vehicle V, and the type of sound is recognized as being similar to the abnormal sound actually generated by the user of the vehicle V. The physical quantity is selected from a drop-down list including multiple options such as the vehicle speed, engine speed, ON / OFF time of the brake light switch, steering angle, and SOC of the high-voltage battery of a hybrid vehicle or an electric vehicle. The driving state of the vehicle V is selected from a drop-down list including multiple options such as starting, idling, stopping, starting, accelerating, constant speed driving, decelerating (brake OFF), braking (brake ON), reversing, and turning. The warm-up effect is selected from a drop-down list including cold, warm, cold and warm. The selection items are selected from a drop-down list including the shift position, driving mode, operation status of the auxiliary equipment, etc. The driving environment information is selected from a drop-down list including road surface conditions such as flat road, uphill road, downhill road, and weather, etc. It goes without saying that not all of the multiple items above are provided by the user of the vehicle V, and the medical interview information can be provided to the extent that the worker, etc. understands.

[0025] The server 20 of the abnormal sound diagnosis system 1 is a computer (information processing device) including a CPU, ROM, RAM, input / output devices, a communication module, etc., and is installed and managed, for example, by an automobile manufacturer that manufactures the vehicle V. The server 20 is configured with an abnormal sound diagnosis unit 21 that diagnoses abnormal sounds generated in the vehicle V by cooperation between hardware such as a CPU and a pre-installed abnormal sound diagnosis application. The abnormal sound diagnosis unit 21 includes a neural network (convolutional neural network) configured by supervised learning (machine learning) so as to diagnose the cause of the abnormal sound generated in the vehicle V and the part that is the source of the abnormal sound based on the interview information and sound time axis data acquired by the mobile terminal 10. Furthermore, in the server 20, when it is found that a new abnormal sound has occurred in the vehicle V, re-learning is performed for the abnormal sound diagnosis unit 21 using the sound time axis data acquired for the new abnormal sound and the contents of each item of the interview information as teacher data.

[0026] Furthermore, the server 20 includes a storage device 22 that stores a database that stores information on a plurality of abnormal sounds that are known to occur in a vehicle for each vehicle model. The database stores information such as time axis data of the sound, the cause of the abnormal sound, the part that is the source of the sound, the contents of the medical interview information provided by the user, etc., and measures to eliminate the abnormal sound, for each of the plurality of abnormal sounds, by linking them to each other. The server 20 also updates the database based on information acquired from a large number of vehicles including the vehicle V, and information on newly discovered abnormal sounds transmitted from the automobile manufacturer (developer, etc.), vehicle dealer, maintenance shop, etc. Note that, as a technology for constructing the abnormal sound diagnosis unit 21, for example, those described in the following papers (1)-(5) or a combination thereof can be used. (1)Unsupervised Filterbank Learning Using Convolutional Restricted Boltzmann Machine for Environmental Sound Classification (2)LEARNING FROM BETWEEN-CLASS EXAMPLES FOR DEEP SOUND RECOGNITION (3)Novel Phase Encoded Mel Filterbank Energies for Environmental Sound Classification (4)Knowledge Transfer from Weakly Labeled Audio using Convolutional Neural Network for Sound Events and Scenes (5)Novel TEO-based Gammatone Features for Environmental Sound Classification

[0027] Next, the abnormal sound diagnosis procedure performed by the abnormal sound diagnosis system 1 will be described.

[0028] When a worker at a vehicle dealership or a repair shop is requested by a user of a vehicle V to eliminate an abnormal sound, the worker listens to the inquiry information from the user and then executes a reproduction test to obtain information necessary for diagnosing the abnormal sound. When executing the reproduction test, the worker connects the mobile terminal 10 to an electronic control device of the vehicle V, and places or fixes the mobile terminal 10 or an external microphone connected to the mobile terminal 10 in an appropriate position on the vehicle V. The worker also starts an abnormal sound diagnosis support application and turns on the start switch of the vehicle V. Accordingly, the mobile terminal 10 acquires information such as the vehicle identification number or chassis number of the vehicle V from the electronic control device. Furthermore, the worker taps a recording start button displayed on the display unit 11 and drives (operates) the vehicle V on a roadway or a test stand to reproduce the driving state in which the abnormal sound occurred based on the inquiry information from the user of the vehicle V.

[0029] While the vehicle V is running (operating), the sound acquisition unit 14 of the mobile terminal 10 acquires time axis data of sound emitted from the vehicle V at predetermined time intervals (micro time) and stores the data in the auxiliary storage device M, and the vehicle state acquisition unit 15 acquires vehicle state data specified by the worker in response to medical interview information from the user of the vehicle V at predetermined time intervals (micro time) from the electronic control device of the vehicle V in synchronization with the acquisition of the sound time axis data by the sound acquisition unit 14 and stores the vehicle state data in the auxiliary storage device M. Then, when the worker taps the recording stop button displayed on the display unit 11, the acquisition of the sound time axis data and vehicle state data is completed, and the mobile terminal 10 executes a series of processes shown in FIG.

[0030] 3, after the reproduction test is completed, the calculation processing unit 16 and the voice recognition unit 17 of the mobile terminal 10 each acquire time axis data of the sound acquired by the sound acquisition unit 14 (step S100). The calculation processing unit 16 performs STFT on the acquired time axis data of the sound to acquire a spectrogram showing the relationship between time, frequency, and sound pressure (step S110). In addition, in step S110, the voice recognition unit 17 performs voice recognition processing on the acquired time axis data of the sound to extract a speaking time period including a human voice from the time axis data of the sound.

[0031] Next, the diagnostic range setting unit 18 of the mobile terminal 10 judges whether or not the speech time period has been extracted from the time axis data of the sound by the voice recognition unit 17 (step S120). If it is judged that the voice recognition unit 17 has extracted the speech time period (step S120: YES), the diagnostic range setting unit 18 judges whether or not the number of speech time periods extracted by the voice recognition unit 17 is two (step S130). If it is judged that the voice recognition unit 17 has extracted two speech time periods (step S130: YES), the diagnostic range setting unit 18 sets the range between the two speech time periods extracted by the voice recognition unit 17 in the spectrogram acquired by the calculation processing unit 16 as the diagnostic range to be diagnosed by the abnormal sound diagnosis unit 21 of the server 20. That is, when the calculation processing unit 16 acquires a spectrogram as shown in FIG. 4 and the time period from time t1 to time t2 and the time period from time t3 to time t4 are speaking time periods, the diagnostic range setting unit 18 sets the range of the spectrogram from time t2 to time t3 as the diagnostic range.

[0032] Also, in step S140, the diagnostic range setting unit 18 instructs the display control unit 19 of the mobile terminal 10 to display a spectrogram on the display unit 11. As shown in FIG. 5, the display control unit 19 causes the display unit 11 to display the spectrogram so that only the diagnostic range set by the diagnostic range setting unit 18 is visible by graying out two vocalization time periods (a time period from time t1 to time t2 and a time period from time t3 to time t4) extracted by the voice recognition unit 17. That is, the spectrogram is displayed on the display unit 11 so that vocalization time periods including human voices and time periods not including human voices are visually distinguished. In this embodiment, the spectrogram is a color map in which the horizontal axis is the time axis and the vertical axis is the frequency axis, and the relationship between time and sound pressure level is shown for each frequency by color-coding the sound pressure level.

[0033] Furthermore, the display control unit 19 displays a confirmation message on the display unit 11 to allow the operator to confirm whether the diagnostic range of the spectrogram is acceptable or not, and judges whether the diagnostic range has been adjusted by the operator (step S160). When it is judged that the diagnostic range has been adjusted (narrowed) by the operator (step S160: YES), the display control unit 19 acquires the diagnostic range adjusted by the operator, and displays the spectrogram on the display unit 11 so that only the diagnostic range selected by the operator is visible by graying out the portion not selected by the operator (step S170). When the adjustment of the diagnostic range by the operator is completed, the diagnostic range setting unit 18 determines the range adjusted by the operator as the diagnostic range of the spectrogram (step S180). Furthermore, when it is judged that the diagnostic range has not been adjusted by the operator (step S160: NO), the diagnostic range setting unit 18 determines the range set in step S140 as the diagnostic range of the spectrogram (step S180).

[0034] On the other hand, if the speech recognition unit 17 has not extracted a speaking time period from the time axis data of the sound (step S120: NO), and if the number of speaking time periods extracted by the speech recognition unit 17 is one or three or more (step S130: NO), the display control unit 19 causes the display unit 11 to display the spectrogram acquired by the calculation processing unit 16 and also causes the display unit 11 to display a message (instruction) for selecting a diagnostic range. In step S155, if the number of speaking time periods extracted by the speech recognition unit 17 is one or three or more, the display control unit 19 causes the display unit 11 to gray out the speaking time period extracted by the speech recognition unit 17 and display a message (instruction) for selecting a diagnostic range from a range corresponding to a time period other than the speaking time period of the spectrogram.

[0035] The operator selects (specifies) a diagnostic range of the spectrogram on the display unit 11 in response to a selection instruction on the display unit 11, and the display control unit 19 acquires the diagnostic range selected by the operator (step S175). Furthermore, in step S175, the display control unit 19 causes the display unit 11 to redisplay the spectrogram so that only the diagnostic range selected by the operator is visible by graying out the portions not selected by the operator. Furthermore, the diagnostic range setting unit 18 determines the range selected by the operator as the diagnostic range of the spectrogram (step S180).

[0036] After the diagnostic range is determined, the display control unit 19 causes the display unit 11 to display a message instructing the operator to input medical interview information, and the medical interview information acquisition unit 13 determines the information input by the operator as the final medical interview information after the operator has completed inputting the medical interview information (step S190). After the medical interview information is determined, an information transmission button is displayed on the display unit 11, and when the operator taps the information transmission button, information required for diagnosing the abnormal sound is transmitted from the communication module 12 of the mobile terminal 10 to the server 20 (step S200). In this embodiment, the information transmitted from the mobile terminal 10 to the server 20 includes the time axis data of the sound acquired by the sound acquisition unit 14, the vehicle state data acquired by the vehicle state acquisition unit 15, information defining the diagnostic range determined in step S180, and the medical interview information determined in step S190.

[0037] When the information required for diagnosing the abnormal sound is transmitted from the mobile terminal 10 to the server 20, the abnormal sound diagnosis unit 21 of the server 20 diagnoses the cause of the abnormal sound generated in the vehicle V based on the information provided from the mobile terminal 10, and transmits the diagnosis result to the mobile terminal 10. The diagnosis result includes the cause of the abnormal sound generated in the vehicle V, the part that is the source of the abnormal sound, and measures to eliminate the abnormal sound read from the storage device 22. When the diagnosis result from the server 20 is received by the mobile terminal 10 (step S210), the diagnosis result is displayed on the display unit 11 (step S220), and a series of processes executed by the mobile terminal 10 for diagnosing the abnormal sound is completed. By executing the process shown in FIG. 3, the worker can accurately explain the diagnosis result from the server 20 to the user of the vehicle V, and promptly proceed with measures to deal with the abnormal sound.

[0038] As described above, in the abnormal sound diagnosis system 1, the calculation processing unit 16 of the mobile terminal 10 acquires a spectrogram showing the relationship between time, frequency, and sound pressure from the time axis data of the sound emitted from the vehicle V (step S110). In addition, the voice recognition unit 17 extracts a speech time period including human voice from the time axis data of the sound (step S110). Here, the frequency range of human voice is relatively wide, and the range corresponding to the speech time period of the spectrogram includes a lot of noise, so it is not suitable as the diagnosis range of the server 20 as a diagnosis device, i.e., the abnormal sound diagnosis unit 21. In light of this, in the abnormal sound diagnosis system 1, the spectrogram is displayed on the display unit 11 so that the speech time period including the human voice is grayed out so that the speech time period and the time period not including the human voice are visually distinguished (steps S150, S155). This effectively prevents the operator (user of the abnormal sound diagnosis system 1) from selecting the range corresponding to the speech time period of the spectrogram as the diagnosis range of the server 20. As a result, the influence of sounds other than the abnormal sound, such as human voices, on the abnormal sound diagnosis can be reduced, thereby making it possible to further improve the accuracy of abnormal sound diagnosis.

[0039] Moreover, the display control unit 19 of the mobile terminal 10 makes the range of the spectrogram corresponding to the speaking time period unselectable on the display unit 11 by graying out the speaking time period. This makes it possible to exclude the range of the spectrogram corresponding to the speaking time period from the diagnosis range of the server 20.

[0040] Furthermore, when two vocalization time periods are extracted by the voice recognition unit 17, the display control unit 19 causes the display unit 11 to display the range between the two vocalization time periods in the spectrogram as the diagnosis range of the server 20 (steps S140-S150). When acquiring sound time axis data for diagnosing an abnormal sound, human voices (conversations) in the vehicle cabin are often recorded by the mobile terminal 10 immediately after the vehicle V starts running (operating). In addition, if an abnormal sound occurs while acquiring the sound time axis data, the voice of a person who reacts to the abnormal sound immediately after the abnormal sound occurs may be recorded by the mobile terminal 10. In other words, when two vocalization time periods are extracted by the voice recognition unit 17, the possibility that the timing of the abnormal sound occurs is high in the range between the two vocalization time periods in the spectrogram. Therefore, if the range between the two speech time periods in the spectrogram is displayed in advance on the display unit 11 as the diagnosis range of the server 20 (steps S140-S150), the abnormal sound diagnosis unit 21 can diagnose this diagnosis range after obtaining the operator's confirmation, thereby obtaining more accurate diagnosis results. In addition, in the abnormal sound diagnosis system 1, if the operator performing the test speaks before and after the occurrence of an abnormal sound, it becomes possible to determine the diagnosis range of the server 20 (abnormal sound diagnosis unit 21) using the operator's voice as a trigger.

[0041] Furthermore, when one or three or more speaking time periods are extracted by the voice recognition unit 17 (step S130: NO), the display control unit 19 of the mobile terminal 10 causes the display unit 11 to display a message for selecting the diagnostic range of the server 20 from a range corresponding to the time periods other than the speaking time periods of the spectrogram (step S155). In other words, when one or three or more speaking time periods are extracted by the voice recognition unit 17, if the diagnostic range of the server 20 is determined on the mobile terminal 10 side (system side) based on the speaking time periods, the diagnostic range may be too wide or may not include the timing of the occurrence of abnormal sounds. Therefore, in such a case, it is possible to allow the operator to select the diagnostic range of the spectrogram by displaying a message for selecting the diagnostic range of the server 20 from a range corresponding to the time periods other than the speaking time periods of the spectrogram on the display unit 11.

[0042] 3, at least one of the medical inquiry information and the vehicle condition data may be acquired by the mobile terminal 10 in addition to the time axis data of the sound, and at step S175, the diagnostic range may be set by the mobile terminal 10 based on at least one of the medical inquiry information and the vehicle condition data in response to an instruction (intention) from the worker. In this case, the estimated occurrence time period of the abnormal sound acquired based on at least one of the medical inquiry information and the vehicle condition data may be set as the diagnostic range of the spectrogram. In addition, a frequency range corresponding to the onomatopoeia selected in the medical inquiry information may be acquired as the estimated frequency range of the abnormal sound generated in the vehicle V, and the estimated frequency range may be transmitted to the server 20 together with the diagnostic range.

[0043] Furthermore, the abnormal sound diagnosis system 1 includes a display unit 11, a medical interview information acquisition unit 13 that acquires medical interview information, a sound acquisition unit 14 that acquires time axis data of sound, a calculation processing unit 16, a voice recognition unit 17, and a display control unit 19, as well as a mobile terminal 10 that exchanges information with a server 20 through communication. This makes it possible to easily acquire time axis data of sound and reduces the load on the mobile terminal 10, making it possible to obtain more accurate diagnosis results from the server 20. However, the abnormal sound diagnosis support application installed on the mobile terminal 10 may also be installed on a tablet terminal, a laptop or desktop personal computer, or the like, and the tablet terminal or the like may be used instead of the mobile terminal 10. The abnormal sound diagnosis system 1 may also be configured as a single information processing device such as a personal computer. [Industrial Applicability]

[0044] The invention disclosed herein is extremely useful for diagnosing abnormal noises occurring in a vehicle. [Explanation of symbols]

[0045] 1 abnormal sound diagnosis system, 10 mobile terminal, 11 display unit, 12 communication module, 13 medical interview information acquisition unit, 14 sound acquisition unit, 15 vehicle state acquisition unit, 16 calculation processing unit, 17 voice recognition unit, 18 diagnosis range setting unit, 19 display control unit, 20 server (information processing device), 21 abnormal sound diagnosis unit, 22 storage device, V vehicle.

Claims

1. An abnormal sound diagnosis system including a diagnosis device that diagnoses an abnormal sound generated in a vehicle based on data of a sound emitted from the vehicle and questioning information regarding the abnormal sound generated in the vehicle, A processor for acquiring a spectrogram showing the relationship between time, frequency, and sound pressure from the sound data; a voice recognition unit that extracts a speech time period that includes human voices from the sound data by voice recognition; A display unit that displays the spectrogram; a display control unit that causes the spectrogram to be displayed on the display unit so that the speaking time period extracted by the voice recognition unit is visually distinguished from time periods not including human voice; An abnormal sound diagnosis system comprising:

2. The abnormal sound diagnosis system according to claim 1, The display control unit makes a range of the spectrogram corresponding to the speaking time period unselectable on the display unit.

3. The abnormal sound diagnosis system according to claim 1 or 2, the display control unit, when the two speaking time periods are extracted by the voice recognition unit, causes the display unit to display a range between the two speaking time periods in the spectrogram as a diagnostic range of the diagnostic device.

4. The abnormal sound diagnosis system according to claim 3, when one or more of the speaking time periods have been extracted by the voice recognition unit, the display control unit causes the display unit to display an instruction for prompting the user to select a diagnostic range of the diagnostic apparatus from a range of the spectrogram corresponding to other than the speaking time periods.

5. The abnormal sound diagnosis system according to claim 1 or 2, An abnormal sound diagnosis system including the display unit, a sound acquisition unit for acquiring the sound data, a medical interview information acquisition unit for acquiring the medical interview information, the arithmetic processing unit, the voice recognition unit, and the display control unit, and also including a portable terminal for exchanging information with the diagnostic device via communication.

Images