Abnormal noise data processing method and abnormal noise data processing device for vehicle

Abstract

To effectively utilize collected sound data posteriorly.SOLUTION: An abnormal noise data processing device for a vehicle includes: a sound file database 2 that stores a large number of sound files containing sound data acquired for various components during the abnormal noise inspection in a completed vehicle inspection process; a sound-to-onomatopoeia conversion model 3 being a model that converts the sound data acquired in the sound file database 2 into an onomatopoeia; an onomatopoeia conversion unit 4 that utilizes the sound-to-onomatopoeia conversion model 3 to convert the sound data into the onomatopoeia; an onomatopoeia-abnormal noise level database 5 that includes the correspondence among the onomatopoeia, vehicle abnormal noise, and an abnormal noise occurrence possibility portion; an abnormal noise possibility level determination unit 6 that determines the abnormal noise possibility level of the sound data by referring to the known relationship between each onomatopoeia and the vehicle components or portions; and a result output unit 7 that outputs information including the sound data, the onomatopoeia, and the abnormal noise possibility level.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a method and device for acquiring sound data and onomatopoeia contained in the sound data during an inspection process for a vehicle such as an automobile, and outputting information including the sound data and the onomatopoeia. [Background technology]

[0002] For example, in order to efficiently solve problems related to abnormal vehicle noise, when the sound emitted from a vehicle is measured using a microphone during the final stage of a finished vehicle inspection on an automobile production line, a large amount of sound data is accumulated. In order to utilize such a large amount of sound data after the fact, some kind of data processing is required.

[0003] Patent Document 1 also discloses an environmental sound search method using onomatopoeias. In this search method, sounds made by a user are acquired as sound data, user onomatopoeias contained in the sound data are detected, and environmental sounds containing environmental onomatopoeias similar to the user onomatopoeias are extracted. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 6013951 Summary of the Invention [Problem to be solved by the invention]

[0005] The environmental sound search method of Patent Document 1 does not have the concept that the user's sound is an allophone, and merely extracts environmental sounds that contain environmental onomatopoeias similar to the user's onomatopoeia. [Means for solving the problem]

[0006] In the abnormal sound data processing method according to the present invention, sounds generated from a vehicle are acquired as sound data, and onomatopoeias contained in the sound data are detected. Furthermore, in this abnormal sound data processing method, an abnormal sound possibility level of the sound data is determined by referring to known associations between onomatopoeias and vehicle parts or locations, and information including the sound data, the onomatopoeia, and the abnormal sound possibility level is output. Effect of the Invention

[0007] According to the present invention, since the sound data is output in association with the onomatopoeia and the abnormal sound possibility level, the collected sound data can be effectively utilized afterwards. [Brief description of the drawings]

[0008] [Figure 1] FIG. 2 is a functional block diagram of the abnormal sound data processing device according to the first embodiment. [Diagram 2] FIG. 13 is an explanatory diagram illustrating an example of a part of an onomatopoeia-allophone level database. [Diagram 3] FIG. 2 is an explanatory diagram showing a display example in a result output unit in the first embodiment. [Figure 4] 4 is a flowchart showing a process flow in the first embodiment. [Diagram 5] FIG. 11 is a functional block diagram of an abnormal sound data processing device according to a second embodiment. [Figure 6] FIG. 13 is an explanatory diagram showing a display example in a result output unit in the second embodiment. [Figure 7] 13 is a flowchart showing a process flow according to a second embodiment. [Figure 8] FIG. 11 is a functional block diagram of an abnormal sound data processing device according to a third embodiment. [Figure 9] FIG. 13 is an explanatory diagram showing a display example in a result output unit according to the third embodiment. [Figure 10] 13 is a flowchart showing a process flow according to a third embodiment. [Figure 11] FIG. 13 is a functional block diagram of an abnormal sound data processing device according to a fourth embodiment. [Figure 12]FIG. 13 is an explanatory diagram showing a display example in a result output unit according to the fourth embodiment. [Figure 13] 13 is a flowchart showing a process flow according to a fourth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] An embodiment in which the present invention is applied to the finished vehicle inspection process, which is the final stage of an automobile production line, will be described below. In general, in the finished vehicle inspection process, inspectors test the finished vehicle on free rollers, inspecting various parts including the engine, brakes, etc. This inspection includes inspection of abnormal noises of the various parts.

[0010] 1 shows a functional block diagram of an allophone data processing device of Example 1. The allophone data processing device includes a search condition input unit 1, a voice file database 2, a sound-to-onomatopoeia conversion model 3, an onomatopoeia conversion unit 4, an onomatopoeia-allophone level database (allophone database) 5, an allophone possibility level determination unit 6, and a result output unit 7.

[0011] The search condition input unit 1 has a function of inputting search conditions for extracting desired sound data from the sound file database 2 in which a large amount of sound data collected daily is stored. The search condition input unit 1 is provided on a device (not shown) for presenting visual information, such as a liquid crystal display or an organic EL display. In this embodiment, the search conditions are the date and time range of the sound data to be searched and the abnormal sound possibility level of the search target, which will be described later. Here, the date and time range can be a range from a certain date and time to another date and time. The "date and time" refers to the date and time when the sound file was generated, and the sound file itself may contain information on the date and time of generation, or the file name, data header, footer, etc. may contain information on the date and time of sound acquisition. In addition, when inputting the date and time range, only the date may be input without entering the time. In another example, in order to investigate the trend of abnormal sounds on a daily basis in the same time zone, for example, "18:00 to 21:00 in July 2022" may be input as the date and time range. Furthermore, to further narrow down the search conditions, a day of the week condition may be added, for example, by inputting "18:00 to 21:00 on Monday to Friday in July 2022." When inputting the abnormal sound possibility level, an upper limit or lower limit of the abnormal sound possibility level, for example, "2" or more or "2" or less, is used. Instead of using the upper limit or lower limit of the abnormal sound possibility level, a range of the abnormal sound possibility level, for example, "2 to 3," may be input. In this embodiment, the content input to the search condition input unit 1 is "2022 / 6 / 1 to 6 / 2, abnormal sound possibility level 2 or more."

[0012] The sound file database 2 is constructed by accumulating a large number of sound files including sound data acquired for each of various parts such as engines and brakes during abnormal sound inspection in the finished vehicle inspection process. The sound data is acquired by collecting sounds using a microphone, which is a sound acquisition unit, during abnormal sound inspection of each of various parts, and acquiring the collected sounds as sound data. Here, after acquiring sound data during abnormal sound inspection of each part, the sound of the part to be inspected is extracted from the sound data and cut out as sound data of the part. The sound acquired by the microphone may include sounds other than the part, and the sound is separated and extracted from the sound. For example, the sound of the part is identified using a known method such as FFT (Fast Fourier Transform) or wavelet analysis. The sound file database 2 is provided, for example, in a dedicated server or a storage means on the cloud.

[0013] Moreover, each sound data collected from various parts is linked to the ID of the vehicle. The sound data is used after the fact to verify abnormal sounds in a specific time period based on the search conditions input in the search condition input unit 1. Moreover, the sound data is not simply collected to verify abnormal sounds after the fact, but may be collected for after-the-fact verification during a so-called sensory inspection, which is an inspection based on the senses of an inspector. In this case, the inspector performs a search after the fact based on, for example, a memory that an abnormal sound was heard during an abnormal sound inspection of a specific part performed at a specific time on a certain date and time, and searches for an onomatopoeia corresponding to the abnormal sound from the result output unit 7. Moreover, the abnormal sound inspection may be one that inspects abnormal sounds in real time using AI technology. Examples of sources of abnormal sounds include the engine, brakes, alternator, tires, belts, etc.

[0014] The sound-onomatopoeia conversion model 3 is a model for converting sound data acquired in the audio file database 2 into onomatopoeia. In other words, the sound-onomatopoeia conversion model 3 is a model for attaching a label to each piece of sound data indicating which onomatopoeia it is. In the sound-onomatopoeia conversion model 3, the total number of onomatopoeias (excluding human voices and animal cries) listed in a general onomatopoeia dictionary that includes multiple onomatopoeias, 398 in this embodiment, is defined. All of these 398 onomatopoeias are written in Japanese kana. The sound-onomatopoeia conversion model 3 utilizes a method using a spectrogram envelope and reverberation time. For example, for sound data having a vibration characteristic of a chatter of 1 to 4 kHz, it matches the characteristics of the sound "key", so that the sound data having the above vibration characteristic can be made to correspond to the onomatopoeia "key".

[0015] The onomatopoeia conversion unit 4 acquires sound data for the date and time range input to the search condition input unit 1 from the audio file database 2, and converts the sound data into onomatopoeias using the sound-onomatopoeia conversion model 3. In other words, the onomatopoeia conversion unit 4 functions as a detection unit that detects onomatopoeias in the sound data of the audio files extracted from the audio file database 2 based on the search conditions. The onomatopoeia conversion unit 4 is also configured to be able to arbitrarily change the time unit for converting the sound data into onomatopoeias depending on the usage situation.

[0016] The onomatopoeia-abnormal sound level database 5 stores information including a correspondence between a plurality of types of onomatopoeia prepared in advance, the above-mentioned 398 types of onomatopoeia (onomatopoeia labels) in this embodiment, information indicating whether each onomatopoeia corresponds to an abnormal vehicle sound ("Whether it is an abnormal vehicle sound" in FIG. 2), and if it corresponds to an abnormal vehicle sound, which part of the vehicle each onomatopoeia is likely to represent ("Possible part of abnormal sound occurrence" in FIG. 2). This information is created from materials summarizing the correlation between onomatopoeia indicating abnormal sounds and the occurrence conditions of abnormal sounds, such as materials from automobile-related academic societies and educational materials from automobile mechanic schools.

[0017] FIG. 2 is an explanatory diagram showing an example of a part of the onomatopoeia-abnormal sound level database 5, and shows "whether it is an abnormal vehicle sound" and "possible abnormal sound generating parts" for four of the 398 onomatopoeias, namely "katakata", "key", "pee" and "tonton". Here, in the example shown in FIG. 2, "katakata", "key" and "tonton" which are onomatopoeias related to vehicles, and vehicle parts or parts which may be the source of each onomatopoeia are collected, and further, for "pee", which is an onomatopoeia not related to vehicles, the source is left blank, and the database is constructed in such a way that. Note that FIG. 2 actually contains information on 137 onomatopoeias related to vehicles, including "katakata", "key" and "tonton", and 261 onomatopoeias not related to vehicles, including "pee".

[0018] As shown in FIG. 2, when the onomatopoeia "katakataka" is detected, it is determined that the noise is an abnormal vehicle noise, and it is highly likely that this abnormal noise is generated by the alternator or engine. When the onomatopoeia "kii" is detected, it is determined that the noise is an abnormal vehicle noise, and it is highly likely that this abnormal noise is generated by the brakes or tires. When the onomatopoeia "pee" is detected, it is determined that the noise is not an abnormal vehicle noise. When the onomatopoeia "tonton" is detected, it is determined that the noise is an abnormal vehicle noise, and it is highly likely that this abnormal noise is generated by the engine.

[0019] The abnormal sound possibility level determination unit 6 determines the abnormal sound possibility level of the sound data by referring to the known association between each onomatopoeia converted (detected) by the onomatopoeia conversion unit 4 and a vehicle part or part. The abnormal sound possibility level determination unit 6 determines whether the detected onomatopoeia exists in the onomatopoeia-abnormal sound level database 5 by referring to the onomatopoeia-abnormal sound level database 5. If the onomatopoeia exists, the abnormal sound possibility level determination unit 6 calculates the abnormal sound possibility level indicated by a numerical value. For example, a reference frequency characteristic is determined as a reference for each onomatopoeia, and the abnormal sound possibility level is calculated based on the degree of deviation from this reference frequency characteristic. For example, when a reference is determined as the frequency characteristic of "katakata" that "a fluctuating sound of about 10 Hz continues," the abnormal sound possibility level is set to the maximum value ("3" in this embodiment) when a fluctuating sound of 10 Hz is detected, and the abnormal sound possibility level is decreased as the fluctuating sound moves away from 10 Hz, thereby calculating the abnormal sound possibility level.

[0020] Moreover, if the detected onomatopoeia does not exist in the onomatopoeia-alien sound level database 5, the abnormal sound possibility level determining unit 6 sets the abnormal sound possibility level to a relatively low value, 0 in this embodiment.

[0021] The result output unit 7 outputs information including the search conditions, in this embodiment "2022 / 6 / 1-6 / 2, abnormal sound possibility level 2 or higher", as well as the search results, for example, the date and time, the audio file name including the sound data, the detected onomatopoeia, and the abnormal sound possibility level. The result output unit 7 is provided in a device (not shown) for presenting visual information, for example, a liquid crystal display or an organic EL display. The search results are displayed in list form as shown in FIG. 3, and in this embodiment, are displayed in order of the generation date and time of the audio file. Also, instead of displaying the search results in order of the generation date and time of the audio file, the search results may be sorted by other items of the search results, for example, in order of the high abnormal sound possibility level or the Japanese syllabary order of the onomatopoeia. Also, the search results may be sorted by using the abnormal sound possibility level and the onomatopoeia in combination.

[0022] In this embodiment, rows containing abnormal sound possibility levels that match the search conditions are displayed more emphasized than rows containing abnormal sound possibility levels that do not match the search conditions. More specifically, as shown in FIG. 3, the background of rows containing abnormal sound possibility levels that match the search conditions is shown in white, while the background of rows containing abnormal sound possibility levels that do not match the search conditions is shown in gray. When displaying the search results, rows containing abnormal sound possibility levels that do not match the search conditions may be shaded or deleted. Alternatively, multiple rows containing abnormal sound possibility levels that match the search conditions may be arranged together at the top of the screen of the result output unit 7, and multiple rows containing abnormal sound possibility levels that do not match the search conditions may be arranged together at the bottom of the screen of the result output unit 7. Alternatively, the characters in the rows containing abnormal sound possibility levels that match the search conditions may be displayed larger or in bold.

[0023] The search results may be displayed in the form of icons instead of in a list format. In this case, for example, if the search conditions are met, the icon is displayed in red, whereas if the search conditions are not met, the icon is displayed in a neutral color.

[0024] FIG. 4 is a flow chart showing the flow of processing performed by the abnormal sound data processing device of the first embodiment.

[0025] First, in step S101, search conditions (date and time range to be searched, abnormal sound possibility level), in this embodiment, "2022 / 6 / 1 to 6 / 2, abnormal sound possibility level 2 or higher" are input.

[0026] Next, in step S102, sound data that meets the search conditions is obtained from the sound file database 2.

[0027] Then, the process proceeds to step S103, where the sound data is converted into onomatopoeias using the sound-onomatopoeia conversion model 3. That is, onomatopoeias are detected from the sound data of the sound files extracted from the sound file database based on the search conditions.

[0028] After the onomatopoeia is detected, in step S104, it is determined whether or not the detected onomatopoeia exists in the onomatopoeia-altered sound level database 5. If the detected onomatopoeia does not exist in the onomatopoeia-altered sound level database 5, the process proceeds to step S105, where the abnormal sound possibility level is set to a relatively low value, 0 in this embodiment.

[0029] Furthermore, if it is determined in step S104 that the detected onomatopoeia exists in the onomatopoeia-altere sound level database 5, the process proceeds to step S106, in which the abnormal sound possibility level of the sound data determined to be an onomatopoeia existing in the onomatopoeia-altere sound level database 5 is obtained.

[0030] Next, in step S107, the voice file names are sorted according to the search conditions. In this embodiment, the voice file names are sorted in order of the search target date and time, that is, in order of the file creation date and time.

[0031] Then, in step S108, the file creation date and time, the audio file name, the onomatopoeia, and the abnormal sound possibility level are displayed as the search results together with the search conditions. That is, the screen shown in FIG.

[0032] As described above, in the first embodiment, the sound generated from the vehicle is acquired as sound data using a microphone, and the onomatopoeia contained in the sound data is detected. Then, the abnormal sound possibility level of the sound data is determined by referring to the known association between the onomatopoeia and the vehicle parts or parts, and the result output unit 7 outputs information including the sound data, the onomatopoeia, and the abnormal sound possibility level. Therefore, the onomatopoeia and the abnormal sound possibility level are linked to the sound data and output, so that the collected sound data can be effectively used after the fact. When using the sound data after the fact, the worker engaged in the finished vehicle inspection can intuitively and sensuously search for onomatopoeia related to the abnormal sound of the vehicle. Moreover, such a search for abnormal sounds using onomatopoeia is useful for sharing intuition and sensuality with other workers, so that information exchange with other workers can be easily performed.

[0033] In this embodiment, the sound file database 2 is constructed by accumulating a large number of sound files including sound data acquired for each of various vehicle parts during the abnormal sound inspection in the finished vehicle inspection process. Then, onomatopoeia detection is performed on the sound data of the sound files extracted from the sound file database based on the search conditions. As a result, onomatopoeias are detected for each of the multiple sound data, and the result output unit 7 outputs information including each sound data, each onomatopoeia, and each abnormal sound possibility level. Therefore, the worker can grasp all the onomatopoeias related to each abnormal sound of the vehicle by checking only once after the fact, so that the time required for searching for abnormal sounds can be shortened compared to the case of directly listening to each of the multiple sound files to search for abnormal sounds.

[0034] Furthermore, in this embodiment, the onomatopoeia-abnormal sound level database 5 is constructed in a form that compiles onomatopoeias related to a vehicle and vehicle parts or portions that may be the source of the onomatopoeia. The abnormal sound possibility level determination unit 6 then determines the abnormal sound possibility level by referring to the onomatopoeia-abnormal sound level database 5. Therefore, it is possible to efficiently calculate the abnormal sound possibility level after knowing that the onomatopoeia is related to an abnormal sound of the vehicle and further identifying the source of the onomatopoeia.

[0035] Furthermore, in this embodiment, the abnormal sound possibility level determination unit 6 sets the abnormal sound possibility level to a relatively low value if the detected onomatopoeia does not exist in the onomatopoeia-abnormal sound level database 5. In this case, since the detected onomatopoeia is not associated with the vehicle's abnormal sound, by setting the abnormal sound possibility level to a low value, the operator can eliminate information containing this low value and improve search efficiency.

[0036] FIG. 5 is a functional block diagram of the abnormal sound data processing device of the second embodiment.

[0037] The abnormal sound data processing device of the second embodiment is configured by adding a sound pressure level acquiring section 8 between the search condition input section 1 and the onomatopoeia converting section 4 of the abnormal sound data processing device of the first embodiment.

[0038] The sound pressure level acquisition unit 8 acquires each sound data in the date and time range from the audio file database 2, and acquires an average sound pressure level for each sound data. The average sound pressure level is acquired, for example, in the same way as a general sound level meter, by applying a frequency filter such as an A-weighting based on the human hearing characteristics to change the sound pressure and then calculating the average sound pressure level. The average sound pressure level may also be acquired by calculating the average sound pressure level using sampling data corresponding to the time characteristics. Instead of the average sound pressure level, the sound pressure level acquisition unit 8 may acquire a maximum sound pressure level, a minimum sound pressure level, or other sound pressure levels according to the situation of the abnormal sound inspection. For example, during the abnormal sound inspection of an alarm, a feature is that a loud sound occurs only for a moment, so the maximum sound pressure level when this feature appears is acquired.

[0039] In the second embodiment, the abnormal sound possibility level determination unit 6 corrects the abnormal sound possibility level based on the average sound pressure level acquired by the sound pressure level acquisition unit 8. The abnormal sound possibility level determination unit 6 judges whether the average sound pressure level exceeds a predetermined specified value. In this embodiment, the predetermined specified value is "+30 dB". The reason why the predetermined specified value is set to +30 dB is based on the fact that a sound pressure level of +30 dB or less is a volume that is difficult for humans to recognize, and therefore there is a high possibility that humans will not be able to recognize it as an abnormal sound. In addition, instead of +30 dB, the predetermined specified value may be set to "+50 dB", which is the noise level in a residential area late at night. In addition, the specified value is not limited to +30 dB or +50 dB, and can be arbitrarily changed depending on the external environment, the type of abnormal sound to be found, and the like. When the average sound pressure level is +30 dB or less, the abnormal sound possibility level determination unit 6 corrects the abnormal sound possibility level to be one level lower. Furthermore, when the average sound pressure level exceeds +30 dB, the abnormal sound possibility level determination unit 6 maintains the abnormal sound possibility level without correcting it. Also, when the average sound pressure level exceeds +30 dB, instead of maintaining the abnormal sound possibility level, it may correct it to be one level higher.

[0040] Also, instead of using one predetermined specified value as a comparison standard with the average sound pressure level, the specified value may be divided into a plurality of stages. In one example, the specified value may be divided into three stages, for example, "less than +30 dB", "+30 dB to +50 dB", and "greater than +50 dB". In this case, if the average sound pressure level is less than +30 dB, the abnormal sound possibility level is corrected to be one stage lower, if the average sound pressure level is +30 dB to +50 dB, the abnormal sound possibility level is maintained without correction, and if the average sound pressure level is greater than 50 dB, the abnormal sound possibility level is corrected to be one stage higher.

[0041] As shown in FIG. 6, the result output unit 7 outputs a sound pressure level comparison with normal environmental sounds (average sound pressure level acquired by the sound pressure level acquisition unit 8) as a search result in addition to the date and time, the audio file name, the detected onomatopoeia, and the abnormal sound possibility level.

[0042] FIG. 7 is a flow chart showing the flow of processing by the abnormal sound data processing device of the second embodiment.

[0043] In step S101, search conditions (date and time range to be searched, abnormal sound possibility level) are input, and in step S102, sound data that meets the search conditions is obtained from the sound file database 2.

[0044] Next, in step S201, an average sound pressure level is obtained for each sound data. The average sound pressure level is obtained by calculating the average sound pressure level using a frequency filter such as the above-mentioned A-weighting, or by calculating the average sound pressure level using sampling data corresponding to the above-mentioned time characteristics.

[0045] After obtaining the average sound pressure level, in step S103, the sound data is converted into an onomatopoeia using the sound-onomatopoeia conversion model 3, and in step S104, it is determined whether or not the detected onomatopoeia exists in the onomatopoeia-altered sound level database 5. Depending on the determination result, in step S105, the abnormal sound possibility level is set to 0, or in step S106, the abnormal sound possibility level of the sound data determined to be an onomatopoeia existing in the onomatopoeia-altered sound level database 5 is obtained.

[0046] Then, in step S202, it is determined whether the average sound pressure level of each sound data exceeds +30 dB. If the average sound pressure level is +30 dB or less, the process proceeds to S203, where the abnormal sound possibility level is corrected to be one level lower.

[0047] If it is determined in step S202 that the average sound pressure level exceeds +30 dB, the process proceeds to step S107, where the audio file names are rearranged in accordance with the search conditions.

[0048] Then, in step S204, the search results are displayed, including the file creation date and time, the audio file name, the detected onomatopoeia, the average sound pressure level, and the abnormal sound possibility level.

[0049] As described above, in the second embodiment, the abnormal sound possibility level determination unit 6 corrects the abnormal sound possibility level based on the average sound pressure level acquired by the sound pressure level acquisition unit 8. In the first embodiment, it was possible to grasp the type of abnormal sound, but in the second embodiment, it is possible to grasp the loudness of the abnormal sound as well. Therefore, a "sound pressure level comparison with normal environmental sound" indicating the sound pressure level is added as an output result output to the result output unit 7, making it possible to perform a quantitative and detailed search for audio files.

[0050] In addition, in the first embodiment, even if the abnormal sound is small enough that humans do not notice it, if an onomatopoeia related to the abnormal sound of an automobile is detected, there is a risk that the abnormal sound possibility level will be set high. However, in the second embodiment, by correcting the abnormal sound possibility level based on the average sound pressure level, it is possible to reduce the abnormal sound possibility level for the abnormal sound that is small enough that humans do not notice it, thereby suppressing overdetection of abnormal sounds. Furthermore, it is possible to obtain search results that are adapted to human senses, such as preventing the operator from overlooking an abnormal sound when the abnormal sound possibility level would be set low if the abnormal sound were judged based on onomatopoeia, but an abnormal sound is actually very loud.

[0051] FIG. 8 is a functional block diagram of an abnormal sound data processing device according to the third embodiment.

[0052] The abnormal sound data processing device of the third embodiment is configured by adding a vehicle (CAN) signal acquisition unit 9 between the onomatopoeia conversion unit 4 and the abnormal sound possibility level determination unit 6 of the abnormal sound data processing device of the first embodiment.

[0053] In the third embodiment, when acquiring sound data in an abnormal sound inspection of each part of a vehicle during a finished vehicle inspection, information on vehicle signals, such as one or more signals related to an engine, brake, motor, horn, muffler, oil, connector, or belt, is acquired, and the acquired signals are linked to the sound data. For example, when the vehicle signal is a signal related to an engine, the engine speed is used as the vehicle signal. Also, when the vehicle signal is a signal related to a brake, whether or not the brake is depressed is used as the vehicle signal.

[0054] In addition to the date and time range of the search target and the possibility level of the abnormal sound of the search target, the vehicle state of the search target is input to the search condition input unit 1 as a search condition. Here, the "vehicle state of the search target" indicates the state of the vehicle and driving operation at the time of acquiring the sound data. The vehicle state of the search target is, for example, one of the maximum value, minimum value, median value, average value, and fluctuation range of a specific vehicle signal within a predetermined period at the time of acquiring the sound data. When the vehicle state is a fluctuation range, the vehicle state is, for example, an engine speed of 3000 to 5000 rpm. In addition, the vehicle state of the search target may not be the one value, but may be the state of change of the vehicle signal, for example, an upward trend, a flat trend, a downward trend, etc. In this case, the vehicle state is, for example, a state in which the engine speed is on the upward trend. The vehicle state of the search target may be a combination of one value of a specific vehicle signal and information on another vehicle signal. In this case, the vehicle state is, for example, a state in which the engine speed is 5000 rpm or less and the brake is not depressed. In this embodiment, the vehicle condition to be searched for that is input to the search condition input unit 1 is "engine speed 2000 to 4000 rpm, no brake applied."

[0055] The vehicle (CAN) signal acquisition unit 9 acquires, by CAN communication, a vehicle signal linked to the past sound data extracted according to the search conditions.

[0056] The abnormal sound possibility level determination unit 6 sets the abnormal sound possibility level to a value higher than the abnormal sound possibility level when it is determined that the onomatopoeia detected by the onomatopoeia conversion unit 4 does not exist in the onomatopoeia-abnormal sound level database 5 in relation to the part of the vehicle from which the sound data was acquired. As described above, a vehicle signal is also acquired when the sound data is acquired, and this vehicle signal is linked to the sound data, so that the part of the vehicle from which the sound data was acquired can be specified. When setting the abnormal sound possibility level, for example, the onomatopoeia "key" extracted from the sound data acquired in the brake inspection is likely to be an abnormal sound related to the brakes, so the abnormal sound possibility level is set to "2". In addition, when the onomatopoeia "katakata" is detected, although it is an abnormal sound related to the automobile, it is less related to abnormal sounds related to the brakes, so the abnormal sound possibility level is set to "1".

[0057] Furthermore, the abnormal sound possibility level determination unit 6 corrects the abnormal sound possibility level based on whether or not the vehicle signal acquired when acquiring the sound data corresponds to the vehicle signal acquired by the vehicle (CAN) signal acquisition unit 9. For example, if the vehicle signal acquired when acquiring the sound data is "engine speed 3000 rpm, brake not applied," and the vehicle signal acquired by the vehicle (CAN) signal acquisition unit 9 according to the input conditions is "engine speed 2000-4000 rpm, brake not applied," it is determined that the vehicle signal acquired when acquiring the sound data corresponds to the vehicle signal of the vehicle (CAN) signal acquisition unit 9. Accordingly, the abnormal sound possibility level determination unit 6 maintains the abnormal sound possibility level without correcting it. Note that the abnormal sound possibility level determination unit 6 may correct the abnormal sound possibility level to be one step higher rather than maintaining it. Also, for example, if the vehicle signal acquired when acquiring sound data is "engine speed 5000 rpm, brake not applied," and the vehicle signal acquired by vehicle (CAN) signal acquisition unit 9 in accordance with the input conditions is "engine speed 2000-4000 rpm, brake not applied," it is determined that the vehicle state input as the search condition does not match the vehicle signal of vehicle (CAN) signal acquisition unit 9. Accordingly, abnormal sound possibility level determination unit 6 corrects the abnormal sound possibility level to be one level lower.

[0058] As shown in FIG. 9, the result output unit 7 outputs the engine speed and the brake (whether the brake is applied) as search results in addition to the date and time, the voice file name, the detected onomatopoeia, and the abnormal sound possibility level. When outputting, the search results may be sorted according to the vehicle signal. In this case, for example, the search results may be sorted so that the closer to 3000 rpm, the midpoint of the engine speed range of 2000 to 4000 rpm, the higher the list, and the further from the midpoint the lower the list. Also, instead of sorting the search results according to the vehicle signal, the search results may be sorted using a combination of the vehicle signal and the abnormal sound possibility level.

[0059] FIG. 10 is a flow chart showing the flow of processing performed by the abnormal sound data processing device of the third embodiment.

[0060] In step S301, search conditions (date and time range to be searched, abnormal noise possibility level, vehicle condition) are input.

[0061] Next, in step S102, sound data that meets the search criteria is obtained from the sound file database 2.

[0062] Then, in step S302, a vehicle signal linked to the past sound data extracted according to the search conditions is acquired by CAN communication.

[0063] After acquiring this vehicle signal, in step S103, the sound data is converted into an onomatopoeia using the sound-onomatopoeia conversion model 3, and in step S104, it is determined whether or not the detected onomatopoeia exists in the onomatopoeia-abnormal sound level database 5. Depending on the determination result, in step S105, the abnormal sound possibility level is set to 0, or in step S106, the abnormal sound possibility level of the sound data determined to be an onomatopoeia existing in the onomatopoeia-abnormal sound level database 5 is acquired.

[0064] Next, in step S303, it is determined whether or not the vehicle signal acquired when the sound data was acquired matches the vehicle signal acquired by the vehicle (CAN) signal acquisition unit 9. If the vehicle signal acquired when the sound data was acquired does not match the vehicle signal of the vehicle (CAN) signal acquisition unit 9, the process proceeds to step S304, where the abnormal sound possibility level is corrected to be one level lower.

[0065] Furthermore, if it is determined in step S303 that the vehicle signal at the time the sound data was acquired matches the vehicle signal of the vehicle (CAN) signal acquisition unit 9, the process proceeds to step S305, where the audio file names are rearranged according to the search conditions (in this embodiment, date and time, and braking), and in step S306, the search results are displayed, including the file creation date and time, the audio file name, the detected onomatopoeia, the engine speed, the braking, and the abnormal noise possibility level.

[0066] As described above, in the third embodiment, the vehicle (CAN) signal acquisition unit 9 acquires information on the vehicle signal associated with the sound data at the time of acquisition of the sound data. Then, the abnormal sound possibility level determination unit 6 corrects the abnormal sound possibility level based on the information on the vehicle signal acquired by the vehicle (CAN) signal acquisition unit 9. This allows for preferential search for abnormal sounds in a vehicle state similar to the vehicle state input as a search condition. In addition, the abnormal sound possibility level is lowered for abnormal sounds in a vehicle state different from the vehicle state input as a search condition, making them less likely to be searched for, so that when searching for abnormal sounds caused by a specific vehicle operation, etc., the search time for the abnormal sounds to be searched for can be shortened.

[0067] In the third embodiment, the vehicle state and the onomatopoeia are simultaneously displayed on the result output unit 7, so that not only the type of abnormal sound contained in the audio file, but also information including language and numerical values ​​regarding the operation and vehicle state in which the abnormal sound was obtained is obtained. This makes it easier to imagine the scene in which the sound data was obtained, makes it easier to share information with other workers, and makes it easier to find the desired audio file. The vehicle state can also be used as one of the features indicating the occurrence of an abnormal sound.

[0068] FIG. 11 is a functional block diagram of an abnormal sound data processing device according to the fourth embodiment.

[0069] The abnormal sound data processing device of the fourth embodiment is configured by adding an environmental signal acquisition unit 10 between the onomatopoeia conversion unit 4 and the abnormal sound possibility level determination unit 6 of the abnormal sound data processing device of the first embodiment.

[0070] In the fourth embodiment, when acquiring sound data during the abnormal sound inspection of each part of a vehicle during a finished vehicle inspection, information on environmental conditions (environmental signal information), such as one or more signals related to temperature, humidity, sunlight, wind speed, and wind direction, is acquired, and the acquired signals are linked to the sound data.

[0071] Further, in addition to the date and time range of the search target and the abnormal noise possibility level of the search target, the environmental condition of the search target is input to the search condition input unit 1 as a search condition. Here, the environmental condition of the search target is, for example, any one or more pieces of information related to the environment inside or outside the vehicle, such as temperature, humidity, sunlight, wind speed, wind direction, etc. If the environmental condition is temperature, the environmental condition is, for example, "outside temperature 30 to 35 degrees". If the environmental condition is wind speed, the environmental condition is, for example, "maximum wind speed 5 m or less". In this embodiment, the environmental condition of the search target input to the search condition input unit 1 is "temperature 25 degrees or more, humidity 50 to 70%".

[0072] The environmental signal acquisition unit 10 acquires an environmental signal linked to past sound data extracted according to a search condition.

[0073] The abnormal sound possibility level determination unit 6 corrects the abnormal sound possibility level based on whether or not the environmental signal acquired when acquiring the sound data matches the environmental signal acquired by the environmental signal acquisition unit 10. For example, if the environmental signal acquired when acquiring the sound data is "temperature 27 degrees, humidity 60%", and the environmental signal acquired by the environmental signal acquisition unit 10 based on the input conditions is "temperature 25 degrees or higher, humidity 50 to 70%, it is determined that the environmental signal acquired when acquiring the sound data matches the environmental signal of the environmental signal acquisition unit 10. Accordingly, the abnormal sound possibility level determination unit 6 maintains the abnormal sound possibility level without correcting it. Alternatively, the abnormal sound possibility level determination unit 6 may correct the abnormal sound possibility level to be one step higher rather than maintaining it. Also, for example, if the environmental signal acquired when acquiring sound data is "temperature 23 degrees, humidity 40%," and the environmental signal acquired by environmental signal acquisition unit 10 based on the input conditions is "temperature 25 degrees or higher, humidity 50 to 70%," it is determined that the vehicle signal input as the search condition does not match the environmental signal of environmental signal acquisition unit 10. Accordingly, abnormal sound possibility level determination unit 6 corrects the abnormal sound possibility level to be one level lower.

[0074] As shown in Fig. 12, the result output unit 7 outputs the temperature and humidity as search results in addition to the date and time, the audio file name, the detected onomatopoeia, and the abnormal sound possibility level. When outputting, the search results may be sorted according to the environmental signal. In this case, the search results may be sorted so that the closer to 60%, which is the median value of the humidity range of 50 to 70%, the higher the list is positioned, and the further from the median value the lower the list is positioned. Also, instead of sorting the search results according to the environmental signal, the search results may be sorted using a combination of the environmental signal and the abnormal sound possibility level.

[0075] FIG. 13 is a flow chart showing the flow of processing performed by the abnormal sound data processing device of the third embodiment.

[0076] In step S401, search conditions (date and time range, abnormal noise possibility level, environmental conditions) are input.

[0077] Next, in step S102, sound data that meets the search criteria is obtained from the sound file database 2.

[0078] Then, in step S402, an environmental signal linked to the past sound data extracted according to the search conditions is acquired.

[0079] After acquiring this vehicle signal, in step S103, the sound data is converted into an onomatopoeia using the sound-onomatopoeia conversion model 3, and in step S104, it is determined whether or not the detected onomatopoeia exists in the onomatopoeia-abnormal sound level database 5. Depending on the determination result, in step S105, the abnormal sound possibility level is set to 0, or in step S106, the abnormal sound possibility level of the sound data determined to be an onomatopoeia existing in the onomatopoeia-abnormal sound level database 5 is acquired.

[0080] Next, in step S403, it is determined whether or not the environmental signal acquired when the sound data was acquired matches the environmental signal acquired by the environmental signal acquisition unit 10. If the environmental signal acquired when the sound data was acquired does not match the environmental signal of the environmental signal acquisition unit 10, the process proceeds to step S404, where the abnormal sound possibility level is corrected to be one step lower.

[0081] If it is determined in step S403 that the environmental signal at the time of acquiring the sound data matches the environmental signal of the environmental signal acquiring unit 10, the process proceeds to step S405, where the sound file names are rearranged according to the search conditions (date and time, humidity). Then, in step S406, the search results are displayed, including the file creation date and time, the voice file name, the detected onomatopoeia, the temperature, the humidity, and the abnormal sound possibility level.

[0082] As described above, in the fourth embodiment, the environmental signal acquisition unit 10 acquires information on the environmental conditions (environmental signals) associated with the sound data at the time of acquisition of the sound data. Then, the abnormal sound possibility level determination unit 6 corrects the abnormal sound possibility level based on the information on the environmental signals acquired by the environmental signal acquisition unit 10. This makes it possible to preferentially search for abnormal sounds under environmental conditions similar to the environmental conditions input as search conditions. Furthermore, for abnormal sounds under environmental conditions different from the environmental conditions input as search conditions, the abnormal sound possibility level is low and they are less likely to be searched for, so that when searching for abnormal sounds caused by a specific environment, the search time for the abnormal sounds to be searched for can be shortened.

[0083] In the fourth embodiment, the environmental conditions and the onomatopoeia are simultaneously displayed on the result output unit 7, so that not only is it possible to know what kind of abnormal sound is contained in the audio file, but also the environment in which the abnormal sound was obtained, as information including language and numerical values. This makes it easier to imagine the scene in which the sound data was obtained, makes it easier to share information with other workers, and makes it easier to find the desired audio file. The environmental conditions can also be used as one of the features indicating the occurrence of an abnormal sound.

[0084] In the above embodiments, examples have been described in which sound data is acquired during finished vehicle inspection. However, the present invention can also be applied to examples in which sound data is acquired during abnormal sound inspection in a repair shop, for example, other than finished vehicle inspection. [Explanation of symbols]

[0085] 1. Search criteria input section 2. Audio file database 3. Sound-to-onomatopoeia conversion model 4. Onomatopoeia conversion section 5. Onomatopoeia-Allophone Level Database (Allophone Database) 6. Noise probability level determination section 7. Result output section 8. Sound pressure level acquisition section 9. Vehicle (CAN) signal acquisition section 10···Environmental signal acquisition unit

Claims

1. The sound generated by the vehicle is collected as sound data, Detect onomatopoeia contained in this sound data, determining an abnormal sound possibility level of the sound data by referring to a known association between the onomatopoeia and a vehicle part or portion; outputting information including the sound data, the onomatopoeia, and the abnormal sound possibility level; A method for processing abnormal noise data from a vehicle.

2. 2. The method for processing abnormal sound data of a vehicle according to claim 1, wherein the sound is acquired during a completed vehicle inspection of the vehicle.

3. constructing an audio file database in which a large number of audio files including the audio data are stored; 2. The method for processing abnormal sound data of a vehicle according to claim 1, further comprising detecting onomatopoeia from the sound data of the sound files extracted from the sound file database based on a search condition.

4. A database of abnormal sounds is constructed that compiles onomatopoeias related to the vehicle and vehicle parts or locations that may be the source of the abnormal sounds, 2. The method for processing abnormal sound data of a vehicle according to claim 1, further comprising: determining the abnormal sound likelihood level by referring to the abnormal sound database.

5. 5. The method for processing abnormal sound data of a vehicle according to claim 4, further comprising: setting an abnormal sound possibility level to a relatively low value if the detected onomatopoeia does not exist in the abnormal sound database.

6. The method for processing abnormal sound data of a vehicle according to claim 1 , further comprising: correcting the abnormal sound possibility level based on a sound pressure level of the sound data.

7. Acquire information on a vehicle signal associated with the sound data at the time of acquisition of the sound data; 2. The method for processing abnormal sound data of a vehicle according to claim 1, further comprising correcting the abnormal sound possibility level based on information of the vehicle signal.

8. Acquire information on the environmental conditions at the time of acquisition of the sound data, the information being associated with the sound data; 2. The method for processing abnormal sound data for a vehicle according to claim 1, further comprising correcting an abnormal sound possibility level based on the information on the environmental conditions.

9. a sound acquisition unit that acquires sound generated from a vehicle as sound data; A detection unit that detects onomatopoeia contained in the sound data; a determination unit that determines an abnormal sound possibility level of the sound data by referring to a known association between the onomatopoeia and a vehicle part or portion; an output unit that outputs information including the sound data, the onomatopoeia, and the abnormal sound possibility level.

Images