Abnormal sound identifying device, abnormal sound identifying method, and abnormal sound identifying program

Abstract

To provide an abnormal noise specification device, an abnormal noise specification method and an abnormal noise specification program, which highly precisely specify abnormal noise by using artificial intelligence.SOLUTION: An abnormal noise specification method executes the steps of: specifying frequency-time data of sound recorded in a vehicle; inputting the specified frequency-time data to a learned model, allowing the learned model to specify a type of abnormal noise included in the recorded sound and allowing the learned model to specify a basis range indicating a frequency range and a time range which are used for specifying the type of the abnormal noise; designating a designation range indicating the frequency range and the time range in the specified frequency-time data; and determining whether or not to allow an output device to output the type of the abnormal noise in processing for determining whether or not the basis range and the designation range are overlapped as at least one determination element.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The technology disclosed in this specification relates to an abnormal sound identifying device, an abnormal sound identifying method, and an abnormal sound identifying program.

[0002] Patent Document 1 discloses a technique for identifying abnormal sounds from sounds recorded by an image forming device. This technique converts sounds recorded by the image forming device into frequency-time data that indicates changes over time in the frequency spectrum. Furthermore, a fast Fourier transform is performed on the frequency-time data in the time axis direction. A user can identify the type of abnormal sound (e.g., the source of the abnormal sound) based on the analysis results of the fast Fourier transform and a database of abnormal sounds that have occurred in the past. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2017-110919 Summary of the Invention [Problem to be solved by the invention]

[0004] As described above, in Patent Document 1, a user (i.e., a person) identifies the type of abnormal sound. In response to this, the inventors of the present application have conducted an experiment in which an artificial intelligence is used to identify abnormal sounds from sounds recorded in a vehicle. In the experiment, there are cases in which the artificial intelligence identifies abnormal sounds that are not actually occurring. This specification proposes a technology for accurately identifying abnormal sounds using artificial intelligence. [Means for solving the problem]

[0005] The abnormal sound identification device disclosed in this specification includes a calculation device that can access a trained model of artificial intelligence, and an output device. The calculation device executes the following steps: identifying frequency-time data indicating time changes in the frequency spectrum of a sound recorded in a vehicle; inputting the identified frequency-time data into the trained model, causing the trained model to identify the type of abnormal sound contained in the sound based on the input frequency-time data, and causing the trained model to identify a ground truth range indicating the frequency range and time range used to identify the type of abnormal sound within the input frequency-time data; specifying a designated range indicating the frequency range and time range within the identified frequency-time data; and determining whether to cause the output device to output the type of abnormal sound through a determination process that includes, as at least one determination element, whether the ground truth range and the designated range overlap.

[0006] The above-mentioned "step of identifying frequency-time data indicating time changes in the frequency spectrum of the sound recorded in the vehicle" may be a step in which the arithmetic device calculates the frequency-time data based on the sound recorded in the vehicle, or a step in which the frequency-time data calculated by an external device is input to the arithmetic device.

[0007] Furthermore, the "trained model" may be stored anywhere as long as it is accessible by the computing device. For example, the trained model may be stored in a storage device inside the abnormal sound identification device, or in a storage device on a network that is accessible by the computing device.

[0008] The "designated range" may be designated according to an input from the user, or may be designated autonomously by the computing device according to a predetermined algorithm.

[0009] In this abnormal sound identification device, once the arithmetic device identifies the frequency-time data, it inputs the identified frequency-time data into a trained model. The trained model then identifies the type of allophone contained in the sound based on the frequency-time data. At this stage, the accuracy of the type of allophone identified by the trained model is not very high. That is, the trained model may identify a type of allophone that does not actually occur. The trained model also identifies a ground truth range that indicates the frequency range and time range used to identify the type of allophone within the frequency-time data. After identifying the trained model, the arithmetic device also specifies a designated range that indicates the frequency range and time range within the identified frequency-time data. The designated range is specified by user operation, a predetermined algorithm, or the like. The designated range can be a frequency range and time range corresponding to the allophone. In this way, the designated range is specified separately from the ground truth range identified by the trained model. Once the arithmetic device has determined the ground truth range and the designated range, it determines whether or not to cause the output device to output the type of allophone identified by the trained model, using a determination process that includes, as at least one determination element, whether or not the ground truth range and the designated range overlap. If the grounds range and the specified range overlap, the type of abnormal sound identified by the trained model is likely to be correct, whereas if the grounds range and the specified range do not overlap, the type of abnormal sound identified by the trained model is likely to be incorrect. Therefore, by determining whether to output the type of abnormal sound to the output device using a determination process that includes as at least one determination element whether the grounds range and the specified range overlap, it is possible to prevent the wrong type of abnormal sound from being output to the output device. This abnormal sound identification device can identify the type of abnormal sound contained in sounds recorded in a vehicle with higher accuracy. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a block diagram of an abnormal sound identifying device 10. [Figure 2] FIG. 4 is an explanatory diagram of a recording device 42. [Figure 3] 10 is a flowchart showing an abnormal noise identification method. [Figure 4] A diagram showing STFT data 54. [Figure 5] A diagram showing the grounds range 60. [Figure 6] FIG. 10 is a diagram showing a grounds range 60 in which the contour is determined. [Figure 7] FIG. 10 is a diagram showing a screen for selecting a specified range 64. [Figure 8] 10 is a flowchart showing a determination process based on a grounds range 60 and a specified range 64. [Figure 9] FIG. 2 is a diagram showing the positional relationship between a grounds range 60 and a specified range 64. [Figure 10] FIG. 2 is a diagram showing the positional relationship between a grounds range 60 and a specified range 64. [Figure 11] FIG. 2 is a diagram showing the positional relationship between a grounds range 60 and a specified range 64. [Figure 12] FIG. 2 is a diagram showing the positional relationship between a grounds range 60 and a specified range 64. [Figure 13] FIG. 10 is a diagram showing a display screen of the abnormal noise identification results. [Figure 14] 10 is a flowchart showing a determination process according to a modified example. [Figure 15] 10 is a flowchart showing a determination process according to a modified example. DETAILED DESCRIPTION OF THE INVENTION

[0011] In the example abnormal sound identifying device disclosed in this specification, the arithmetic device may cause the output device to output the type of the abnormal sound when the grounds range and the specified range overlap.

[0012] In the example abnormal sound identifying device disclosed in this specification, the arithmetic device may not cause the output device to output the type of abnormal sound when the grounds range and the specified range do not overlap.

[0013] The abnormal sound identification device according to the example disclosed herein may further include an input device. In this case, the arithmetic unit may specify the specified range in response to an input from a user using the input device.

[0014] According to this configuration, the range in which the user has determined that an abnormal sound occurs can be designated as the designated range. Therefore, by determining whether the designated range in which the user has determined that an abnormal sound occurs overlaps with the ground truth range identified by the trained model, it is possible to accurately determine whether the type of abnormal sound identified by the trained model is correct.

[0015] The abnormal sound identification device according to the example disclosed herein may further include a speaker. In this case, the computing device may further execute a step of causing the speaker to emit a sound within the specified range after the specified range is specified and before determining whether the basis range and the specified range overlap.

[0016] With this configuration, when the user designates a designated range, the user can hear the sound within the designated range. Therefore, the user can determine whether the designated range has been correctly designated, and if not, can designate the designated range again.

[0017] In the example abnormal sound identification device disclosed in the present specification, the arithmetic unit may further execute a step of calculating a contour determination ground range by determining a contour of the ground range. In this case, the arithmetic unit may determine whether the contour determination ground range and the specified range overlap in the determination process.

[0018] In some cases, the contour of the grounds range identified by the trained model is blurred. In such cases, the computing device calculates a contour-confirmed grounds range that confirms the contour of the grounds range, thereby making it possible to clearly determine whether the contour-confirmed grounds range and the specified range overlap.

[0019] In the example of the abnormal sound identification device disclosed in the present specification, the trained model may be configured by a convolutional network.

[0020] In the example allophone identification device disclosed in the present specification, the trained model may identify the evidence range using gradient-weighted class activation mapping. [Example]

[0021] The abnormal sound identification device 10 shown in FIG. 1 identifies abnormal sounds from sounds recorded in a vehicle. The abnormal sound identification device 10 is configured as a so-called computer. The abnormal sound identification device 10 has an arithmetic unit 12, a storage device 16, a monitor 18, a speaker 20, an input device 22, a network interface 24, etc. The arithmetic unit 12 is configured with a CPU (central processing unit), memory, etc. The arithmetic unit 12 is connected to the storage device 16, the monitor 18, the speaker 20, the input device 22, and the network interface 24. The storage device 16 is configured with a hard disk drive, a solid state drive, etc. The storage device 16 stores an abnormal sound identification program 50. The arithmetic unit 12 executes the abnormal sound identification program 50. The input device 22 is configured with a mouse, a keyboard, etc. A user operates the input device 22 to input a signal to the arithmetic unit 12. The arithmetic unit 12 controls the monitor 18 and the speaker 20. The arithmetic unit 12 is also connected to a network line 30 via a network interface 24. The network line 30 may be the Internet or an intranet. A storage device 32 and the like are connected to the network line 30. At least one of the storage device 16 and the storage device 32 stores a trained model 52. Regardless of whether the trained model 52 is stored in the storage device 16 or the storage device 32, the arithmetic unit 12 can access the trained model 52. The trained model 52 is an artificial intelligence and is configured using a convolutional neural network (CNN). The trained model 52 is a model trained to identify allophones from frequency-time data (data indicating changes in the frequency profile of a sound over time). In this embodiment, short-time Fourier transform data (hereinafter referred to as STFT data) is used as the frequency-time data. When STFT data is input to the trained model 52, the trained model 52 identifies the type of allophone contained in the sound based on the STFT data.The trained model 52 also has a function called gradient-weighted class activation mapping (Grad-CAM). When identifying the type of allophone based on the STFT data, the trained model 52 uses Grad-CAM to identify the frequency range and time range from the STFT data that are the basis for identifying the type of allophone.

[0022] The storage device 16 can store sound data 56. The sound data 56 is generated by the sound recording device 42 shown in FIG. 2. The sound recording device 42 is mounted on the vehicle 40. The sound recording device 42 records sounds generated by the vehicle 40 while the vehicle 40 is running. The sound recording device 42 may record sounds inside the vehicle cabin or outside the vehicle cabin (for example, in the engine compartment). The sound recording device 42 saves the recorded sounds as sound data 56 in a portable storage device. The sound data 56 is data representing waveforms of sound vibrations (i.e., air vibrations). The sound data 56 includes vehicle road noise, engine noise, and operating sounds of components (for example, an alternator, a water pump, a turbo, a VSV (vacuum switching valve)), etc. Furthermore, if an abnormal sound is generated in the vehicle 40 for some reason, the abnormal sound is also included in the sound data 56. By connecting a portable storage device to the abnormal sound identification device 10, the sound data 56 can be input to the abnormal sound identification device 10. The sound data 56 can also be input to the abnormal sound identifying device 10 via the network line 30. As shown in FIG.

[0023] Next, we will explain the abnormal sound identification method executed by the abnormal sound identification device 10. The abnormal sound identification device 10 executes the abnormal sound identification method shown in Fig. 3 by running an abnormal sound identification program 50. When the user performs a predetermined operation using the input device 22, the arithmetic device 12 starts the abnormal sound identification program 50 (i.e., the abnormal sound identification method in Fig. 3).

[0024] In step S2, the arithmetic unit 12 displays a window or the like on the monitor 18 to instruct the user to select sound data 56. The user can use the input device 22 to select any sound data 56 from the sound data 56 stored in the storage device 16.

[0025] In step S4, the arithmetic device 12 performs a short-time Fourier transform on the sound data 56 selected in step S2. As a result, the arithmetic device 12 calculates the STFT data 54 shown in FIG. 4. The vertical axis of the STFT data 54 indicates the frequency of the sound, and the horizontal axis of the STFT data 54 indicates time. Furthermore, the color of each pixel in the STFT data 54 indicates the sound pressure level (dB). In other words, the STFT data 54 is data that indicates the change over time in the frequency spectrum of the sound. The arithmetic device 12 calculates the STFT data 54 as image data.

[0026] In step S6, the calculation device 12 inputs the STFT data 54 calculated in step S4 to the trained model 52. The trained model 52 then extracts features from the input STFT data 54 and identifies the abnormal sound contained in the STFT data 54 (more specifically, the abnormal sound contained in the sound represented by the STFT data 54) based on the extracted features. That is, the trained model 52 identifies the abnormal sound and, at the same time, identifies the type of the abnormal sound. The trained model 52 identifies the type of abnormal sound as abnormal sound A, abnormal sound B, etc. In a more specific example, the trained model 52 identifies the abnormal sound as abnormal sound from an alternator, abnormal sound from a water pump, abnormal sound from a turbo, abnormal sound from a VSV, etc. That is, the type of abnormal sound represents the source of the abnormal sound. Furthermore, while identifying the type of abnormal sound, the trained model 52 also uses Grad-CAM to identify a basis range, which is a frequency range and a time range that serve as the basis for identifying the type of abnormal sound. That is, the trained model 52 sets an importance level for each pixel in the STFT data 54, and identifies the type of allophone so that pixels with higher importance have a greater influence on the output result (i.e., the type of allophone that is identified). A ground truth range is a collection of pixels with higher importance in the STFT data 54. For example, a ground truth range 60 is identified as shown in FIG. 5 for the STFT data 54 shown in FIG. 4. Since the importance level of each pixel is different, the outline of the ground truth range 60 is blurred. Note that in step S6, the trained model 52 may identify multiple types of allophone. In this case, multiple ground truth ranges 60 are also identified.

[0027] In step S8, the arithmetic device 12 binarizes the importance set for each pixel of the STFT data 54 using a predetermined threshold as a reference. As a result, the arithmetic device 12 clearly determines the outline of the grounds range 60, as shown in Fig. 6. From step S8 onwards, the arithmetic device 12 uses the grounds range with the determined outline as the grounds range 60.

[0028] In step S10, the arithmetic device 12 displays the STFT data 54 on the monitor 18 as shown in FIG. 7. The user can select a frequency range and a time range on the STFT data 54 displayed on the monitor 18 by operating the input device 22. Hereinafter, the range selected in step S10 will be referred to as a designated range 64. For example, the designated range 64 is selected as shown in FIG. 7. In step S10, the user can select a range in the STFT data 54 that is thought to correspond to an abnormal sound as the designated range 64. The user can select the designated range 64 based on their own experience while visually checking the STFT data 54. Also, as shown in FIG. 7, a play button 90 and a confirm button 92 are displayed next to the STFT data 54. When the play button 90 is pressed with the designated range 64 selected, the arithmetic device 12 plays the sound within the designated range 64 through the speaker 20. Therefore, the user can determine whether the designated range 64 contains an abnormal sound by listening to the sound within the designated range 64. This makes it easy to select the range that is thought to correspond to an abnormal sound as the designated range 64. The user can press the confirm button 92 with the designated range 64 selected. When the confirm button 92 is pressed, the arithmetic unit 12 confirms the selected designated range 64 and proceeds to the next step. In this way, in step S10, the range that the user has determined to correspond to an abnormal noise is designated as the designated range 64. Note that it is also possible to designate multiple ranges as the designated range 64 in step S10.

[0029] In step S12, a determination process is performed based on the grounds range 60 identified in step S6 and the specified range 64 specified in step S10. Then, the type of abnormal noise is displayed on the monitor 18 according to the determination process. Fig. 8 shows the details of step S12. As shown in Fig. 8, in step S12, the calculation device 12 executes steps S20 to 28.

[0030] In step S20, the calculation device 12 selects one allophone from among the allophones (i.e., types of allophones) identified in step S6 by the trained model 52. If there is only one allophone identified in step S6, that allophone is selected.

[0031] In step S22, the calculation device 12 determines whether the grounds range 60 corresponding to the selected abnormal noise overlaps with the specified range 64 or not.

[0032] For example, as shown in FIG. 9, if the selected abnormal noise grounds range 60 and the specified range 64 overlap, the calculation device 12 determines YES in step S22.

[0033] Furthermore, as shown in FIG. 10, if the selected reason range 60 of the abnormal noise and the specified range 64 do not overlap, the calculation device 12 determines NO in step S22.

[0034] 11 also shows a case where there are multiple grounds ranges 60a, 60b. In this case, if the grounds range 60 of the selected abnormal noise is the grounds range 60a, the grounds range 60a overlaps with the specified range 64, and so the calculation device 12 determines YES in step S22. If the grounds range 60 of the selected abnormal noise is the grounds range 60b, the grounds range 60b does not overlap with the specified range 64, and so the calculation device 12 determines NO in step S22.

[0035] 12 shows a case where multiple designated ranges 64a, 64b exist. In this case, the calculation device 12 determines YES in step S22 if the grounds range 60 of the selected abnormal noise overlaps with either the designated range 64a, 64b. On the other hand, the calculation device 12 determines NO in step S22 if the grounds range 60 of the selected abnormal noise overlaps with neither the designated range 64a, 64b.

[0036] If the determination in step S22 is YES, the processing device 12 executes step S24. In step S24, the processing device 12 displays the type of abnormal noise selected in step S20 on the monitor 18. For example, if the selected abnormal noise is abnormal noise A, then in step S24, "Abnormal noise A has been detected" is displayed on the monitor 18 as shown in FIG. 13. On the other hand, if the determination in step S22 is NO, the processing device 12 executes step S26. In step S26, the processing device 12 does not display the type of abnormal noise selected in step S20 on the monitor 18.

[0037] In step S28, the calculation device 12 determines whether or not processing has been completed for all abnormal sounds identified by the trained model 52. If the result in step S28 is NO, step S20 is executed again. In this case, an abnormal sound for which processing has not been completed is selected in step S20. Therefore, steps S20 to S28 are repeated until processing for all abnormal sounds is completed. Therefore, if multiple abnormal sounds have been identified by the trained model 52, only those abnormal sounds for which a YES result has been returned in step S22 are displayed on the monitor 18. Furthermore, if there are no abnormal sounds for which a YES result has been returned in step S22, the monitor 18 displays "An abnormal sound could not be detected." When processing for all abnormal sounds identified by the trained model 52 has been completed, the calculation device 12 ends the abnormal sound detection process.

[0038] As described above, the computing device 12 displays the type of abnormal sound being selected on the monitor 18 when the grounds range 60 of the selected abnormal sound overlaps with the specified range 64 specified by the user, and does not display the type of abnormal sound being selected on the monitor 18 when the grounds range 60 of the selected abnormal sound does not overlap with the specified range 64 specified by the user. Therefore, of the abnormal sounds identified by the trained model 52, those that match the abnormal sound recognized by the user are displayed on the monitor 18, and of the abnormal sounds identified by the trained model 52, those that the user does not recognize are not displayed on the monitor 18. Therefore, the user can check the type of abnormal sound that he or she recognizes as an abnormal sound on the monitor 18.

[0039] According to the abnormal sound identification device 10 of the embodiment, it is possible to select only those abnormal sounds that the user recognizes as abnormal sounds from among the abnormal sounds identified by the trained model 52, and display the type of those abnormal sounds on the monitor 18. Therefore, it is possible to accurately identify the type of abnormal sound that is actually occurring in the vehicle 40.

[0040] Furthermore, the abnormal sound identification device 10 of the embodiment can play back the sound within the specified range 64 after the user selects the specified range 64. Therefore, the user can easily identify the range corresponding to the abnormal sound as the specified range 64.

[0041] Furthermore, in the abnormal sound characteristics device of the embodiment, the calculation device 12 determines the contour of the grounds range 60 identified by the trained model 52, so it is possible to accurately determine whether the grounds range 60 and the specified range 64 overlap.

[0042] 8, if it is determined in step S22 that the grounds range 60 and the specified range 64 overlap, the selected abnormal noise is displayed on the monitor 18 in step S24. However, as shown in FIG. 14, step S23, in which another determination is made, may be present between step S22 and step S24. In FIG. 14, even if the grounds range 60 and the specified range 64 overlap, if the selected abnormal noise does not satisfy criterion condition 1 (i.e., if the answer is NO in step S23), the selected abnormal noise is not displayed on the monitor 18. In this way, even if the answer is YES in step S22, the selected abnormal noise may not be displayed on the monitor 18.

[0043] 8, if it was determined in step S22 that the grounds range 60 and the specified range 64 did not overlap, the selected abnormal noise was not displayed on the monitor 18 in step S26. However, as shown in FIG. 15, step S25, in which another determination is made, may be present between step S22 and step S26. In FIG. 15, even if the grounds range 60 and the specified range 64 do not overlap, if the selected abnormal noise satisfies criterion condition 2 (i.e., if the answer is YES in step S25), the selected abnormal noise is displayed on the monitor 18. In this way, even if the answer is NO in step S22, the selected abnormal noise may be displayed on the monitor 18. Also, FIGS. 14 and 15 may be combined.

[0044] In the above-described embodiment, the computing device 12 displays the type of abnormal noise on the monitor 18. However, the computing device 12 may also output the type of abnormal noise to another device. For example, the computing device 12 may output the type of abnormal noise to another device via the network line 30.

[0045] Furthermore, in the above-described embodiment, the STFT data 54 was image data. That is, the trained model 52 identified the type of abnormal noise and the range of grounds for the abnormal noise based on the STFT data 54, which is image data. However, the data format of the STFT data 54 is not limited to image data. For example, the STFT data 54 may be data that indicates the numerical value of the sound pressure level for each time and frequency.

[0046] Furthermore, in the above-described embodiment, the designated range 64 was specified by a user operation. However, the arithmetic device 12 may also specify the designated range 64 in accordance with a predetermined algorithm. For example, a range with a high sound pressure level may be automatically specified as the designated range 64. Even with this configuration, the type of abnormal noise to be ultimately output can be selected using the basis range 60 and the designated range 64 specified by different algorithms, thereby enabling the type of abnormal noise to be identified with high accuracy.

[0047] Although the embodiments have been described in detail above, these are merely examples and do not limit the scope of the claims. The technology described in the claims includes various modifications and variations of the specific examples exemplified above. The technical elements described in this specification or drawings exhibit technical utility alone or in various combinations, and are not limited to the combinations described in the claims at the time of filing. Furthermore, the technology exemplified in this specification or drawings simultaneously achieves multiple objectives, and achieving one of these objectives itself has technical utility. [Explanation of symbols]

[0048] 10: Abnormal noise identification device 12: Arithmetic device 16:Storage device 18: Monitor 20: Speaker 22: Input device 24: Network interface 30: Network line 32: Storage device 40: Vehicle 42: Recording device 50: Abnormal noise identification program 52: Trained model 54: STFT data 56: Sound data 60: Scope of evidence 64: Specified range

Claims

1. An abnormal sound identifying device, A computing device that can access a trained model of artificial intelligence; an output device; and The computing device identifying frequency-time data indicative of time-varying frequency spectrum of sounds recorded from the vehicle; inputting the identified frequency-time data into the trained model, causing the trained model to identify the type of allophone contained in the sound based on the input frequency-time data, and causing the trained model to identify a ground range indicating the frequency range and time range used to identify the type of allophone from the input frequency-time data; a step of specifying, by a user operation or a predetermined algorithm, a specified range indicating a frequency range and a time range determined to correspond to an abnormal noise in the identified frequency-time data, without the grounds range being specified; a step of determining whether or not to cause the output device to output the type of abnormal noise in a determination process that includes, as at least one of determination elements, whether or not the grounds range and the specified range overlap; Run the computing device causes the output device to output the type of the abnormal noise when the grounds range and the specified range overlap; the computing device does not cause the output device to output the type of abnormal noise when the grounds range and the specified range do not overlap; Abnormal noise identification device.

2. further comprising an input device; The abnormal sound identifying device according to claim 1 , wherein the arithmetic unit specifies the specified range in response to an input from a user using the input device.

3. Further comprising a speaker; the calculation device further executes a step of making the speaker emit a sound within the specified range after the specified range is specified and before determining whether the ground range and the specified range overlap. The abnormal noise identifying device according to claim 1 or 2.

4. the calculation device further executes a step of calculating a contour-determined ground range in which a contour of the ground range is determined; the calculation device determines whether or not the contour determination ground range and the specified range overlap in the determination process. The abnormal noise identifying device according to any one of claims 1 to 3.

5. The abnormal sound identification device according to any one of claims 1 to 4, wherein the trained model is configured by a convolutional network.

6. The allophone identification device of claim 5 , wherein the trained model identifies the evidence range using gradient-weighted class activation mapping.

7. 7. The abnormal sound identifying device according to claim 1, wherein in the step of specifying the specified range, the arithmetic device prompts a user to specify the specified range without indicating the grounds range.

8. An abnormal sound identification method for identifying an abnormal sound from a sound recorded in a vehicle using a calculation device and an output device that can access a trained model of artificial intelligence, The computing device identifies frequency-time data indicating a time change in the frequency spectrum of the sound; a step in which the arithmetic device inputs the identified frequency-time data into the trained model, causes the trained model to identify the type of allophone contained in the sound based on the input frequency-time data, and causes the trained model to identify a ground range indicating the frequency range and time range used to identify the type of allophone from the input frequency-time data; a step in which the arithmetic device, in a state in which the grounds range is not specified, specifies, by a user operation or a predetermined algorithm, a specified range indicating a frequency range and a time range determined to correspond to an abnormal noise in the identified frequency-time data; a step in which the arithmetic device determines whether or not to cause the output device to output the type of abnormal noise through a determination process that includes, as at least one determination element, whether or not the grounds range and the specified range overlap; and the computing device causes the output device to output the type of the abnormal noise when the grounds range and the specified range overlap; the computing device does not cause the output device to output the type of abnormal noise when the grounds range and the specified range do not overlap; How to identify abnormal noises.

9. The abnormal noise identifying method according to claim 8 , wherein in the step of specifying the specified range, the arithmetic device prompts the user to specify the specified range without the ground range being indicated.

10. An abnormal noise identification program, A computing device that can access a trained model of artificial intelligence and is connected to an output device, identifying frequency-time data indicative of time-varying frequency spectrum of sounds recorded from the vehicle; inputting the identified frequency-time data into the trained model, causing the trained model to identify the type of allophone contained in the sound based on the input frequency-time data, and causing the trained model to identify a ground range indicating the frequency range and time range used to identify the type of allophone from the input frequency-time data; a step of allowing a user to specify, by a user operation or a predetermined algorithm, a specified range that indicates a frequency range and a time range that are determined to correspond to an abnormal noise in the identified frequency-time data, without the grounds range being specified; a step of determining whether or not to cause the output device to output the type of abnormal noise in a determination process that includes, as at least one of determination elements, whether or not the grounds range and the specified range overlap; Execute the computing device causes the output device to output the type of the abnormal noise when the grounds range and the specified range overlap; the computing device does not cause the output device to output the type of abnormal noise when the grounds range and the specified range do not overlap; Abnormal noise identification program.

11. The abnormal noise identification program according to claim 10 , wherein the step of causing the user to specify the specified range causes the computing device to cause the user to specify the specified range without the basis range being indicated.

Images