Abnormality determination system

Abstract

Provided is an abnormality determination system capable of easily determining the presence or absence of an abnormality in the air pressure or the like of a tire at low cost. This abnormality determination system comprises: a sound collection unit that collects sound generated when an external force is applied to a tire; an extraction unit that extracts a tire sound of the tire from the sound information acquired by the sound collection unit; a determination unit that determines the presence or absence of an abnormality of the tire from the tire sound information; and a display unit that displays the results of the determination of the presence or absence of the abnormality of the tire performed by the determination unit.

Description

Anomaly detection system

[0001] The present invention relates to a tire abnormality detection system that uses sound generated when an external force is applied to a tire, and more particularly to an abnormality detection system that determines tire pressure and other factors.

[0002] Maintaining proper vehicle condition, including tire pressure, is crucial for vehicles such as passenger cars, taxis, trucks, buses, and trailers. In particular, taxis, trucks, buses, and trailers are required to transport passengers or cargo safely and efficiently. Therefore, to avoid disruptions to operations, for example, tire pressure is checked before driving. Tire pressure is widely measured directly using a tire pressure gauge. However, measuring tire pressure using a tire pressure gauge requires pressing the gauge against the tire's air valve (valve stem), removing the valve cap, and then reattaching it after measurement, making the process cumbersome. Furthermore, if the tire pressure gauge is not properly pressed against the air valve, air may leak from the tire.

[0003] Therefore, a method that does not directly measure the tire pressure has been proposed. For example, in Patent Document 1, there is a transmitting means with a sensor that is attached to each wheel of a vehicle, detects the tire pressure of the attached wheel, and wirelessly transmits the detection result information; a receiving means with a display function that is attached near each wheel of the vehicle body, wirelessly receives the detection result information related to the wheel, and has an integrated display output unit that can be visually recognized from outside the vehicle; a warning level determination is implemented to determine whether the tire pressure is within a predetermined warning pressure range based on the detection result information input from the receiving means with a display function, and when it is determined that the tire pressure is within the warning pressure range when the ignition switch is OFF, control means outputs warning display command information for causing the receiving means with a display function, which is the output source of the input detection result information, to perform a predetermined warning display at the display output unit. By the receiving means with a display function that receives the input of the warning display command information performing the warning display at the display output unit, a tire pressure monitoring system has been proposed that enables a wheel with a tire pressure within the warning pressure range to be recognized from outside the vehicle when the ignition switch is OFF.

[0004] Japanese Patent Application Laid-Open No. 2010-230381

[0005] In the above Patent Document 1, since it is possible for the user to easily confirm the tire pressure adjustment state from outside the vehicle, it is said that the pre-operation inspection by the user becomes easier. However, it is necessary to provide a transmitting means with a sensor for each wheel. Therefore, when the number of wheels (tires) is large, many transmitting means with sensors are required, the configuration of the tire pressure monitoring system becomes large-scale, and the cost increases. In addition, the transmitting means with a sensor is driven by a battery, and when the battery runs out, the tire pressure cannot be monitored. For these reasons, it is desired to be able to grasp the abnormality of the tire pressure with a simple device configuration without incurring costs. The object of the present invention is to provide an abnormality determination system that can simply determine the presence or absence of abnormalities such as tire pressure at low cost.

[0006] The above objectives can be achieved with the following configurations. Invention [1] is an abnormality determination system comprising: a sound collection unit that collects sound generated when an external force is applied to a tire; an extraction unit that extracts tire sound from the sound information acquired by the sound collection unit; a determination unit that determines whether or not there is an abnormality in the tire from the tire sound information; and a display unit that displays the determination result of whether or not there is an abnormality in the tire determined by the determination unit. Invention [2] is the abnormality determination system according to Invention [1], wherein the extraction unit extracts tire sound from the sound information acquired by the sound collection unit using a threshold value based on the sound pressure deviation. Invention [3] is the abnormality determination system according to Invention [1], wherein the extraction unit extracts tire sound from the sound information acquired by the sound collection unit using a first machine learning model that has been trained based on the sound information. Invention [4] is an abnormality determination system according to any one of Inventions [1] to [3], wherein the determination unit determines whether or not there is an abnormality in the tire from the tire sound information using a second machine learning model that has been trained based on the tire sound.

[0007] Invention [5] is an anomaly detection system according to Invention [3], further comprising an additional learning unit for further training a first machine learning model that has been trained, wherein the additional learning unit further trains the first machine learning model using the results of tire sound extraction by the first machine learning model that has been trained. Invention [6] is an anomaly detection system according to Invention [4], further comprising an additional learning unit for further training a second machine learning model that has been trained, wherein the additional learning unit further trains the second machine learning model using the results of the determination of whether or not there is an anomaly in the tire by the second machine learning model that has been trained.

[0008] Invention [7] is an abnormality determination system according to any one of Inventions [1] to [6], wherein the tire is attached to a wheel, the wheel is attached to the vehicle by nuts, the extraction unit extracts the nut sound of the nuts from the sound information acquired by the sound collection unit using a third machine learning model that has been trained based on sound information generated when an external force is applied to the tire acquired by the sound collection unit, the determination unit determines whether or not there is an abnormality in the nuts from the nut sound information, and the display unit displays the determination result of whether or not there is an abnormality in the nuts determined by the determination unit.

[0009] Invention [8] is an abnormality determination system according to Invention [7], wherein the determination unit uses a trained fourth machine learning model based on nut sounds to determine whether or not there is an abnormality in the nut from the information of the nut sounds.

[0010] Invention [9] is an abnormality detection system according to Invention [8], further comprising an additional learning unit that performs additional learning on a trained fourth machine learning model, wherein the additional learning unit further learns the trained fourth machine learning model using the results of the trained fourth machine learning model's determination of whether or not there is an abnormality in the nut. Invention

[10] is an abnormality detection system according to any one of Inventions [1] to [9], wherein the abnormality of the tire determined by the determination unit is at least the tire pressure. Invention

[11] is an abnormality detection system according to any one of Inventions [7] to [9], wherein the abnormality of the nut determined by the determination unit is at least the looseness of the nut.

[0011] Invention

[12] is an abnormality detection system according to any one of Inventions [1] to

[11] , wherein the determination unit determines that it is not possible to determine whether or not there is an abnormality in the tire if it is not possible to determine whether or not there is an abnormality, and the display unit displays the determination result of whether or not there is an abnormality in the tire, or that it is not possible to determine whether or not there is an abnormality. Invention

[13] is an abnormality detection system according to any one of Inventions [1] to

[12] , wherein the determination unit has a storage unit that stores the determination result of whether or not there is an abnormality in the tire determined by the determination unit, and the tire information of the tire for which the presence or absence of an abnormality has been determined as a set. Invention

[14] is an abnormality detection system according to any one of Inventions [1] to

[13] , wherein if the determination unit determines that there is an abnormality in the tire, the notification unit has a notification unit that notifies the tire manager, tire owner, tire seller, tire dealer, owner of the vehicle on which the tire is installed, vehicle manager, vehicle inspector, or vehicle user that there is an abnormality in the tire.

[0012] According to the present invention, it is possible to provide an abnormality detection system that can easily and inexpensively determine whether or not there is an abnormality in tire pressure or the like.

[0013] This is a schematic diagram showing an example of a vehicle fitted with tires that are judged by the abnormality detection system of an embodiment of the present invention. This is a schematic diagram showing a first example of the abnormality detection system of an embodiment of the present invention. (a) is a schematic diagram showing an example of a sound waveform generated when an external force is applied to a tire, and (b) is a schematic diagram showing an example of an extracted region extracted as tire sound. This is a flowchart showing a first example of an abnormality detection method according to the first example of the abnormality detection system of an embodiment of the present invention. This is a flowchart showing a second example of an abnormality detection method according to the first example of the abnormality detection system of an embodiment of the present invention. This is a flowchart showing a third example of an abnormality detection method according to the first example of the abnormality detection system of an embodiment of the present invention. This is a flowchart showing a fourth example of an abnormality detection method according to the second example of the abnormality detection system of an embodiment of the present invention. This is a flowchart showing a fifth example of an abnormality detection method according to the second example of the abnormality detection system of an embodiment of the present invention.

[0014] The abnormality detection system of the present invention will be described in detail below based on preferred embodiments shown in the attached drawings. The figures described below are illustrative for illustrating the present invention and the present invention is not limited to the figures shown below. In the following, unless otherwise specified, sound pressure and time include error ranges that are generally acceptable in the relevant art. The abnormality detection system described in detail below extracts tire sound from the sound generated when an external force is applied to the tire, determines whether or not there is an abnormality in the tire from the tire sound, and displays the result of the determination of whether or not there is an abnormality in the tire. In the abnormality detection system, an abnormality in the tire is at least the tire pressure. That is, the tire pressure is not within the appropriate range. In addition to this, tire abnormalities also include internal tire failures such as separation, wear-related issues such as uneven wear or premature wear, and external damage such as scratches on the sidewall. It is preferable that the abnormality detection system can determine whether or not there is an abnormality in the nut, wheel, or hub, in addition to determining whether or not there is an abnormality in the tire.

[0015] (First example of an abnormality detection system) Figure 1 is a schematic diagram showing an example of a vehicle fitted with tires that are detected by the abnormality detection system of an embodiment of the present invention. Figure 2 is a schematic diagram showing a first example of an abnormality detection system of an embodiment of the present invention. The vehicle 10 shown in Figure 1 has multiple wheels, but the abnormality detection system 11 shown in Figure 2 shows only one wheel 12a as a representative. The vehicle 10 shown in Figure 1 has six wheels 12a to 12f. These six wheels 12a to 12f are each configured with tires 13a to 13f mounted on wheels 14 (see Figure 2). The six tires 13a to 13f are each attached to wheels 14 (see Figure 2), and each wheel 14 is attached to the vehicle 10 by nuts 15 (see Figure 2). Specifically, of the six tires 13a to 13f, tire 13a is the right front wheel tire. Tire 13b is the left front wheel tire. Tire 13c is the right rear outer wheel tire. Tire 13d is the inner right rear tire. Tire 13e is the outer left rear tire. Tire 13f is the inner left rear tire. Tires 13a to 13f are, for example, what are called pneumatic tires. Pneumatic tires include, for example, passenger car tires, heavy-duty tires, truck and bus tires, and light truck tires. The tires judged by the abnormality detection system are not particularly limited, but when determining air pressure, the tires are pneumatic tires. Vehicle 10 is not limited to the configuration of six tires shown in Figure 1. The number of tires is not limited to six, as it depends on the configuration of the vehicle, and the number of tires may exceed six, be four, or be three. The tires judged by the abnormality detection system are not particularly limited to those attached to the above-mentioned vehicle. They may be configurations with two tires, such as a two-wheeled vehicle, or a towed camper van or trailer, or configurations with one tire, such as a unicycle. Unicycles include electric unicycles, but are not limited to those specified. Two-wheeled vehicles include bicycles, motorcycles, and electric two-wheeled vehicles, but are not limited to those specified.

[0016] The abnormality detection system 11 determines whether there are any abnormalities in the tires 13a to 13f in a vehicle 10 equipped with wheels 12a to 12d on which tires 13a to 13f are mounted, for example, when the vehicle 10 is stopped. Note that the determination of whether there are any abnormalities in the tires 13a to 13f is not limited to when they are mounted on the vehicle 10, but may also be performed when they are removed from the vehicle 10. That is, the determination of whether there are any abnormalities in the tires 13a to 13f may be performed when the tires 13a to 13f mounted on wheels (not shown) are removed from the vehicle 10. For example, in the case of pre-ride inspections and pre-operation inspections, the determination of whether there are any abnormalities in the tires 13a to 13f is performed when the tires 13a to 13f are mounted on the vehicle 10. For example, in the case of vehicle maintenance, the determination of whether there are any abnormalities in the tires 13a to 13f is performed when the tires 13a to 13f are removed from the vehicle 10. In this case, each tire is individually determined to be abnormal.

[0017] The abnormality detection system 11 determines whether or not there is an abnormality in the tire by utilizing the sound generated when an external force is applied to the tire. For example, an external force is applied to the tire using the striking hammer 17 shown in Figure 2. The striking hammer 17 may be a general inspection hammer or one specifically designed for this purpose. The abnormality detection system 11 shown in Figure 2 includes, for example, a data processing unit 20. The data processing unit 20 has a determination unit 32 that determines whether or not there is an abnormality in the tire from tire sound information extracted from the sound generated when an external force is applied to the tire. The data processing unit 20 also includes, for example, a notification unit 33 that notifies a predetermined contact of the determination result regarding the presence or absence of an abnormality in the tire. The data processing unit 20 will be described in detail below.

[0018] [Data Processing Unit] The data processing unit 20 includes a sound collection unit 21, an amplifier (AMP) 22, a processing unit 23, a CPU 24, a memory 25, and a display unit 26. The data processing unit 20 is connected to an input unit 27. Furthermore, the data processing unit 20 includes a learning model creation unit 35 and an additional learning unit 36. The sound collection unit 21 collects sound generated when an external force is applied to the tire and obtains sound information. The sound generated when an external force is applied to the tire includes not only the sound produced when the tire is struck with a striking hammer 17, but also ambient sounds around the vehicle.

[0019] The amplifier 22 increases the signal level of the sound picked up by the sound pickup unit 21. The sound pickup unit 21 and the amplifier 22 are not particularly limited, and known components can be used as appropriate. For example, the sound pickup unit 21 may be a microphone (not shown). Alternatively, the sound pickup unit 21 may be a combination of a microphone and a filter that cuts specific frequencies. The microphone may be one built into a mobile device such as a smartphone or tablet, or it may be a microphone specifically designed and manufactured for this purpose. The sound pickup unit 21 is not particularly limited to being located in the data processing unit 20; it may also be attached to the striking hammer 17, for example, on the handle 17a of the striking hammer 17. In this case, the sound pickup unit 21 of the striking hammer 17 is configured to output the sound signal picked up by the data processing unit 20 wirelessly or via a wired connection. Furthermore, the data processing unit 20 is provided with a receiving unit (not shown) to which the sound signal output from the sound pickup unit 21 is input, and the receiving unit outputs the sound signal to the amplifier 22.

[0020] The display unit 26 displays the determination result of whether or not there is a tire abnormality, as determined by the determination unit 32. The display unit 26 displays the determination result of whether or not there is a tire abnormality, or that it is not possible to determine whether or not there is a tire abnormality. As described above, the display unit 26 can also use the display screen of a mobile device such as a smartphone or tablet.

[0021] The data processing unit 20 may be configured as a computer in which each part shown in the processing unit 23 functions when the CPU 24 executes a program stored in the memory 25, or it may be a dedicated device or dedicated terminal in which each part is configured with a dedicated circuit. For example, the data processing unit 20 may be configured to be implemented in a mobile terminal such as a smartphone or tablet. The data processing unit 20 may also be configured to be virtually provided on the cloud. In this case, the data processing unit 20 is configured as a server to be executed on the cloud. In this configuration, for example, the display unit 26 can use the display (display screen) of a mobile terminal such as a smartphone or tablet. The mobile terminal can be used as a device for inputting data to the data processing unit 20 and for displaying the judgment results obtained by the data processing unit 20.

[0022] The processing unit 23 includes, for example, a storage unit 30, an extraction unit 31, a determination unit 32, and a notification unit 33. The storage unit 30 stores sound information (sound data) acquired by the sound collection unit 21 when an external force is applied to the tire. The storage unit 30, like the memory 25 described above, is composed of, for example, a non-volatile memory that allows data to be rewritten. The sound information (sound data) stored in the storage unit 30 is amplified by the amplifier 22.

[0023] The sound information (sound data) supplied from the amplifier 22 is analog data. The storage unit 30 has an AD converter (not shown) that converts the acquired sound information (sound data) into a digital signal. The storage unit 30 samples the acquired sound information (sound data) at a predetermined sampling frequency and converts it into a digital signal. The method for converting the acquired sound information (sound data) into a digital signal is not particularly limited, and various known methods can be used.

[0024] The extraction unit 31 extracts tire noise from the sound information acquired by the sound collection unit 21. The sound information acquired by the sound collection unit 21 is converted into a digital signal in the storage unit 30, and the extraction unit 31 performs tire noise extraction on the digitized sound information. The tire noise extracted by the extraction unit 31 is stored, for example, in the memory 25 or the storage unit 30. The extraction unit 31 extracts tire noise from the sound information acquired by the sound collection unit 21, for example, using a threshold value based on the sound pressure deviation. More specifically, the sound data 40 shown in Figure 3(a) will be used as an example of the sound information acquired by the sound collection unit 21. Here, Figure 3(a) is a schematic diagram of an example of a sound waveform generated when an external force is applied to a tire, and (b) is a schematic diagram showing an example of an extracted region extracted as tire noise. In Figures 3(a) and (b), the vertical axis is sound pressure and the horizontal axis is time. The sound data 40 includes, for example, five sound waveforms 42a to 42e. The five sound waveforms 42a to 42e all show a decrease in sound pressure over time. The five sound waveforms 42a to 42e correspond to tire noise. Thresholds 43a and 43b are set for the sound data 40. For example, the standard deviation of the sound data 40 is used for thresholds 43a and 43b. If there is a signal with a sound pressure above threshold 43a and a signal with a sound pressure below threshold 43b, it is considered tire noise. In this way, tire noise can be extracted using the sound attenuation characteristics.

[0025] For example, for five sound waveforms 42a to 42e, the times at which the threshold is exceeded (44a to 44e) and the times at which the sound pressure becomes zero (45a to 45e) are determined. The interval between the time of point 44a and the time of point 45a where the sound pressure becomes zero is defined as the region 46a where tire noise is generated (see Figure 3(b)). The interval between the time of point 44b and the time of point 45b where the sound pressure becomes zero is defined as the region 46b where tire noise is generated (see Figure 3(b)). The interval between the time of point 44c and the time of point 45c where the sound pressure becomes zero is defined as the region 46c where tire noise is generated (see Figure 3(b)). The interval between the time of point 44d and the time of point 45d where the sound pressure becomes zero is defined as the region 46d where tire noise is generated (see Figure 3(b)). The interval between the time at point 44e and the time at point 45e, where the sound pressure becomes zero, is defined as the region 46e where tire noise is generated (see Figure 3(b)). When classified by frequency characteristics in this way, the interval of tire noise becomes clear, and the accuracy of tire noise extraction can be increased. Alternatively, for example, when extracting tire noise analytically as described above, instead of specifying the times of points 45a to 45e where the sound pressure becomes zero, a predetermined period may be set for points 44a to 44e that exceed a threshold, and the region 46a to 46e where tire noise is generated may be extracted.

[0026] The extraction unit 31 is not particularly limited to analytically extracting tire sounds as described above. For example, the extraction unit 31 may extract tire sounds from the sound information acquired by the sound collection unit 21 using a first machine learning model that has been trained based on sound information. The first machine learning model that has been trained is created, for example, by the learning model creation unit 35 described later. The first machine learning model that has been trained is trained by inputting sound information (digital signals) that include tire sounds as training data. As sound information that includes tire sounds, for example, the sound data 40 shown in Figure 3(a) above may be used. As training data for the first machine learning model, for example, percussion data of tires with known air pressures such as 600 kPa, 700 kPa, 800 kPa, 900 kPa, and 1000 kPa is used. The percussion data of tires with multiple levels corresponds to the correct answer data. The percussion data of multiple levels is called the percussion dataset. Furthermore, the aforementioned tapping sound dataset can also be used as training data for the second machine learning model described later. The first machine learning model may be created for each type of tire, such as normal tires and studless tires, or for each tire product manufactured and sold by each manufacturer. Moreover, the first machine learning model may be created for each vehicle (each vehicle type) on which the tire is mounted. Each vehicle (each vehicle type) refers to, for example, each type of tire, such as for trucks, light trucks, passenger cars, industrial vehicles, construction vehicles, agricultural machinery, and motorcycles. The first machine learning model may be trained by inputting the tire sounds that were determined to be abnormal and the tire sounds that were determined to be normal as training data. For example, if the extraction unit 31 cannot extract the tire sound using the analytical method described above, or if it cannot extract the tire sound using the trained first machine learning model, the display unit 26 will display that it is not possible to determine whether or not there is an abnormality in the tire. This allows inspectors to again apply external force to the tire using the striking hammer 17 or the like to generate sound and extract the tire noise.

[0027] The determination unit 32 shown in Figure 2 determines whether or not there is a tire abnormality based on tire sound information. For example, the sound data 40 shown in Figure 3(a) above has five sound waveforms 42a to 42e that represent tire sounds. For example, a threshold is set for determining whether or not there is a tire abnormality based on the frequency or pitch of the tire sound. This threshold is stored in memory 25, for example, and read out by the determination unit 32. The determination unit 32 compares the obtained tire sound with the preset threshold to determine whether or not there is a tire abnormality. The threshold for determining whether or not there is a tire abnormality is set for each type of tire or product being measured. As mentioned above, tire abnormalities include at least tire pressure, but there are others as well. The threshold is set appropriately according to the type of tire abnormality. For this reason, even if the tire being measured is the same, the threshold may differ depending on whether the tire abnormality is tire pressure or an internal tire malfunction.

[0028] The determination unit 32 is not particularly limited to analytically determining the presence or absence of a tire abnormality as described above. The determination unit 32 can also determine the presence or absence of a tire abnormality from tire sound information using a second machine learning model that has been trained based on tire sound. In some cases, the determination unit 32 may not be able to determine the presence or absence of a tire abnormality using the second machine learning model that has been trained. In this case, the determination unit 32 determines that it cannot determine the presence or absence of a tire abnormality. When the determination unit 32 determines that it cannot determine the presence or absence of a tire abnormality using the second machine learning model that has been trained, it is preferable that it provides a reason and displays it on the display unit 26. Reasons for not being able to determine include, for example, that the external force applied to the tire was small, that the impact on the tire was small, or that the position of the tire to which the external force was applied was inappropriate. For example, if sound pressure information is used from the tire sound information and the sound pressure is small, the reason may be that the external force applied to the tire was small or that the impact on the tire was small, and this may be displayed on the display unit 26. Furthermore, regarding the reasons mentioned above, for example, if frequency information is used from the tire noise information, and the frequency of the tire noise falls outside the range of tire abnormalities, the reason given is that the position of the tire to which the external force is applied is inappropriate, and this is displayed on the display unit 26.

[0029] The trained second machine learning model is created, for example, by the learning model creation unit 35 described later. For example, tire sounds that have been determined to be normal are input to the second machine learning model as training data to train the second machine learning model. In the second machine learning model, for example, similar to the first machine learning model, multiple levels of impact sound data of tires with known air pressures such as 600 kPa, 700 kPa, 800 kPa, 900 kPa, and 1000 kPa are used. Among the multiple levels of impact sound data, those where the air pressure is within the appropriate range for the tire are the correct data, and this correct data is used as training data for the second machine learning model. The second machine learning model may be a general-purpose machine learning model, but it is preferable that it corresponds to the first machine learning model. Specifically, it is preferable to train the second machine learning model using training data obtained from tires used as training data for the first machine learning model. This ensures that the tire type, tire product, or vehicle on which the tire is mounted matches, allowing the second machine learning model to improve its accuracy. Furthermore, it is preferable for both the first and second machine learning models to use training data for tires of the same product to further improve accuracy. Alternatively, the tire sounds that were determined to be abnormal and the tire sounds that were determined to be normal may be used as training data for the second machine learning model.

[0030] Each time the determination unit 32 determines whether or not there is a tire abnormality, it sends the determination result, or if it is not possible to determine whether or not there is a tire abnormality, a message indicating that it is not possible to determine this, to the display unit 26. The display unit 26 displays the determination result, or the message indicating that it is not possible to determine whether or not there is a tire abnormality. The display unit 26 can sequentially display data acquired by the data processing unit 20 and various data handled by the processing unit 23.

[0031] The notification unit 33, for example, has a communication function and includes an external communication unit (not shown) capable of data communication. The notification unit 33 can send emails, etc., to a predetermined contact (email address, etc.) outside the abnormality detection system 11. In this configuration, the notification unit 33 can provide notifications using text, graphics, sound, or voice. The external communication unit of the notification unit 33 uses a known wireless communication module. It is preferable that the notification unit 33 pre-determines notification content, for example, using text, graphics, sound, or voice, according to the result of the tire abnormality detection, such as when there is an abnormality in the tire or when there is no abnormality in the tire. This allows for prompt notification according to the result of the tire abnormality detection.

[0032] If the determination unit 32 determines that there is an abnormality in the tire, it sends a notification to the notification unit 33 that an abnormality has been determined in the tire. When the determination unit 32 determines that there is an abnormality in the tire in this way, the notification unit 33 notifies a designated contact (email address, etc.) that an abnormality has been determined in the tire. The designated contact is, for example, the tire manager, tire owner, tire seller, tire dealer, owner of the vehicle on which the tire is installed, vehicle manager, vehicle inspector, or vehicle user. In this case, the notification unit 33 may also notify the tire manager, tire owner, tire seller, tire dealer, owner of the vehicle on which the tire is installed, vehicle manager, vehicle inspector, or vehicle user of the determination that an abnormality has been determined in the tire via email or other means. In this way, the notification unit 33 can provide the tire manager, tire owner, tire seller, tire dealer, owner of the vehicle on which the tire is installed, vehicle manager, vehicle inspector, or vehicle user with timely information that an abnormality has been determined in the tire. This allows tire managers, tire owners, tire sellers, tire retailers, owners of vehicles on which the tires are installed, vehicle managers, vehicle inspectors, or vehicle users to be prompted to check tires that have been determined to be defective. Vehicle managers include vehicle operations managers. Vehicle users include vehicle drivers and vehicle operation staff.

[0033] The notification content of the notification unit 33 may include instructions prompting a tire check, in addition to the determination that there is an abnormality in the tire. The notification content of the notification unit 33 may also include the reason why it was determined that there is no abnormality in the tire. Furthermore, a mobile terminal such as a smartphone of a tire manager, tire owner, tire seller, tire dealer, owner of a vehicle on which the tire is installed, vehicle manager, vehicle inspector, or vehicle user can also be used as the notification unit 33. The display unit 26 described above can also function as the notification unit 33, in particular, by displaying a warning to notify that there is an abnormality in the tire when the determination unit 32 determines that there is an abnormality in the tire. The notification unit 33 may also be configured to have a buzzer that emits a specific sound, or a speaker that emits a specific sound or voice. In this configuration, the notification unit 33 can notify using sound or voice. As described above, the notification unit 33 can notify whether there is an abnormality in the tire or whether there is no abnormality in the tire using text, graphics, sound, or voice. The notification unit 33 may, for example, change the sound or voice it emits depending on whether there is a problem with the tire or not. The buzzer or speaker is not limited to being provided on the data processing unit 20; for example, a mobile terminal such as a smartphone can also be used as the notification unit 33.

[0034] Furthermore, for example, a threshold may be set for the number of times a tire is determined to be abnormal based on the determination result by the determination unit 32. If the number of times a tire is determined to be abnormal exceeds the set threshold, the display unit 26 will display, for example, that the number of times a tire has been determined to be abnormal is high. This will prompt the user to check the tire that has been determined to be abnormal. The tire that has been determined to be abnormal as described above may be all the tires mounted on the vehicle, or it may be a specific single tire. Also, regarding the determination result of the determination unit 32 regarding the presence or absence of tire abnormality, if a threshold is used to determine the presence or absence of tire abnormality, in addition to the threshold that distinguishes the presence or absence of tire abnormality, multiple determination values ​​may be set for, for example, the frequency of the tire sound or the pitch of the tire sound. The multiple determination values ​​may be set for, for example, on the side where the tire is abnormal and on the side where the tire is not abnormal, relative to the threshold for the frequency of the tire sound or the pitch of the tire sound. In this case, by comparing the obtained tire sound with the pre-set threshold and the multiple determination values ​​and identifying which of the threshold and the multiple determination values ​​the obtained tire sound is closest to, it is possible to determine not only the presence or absence of tire abnormality, but also the degree of deviation from the threshold, i.e., the level of abnormality determination. Furthermore, regarding the determination result of the determination unit 32 regarding the presence or absence of a tire abnormality, when using a trained second machine learning model, the second machine learning model is not a machine learning model that determines whether or not there is a tire abnormality in the obtained tire sound, but rather a machine learning model that determines whether the obtained tire sound falls into one of several levels. One of the multiple levels is set as a threshold. In this case, by identifying which level the obtained tire sound is closest to, it is possible not only to determine whether or not there is a tire abnormality, but also to identify the degree of deviation from the threshold, i.e., the level of abnormality determination. For example, the number of times a tire has been determined to be abnormal and the level of abnormality determination can also be notified by the notification unit 33 to the tire manager, tire owner, tire seller, tire dealer, owner of the vehicle on which the tire is installed, vehicle manager, vehicle inspector, or vehicle user. The notification by the notification unit 33 described above is not particularly limited, but may be in the form of text, graphics, sound, or voice.

[0035] The tire preferably has a recording unit (not shown) on which individual identification information is recorded. By providing a recording unit, it becomes easier to associate tires that have been determined to have abnormalities, and further, it becomes easier to associate tires that have had abnormalities. Therefore, even when the tires are rotated, it becomes easier to track tires that have had abnormalities and tires that have not. For this reason, it is preferable that the storage unit 30 stores the determination result of whether or not a tire has an abnormality, as determined by the determination unit 32, and the tire information of the tire that has been determined to have an abnormality, as a set. The recording unit is used as a means of managing the individual identification information of the tire. The recording unit can be, for example, a two-dimensional barcode, a one-dimensional barcode, a label (sticker), RFID (Radio Frequency Identification), etc. An embedded type RFID is more preferable as the recording unit because it can suppress the risk of reading failure due to abrasion or peeling, or tampering due to post-processing of the tire surface. In addition, by accumulating data by associating individual identification information such as manufacturing information and tire usage history with history information of changes in the tire's condition, it can also be used for quality control information. The recording unit records information related to tire use, such as tire size, date of tire use commencement, date of intermediate inspection, tire wear status, presence or absence of external abnormalities, and tire pressure history. Alternatively, the tire's recording unit may record individual tire identification information, and the aforementioned tire usage information corresponding to the individual tire identification information may be stored, for example, in memory 25 or on a cloud server.

[0036] The input unit 27 is used for inputting instructions and data to operate the data processing unit 20. For example, if the display unit 26 has a touch sensor function, a screen for input can be displayed on the display unit 26 and used as the input unit 27.

[0037] The learning model creation unit 35 creates the first and second machine learning models that have been trained as described above. For example, the learning model creation unit 35 uses the recorded sound information (digital signal) that includes tire noise as described above as training data, inputs it into the first machine learning model, and trains it to create the first machine learning model that has been trained. As training data for the first machine learning model, for example, it uses multiple levels of tapping sound data (ground truth data) of a tire with known air pressure, as described above. The learning model creation unit 35 also uses the tire noise (ground truth data) that has been determined to be normal as training data, inputs it into the second machine learning model, and trains it to create the second machine learning model that has been trained. Note that if the learning model creation unit 35 does not create the first and second machine learning models that have been trained, the learning model creation unit 35 is not necessarily required. The first and second machine learning models that have been trained can also be stored in the memory 25 in advance. The determination unit 32 may read the trained first machine learning model and the trained second machine learning model from the memory 25 and perform a determination of whether there is tire noise and whether there is a tire abnormality.

[0038] Examples of machine learning models that can be used as the first and second machine learning models include CNN (Convolutional Neural Network), DNN (Deep Neural Network), Random Forest, K-Means method, SVM (Support Vector Machine), One-Class SVM (Support Vector Machine), and LOF (Local Outlier Factor). By using these models, it is possible to accurately determine whether or not there is a defect in the tire.

[0039] The additional learning unit 36 ​​performs additional learning on the trained first machine learning model and the trained second machine learning model. The additional learning unit 36 ​​uses the results of tire sound extraction by the trained first machine learning model to further learn the trained first machine learning model. Additional learning of the trained first machine learning model is performed, for example, based on feedback information of the tire sound extraction results. More specifically, if something that is not a tire sound is extracted as a tire sound, this sound data indicating that it is not a tire sound is used as training data to further learn the trained first machine learning model. Also, if something that is a tire sound is not a tire sound, this sound data indicating that it is a tire sound (ground truth data) is used as training data to further learn the trained first machine learning model. Through such additional learning, the accuracy of tire sound extraction by the trained first machine learning model can be improved, and furthermore, the accuracy of tire anomaly detection can also be improved. As mentioned above, in the additional training of the first machine learning model that has already been trained, sound data (ground truth data) indicating that it is a tire sound is used as training data. However, related information may be added to this training data, for example. Related information includes, for example, tire size, type of vehicle to which the tire is attached, tire mounting position, tire individual identification number, air pressure, presence or absence of external abnormalities, tire wear condition, date and time the tire was installed, mileage, driving area, acceleration / deceleration history, and vehicle operation history.

[0040] The additional learning unit 36 ​​further trains the trained second machine learning model using the results of the trained second machine learning model's determination of whether or not there is a tire abnormality. This additional training of the trained second machine learning model is performed, for example, based on feedback information from the tire abnormality determination results. More specifically, if a tire with abnormal tire pressure is determined to be normal, this data is used as training data to further train the trained second machine learning model, representing data where the tire pressure is abnormal, i.e., the tire has an abnormality. Similarly, if a tire with normal tire pressure is determined to be abnormal, this data is used as training data to further train the trained second machine learning model, representing data where the tire pressure is normal (ground truth data). This additional training improves the accuracy of the trained second machine learning model's determination of whether or not there is a tire abnormality, for example, the determination of whether or not there is an abnormality in the tire pressure. Additional training can also be used to handle changes in conditions, such as new tire sizes and changes in the vehicle to which the tires are attached. As mentioned above, in the additional training of the second machine learning model that has already been trained, data where the tire pressure is at a normal value (ground truth data) is used as training data. However, related information may be added to this training data, for example. Related information includes, for example, tire size, type of vehicle on which the tire is attached, tire mounting position, tire individual identification number, air pressure, presence or absence of external abnormalities, tire wear condition, date and time the tire was installed, mileage, driving area, acceleration / deceleration history, and vehicle operation history.

[0041] In the abnormality detection system 11, for example, when an external force is applied to the tire by a striking hammer 17, the sound generated is collected by the sound collection unit 21, the tire sound is extracted from the sound information acquired by the sound collection unit 21 by the extraction unit 31, the determination unit 32 determines whether or not there is an abnormality in the tire from the tire sound information, and the display unit 26 displays the determination result of whether or not there is an abnormality in the tire determined by the determination unit 32. With a simple configuration, the presence or absence of an abnormality in the tire can be determined without incurring costs. Therefore, the presence or absence of abnormalities in tire air pressure, etc., can be determined at low cost and simply.

[0042] When collecting sound generated when an external force is applied to a tire, there are many constraints, such as minimizing white noise, acquiring the impact sound with a certain level of power, and suppressing double impacts due to rebound from the tire. However, in the abnormality detection system 11, by extracting the tire sound with the extraction unit 31, the special work required to collect the impact sound can be omitted, thereby reducing inspection costs. The system collects sound generated when an external force is applied to the tire, and this sound collection section may be continuous throughout the tire or vehicle inspection process, or only during the inspection of the area around the tire. By using a pre-trained first machine learning model, the characteristics of the sound generated when an external force is applied to the tire, such as the characteristics of the impact sound, can be extracted. This allows for the exclusion of ambient noise with a sound pressure greater than the sound generated when an external force is applied to the tire (impact sound), easing constraints due to the surrounding environment during inspection and further improving the accuracy of the judgment. The tire sound may be extracted using a pre-trained first machine learning model after pre-processing such as filtering has been performed on the sound interval (impact interval) generated when an external force is applied to the tire.

[0043] The striking method using the striking hammer 17 is not particularly limited. When striking the tire with the striking hammer 17, the side of the tire is preferable if it is mounted in a way that allows the side of the tire to be struck. If the side of the tire cannot be struck, the tread of the tire is preferable. The striking method is preferable because it improves the accuracy of sound extraction, such as striking multiple times with a constant force and position. The sound collection method is not particularly limited, but it is preferable to fix the position of the sound collection unit (microphone) during striking because it improves the accuracy of sound extraction.

[0044] (First Example of Abnormality Determination Method) FIG. 4 is a flowchart showing a first example of an abnormality determination method according to a first example of an abnormality determination system of an embodiment of the present invention. The first example of the abnormality determination method will be described using the abnormality determination system 11 shown in FIG. 2, but the first example of the abnormality determination method is not particularly limited to the use of the abnormality determination system 11. The first example of the abnormality determination method is, for example, a method for determining the presence or absence of tire abnormalities in a state where the vehicle 10 is stopped. In the first example of the abnormality determination method, as shown in FIG. 4, the sound generated when an external force is applied to the tire is picked up by the sound collection unit 21 (step S10). In step S10, the external force applied to the tire is applied, for example, by hitting the side portion of the tire with the impact hammer 17, and the hitting sound generated at this time is picked up by the sound collection unit 21. Next, the extraction unit 31 extracts the tire sound from the sound information acquired by the sound collection unit 21 (step S12). In step S12, for example, after the signal level of the sound (the hitting sound of the tire) picked up by the sound collection unit 21 is increased by the amplifier 22, it is converted into a digital signal by the storage unit 30. Next, the tire sound is extracted from the hitting sound of the tire (digital signal) by the extraction unit 31. The extraction of the tire sound is performed, for example, by setting threshold values 43a and 43b as shown in the sound data 40 shown in FIG. 3(a) above to extract the tire sound.

[0045] Next, the determination unit 32 determines whether or not there is a tire abnormality based on the tire sound information (step S14). In step S14, the determination unit 32 compares the extracted tire sound with a preset threshold to determine whether or not there is a tire abnormality. Next, the determination result of whether or not there is a tire abnormality determined by the determination unit 32 is displayed on the display unit 26 (step S16). In step S16, if it is determined that there is a tire abnormality, the display unit 26 displays that there is a tire abnormality. If it is determined that there is no tire abnormality, the display unit 26 displays that there is no tire abnormality. The display format for indicating whether or not there is a tire abnormality is not particularly limited and may be in the form of text, or the presence or absence of a tire abnormality may be displayed on the display unit 26 using different colors. If it is determined that there is a tire abnormality, the notification unit 33 may also notify the tire manager, tire owner, tire seller, tire retailer, owner of the vehicle on which the tire is installed, vehicle manager, vehicle inspector, or vehicle user that there is a tire abnormality. The notification by the notification unit 33 described above can be in the form of text, graphics, sound, or voice. For example, the notification may be in the form of text, graphics, sound, or voice depending on whether there is a problem with the tire or not. The result of the tire abnormality determination and the tire information of the tire for which abnormality was determined may be stored together in the storage unit 30.

[0046] (Second Example of Anomaly Detection Method) Figure 5 is a flowchart showing a second example of an anomaly detection method according to the first example of the anomaly detection system of the embodiment of the present invention. The second example of the anomaly detection method will be explained using the anomaly detection system 11 shown in Figure 2, but the second example of the anomaly detection method is not particularly limited to the use of the anomaly detection system 11. The second example of the anomaly detection method is, for example, a method for determining whether or not there is an anomaly in the tires when the vehicle 10 is stopped. The second example of the anomaly detection method differs from the first example of the anomaly detection method in that, as shown in Figure 5, it uses a trained first machine learning model for extracting tire sounds (step S20), and otherwise the process is the same as the first example of the anomaly detection method. For this reason, a detailed explanation will be omitted. In the second example of the anomaly detection method, the tire sound of the tires is extracted from the sound information acquired by the sound collection unit 21 in step S10 (step S20). In step S20, the extraction unit 31 reads the first machine learning model that has been trained from the memory 25 and uses the first machine learning model that has been trained based on sound information to extract tire sounds from the sound information acquired by the sound collection unit 21.

[0047] Next, the determination unit 32 determines whether there is an abnormality in the tire from the tire sound information (step S22). Since step S22 is the same process as step S14 described above, detailed description thereof is omitted. Next, the determination result of whether there is an abnormality in the tire determined by the determination unit 32 is displayed on the display unit 26 (step S24). Since step S24 is the same process as step S16 described above, detailed description thereof is omitted. If it is determined that there is an abnormality in the tire, the notification unit 33 can notify the tire administrator, tire owner, tire dealer, owner of the vehicle to which the tire is attached, vehicle administrator, vehicle inspector, or vehicle user that there is an abnormality in the tire. The notification by the notification unit 33 described above can be in the form of text, graphics, sound, or voice, and for example, the text, graphics, emitted sound, or voice can be changed to notify when there is an abnormality in the tire and when there is no abnormality in the tire. The determination result of whether there is an abnormality in the tire and the tire information of the tire for which the presence or absence of the abnormality is determined may be set and stored in the storage unit 30. In step S20, if the learned first machine learning model cannot extract the tire sound, it is displayed that the presence or absence of an abnormality in the tire cannot be determined.

[0048] (Third example of abnormality determination method) FIG. 6 is a flowchart showing a third example of an abnormality determination method according to the first example of the abnormality determination system of the embodiment of the present invention. The third example of the abnormality determination method will be described using the abnormality determination system 11 shown in FIG. 2, but the third example of the abnormality determination method is not particularly limited to the use of the abnormality determination system 11. The third example of the abnormality determination method is, for example, a method for determining whether there is an abnormality in the tire when the vehicle 10 is in a stopped state. The third example of the abnormality determination method is different from the second example of the abnormality determination method shown in FIG. 5 in that a learned second machine learning model is used for determining whether there is an abnormality in the tire from the tire sound information (step S26) as shown in FIG. 6, and the other steps are the same as those in the second example of the abnormality determination method. Therefore, detailed description thereof is omitted.

[0049] In the third example of the abnormality detection method, the tire sound is extracted from the sound information acquired by the sound collection unit 21 in step S10 (step S20). Step S20 is the same process as step S20 in the second example of the abnormality detection method described above, so a detailed explanation is omitted. Next, the determination unit 32 determines whether or not there is an abnormality in the tire from the tire sound information (step S26). In step S26, the determination unit 32 reads the trained second machine learning model from the memory 25 and uses the trained second machine learning model based on the tire sound to determine whether or not there is an abnormality in the tire from the tire sound information. Next, the determination result of whether or not there is an abnormality in the tire determined by the determination unit 32 is displayed on the display unit 26 (step S28). In step S28, if it was determined in step S26 that there is an abnormality in the tire, the display unit 26 displays that there is an abnormality in the tire. Also in step S28, if it was determined in step S26 that there is no abnormality in the tire, the display unit 26 displays that there is no abnormality in the tire. Furthermore, in step S28, if it is not possible to determine whether or not there is a tire abnormality in step S26, the display unit 26 displays that it is not possible to determine whether or not there is a tire abnormality. The display format for indicating whether or not there is a tire abnormality, whether or not there is a tire abnormality, and whether or not it is not possible to determine whether or not there is a tire abnormality is not particularly limited, and may be in the form of text, or the presence or absence of a tire abnormality and the inability to determine whether or not there is a tire abnormality may be displayed in different colors.

[0050] Furthermore, if a tire abnormality is detected, the notification unit 33 can also notify the tire manager, tire owner, tire retailer, vehicle owner, vehicle manager, vehicle inspector, or vehicle user of the tire abnormality. The notification by the notification unit 33 can be in the form of text, graphics, sound, or voice. For example, the notification may be in the form of text, graphics, sound, or voice depending on whether there is a tire abnormality or not. The result of the tire abnormality detection and the tire information of the tire for which abnormality was detected may be stored together in the storage unit 30. In step S20, if the trained first machine learning model cannot extract tire sounds, it will display that it is not possible to determine whether there is a tire abnormality.

[0051] (Second example of an abnormality detection system) Next, a second example of an abnormality detection system will be described. The second example of an abnormality detection system differs from the abnormality detection system 11 shown in Figure 2 in that, in addition to determining whether or not there is an abnormality in the tire, the extraction unit 31 extracts the nut sound from the sound information acquired by the sound collection unit 21, the determination unit 32 determines whether or not there is an abnormality in the nut from the nut sound information, and the display unit 26 displays the determination result of whether or not there is an abnormality in the nut determined by the determination unit 32. Furthermore, if an abnormality is determined to be found in the nut, the notification unit 33 has the function of notifying, for example, the tire manager or tire owner that there is an abnormality in the nut. Other than these differences, the second example of an abnormality detection system has the same configuration as the abnormality detection system 11 shown in Figure 2.

[0052] A second example of the abnormality detection system is the data processing unit 20 shown in Figure 2, in which the extraction unit 31 further has the function of extracting the nut sound of the nut 15 (see Figure 2) from the sound information acquired by the sound collection unit 21, using a third machine learning model that has been trained based on the sound information acquired by the sound collection unit 21 when an external force is applied to the tire. The determination unit 32 further has the function of determining whether or not there is an abnormality in the nut 15 from the nut sound information. The display unit 26 further has the function of displaying the determination result of whether or not there is an abnormality in the nut 15 determined by the determination unit 32. The notification unit 33 can notify using characters, figures, sounds or voices according to the determination result of whether or not there is an abnormality in the nut 15, specifically, it can notify using characters, figures, sounds or voices whether or not there is an abnormality in the nut 15. For example, the notification unit 33 can change the characters, figures, sounds or voices emitted depending on whether or not there is an abnormality in the nut 15.

[0053] The trained third machine learning model is created, for example, by the training model creation unit 35. The trained third machine learning model is trained by inputting information (digital signals) of recorded sound that includes the sound of a nut as training data. Examples of recorded sound information that includes the sound of a nut include sound data (not shown) of the sound produced when a nut is struck with a striking hammer 17. The training data for the third machine learning model uses, for example, sound data at multiple levels of a nut with a known tightening torque, or sound data at multiple levels of a nut that is known not to be damaged. This sound data at multiple levels of nuts corresponds to the correct answer data. The third machine learning model may be created for each type of nut, or for each size of nut, or for each vehicle to which the nut is attached (for each compatible vehicle type), for example, for trucks, light trucks, passenger cars, industrial vehicles, construction vehicles, agricultural machinery, motorcycles, etc. Alternatively, the nut sounds that indicate an abnormality and the nut sounds that indicate a normal nut may be used as training data and input into a third machine learning model to train the third machine learning model.

[0054] The determination unit 32 determines whether or not there is an abnormality in the nut based on the information from the nut sound. An abnormality in the nut is at least a loose nut. In addition to a loose nut, other abnormalities in the nut include, for example, deformation of the nut, damage such as cracking of the nut, or overtightening of the nut. For example, in sound data (not shown) including the nut sound, a threshold is set for determining whether or not there is an abnormality in the nut, for example, based on the frequency of the nut sound or the pitch of the nut sound. This threshold is stored in advance in, for example, memory 25 and read out by the determination unit 32. The determination unit 32 compares the nut sound with the preset threshold to determine whether or not there is an abnormality in the nut.

[0055] The determination unit 32 can also determine whether or not a nut is abnormal from the nut sound information using a trained fourth machine learning model based on the nut sound. The trained fourth machine learning model is created, for example, by the learning model creation unit 35. The trained fourth machine learning model is trained by inputting nut sounds that have been determined to be normal as training data (ground truth data). The fourth machine learning model may be a general-purpose machine learning model, but it is preferable that it corresponds to the third machine learning model. Specifically, it is preferable to train the fourth machine learning model using training data obtained from nuts used as training data for the third machine learning model. This ensures that the type of nut, the size of the nut, or the vehicle to which the nut is attached matches, thus improving the determination accuracy of the trained fourth machine learning model. Furthermore, it is even more preferable for the third and fourth machine learning models to use training data from nuts of the same product in order to further improve the determination accuracy. Alternatively, the nut sounds that indicate a nut is abnormal and the nut sounds that indicate a nut is not abnormal may be used as training data and input into the fourth machine learning model to train it.

[0056] Furthermore, regarding the determination result of the determination unit 32 regarding the presence or absence of a nut abnormality, when a threshold is used to determine the presence or absence of a nut abnormality, in addition to the threshold that distinguishes the presence or absence of a nut abnormality, multiple determination values ​​can also be set for, for example, the frequency of the nut sound or the pitch of the nut sound. The multiple determination values ​​are set, for example, on the side where the nut is abnormal and on the side where the nut is not abnormal, relative to the threshold of the frequency or pitch of the nut sound. In this case, by comparing the obtained nut sound with the pre-set threshold and the multiple determination values ​​and identifying which value among the threshold and the multiple determination values ​​the obtained nut sound is closest to, it is possible to determine not only the presence or absence of a nut abnormality, but also the degree of deviation from the threshold, i.e., the level of abnormality determination. Furthermore, regarding the determination result of the determination unit 32 regarding the presence or absence of a nut abnormality, when a trained second machine learning model is used, the second machine learning model is not a machine learning model that determines the presence or absence of a nut abnormality for the obtained nut sound, but rather a machine learning model that determines whether the obtained nut sound falls into one of multiple levels. One of the multiple levels is set as the threshold. In this case, by identifying which level the obtained nut sound is closest to, it is possible not only to determine whether or not there is a defect in the nut, but also to determine the degree of deviation from the threshold, i.e., the level of the defect determination. For example, the number of times the nut has been determined to be defective and the level of the defect determination can also be notified by the notification unit 33 to the nut's manager, the nut's owner, the nut's seller, the nut's retailer, the owner of the vehicle to which the nut is installed, the vehicle's manager, the vehicle's inspector, or the vehicle's user. The notification by the notification unit 33 described above is not particularly limited, but may be in the form of text, graphics, sound, or voice.

[0057] The learning model creation unit 35 further has the function of creating the trained third machine learning model and the trained fourth machine learning model described above. For example, the learning model creation unit 35 inputs the information (digital signal) of the sound recorded, which includes the nut sound as described above, as training data into the third machine learning model and trains it to create the trained third machine learning model. As training data for the third machine learning model, as described above, for example, the unit uses tapping data (ground truth data) at multiple levels of nuts with a known tightening torque, or tapping data (ground truth data) at multiple levels of nuts that are known not to be damaged. The learning model creation unit 35 also inputs the nut sound (ground truth data) that has been determined to be normal as training data into the fourth machine learning model and trains it to create the trained fourth machine learning model. The trained third machine learning model and the trained fourth machine learning model can also be stored in the memory 25 in advance. The determination unit 32 may read the trained third machine learning model and the trained fourth machine learning model from the memory 25 and perform a determination of whether there is a nut sound or an abnormality in the nut. For the third and fourth machine learning models, for example, CNN, random forest, K-Means method, SVM, One Class SVM, and LOF can be used, just as with the first and second machine learning models described above.

[0058] The additional learning unit 36 ​​further has the function of performing additional learning on the trained third machine learning model and the trained fourth machine learning model. The additional learning unit 36 ​​uses the results of nut sound extraction by the trained third machine learning model to further learn the trained third machine learning model. Additional learning of the trained third machine learning model is performed, for example, based on feedback information of the nut sound extraction results. More specifically, if something that is not a nut sound is extracted as a nut sound, it is used as training data as sound data indicating that it is not a nut sound, and the trained third machine learning model is further learned. Also, if something that is a nut sound is not a nut sound, it is used as training data as sound data indicating that it is a nut sound (ground truth data), and the trained third machine learning model is further learned. Through such additional learning, the accuracy of nut sound extraction by the trained third machine learning model can be improved, and furthermore, the accuracy of anomaly detection of nuts can also be improved. As mentioned above, in the additional training of the pre-trained third machine learning model, sound data indicating that it is a nut sound (ground truth data) is used as training data. However, related information may be added to this training data, for example. Related information includes, for example, tire size, type of vehicle on which the tire is mounted, tire mounting position, tire individual identification number, air pressure, presence or absence of external abnormalities, tire wear condition, date and time of tire installation, mileage, driving area, acceleration / deceleration history, and vehicle operation history. Furthermore, related information may include wheel size, wheel material, wheel product name, nut size, nut material, nut shape, nut part number, and presence or absence of external damage to the nut.

[0059] The additional learning unit 36 ​​further trains the trained fourth machine learning model using the results of the previously trained fourth machine learning model's determination of whether or not there is an abnormality in the nut. This additional training of the trained fourth machine learning model is performed, for example, based on feedback information from the determination of whether or not there is an abnormality in the nut. More specifically, if a nut that is not properly tightened is determined to be properly tightened, this data is used as training data to further train the trained fourth machine learning model, indicating that the nut is not properly tightened, i.e., that there is an abnormality in the nut. Similarly, if a nut that is properly tightened is determined to be not properly tightened, this data is used as training data to further train the trained fourth machine learning model, indicating that the nut is properly tightened (ground truth data). Through this additional training, the accuracy of the previously trained fourth machine learning model's determination of whether or not there is an abnormality in the nut, for example, the determination of whether or not there is an abnormality in the nut tightening, can be improved. Proper tightening of a nut means that the nut is not loose. For example, additional learning can be used to handle changes in conditions such as the size of the nut, the material of the nut, the shape of the nut, and changes in the vehicle to which the nut is attached. As mentioned above, in the additional learning of the fourth machine learning model that has already been trained, data where the nut is properly tightened (ground truth data) is used as training data, but related information may be added to this training data, for example. Related information includes, for example, tire size, type of vehicle to which the tire is attached, tire mounting position, individual tire identification number, air pressure, presence or absence of external abnormalities, tire wear condition, date and time of tire installation, mileage, driving area, acceleration / deceleration history, and vehicle operation history. Furthermore, related information may include wheel size, wheel material, wheel product name, nut size, nut material, nut shape, nut part number, and presence or absence of external damage to the nut.

[0060] As described above, the second example of the abnormality detection system can achieve the same effects as the first example of the abnormality detection system, and can also detect abnormalities in nuts. However, similar to the tire, the detection of abnormalities in nuts may include cases where it is not possible to determine whether or not there is an abnormality. The display unit 26 can display that it is not possible to determine whether or not there is an abnormality. Furthermore, the second example of the abnormality detection system can also perform the detection of abnormalities in tires and nuts with a simple configuration and without incurring costs. Therefore, the detection of abnormalities such as tire air pressure and loose nuts can be performed easily and at low cost.

[0061] The second example of the anomaly detection system uses a pre-trained first machine learning model and a pre-trained third machine learning model, but it is not limited to this configuration. It is also possible to use a single machine learning model that extracts tire sounds and nut sounds. In this case, for example, the machine learning model is trained using ground truth data as training data for tire sounds and nut sounds, respectively, to obtain a pre-trained machine learning model. Furthermore, the second example of the anomaly detection system uses a pre-trained second machine learning model and a pre-trained fourth machine learning model for detection, but it is not limited to this configuration. It is also possible to use a single machine learning model that detects abnormal tire sounds and abnormal nut sounds. In this case, for example, the machine learning model is trained using ground truth data as training data for tire sounds where the tire is determined not to be abnormal and nut sounds where the nut is determined not to be abnormal, respectively, to obtain a pre-trained machine learning model.

[0062] (Fourth Example of Anomaly Detection Method) Figure 7 is a flowchart showing a fourth example of an anomaly detection method according to a second example of an anomaly detection system of an embodiment of the present invention. The fourth example of an anomaly detection method will be explained using the second example of an anomaly detection system. As described above, the second example of an anomaly detection system has the same basic configuration as the anomaly detection system 11 shown in Figure 2. The fourth example of an anomaly detection method is not particularly limited to using the second example of an anomaly detection system. The fourth example of an anomaly detection method is, for example, a method for determining whether there is an anomaly in the tires and whether there is an anomaly in the nuts when the vehicle 10 is stopped. The fourth example of an anomaly detection method differs from the third example of an anomaly detection method shown in Figure 6 in that, in addition to extracting tire sounds and nut sounds, determining whether there is an anomaly in the tires from the tire sound information and displaying the determination result of whether there is an anomaly in the tires, it also determines whether there is an anomaly in the nuts from the nut sound information and displays the determination result of whether there is an anomaly in the nuts. Otherwise, the process is the same as the third example of an anomaly detection method. For this reason, a detailed explanation will be omitted.

[0063] In the fourth example of the abnormality detection method, as shown in Figure 7, the tire noise and nut noise are extracted from the sound information acquired by the sound collection unit 21 in step S10 (step S30). In step S30, the extraction of tire noise is the same process as in step S20 of the second example of the abnormality detection method described above, so a detailed explanation is omitted. In step S30, the extraction unit 31 reads the learned third machine learning model from the memory 25 and uses the learned third machine learning model, which is based on the sound information generated when an external force is applied to the tire, to extract the nut noise from the sound information acquired by the sound collection unit 21. Next, the determination unit 32 determines whether or not there is an abnormality in the tire from the tire noise information extracted in step S30 (step S26). Step S26 is the same process as in step S26 of the third example of the abnormality detection method described above, so a detailed explanation is omitted. Next, the determination result of whether or not there is an abnormality in the tire, determined by the determination unit 32, is displayed on the display unit 26 (step S28). Step S28 is the same as step S28 in the third example of the abnormality determination method described above, so a detailed explanation of it will be omitted.

[0064] In step S32, the determination unit 32 determines whether there is an abnormality in the nut based on the nut sound information extracted in step S30. In step S32, the determination unit 32 compares the nut sound with a preset threshold to determine whether there is an abnormality in the nut. Next, the determination result of whether there is an abnormality in the nut, as determined by the determination unit 32, is displayed on the display unit 26 (step S34). In step S34, if it was determined in step S32 that there is an abnormality in the nut, the display unit 26 displays that there is an abnormality in the nut. Also in step S34, if it was determined in step S32 that there is no abnormality in the nut, the display unit 26 displays that there is no abnormality in the nut. The display format for indicating whether there is an abnormality in the nut and whether there is no abnormality in the nut is not particularly limited and may be in text or the presence or absence of an abnormality in the nut may be displayed in different colors. If it is determined that there is an abnormality in the tire, the notification unit 33 may also notify the tire manager, tire owner, tire dealer, owner of the vehicle on which the tire is installed, vehicle manager, vehicle inspector, or vehicle user that there is an abnormality in the tire. The notification by the notification unit 33 described above can be in the form of text, graphics, sound, or voice. For example, the notification may be in the form of text, graphics, sound, or voice depending on whether there is a problem with the tire or not. If it is determined that there is a problem with the nut, the notification unit 33 can also notify, for example, the tire manager, tire owner, tire dealer, owner of the vehicle on which the tire is installed, vehicle manager, vehicle inspector, or vehicle user that there is a problem with the nut. For example, the notification may be in the form of text, graphics, sound, or voice depending on whether there is a problem with the nut or not. The determination result of whether there is a problem with the tire, the determination result of whether there is a problem with the nut, and the tire information of the tire for which a problem has been determined may be stored together in the storage unit 30. In step S30, if the trained first machine learning model cannot extract the tire sound, the method for determining whether there is a problem with the tire is terminated.

[0065] (Fifth Example of Anomaly Detection Method) Figure 8 is a flowchart showing a fifth example of an anomaly detection method according to a second example of the anomaly detection system of an embodiment of the present invention. The fifth example of an anomaly detection method will be explained using the second example of the anomaly detection system. As described above, the second example of the anomaly detection system has the same basic configuration as the anomaly detection system 11 shown in Figure 2. The fifth example of an anomaly detection method is not particularly limited to using the second example of the anomaly detection system. The fifth example of an anomaly detection method is, for example, a method for determining whether there is an anomaly in the tires and whether there is an anomaly in the nuts when the vehicle 10 is stopped. The fifth example of an anomaly detection method differs from the fourth example of an anomaly detection method shown in Figure 7 in that, as shown in Figure 8, it uses a trained fourth machine learning model to determine whether there is an anomaly in the nuts from the information of the nut sound (step S36), and otherwise the process is the same as the fourth example of an anomaly detection method. For this reason, a detailed explanation will be omitted.

[0066] In the fifth example of the abnormality detection method, the sound information acquired by the sound collection unit 21 in step S10 is used to extract the tire sound and nut sound of the tire (step S30). Step S30 is the same process as step S30 in the fourth example of the abnormality detection method described above, so a detailed explanation is omitted. Next, the determination unit 32 determines whether or not there is an abnormality in the tire from the tire sound information (step S26). Step S26 is the same process as step S26 in the third example of the abnormality detection method described above, so a detailed explanation is omitted. Next, the determination result of whether or not there is an abnormality in the tire, determined by the determination unit 32, is displayed on the display unit 26 (step S28). Step S28 is the same process as step S28 in the third example of the abnormality detection method described above, so a detailed explanation is omitted. The determination unit 32 determines whether or not there is an abnormality in the nut from the nut sound information extracted in step S30 (step S36). In step S36, the determination unit 32 reads the trained fourth machine learning model from the memory 25 and uses the trained fourth machine learning model based on nut sounds to determine whether or not there is an abnormality in the nut based on the nut sound information.

[0067] Next, the determination result of whether or not there is an abnormality in the nut, as determined by the determination unit 32, is displayed on the display unit 26 (step S38). In step S38, if it was determined in step S36 that there is an abnormality in the nut, the display unit 26 displays that there is an abnormality in the nut. Also in step S38, if it was determined in step S36 that there is no abnormality in the nut, the display unit 26 displays that there is no abnormality in the nut. Also in step S38, if it is not possible to determine whether or not there is an abnormality in the nut in step S36, the display unit 26 displays that it is not possible to determine whether or not there is an abnormality in the nut. The display format for indicating whether or not there is an abnormality in the nut, whether or not there is an abnormality in the nut, and whether or not it is not possible to determine whether or not there is an abnormality in the nut is not particularly limited, and may be in the form of text, or the presence or absence of an abnormality in the nut and the inability to determine whether or not there is an abnormality in the nut may be displayed in different colors. In step S38, if it is not possible to determine whether or not there is an abnormality in the nut in step S36, the display unit 26 displays that it is not possible to determine whether or not there is an abnormality in the nut, but this display is not required. In step S38, if it was determined in step S36 that there was an abnormality in the nut, the display unit 26 may be configured to display that there was an abnormality in the nut.

[0068] Furthermore, if a tire abnormality is detected, the notification unit 33 can also notify the tire manager, tire owner, tire dealer, vehicle owner, vehicle manager, vehicle inspector, or vehicle user of the tire abnormality. The notification by the notification unit 33 can be in the form of text, graphics, sound, or voice. For example, the text, graphics, sound, or voice may be changed depending on whether there is a tire abnormality or not. If a nut abnormality is detected, the notification unit 33 can also notify the tire manager, tire owner, tire dealer, vehicle owner, vehicle manager, vehicle inspector, or vehicle user of the nut abnormality. For example, the text, graphics, sound, or voice may be changed depending on whether there is a nut abnormality or not. The results of the tire abnormality detection, the results of the nut abnormality detection, and the tire information of the tire for which abnormality was detected may be stored together in the storage unit 30. If the trained first machine learning model fails to extract tire noise in step S30, the method for determining whether or not there is a tire abnormality is terminated.

[0069] The first to fifth examples of the abnormality detection method described above are all performed when the vehicle 10 is stopped, but the timing of the abnormality detection method is, for example, before the vehicle is put into operation. In the third to fifth examples of the abnormality detection method described above, in step S28, if it is not possible to determine whether or not there is an abnormality in the tire in step S26, the display unit 26 displays that it is not possible to determine whether or not there is an abnormality in the tire, but this display is not required. In step S28, the display unit 26 may only display that there is an abnormality in the tire if it is determined in step S26 that there is an abnormality in the tire. Also, for example, the notification unit 33 may display text or a graphic, or emit a sound or voice, only if there is an abnormality in the tire. Also, for example, the notification unit 33 may only display text or a graphic, or emit a sound or voice, only if there is an abnormality in the nut.

[0070] In the first and second examples of the abnormality detection system described above, the presence or absence of a wheel abnormality may also be determined based on the wheel's sound. In this case, the sound collection unit collects the sound generated when an external force is applied to the tire, and the extraction unit extracts the wheel's sound from the sound information acquired by the sound collection unit. The determination unit determines whether or not there is a wheel abnormality based on the wheel sound information. The determination result of whether or not there is a wheel abnormality, as determined by the determination unit, is displayed on the display unit. The extraction of the wheel sound and the determination of whether or not there is a wheel abnormality based on the wheel sound information can utilize machine learning models, similar to the tire abnormality determination described above. The abnormality detection method can also determine whether or not there is a wheel abnormality and display the determination result. In addition to the wheel, the presence or absence of abnormalities in the hub and bolts can also be determined in the same way as the tire abnormality determination and the determination result of whether or not there is a hub and bolt abnormality can be displayed. If an abnormality is detected in the hub or bolt, the notification unit 33 can also notify, for example, the tire manager, tire owner, tire retailer, owner of the vehicle on which the tire is installed, vehicle manager, vehicle inspector, or vehicle user of the abnormality in the hub or bolt. For example, the notification can be made using text, graphics, sound, or voice depending on the result of the determination of whether or not there is an abnormality in the hub or bolt.

[0071] The first to fifth examples of the abnormality detection method described above all determine whether or not there is an abnormality in the tire, but the level of the tire abnormality may also be determined and displayed. The level of the tire abnormality can be determined as described above. Similarly, the fourth and fifth examples of the abnormality detection method described above, which determine whether or not there is an abnormality in the nut, all determine whether or not there is an abnormality in the nut, but the level of the nut abnormality may also be determined and displayed. The level of the nut abnormality can be determined as described above.

[0072] The present invention is basically configured as described above. Although the abnormality detection system of the present invention has been described in detail above, the present invention is not limited to the embodiments described above, and various improvements or modifications may be made without departing from the spirit of the present invention.

[0073] 10 Vehicle 11 Anomaly detection system 12a, 12b, 12c, 12d, 12e, 12f Wheel 13a, 13b, 13c, 13d, 13e, 13f Tire 14 Wheel 15 Nut 17 Striking hammer 17a Handle 20 Data processing unit 21 Sound collection unit 22 Amplifier 23 Processing unit 24 CPU 25 Memory 26 Display unit 30 Storage unit 31 Extraction unit 32 Judgment unit 33 Notification unit 35 Learning model creation unit 36 ​​Additional learning unit 40 Sound data 42a-42e Sound waveform 43a, 43b Threshold 44a, 44b, 44c, 44d, 44e Points 45a, 45b, 45c, 45d, 45e Points 46a, 46b, 46c, 46d, 46e areas

Claims

1. An abnormality detection system comprising: a sound collection unit that collects sound generated when an external force is applied to a tire; an extraction unit that extracts tire sound from the sound information acquired by the sound collection unit; a determination unit that determines whether or not there is an abnormality in the tire from the tire sound information; and a display unit that displays the determination result of whether or not there is an abnormality in the tire determined by the determination unit.

2. The abnormality determination system according to claim 1, wherein the extraction unit extracts the tire sound from the sound information acquired by the sound collection unit using a threshold value based on the sound pressure deviation.

3. The abnormality detection system according to claim 1, wherein the extraction unit extracts the tire sound from the sound information acquired by the sound collection unit using a first machine learning model that has been trained based on the sound information.

4. The abnormality determination system according to claim 1, wherein the determination unit determines whether or not there is an abnormality in the tire from the tire sound information using a second machine learning model that has been trained based on the tire sound.

5. An anomaly detection system according to claim 3, comprising an additional learning unit for further training a first machine learning model that has already been trained, wherein the additional learning unit further trains the first machine learning model using the results of the extraction of the tire sound by the first machine learning model that has already been trained.

6. An anomaly detection system according to claim 4, further comprising an additional learning unit for additionally training a trained second machine learning model, wherein the additional learning unit further trains the trained second machine learning model using the determination result of whether or not the tire has an anomaly by the trained second machine learning model.

7. The abnormality determination system according to claim 1, wherein the tire is attached to a wheel, the wheel is attached to a vehicle by nuts, the extraction unit extracts the nut sound of the nuts from the sound information acquired by the sound acquisition unit using a third machine learning model that has been trained based on sound information generated when an external force is applied to the tire, the determination unit determines whether or not there is an abnormality in the nuts from the nut sound information, and the display unit displays the determination result of whether or not there is an abnormality in the nuts determined by the determination unit.

8. The abnormality determination system according to claim 7, wherein the determination unit determines whether or not there is an abnormality in the nut from the information of the nut sound using a fourth machine learning model that has been trained based on the nut sound.

9. An anomaly determination system according to claim 8, further comprising an additional learning unit for further training a trained fourth machine learning model, wherein the additional learning unit further trains the trained fourth machine learning model using the determination result of whether or not the nut has an anomaly by the trained fourth machine learning model.

10. The abnormality determination system according to claim 1, wherein the abnormality of the tire determined by the determination unit is at least the air pressure of the tire.

11. The abnormality determination system according to claim 7, wherein the abnormality of the nut determined by the determination unit is at least a loosening of the nut.

12. The abnormality determination system according to claim 1, wherein the determination unit determines that it is not possible to determine whether or not there is an abnormality in the tire if it is not possible to determine whether or not there is an abnormality, and the display unit displays the result of determining whether or not there is an abnormality in the tire, or that it is not possible to determine whether or not there is an abnormality.

13. An abnormality determination system according to any one of claims 1 to 12, further comprising a storage unit that stores as a set the determination result of whether or not the tire has an abnormality, determined by the determination unit, and tire information of the tire for which the presence or absence of an abnormality has been determined.

14. An abnormality detection system according to any one of claims 1 to 12, further comprising a notification unit that, when the determination unit determines that there is an abnormality in the tire, notifies the manager of the tire, the owner of the tire, the seller of the tire, the tire dealer, the owner of the vehicle on which the tire is installed, the manager of the vehicle, the inspector of the vehicle, or the user of the vehicle that there is an abnormality in the tire.

Images