Information processing method and information processing device
The information processing method and device improve tire failure detection by using infrared imaging to estimate damage depth and notify users, addressing inefficiencies in current methods and reducing costs and downtime in mining and other industries.
Patent Information
- Application Number
- PCT/JP2023/041842
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-31
- Filing Date
- 2023-11-21
- Publication Date
- 2024-12-05
AI Technical Summary
Current tire failure detection technologies are inefficient and costly, particularly in mining vehicles, where sudden tire breakdowns lead to decreased productivity and regular inspections are necessary to prevent unexpected failures.
An information processing method and device that uses an infrared camera to obtain thermoimages of tires, identifies damaged areas, determines surface temperatures, estimates damage depth, and notifies users of relative temperature relationships to automate tire failure determination, improving detection accuracy and reducing inspection burdens.
Enhances the accuracy and efficiency of tire failure detection, reducing downtime and costs by automating the process and allowing for proactive maintenance, thereby improving productivity and safety in mining and other industries.
Smart Images

Figure JP2023041842_05122024_PF_FP_ABST
Abstract
Description
Information processing method and information processing device Cross-reference to related applications
[0001] This application claims priority from Japanese Patent Application No. 2023-090197 (filed May 31, 2023), the entire disclosure of which is incorporated herein by reference.
[0002] The present disclosure relates to an information processing method and an information processing device.
[0003] There are known technologies for determining tire damage. For example, Patent Literature 1 discloses a tire damage detection system that detects the size of a damaged portion of a tire based on the diameter of a rim wheel, based on image data of the tire mounted on the rim wheel.
[0004] Japanese Patent Application Laid-Open No. 2019-202729
[0005] However, there is a need for further improvements in the usefulness of technology for determining tire failure. For example, if a mining vehicle used at a mining site experiences a sudden tire failure, productivity is reduced due to the need to transport the vehicle and replace the tire. In addition, regular tire inspections by workers to prevent sudden failures are a cost burden. Therefore, there is a need for automation of tire failure determination.
[0006] The purpose of the present disclosure, made in consideration of the above circumstances, is to improve the usefulness of technology for determining tire failures.
[0007] [1] An information processing method according to one embodiment of the present disclosure is an information processing method executed by an information processing device, the information processing device including a control unit, a communication unit, and a memory unit, and capable of communicating with a network via the communication unit, and the information processing device includes, by the control unit, acquiring a thermal image of a tire from an infrared camera, identifying the position of a damaged portion in the tire from the thermal image, identifying the surface temperature of the damaged portion from the thermal image, and estimating the depth of the damaged portion from the position and the surface temperature.
[0008] [2] An information processing method according to one embodiment of the present disclosure is the information processing method described in [1] above, wherein the tire includes a carcass ply, the carcass ply includes a folded portion, and the information processing method preferably includes determining whether the position of the damaged portion is in an area where the folded portion is located, and estimating the depth from the determination result and the surface temperature.
[0009] [3] An information processing method according to an embodiment of the present disclosure is the information processing method described in [2] above, and preferably includes: storing a correspondence between the surface temperature and the depth in the memory unit for each of a case where the position is in an area with the folded portion and a case where the position is in an area without the folded portion; and estimating the depth from the correspondence.
[0010] [4] An information processing method according to an embodiment of the present disclosure is the information processing method described in any one of [1] to [3] above, and preferably includes: acquiring the tire cavity temperature; and estimating the depth of the damaged portion based on the position, the surface temperature, and the cavity temperature.
[0011] [5] An information processing method according to an embodiment of the present disclosure is the information processing method described in any one of [1] to [4] above, and preferably includes: determining, from the thermal image, a relative relationship between the surface temperatures obtained for each of a plurality of damaged portions in the tire; and notifying an alert to a user based on the determination result of the relative relationship.
[0012] [6] An information processing device according to one embodiment of the present disclosure is an information processing device that includes a control unit, a communication unit, and a memory unit, and is capable of communicating with a network via the communication unit, wherein the control unit performs operations including: acquiring a thermal image of a tire from an infrared camera; identifying the position of a damaged portion in the tire from the thermal image; identifying the surface temperature of the damaged portion from the thermal image; and estimating the depth of the damaged portion from the position and the surface temperature.
[0013] According to the present disclosure, the usefulness of the technology for determining tire faults can be improved.
[0014] It is a diagram showing a schematic configuration of an information processing system according to an embodiment of the present disclosure. It is a block diagram showing the configuration of the information processing device shown in FIG. It is a diagram showing an example of a thermo image. It is a diagram showing a cross-sectional view of a tire, etc. It is a flowchart showing the operation of the information processing device.
[0015] An information processing system S according to an embodiment of the present disclosure will be described below with reference to the drawings. Note that the drawings are schematic and the ratios of dimensions may differ from those of the actual system.
[0016] An overview of an information processing system S according to this embodiment will be described with reference to Fig. 1. As shown in Fig. 1, the information processing system S includes an information processing device 10 and an infrared camera 20 that can communicate with each other. Fig. 1 shows one information processing device 10 and one infrared camera 20. However, the information processing system S may include any number of information processing devices 10 and any number of infrared cameras 20.
[0017] The information processing device 10 is configured with one or more computers. In this embodiment, the information processing device 10 is described as being configured with one computer. However, the information processing device 10 may be configured with multiple computers, such as a cloud computing system. In the present disclosure, the information processing device 10 may be referred to as a "tire malfunction determination device."
[0018] The infrared camera 20 is configured with a computer including one or more cameras. The infrared camera 20 may be, for example, a thermography camera capable of capturing thermal images. The images captured by the infrared camera 20 may be still images or videos. The infrared camera 20 generates a thermal image of the tire and transmits it to the information processing device 10. In this embodiment, the thermal image of the tire captures at least a portion of the tire. In addition to at least a portion of the tire, the thermal image may also capture at least a portion of the vehicle on which the tire is mounted.
[0019] The infrared camera 20 may be, for example, a fixed camera installed on the vehicle's travel route at the mine site. This allows the infrared camera 20 to capture images of the vehicle traveling on the travel route, and is less likely to reduce the vehicle's operating rate and productivity at the mine. However, the infrared camera 20 is not limited to a fixed camera and may be a mobile camera such as a camera mounted on a drone or a tablet terminal that can be carried by a person.
[0020] The network 50 is any communication network that allows mutual communication between the information processing device 10 and the infrared camera 20. The network 50 may be, for example, the Internet, a mobile communication network, a fixed communication network, a LAN (Local Area Network), or a combination of these.
[0021] In the present disclosure, the tire is not particularly limited, but may be an OR (Off The Road) tire mounted on a mining vehicle such as a transport vehicle, construction vehicle, work vehicle, or heavy machinery vehicle used at a mining site, etc. However, the tire may be a tire other than an OR tire.
[0022] In the present disclosure, the vehicle is, for example, a mining vehicle used at a mining site, etc. However, the vehicle is not limited to the above-described mining vehicle, and may be any vehicle capable of being fitted with tires, such as a transport vehicle, a construction vehicle, a work vehicle or heavy machinery vehicle, a bus, a passenger car, a motorcycle, a bicycle, or an airplane.
[0023] The configuration of the information processing device 10 will be described with reference to Fig. 2. Fig. 2 is a block diagram showing the configuration of the information processing device 10. The information processing device 10 includes a communication unit 11, an output unit 12, an input unit 13, a storage unit 14, and a control unit 15. In the information processing device 10, the communication unit 11, the output unit 12, the input unit 13, the storage unit 14, and the control unit 15 are connected to each other via wire or wirelessly so as to be able to communicate with each other.
[0024] The communication unit 11 includes a communication module for connecting to the network 50. The communication module is a communication module compatible with mobile communication standards such as 4G (4th Generation) or 5G (5th Generation). The communication module may be a communication module compatible with standards such as wired LAN or wireless LAN. The communication module may be a communication module compatible with short-range wireless communication standards such as Wi-Fi (registered trademark), Bluetooth (registered trademark), or infrared communication. In this embodiment, the information processing device 10 is connected to the network 50 via the communication unit 11. This allows the information processing device 10 to communicate with the infrared camera 20, other computers, etc.
[0025] The output unit 12 includes one or more output devices, such as a display, a speaker, a lamp, etc. As a result, the output unit 12 outputs an image, sound, light, etc.
[0026] The input unit 13 includes one or more input devices. The input devices are, for example, a touch panel, a camera, a microphone, etc. The input unit 13 receives, for example, an input operation by a user of the information processing device 10.
[0027] The storage unit 14 is, for example, a semiconductor memory, a magnetic memory, an optical memory, or the like. The storage unit 14 functions, for example, as a main storage device, an auxiliary storage device, or a cache memory. The storage unit 14 stores any information used in the operation of the information processing device 10. For example, the storage unit 14 stores a system program, an application program, embedded software, a database, or the like. The information stored in the storage unit 14 may be updatable, for example, with information obtained from the network 50 via the communication unit 11.
[0028] For example, the storage unit 14 may store tire identification information for one or more tires that are the subject of failure determination. In the present disclosure, tire identification information is also referred to as a tire ID (Identifier). The tire identification information for a tire is information that can uniquely identify the tire. The tire identification information for a tire is, for example, uniquely generated by the information processing device 10, but is not limited to this and may be the tire's serial number, the vehicle number of the vehicle on which the tire is mounted, or the like. Furthermore, the storage unit 14 may store tire-related information related to the tire in association with the tire identification information.
[0029] The tire-related information includes any information related to the tire. The tire-related information may include, for example, information about the distance from the tire surface to the carcass ply, tire damage information, tire configuration information, information about the vehicle on which the tire is mounted, or information about the position of the tire on the vehicle. The tire damage information may include, for example, the position, shape, depth, and registration date and time of damage previously sustained to the tire. The tire configuration information includes, for example, the tire type, model number, material properties, tread pattern, belt angle, size, or weight. The information about the vehicle on which the tire is mounted includes vehicle identification information, type, model number, engine displacement, number of tires mounted, or number of shafts.
[0030] The control unit 15 includes one or more processors. The processor may be, for example, a general-purpose processor such as a CPU (Central Processing Unit) or a dedicated processor specialized for specific processing. The control unit 15 is not limited to a processor and may include one or more dedicated circuits. The dedicated circuits may be, for example, an FPGA (Field-Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit). The control unit 15 controls each component to realize the functions of the information processing device 10, including the functions of the components such as the communication unit 11, the output unit 12, the input unit 13, and the memory unit 14 described above.
[0031] The control unit 15 of the information processing device 10 acquires a thermo image from the infrared camera 20 via the communication unit 11 and the network 50 at any time.
[0032] 3 shows an example of a thermo image. The thermo image is a captured image of a tire TR. The control unit 15 performs image analysis on the thermo image to identify the location of a damaged portion 31 on the surface of the tire TR and the surface temperature of the damaged portion of the tire TR. Any method of identification may be used.
[0033] 4 shows a cross-sectional view of a tire TR. The tire TR includes a carcass ply 41 therein. The carcass ply 41 includes a turned-up portion 45. Therefore, the tire TR includes a region 42 with the turned-up portion 45 and a region 43 without the turned-up portion 45.
[0034] A turn-up portion 45 of the carcass ply 41 is present on the inner side of the damaged portion 46 of the surface 44. No turn-up portion 45 is present on the inner side of the damaged portion 47. In this case, the surface temperatures of the damaged portion 46 and the damaged portion 47 change differently. Graph A in FIG. 4 shows the relationship between the surface temperature of the damaged portion 46 and the distance from the carcass ply 41 (here, the turn-up portion 45) to the tip 48 of the damaged portion 46 in region 42. Graph B in FIG. 4 shows the relationship between the surface temperature of the damaged portion 47 and the distance from the carcass ply 41 to the tip 49 of the damaged portion 47 in region 43. As shown in graphs A and B, the surface temperature of the damaged portion 46 increases by a larger amount (i.e., sensitivity) as the distance from the tip to the carcass ply 41 decreases than the surface temperature of the damaged portion 47. In other words, even at the same distance, the surface temperature of the damaged portion is relatively high in the region where the turn-up portion 45 is present. This is because the damaged portion is thought to be affected by the heat of the turn-up portion 45.
[0035] The correspondence between the distance from the carcass ply 41 to the tip of the damaged portion and the surface temperature of the damaged portion is stored in the memory unit 14 for both the case where the damaged portion is located in an area of the folded portion 45 (for example, graph A in Figure 4) and the case where the damaged portion is not located in an area of the folded portion 45 (for example, graph B in Figure 4).
[0036] The area of the tire TR that is prone to failure is the area where the turn-up portion 45 is located. Therefore, the control unit 15 determines whether the location of the damaged portion is in the area where the turn-up portion 45 is located based on the location of the identified damaged portion and tire-related information stored in the memory unit 14. Based on the determination result, the control unit 15 determines in the memory unit 14 a correspondence relationship between the distance from the carcass ply 41 to the tip of the damaged portion and the surface temperature of the damaged portion. The control unit 15 applies the surface temperature identified from the thermal image to the correspondence relationship between the distance from the carcass ply 41 to the tip of the damaged portion and the surface temperature of the damaged portion to determine the distance from the carcass ply 41 to the tip of the damaged portion. The control unit 15 estimates the depth of the damaged portion by subtracting the determined distance from the distance from the surface 44 of the tire TR to the carcass ply 41. The units of distance and depth may be, for example, [mm] or [cm]. This configuration allows the control unit 15 to improve the accuracy of estimating the depth of the damaged portion and thereby improve the usefulness of the technology for determining tire failures.
[0037] Alternatively, the control unit 15 may estimate the depth of the damaged portion based on whether the damaged portion is located in a region where the turn-back portion 45 is present and the surface temperature of the damaged portion. With this configuration, the control unit 15 estimates the depth taking into account whether the damaged portion is located in a turn-back portion, thereby further improving the estimation accuracy. The estimation may be performed using any depth estimation algorithm (e.g., a calculation formula), or may be performed based on the correspondence between whether the damaged portion is located in a region where the turn-back portion 45 is present and the surface temperature of the damaged portion. With this configuration, the control unit 15 can employ different estimation methods depending on whether the damaged portion is located in a turn-back portion, thereby further improving the estimation accuracy. The depth estimation algorithm may be constructed using a statistical method such as machine learning or deep learning, using the surface temperature of the damaged portion and the depth of the damaged portion measured by a human or the like as training data.
[0038] Additionally or alternatively, the control unit 15 may acquire the cavity temperature of the tire TR. This configuration allows the control unit 15 to further improve estimation accuracy. The cavity temperature is acquired via the network 50 from a tire pressure monitoring system (TPMS) installed in the tire TR. The control unit 15 may estimate the depth of the damaged portion based on the position of the damaged portion, the surface temperature of the damaged portion, and the cavity temperature. Any depth estimation algorithm may be used to estimate the depth.
[0039] Additionally or alternatively, the control unit 15 may determine, from the thermal image, a relative relationship between the surface temperatures acquired for each of the multiple damaged portions of the tire TR. For example, the control unit 15 may determine whether the thermal image contains a relatively high outlier exceeding a first threshold. The control unit 15 may issue an alert to the user based on the determination result of the relative relationship. For example, the control unit 15 may issue an alert to a user terminal operated by the user via the network 50. The alert may include, for example, prompting the user to inspect the damaged portion corresponding to the outlier, or identifying and presenting the location of the damaged portion corresponding to the outlier. With this configuration, the control unit 15 can motivate the user to inspect more carefully the damaged portion with a particularly high surface temperature among the multiple damaged portions, and can also prompt the user to simplify or omit inspection of the damaged portion with a low surface temperature among the multiple damaged portions. This improves the usefulness of the technology for determining tire faults.
[0040] Additionally or alternatively, the control unit 15 may transmit damage information regarding damage to the surface of the tire TR to any other terminal via the communication unit 11 and the network 50. The damage information may include, for example, the location, number, or depth of the damaged portions. The damage information is stored in the storage unit 14 in association with the damage identification information. The damage information is output by any other terminal as an image, sound, light, or the like. This allows the information processing device 10 to notify users of the information processing system S of the damage information, allowing them to easily understand the damage. Thus, the information processing device 10 can plan inspection, repair, replacement, etc. of the tire TR before the tire TR breaks down. In particular, when the tire TR is replaced, the tire TR may be replaced with a different tire TR according to the damage information of the tire TR. For example, if the surface temperature of the damaged portion of the tire TR is higher than the temperature tolerance set for the tire TR, the tire TR may be replaced with a different type of tire TR with higher heat resistance. Alternatively, if the number of damaged portions of the tire TR is greater than a predetermined number, the tire TR may be replaced with a different type of tire TR that is less susceptible to damage. In this way, the user of the information processing system S can select a tire TR that is suitable for use at a mining site based on the damage information of the tire TR.
[0041] FIG. 5 shows a flowchart of an information processing method performed by the information processing device 10.
[0042] In step S1, the control unit 15 of the information processing device 10 acquires a thermographic image of the tire TR from the infrared camera 20. The acquisition of the thermographic image may be performed periodically or irregularly. In step S2, the control unit 15 identifies the position of a damaged portion in the tire TR and the surface temperature of the damaged portion from the thermographic image. In step S3, the control unit 15 estimates the depth of the damaged portion from the identified position and surface temperature.
[0043] Although the present disclosure has been described based on the drawings and embodiments, it should be noted that those skilled in the art can make various modifications and alterations based on the present disclosure. Therefore, it should be noted that these modifications and alterations are included within the scope of the present disclosure. For example, the configurations or functions included in each embodiment can be rearranged so as not to cause logical inconsistencies. Furthermore, the configurations or functions included in each embodiment can be used in combination with other embodiments, and multiple configurations or functions can be combined into one, divided, or partially omitted.
[0044] Also, for example, an embodiment is possible in which a general-purpose computer functions as the information processing device 10 according to the above-described embodiment. Specifically, a program describing the processing content for realizing each function of the information processing device 10 according to the above-described embodiment is stored in the memory of the general-purpose computer, and the program is read and executed by a processor. Therefore, the present disclosure can also be realized as a program executable by a processor or a non-transitory computer-readable medium storing the program. Non-transitory computer-readable media include, for example, a magnetic recording device, an optical disk, a magneto-optical recording medium, or a semiconductor memory.
[0045] According to the present disclosure, the usefulness of the technology for determining tire faults can be improved.
[0046] [Contribution to the United Nations-led Sustainable Development Goals (SDGs)] The SDGs have been proposed to realize a sustainable society. One embodiment of the present invention is thought to be a technology that can contribute to "No. 9 - Build resilient infrastructure for industry, innovation and other goals."
[0047] S: Information processing system
Claims
1. An information processing method executed by an information processing device, the information processing device including a control unit, a communication unit, and a memory unit, and capable of communicating with a network via the communication unit, the information processing method including, by the control unit, obtaining a thermographic image of a tire from an infrared camera, identifying a position of a damaged portion in the tire from the thermographic image, identifying a surface temperature of the damaged portion from the thermographic image, and estimating a depth of the damaged portion from the position and the surface temperature.
2. An information processing method according to claim 1, wherein the tire includes a carcass ply, the carcass ply includes a folded portion, and the information processing method includes determining whether the position of the damaged portion is in an area where the folded portion is located, and estimating the depth from the determination result and the surface temperature.
3. An information processing method according to claim 2, comprising: storing in the memory unit a correspondence relationship between the surface temperature and the depth for each of the cases where the position is in an area with the folded portion and where the position is in an area without the folded portion; and estimating the depth from the correspondence relationship.
4. An information processing method according to claim 1, comprising: acquiring the tire cavity temperature; and estimating the depth of the damaged portion based on the position, the surface temperature, and the cavity temperature.
5. An information processing method according to claim 1, comprising: determining a relative relationship between the surface temperatures obtained for each of a plurality of damaged portions of the tire from the thermo image; and notifying a user of an alert based on the result of determining the relative relationship.
6. An information processing device including a control unit, a communication unit, and a memory unit, and capable of communicating with a network via the communication unit, wherein the control unit performs operations including: acquiring a thermographic image of a tire from an infrared camera; identifying a position of a damaged portion in the tire from the thermographic image; identifying a surface temperature of the damaged portion from the thermographic image; and estimating a depth of the damaged portion from the position and the surface temperature.
Citation Information
Patent Citations
Tire uneven profile measuring device, tire uneven profile determining system, tire uneven profile measuring method, and tire uneven profile determining method
JP2008191087A
Image processing method, image processor, image processing program, and recording medium
JP2011138208A
Tire state evaluation system and tire state evaluation method
JP2016003964A
Vehicle-based multimode discovery
US20150232065A1
Tire degradation evaluation system, and method and program thereof
WO2018207265A1
