Temperature detection device, temperature detection method, and program

Abstract

This invention provides a temperature detection device, a temperature detection method, and a program that can more easily detect the temperature of equipment laid around railway tracks. [Solution] A temperature detection device for detecting the temperature of equipment laid at a predetermined location around a railway track comprises: an acquisition unit that acquires a visible light image of the area around the railway track captured by a visible light camera and a temperature image of the area around the railway track captured by a far-infrared camera; an estimation target area identification unit that identifies a predetermined area on the visible light image, including the location where the equipment is laid, as an estimation target area; an estimation unit that estimates the equipment area on the visible light image of the equipment present in the estimation target area; a matching unit that associates feature points of the visible light image and the temperature image; and a detection unit that identifies a temperature area on the temperature image corresponding to the equipment area based on the matching of feature points by the matching unit, and detects the temperature of the temperature area as the temperature of the equipment.

Description

Technical Field

[0001] The present invention relates to a temperature detection device, a temperature detection method, and a program.

Background Art

[0002] Among the facilities laid along railway lines, many are powered on, and abnormal heat generation or ignition may occur due to the mixing of impurities into the contact surface, ground faults caused by component contact, or damage. Therefore, it is desirable to measure the temperature of the facilities as part of the maintenance of the facilities along the line and check for abnormalities.

[0003] Therefore, in Patent Document 1, a technique for locking a temperature sensor to a pantograph fitting is disclosed. This makes it easier to visually identify an abnormal temperature when the pantograph fitting generates heat due to an abnormal current or the like.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, in Patent Document 1, since the temperature sensor needs to be locked to each pantograph fitting, for example, the initial cost and maintenance cost increase.

[0006] The present invention has been made in view of such problems, and an object thereof is to provide a temperature detection device, a temperature detection method, and a program that can more easily detect the temperature of facilities laid around a railway line.

Means for Solving the Problems

[0007] One aspect of the present invention is a temperature detection device for detecting the temperature of equipment laid at a predetermined location around a railway track, and is characterized by comprising: an acquisition unit that acquires a visible light image of the area around the railway track captured by a visible light camera and a temperature image of the area around the railway track captured by a far-infrared camera; an estimation target area identification unit that identifies a predetermined area on the visible light image, including the location where the equipment is laid, as an estimation target area; an estimation unit that estimates the equipment area on the visible light image of the equipment located in the estimation target area; a matching unit that associates feature points of the visible light image and the temperature image; and a detection unit that identifies a temperature area on the temperature image corresponding to the equipment area based on the association of feature points by the matching unit, and detects the temperature of the temperature area as the temperature of the equipment.

[0008] One aspect of the present invention is a temperature detection method for detecting the temperature of equipment laid at a predetermined location around a railway track, characterized by comprising: an acquisition step of acquiring a visible light image of the area around the railway track captured by a visible light camera and a temperature image of the area around the railway track captured by a far-infrared camera; an estimation target area identification step of identifying a predetermined area on the visible light image that includes the location where the equipment is laid as an estimation target area; an estimation step of estimating the equipment area on the visible light image of the equipment located in the estimation target area; a matching step of associating feature points between the visible light image and the temperature image; and a detection step of identifying a temperature area on the temperature image corresponding to the equipment area based on the association of feature points by the matching step, and detecting the temperature of the temperature area as the temperature of the equipment.

[0009] A program according to one aspect of the present invention is a program that can be executed by a computer capable of exchanging information with an input / output device to detect the temperature of equipment laid at a predetermined location around a railway line, and is characterized by including an acquisition step of acquiring a visible light image of the area around the railway line captured by a visible light camera and a temperature image of the area around the railway line captured by a far-infrared camera; an estimation target area identification step of identifying a predetermined area on the visible light image that includes the location where the equipment is laid as an estimation target area; an estimation step of estimating the equipment area on the visible light image of the equipment located in the estimation target area; a matching step of associating feature points between the visible light image and the temperature image; and a detection step of identifying a temperature area on the temperature image corresponding to the equipment area based on the association of feature points by the matching step, and detecting the temperature of the temperature area as the temperature of the equipment, wherein the program causes a computer to execute the processing. [Effects of the Invention]

[0010] According to the present invention, the temperature of equipment laid around railway tracks can be detected. [Brief explanation of the drawing]

[0011] [Figure 1] Figure 1 shows the external configuration of the temperature detection system 1. [Figure 2] Figure 2A is an example of a visible light image, and Figure 2B is an example of a temperature image. [Figure 3] Figure 3 is a block diagram showing the internal configuration of the temperature detection device 13. [Figure 4] Figure 4 is a flowchart showing the flow of the temperature detection process performed by the temperature detection device 13. [Figure 5] Figure 5 shows an example of the area to be estimated. [Figure 6] Figure 6 is an enlarged view of the estimated target area in Figure 5. [Figure 7] Figure 7 is an illustrative diagram showing the correspondence between the feature points in the visible light image on the left and the feature points in the temperature image on the right, connected by straight lines. [Figure 8]FIG. 8 is a diagram showing a temperature region T of an impedance bond on a temperature image corresponding to an equipment region V2 of the impedance bond in the visible light image on the left side.

Embodiments for Carrying Out the Invention

[0012] [One Embodiment] (Configuration of Temperature Detection System 1) FIG. 1 illustrates a temperature detection system 1 according to an embodiment of the present invention. FIG. 1 is a diagram showing an example of the use of the temperature detection system 1.

[0013] The temperature detection system 1 includes a visible light camera 11, an infrared camera 12, and a temperature detection device 13.

[0014] The visible light camera 11 is provided on a vehicle 3 traveling on a line 2, captures visible light around the line 2, and creates a visible light image that is an image reflecting colors and brightness that can be directly seen by humans.

[0015] The infrared camera 12 is provided on the vehicle 3, detects infrared rays along the line 2, and creates a temperature image based on the temperature difference.

[0016] A in FIG. 2 is an example of a visible light image, and B in FIG. 2 is an example of a temperature image. Although the temperature image in FIG. 2B is shown as a black-and-white image, originally, a high-temperature portion is displayed in red or white, and a low-temperature portion is displayed in blue or black. The visible light image and the temperature image include an image of an impedance bond P as equipment laid at a predetermined position around the line 2. The impedance bond P is a device used in an electric railway system to safely maintain the operation of a train while appropriately separating a signal circuit and a power supply circuit. The lead line and the symbol P in FIG. 2A are added for explanation.

[0017] The temperature detection device 13 detects the temperature (for example, the surface temperature of the impedance bond P) of the equipment laid at a predetermined position around the line 2 based on the visible light image captured by the visible light camera 11 and the temperature image captured by the far-infrared camera 12. As shown in FIG. 1, the temperature detection device 13 may be provided on the vehicle 3 or may be provided outside the vehicle 3.

[0018] (Internal configuration of the temperature detection device 13) FIG. 3 is a block diagram showing the internal configuration of the temperature detection device 13. The temperature detection device 13 includes a communication unit 21, a storage unit 22, and a control unit 23.

[0019] The communication unit 21 communicates with the visible light camera 11, the far-infrared camera 12, etc. by wire or wirelessly. The communication unit 21 may be realized by short-range wireless communication such as Bluetooth (registered trademark) or Wi-Fi (registered trademark). Further, the communication unit 21 can also perform communication with the visible light camera 11, the far-infrared camera 12, etc. via the Internet or the like by wide-area wireless communication.

[0020] The storage unit 22 has, for example, a ROM (Read Only Memory), a RAM (Random Access Memory), and a non-volatile memory. The storage unit 22 stores the control application program of the temperature detection device 13, various information necessary for its execution, and information generated by processing.

[0021] The control unit 23 is composed of a CPU (Central Processing Unit), the storage unit 22 (ROM (Read Only Memory), RAM (Random Access Memory), non-volatile memory, etc.), hardware, and other elements. By executing a control application program (not shown) stored in the storage unit 22, the control unit 23 controls the entire temperature detection device 13 and functions as an acquisition unit 31, an estimation target area specifying unit 32, an estimation unit

[0022] The acquisition unit 31 acquires a visible light image of the area around the railway track 2 captured by the visible light camera 11, and a temperature image of the area around the railway track 2 captured by the far-infrared camera. The acquisition unit 31 stores the acquired visible light image and temperature image in the storage unit 22.

[0023] The estimation target area identification unit 32 identifies a predetermined area on the visible light image that includes the location where the temperature detection equipment is installed, as the estimation target area. Specifically, the estimation target area identification unit 32 identifies the location where the equipment is installed on the visible light image and identifies a predetermined area including the identified location as the estimation target area. The estimation target area identification unit 32 is, for example, AI (Artificial Intelligence), and the estimation target area is identified using a machine learning model based on deep learning such as YOLOX. Temperature images have a large contrast that changes depending on the surrounding temperature environment, making it costly to create a dataset and difficult to identify the estimation target area using a machine learning model. Therefore, the estimation target area identification unit 32 identifies the estimation target area from the visible light image.

[0024] Furthermore, the estimated target area identification unit 32 may identify the names of the equipment included in the estimated target area.

[0025] The estimation unit 33 estimates the equipment area on the visible light image of equipment located in the estimation target area identified by the estimation target area identification unit 32. Specifically, the estimation unit 33 estimates the area occupying the largest area within the estimation target area as the area where equipment is visible (hereinafter referred to as the equipment area) on a pixel-by-pixel basis. The estimation unit 33 is, for example, an AI, and performs the estimation of the equipment area using a machine learning model based on Segment Anything or the like.

[0026] The matching unit 34 extracts feature points from the visible light image and the temperature image, respectively, and associates each feature point. Specifically, the matching unit 34 detects characteristic points from each image using a feature point detection algorithm. Next, the matching unit 34 assigns a descriptor with unique characteristics to each pixel of the detected feature point. The descriptor is described by combining the brightness gradient within the surrounding image patch of the feature point, the orientation of the feature, color information, etc. The matching unit 34 evaluates the similarity of the descriptors between the feature points detected from the visible light image and the feature points detected from the temperature image, and identifies the feature points with the highest similarity as corresponding points.

[0027] Based on the feature point correspondence by the matching unit 34, the detection unit 35 identifies a temperature region in the temperature image that corresponds to the equipment area in the visible light image estimated by the estimation unit 33, and detects the temperature of the temperature region as the equipment temperature. As described above, in the temperature image, areas with high temperature are displayed in red or white, and areas with low temperature are displayed in blue or black, so the detection unit 35 can detect the equipment temperature based on the color of the temperature region.

[0028] Furthermore, the temperature detected by the detection unit 35 may be linked to the name of the equipment identified by the estimated target area identification unit 32 and output to the user. In addition, if the detected temperature is above a predetermined value, a message indicating that abnormal heat generation has occurred may be output. This allows the user to understand the temperature of each piece of equipment and check for the presence or absence of abnormal heat generation.

[0029] (Temperature detection process) Next, the temperature detection process performed by the temperature detection device 13 will be explained with reference to the flowchart in Figure 4.

[0030] The temperature detection process starts, for example, when a start button (not shown in the diagram) is pressed.

[0031] In step S11, the acquisition unit 31 acquires a visible light image captured by the visible light camera 11 and a temperature image captured by the far-infrared camera 12. The acquisition unit 31 acquires a visible light image of the area around the railway tracks, for example, as shown in Figure 2A, and a temperature image as shown in Figure 2B.

[0032] In step S12, the estimation target area identification unit 32 identifies a predetermined area on the visible light image that includes the location where the equipment is installed as the estimation target area. In the example in Figure 2, as shown in Figure 5, the estimation target area identification unit 32 identifies a rectangular area on the visible light image surrounding the image of impedance bond P, which is one of the pieces of equipment, as the estimation target area V1. Note that the leader lines, symbols P and V1, and dashed lines in Figure 5 are added for illustrative purposes.

[0033] In step S13, the estimation unit 33 estimates the area occupying the largest area from the estimation target area as the equipment area of ​​the equipment, on a pixel-by-pixel basis. In the example of Figure 5, the area with hatching in Figure 6, which is an enlarged view of a part of the estimation target area V1 shown in Figure 5, is estimated as the equipment area V2 of the impedance bond P. Note that the leader line, the symbol V2, and the hatch in Figure 6 are added for illustrative purposes.

[0034] In step S14, the matching unit 34 extracts feature points from the visible light image and the temperature image, respectively, and associates each feature point from the visible light image and the temperature image. Figure 7 is an image diagram showing the association between the feature points of the visible light image (left) and the feature points of the temperature image (right) shown in Figure 2, connected by straight lines.

[0035] In step S15, the detection unit 35 identifies the temperature region in the temperature image that corresponds to the equipment region in the visible light image estimated by the estimation unit 33, based on the feature point correspondence by the matching unit 34. When the feature points are matched as shown in Figure 7, the temperature region T of impedance bond P in the right temperature image that corresponds to the equipment region V2 of impedance bond P in the left visible light image is identified, as shown in Figure 8. Note that the leader lines, symbols V2, T, hatches, and arrows in Figure 8 are added for illustrative purposes.

[0036] In step S16, the detection unit 35 detects the temperature of the temperature region as the temperature of the equipment. In the example in Figure 8, the temperature of the temperature region T on the right temperature image, which corresponds to the equipment region V2 of the impedance bond P in the visible light image on the left, is detected as the temperature of the impedance bond P.

[0037] The subsequent processing is now complete.

[0038] [Differentiation] In the example shown in Figure 2, the case of detecting the temperature of a single impedance bond P was explained as an example, but there may be multiple temperature detection devices, and these may be configured as appropriate.

[0039] Furthermore, although the area on the visible light image of the entire impedance bond P was described as the estimation target area, the estimation target area identification unit 32 can identify the estimation target area on a pixel-by-pixel basis, so it is also possible to identify a predetermined part of the equipment as the estimation target area and detect the temperature of that part.

[0040] [Supplementary explanation of the embodiment]

[0041] The embodiments described above are all preferred examples of the present invention. The numerical values, components, arrangement positions and connection configurations of the components, and processing order in the flowcharts shown in the following embodiments are examples only and are not intended to limit the present invention. Furthermore, the figures are not necessarily strictly illustrative.

[0042] The series of processes described above can be executed by hardware or by software. When the series of processes are executed by software, the programs that make up the software are installed from a program storage medium onto a computer that is built into dedicated hardware, or onto a general-purpose personal computer, for example, which can perform various functions by installing various programs.

[0043] The programs executed by the computer may be programs that are processed chronologically in the order described herein, or they may be programs that are processed in parallel or at necessary times, such as when a call is made.

[0044] [Note] The contents described in some of the embodiments above can be understood, for example, as follows:

[0045] (1) In a temperature detection device 13 (Figure 3) that detects the temperature of equipment laid at a predetermined location around the railway tracks, The acquisition unit 31 acquires a visible light image of the area around the railway tracks captured by a visible light camera, and a temperature image of the area around the railway tracks captured by a far-infrared camera. An estimation target area identification unit 32 identifies a predetermined area on the visible light image, including the location where the equipment is installed, as the estimation target area V1 (Figure 5), An estimation unit 33 estimates the equipment area V2 (Figure 6) on the visible light image of the equipment located in the estimation target area V1, A matching unit 34 that associates the feature points of the visible light image and the temperature image (Figure 7), Based on the correspondence of the feature points by the matching unit 34, the detection unit 35 identifies the temperature region T (Figure 8) on the temperature image corresponding to the equipment region V2 and detects the temperature of the temperature region T as the temperature of the equipment. It is equipped with.

[0046] Because the contrast of thermal images changes considerably depending on the surrounding temperature environment, identifying equipment from thermal images requires a tremendous amount of effort to create a dataset. Therefore, with the configuration described above, a predetermined area including the location where the equipment is laid is identified as the target area V1, the equipment area V2 (Figure 6) on the visible light image of the equipment located in the target area V1 is estimated, and the temperature of the temperature area T on the thermal image corresponding to the equipment area V2 is detected. This makes it easier to detect the temperature of equipment laid at a predetermined location around the railway line compared to creating a dataset to identify equipment from thermal images. Furthermore, since the equipment area V2 is estimated from the estimated target area V1, which includes the location where the equipment is installed, on the visible light image, the equipment area V2 can be easily identified compared to, for example, the case where image processing is applied to the entire visible light image to identify the equipment area V2.

[0047] (2) Identification of the estimated target area V1 based on location identification The estimation target area identification unit 32 can identify the location on the visible light image where the equipment is installed, and can identify a predetermined area including the identified location as the estimation target area V1 (Figure 5).

[0048] To create training data for estimating the areas of railway-specific equipment using image processing from visible light images, it is necessary to prepare a large number of images that have been color-coded by category at the pixel level. However, by identifying the location where the equipment is installed and designating a predetermined area including the identified location as the estimation target area V1, the range for estimating the equipment area V2 is narrowed down to the estimation target area V1, thus streamlining the creation of training data.

[0049] (3) Use of segment anything The estimation unit 33 is an AI that estimates the area occupying the largest area among the estimation target areas as the equipment area on a pixel-by-pixel basis.

[0050] In this way, the equipment area is estimated on a pixel-by-pixel basis, making it possible to detect the temperature of any part of the equipment area, for example. [Explanation of symbols]

[0051] 1. Temperature detection system 2 tracks 3 vehicles 11. Visible light camera 12 Far-infrared camera 13. Temperature detection device 21 Communications Department 22 Memory section 23 Control Unit 31 Acquisition Department 32. Estimated Target Area Identification Unit 33 Estimation part 34 Matching Section 35 Detection unit

Claims

1. In a temperature detection device that detects the temperature of equipment installed at a predetermined location around a railway track, An acquisition unit that acquires a visible light image of the area around the railway tracks captured by a visible light camera, and a temperature image of the area around the railway tracks captured by a far-infrared camera, An estimation target area identification unit identifies a predetermined area on the visible light image that includes the location where the equipment is installed as an estimation target area, An estimation unit that estimates the equipment area on the visible light image of the equipment located in the estimated target area, A matching unit that associates the feature points of the visible light image with the temperature image, A detection unit identifies a temperature region on the temperature image corresponding to the equipment area based on the matching of the feature points by the matching unit, and detects the temperature of the temperature region as the temperature of the equipment. A temperature detection device characterized by comprising the following features.

2. In the temperature detection device according to claim 1, The estimation target area identification unit identifies the location on the visible light image where the equipment is installed, and identifies a predetermined area including the identified location as the estimation target area. A temperature detection device characterized by the following features.

3. In the temperature detection device according to claim 1, The estimation unit is an AI that estimates the area occupying the largest area among the areas to be estimated as the equipment area on a pixel-by-pixel basis. A temperature detection device characterized by the following features.

4. In a temperature detection method for detecting the temperature of equipment laid at a predetermined location around a railway track, The acquisition step involves acquiring a visible light image of the area around the railway tracks captured by a visible light camera, and a thermal image of the area around the railway tracks captured by a far-infrared camera. An estimation target area identification step, which identifies a predetermined area on the visible light image that includes the location where the equipment is installed as an estimation target area, Estimation step of estimating the equipment area on the visible light image of the equipment located in the estimation target area, A matching step that associates the feature points of the visible light image with the temperature image, A detection step is performed to identify a temperature region on the temperature image corresponding to the equipment region based on the matching step of the feature points, and to detect the temperature of the temperature region as the temperature of the equipment. A method for detecting temperature, characterized by including the following:

5. In a program that can be executed by an input / output device and a computer capable of exchanging information, for detecting the temperature of equipment laid at a predetermined location around a railway track, The acquisition step involves acquiring a visible light image of the area around the railway tracks captured by a visible light camera, and a thermal image of the area around the railway tracks captured by a far-infrared camera. An estimation target area identification step, which identifies a predetermined area on the visible light image that includes the location where the equipment is installed as an estimation target area, Estimation step of estimating the equipment area on the visible light image of the equipment located in the estimation target area, A matching step that associates the feature points of the visible light image with the temperature image, A detection step is performed to identify a temperature region on the temperature image corresponding to the equipment region based on the matching step of the feature points, and to detect the temperature of the temperature region as the temperature of the equipment. A program that causes a computer to perform a process, characterized by including [a certain element].

Images