A track temperature monitoring method, system and device based on image recognition

Abstract

The application provides a track temperature monitoring method, system and device based on image recognition, relates to the technical field of railway track safety monitoring, and mainly comprises the following steps: arranging infrared cameras, temperature sensors and temperature identification targets and the like devices in a monitoring area; collecting radiation brightness images of monitoring objects, activity blackbody shutter radiation brightness images, radiation brightness images and temperatures of cooperative blackbodies; determining the quantitative relationship between radiation brightness and energy grayscale based on the calibration blackbody; generating energy grayscale images through image conversion and non-uniformity correction; correcting the energy grayscale images based on the radiation brightness images and the temperatures of the cooperative blackbodies; and solving the temperature of the monitoring object through fitting calculation. The scheme improves the precision and efficiency of track temperature monitoring, reduces the labor intensity, improves the intelligent level of railway operation and maintenance work, and further guarantees the safe and reliable operation of railways.

Description

Technical Field

[0001] This invention relates to the field of railway track safety monitoring technology, and in particular to a track temperature monitoring method, system and device based on image recognition. Background Technology

[0002] Railway tracks traversing various regions experience fluctuations in rail temperature, ballastless track bed temperature, and beam temperature due to the influence of the natural environment and train operation. These temperature changes affect the stress on the track structure, leading to gradual track deformation and posing safety hazards. Furthermore, the tracks are often located in sparsely populated areas with poor transportation, making manual inspection and maintenance extremely difficult. Therefore, intelligent methods are urgently needed to monitor track temperature, promptly grasp its status, and establish a sound safety early warning mechanism to provide strong support for the overall safe operation of the line. Summary of the Invention

[0003] The purpose of this invention is to provide a method, system, and apparatus for monitoring track temperature based on image recognition, so as to solve at least one of the above-mentioned technical problems existing in the prior art.

[0004] To address the aforementioned technical problems, this invention provides a method, system, and apparatus for monitoring track temperature based on image recognition. The method includes:

[0005] Step 1: Deploy infrared cameras, temperature sensors, and temperature identification targets in the monitoring area to collect images of the monitored object's radiance, the radiance of a moving blackbody shutter, the radiance of a cooperative blackbody, and the temperature of the cooperative blackbody. The monitored object may include track structure components, ballastless track beds, bridge structure components, etc. The temperature identification target is placed on the surface of the monitored object that can be captured by the infrared camera. The blackbody refers to a radiation source used to provide a calibration temperature at a specified emissivity as a temperature reference for the infrared measurement equipment. The moving blackbody shutter is built into the shutter position of the infrared camera. The cooperative blackbody is placed within the field of view of the infrared camera. The temperature sensor may be built into the infrared camera to measure the temperature of the cooperative blackbody.

[0006] Step 2: Acquire the radiance image and temperature of the calibrated blackbody, calculate the quantitative relationship between radiance and energy grayscale, and based on the quantitative relationship and the radiance image of the moving blackbody shutter, convert it into the energy grayscale image of the calibrated blackbody and the energy grayscale image of the moving blackbody shutter. The calibrated blackbody is a temperature calibration reference used for radiation calibration and can be set in a stable environment such as a laboratory or testing room.

[0007] Step 3: Based on the reference image, the quantitative relationship, and the grayscale image of the active blackbody shutter energy, calculate and update the gain parameters and bias parameters of the infrared camera to perform non-uniformity correction on the image.

[0008] Step 4: Based on the radiance image of the monitored object and the radiance image of the cooperating blackbody, generate the energy grayscale image of the monitored object and the energy grayscale image of the cooperating blackbody through conversion and non-uniformity correction;

[0009] Step 5: Acquire several sets of calibration blackbody energy grayscale images and calibration blackbody temperatures at varying temperatures, statistically analyze the average grayscale value of the calibration blackbody, calculate the first fitting curve, and calculate the correction constant based on the first fitting curve, the cooperative blackbody temperature, and the cooperative blackbody energy grayscale images. Correct the energy grayscale image of the monitored object based on the correction constant.

[0010] Step 6: Based on the calibration blackbody temperature and the calibration blackbody average gray value, calculate the second fitting curve; based on the corrected energy grayscale image of the monitored object and the second fitting curve, calculate the temperature of the monitored object.

[0011] Step 7: Based on the temperature of the monitored object, determine and generate alarm information.

[0012] Through the above steps, the temperature of the monitored object can be intelligently calculated using image vision, and alarm information can be generated to achieve the purpose of monitoring the temperature of targets such as tracks.

[0013] Furthermore, when the infrared camera is in operation, it acquires images of the radiance of the active blackbody shutter by periodically performing instantaneous shutter opening and closing operations.

[0014] In one feasible implementation, step 2 specifically includes the following steps:

[0015] Step 201: Based on the calibrated blackbody temperature, calculate the radiance value L(λ). The specific formula is as follows:

[0016]

[0017] Where c1 represents the first radiation constant, c ２ Let λ represent the second radiation constant, λ represent the radiation wavelength, n represent the air refractive index, T represent the calibrated blackbody temperature, and λ1 represent light of one wavelength, while λ2 represent light of another wavelength.

[0018] Step 202: Based on the radiance value L(λ), calculate the narrowband responsivity Δλ. The specific formula is as follows:

[0019]

[0020] Among them, R T T represents the narrowband responsivity Δλ. int S represents the integration time of the infrared camera. e (λ) represents the number of electrons generated by one pixel of an infrared camera;

[0021] Step 203, based on narrowband responsivity R T Given the radiance value L(λ), calculate the energy grayscale value;

[0022] V=R T L(λ)ΔS / r 2 ;

[0023] Where V represents the energy gray value, ΔS represents the effective area of ​​the calibration blackbody, and r represents the perpendicular distance between the infrared camera and the surface of the calibration blackbody, with the angle between the normal of the calibration blackbody surface and r being 0.

[0024] Step 204: Based on the quantitative relationship between radiance and energy grayscale and the radiance image of the active blackbody shutter, obtain the calibrated blackbody energy grayscale image and the active blackbody shutter energy grayscale image.

[0025] By following the steps above, changes in the external environment can be dynamically perceived, resulting in more accurate output results.

[0026] In one feasible implementation, step 3 specifically includes the following steps:

[0027] Step 301: Extract the grayscale image Φ of the current active blackbody shutter energy. st ;

[0028] Step 302: Using the reference image Φ built into the infrared camera H and the grayscale image Φ of the active blackbody shutter energy st Calculate the gain parameter G i，j (t) and bias parameter O i，j (t), the specific formula can be:

[0029]

[0030]

[0031] Where, x i，j (Φ H ) represents the reference image Φ H Acquire the output value of the infrared focal plane; V H Represents the reference image Φ H Average response:

[0032] xi，j (Φ st ) represents the grayscale image of the active blackbody shutter energy. st Acquire the output value of the infrared focal plane; V s (t) represents the grayscale image of the active blackbody shutter energy Φ. st Average response:

[0033]

[0034] Step 303: Update the gain and bias parameters of the infrared camera. The specific formula can be:

[0035] y i，j (t+n)=G i，j (t)x i，j (t+n)+O i，j (t);

[0036] Among them, y i，j (t+n) represents the energy grayscale output value of the infrared camera, x i，j (t+n) represents the energy grayscale input value of the infrared camera.

[0037] The above steps can be used to correct for non-uniformity in the output.

[0038] In one feasible implementation, step 5 specifically includes the following steps:

[0039] Step 501: Acquire several sets of calibration blackbody energy grayscale images and calibration blackbody temperatures at varying temperatures, and statistically analyze the average grayscale value of the calibration blackbody.

[0040] Step 502: Based on the calibration blackbody temperature and the average gray value of the calibration blackbody, calculate the first fitting curve. The specific formula can be:

[0041] y(T)=A0+A1x+A2x 2 +A3x 3 +A4x 4 +A5x 5 ;

[0042] Where y(T) represents the average gray value, and x represents the temperature value.

[0043] A0, A1, A2, A3, A4, and A5 all represent the coefficients of the polynomial;

[0044] Step 503: Based on the grayscale images of the cooperating blackbody temperature and energy, and the first fitted curve, calculate the grayscale correction constant y0. The specific formula can be:

[0045] y0=y h -yh′ ;

[0046] Among them, y h y represents the actual grayscale value in the cooperating blackbody energy grayscale image. h′ This represents the average gray value of the cooperating blackbody;

[0047] Step 504: Correct the energy grayscale image based on the correction constant y0.

[0048] Further, the specific method of step 501 can be as follows: change the calibration blackbody temperature according to a preset temperature interval, record the current calibration blackbody temperature after the temperature stabilizes, and collect the current calibration blackbody energy grayscale image. Then perform the next set of temperature changes until several sets of data are obtained. Based on the grayscale value of the calibration blackbody energy grayscale image, calculate the average grayscale value of the calibration blackbody.

[0049] Furthermore, the correction method subtracts the correction constant y0 from the gray value of each pixel in the energy grayscale image.

[0050] Through the above steps, the average gray value can be calculated based on the temperature value, and the energy gray value image can be corrected by a correction constant, thus taking into account the influence of environmental radiation and shell radiation, and obtaining a more accurate output result.

[0051] In one feasible implementation, step 6 specifically includes: performing polynomial fitting based on the calibrated blackbody temperature and the average gray value of the calibrated blackbody to calculate a second fitting curve, the specific formula of which can be:

[0052] C(T) = B0 + B1y + B2y 2 +B3y 3 +B4y 4 +B5y 5 ;

[0053] Where C(T) represents the temperature value, and y represents the grayscale value.

[0054] B0, B1, B2, B3, B4, and B5 all represent the coefficients of the polynomial;

[0055] The temperature of the monitored object is calculated based on the corrected grayscale image of the monitored object and the second fitted curve.

[0056] By following the steps above, the temperature value can be accurately calculated based on the grayscale value.

[0057] Furthermore, the process of judging temperature and generating alarm information includes: generating a warning message when the temperature at a certain location exceeds a first preset range; generating a regular alarm message when the temperature exceeds a second preset range; and generating an emergency alarm message when the temperature exceeds a third preset range.

[0058] On the other hand, based on the same idea, the present invention also provides an image recognition-based track temperature monitoring system, including a data acquisition module, a data processing module, and a data generation module:

[0059] The data acquisition module includes an image acquisition unit and a temperature acquisition unit:

[0060] The image acquisition unit is used to acquire images of the radiance of the monitored object, images of the active blackbody shutter, images of the cooperative blackbody, and images of the calibrated blackbody.

[0061] The temperature acquisition unit is used to acquire the calibrated blackbody temperature and the cooperative blackbody temperature;

[0062] The data processing module includes an image calibration unit, a parameter correction unit, an image conversion unit, an image correction unit, a temperature calculation unit, and an alarm information unit.

[0063] The image calibration unit calculates the quantitative relationship between radiance and energy grayscale based on the calibrated blackbody radiance image and the calibrated blackbody temperature. Based on the quantitative relationship and the active blackbody shutter radiance image, it converts the calibrated blackbody energy grayscale image and the active blackbody shutter energy grayscale image.

[0064] The parameter correction unit calculates and outputs gain parameters and bias parameters based on the reference image, the quantitative relationship, and the grayscale image of the active blackbody shutter energy, for image non-uniformity correction.

[0065] The image conversion unit generates an energy grayscale image of the monitored object and an energy grayscale image of the cooperating blackbody based on the radiation brightness image of the monitored object and the radiation brightness image of the cooperating blackbody, the gain parameter and the bias parameter, through conversion and non-uniformity correction.

[0066] The image correction unit calculates the average gray value of the calibrated blackbody based on several sets of calibrated blackbody energy grayscale images and calibrated blackbody temperatures, obtains a first fitting curve, calculates a correction constant based on the first fitting curve, the cooperating blackbody temperature and the cooperating blackbody energy grayscale image, and corrects the energy grayscale image of the monitored object based on the correction constant.

[0067] The temperature calculation unit calculates a second fitting curve based on the calibrated blackbody temperature and the average gray value of the calibrated blackbody, and calculates the temperature of the monitored object based on the corrected energy grayscale image of the monitored object and the second fitting curve.

[0068] The alarm information unit is used to determine the temperature of the monitored object and generate alarm information;

[0069] The data generation module is used to output the alarm information to external systems.

[0070] On another front, based on the same idea, the present invention also provides an image recognition-based track temperature monitoring device, the device comprising a monitoring point, an infrared camera, a control box, and a power supply.

[0071] The monitoring points are used to mark the location of the monitored objects, and temperature recognition targets can be set.

[0072] The infrared camera is installed on the outside of the track and has a built-in temperature sensor to collect data such as the radiance image of the monitored object, the radiance image of the moving blackbody shutter, the radiance image of the cooperating blackbody, and the temperature of the cooperating blackbody.

[0073] The control box is electrically connected to the infrared camera, receives the images and temperatures collected by the infrared camera, processes the images using the above-mentioned image recognition-based track temperature monitoring method, calculates the temperature of the monitored object, determines and generates alarm information and sends it out.

[0074] The power supply equipment is electrically connected to the control box and the infrared camera, and is used to provide power to the control box and the infrared camera.

[0075] Furthermore, the control box is also equipped with a wireless communication device;

[0076] Furthermore, the monitoring points include beam monitoring points, sleeper monitoring points, rail monitoring points, and longitudinal beam monitoring points:

[0077] The beam monitoring points are set at the joints between beams to monitor the temperature of the beam near the joint.

[0078] The steel sleeper monitoring points are set on the surface of the steel sleeper to monitor the temperature of the steel sleeper;

[0079] The rail monitoring points include switch rail monitoring points and stock rail monitoring points: the switch rail monitoring points are located at the end of the switch rail closest to the infrared camera; the stock rail monitoring points are located at the end of the stock rail closest to the infrared camera.

[0080] The monitoring point on the steel longitudinal beam is located at the end of the steel longitudinal beam closest to the infrared camera.

[0081] By adopting the above technical solution, the present invention has the following beneficial effects:

[0082] The image recognition-based track temperature monitoring method, system, and device provided by this invention automatically and quickly identifies the temperature of various parts of the track through intelligent image recognition, realizing non-contact automatic monitoring, generating alarm information in a timely manner, improving monitoring accuracy and efficiency, reducing manual labor intensity, enhancing the intelligent level of railway operation and maintenance, and further ensuring the safe and reliable operation of railways. Attached Figure Description

[0083] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0084] Figure 1 A flowchart of an image recognition-based orbital temperature monitoring method provided in an embodiment of the present invention;

[0085] Figure 2 This is a schematic diagram of a temperature recognition target patch provided in an embodiment of the present invention;

[0086] Figure 3 This is a diagram illustrating the use of cooperative blackbody and calibrated blackbody in an embodiment of the present invention.

[0087] Figure 4 for Figure 1 The detailed flowchart for step 2;

[0088] Figure 5 for Figure 1 The detailed flowchart for step 3;

[0089] Figure 6 for Figure 1 The detailed flowchart for step 5;

[0090] Figure 7 A diagram of an image recognition-based track temperature monitoring system provided in an embodiment of the present invention;

[0091] Figure 8 A diagram of an image recognition-based track temperature monitoring device provided in an embodiment of the present invention.

[0092] Figure label:

[0093] 1-Left line beam temperature monitoring point; 2-Right line beam temperature monitoring point; 3-Left line steel sleeper temperature monitoring point; 4-Right line steel sleeper temperature monitoring point; 5-Left line switch rail temperature monitoring point; 6-Right line switch rail temperature monitoring point; 7-Left line main rail temperature monitoring point; 8-Right line main rail temperature monitoring point; 9-Left line outer steel longitudinal beam temperature monitoring point; 10-Left line inner steel longitudinal beam temperature monitoring point; 11-Right line inner steel longitudinal beam temperature monitoring point; 12-Right line outer steel longitudinal beam temperature monitoring point; 13-Left line infrared camera; 14-Right line infrared camera; 15-Left line infrared camera pole; 16-Right line infrared camera pole; 17-Control box; 18-Power supply equipment; 19-Left line cable bundle; 20-Right line cable bundle. Detailed Implementation

[0094] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0095] In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0096] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0097] The present invention will be further explained below with reference to specific embodiments.

[0098] Firstly, such as Figure 1 As shown in the figure, an embodiment of the present invention provides a method, system, and apparatus for monitoring track temperature based on image recognition. The method includes:

[0099] Step 1: Deploy infrared cameras, temperature sensors, and temperature recognition targets in the monitoring area to collect images of the monitored object's radiance, moving blackbody shutter radiance, cooperative blackbody radiance, and cooperative blackbody temperature. The monitored object may include track structure components, ballastless track beds, bridge structure components, etc. The temperature recognition target is placed on the surface of the monitored object that can be captured by the infrared cameras, such as a carbon black coating applied to the surface of a rail. Figure 2 As shown; the blackbody refers to a radiation source used to provide a calibration temperature at a specified emissivity as a temperature reference for infrared measuring equipment; the active blackbody shutter is built into the shutter position of the infrared camera; the cooperating blackbody is placed within the field of view of the infrared camera; the temperature sensor can also be built into the infrared camera to measure the temperature of the cooperating blackbody;

[0100] Step 2: Acquire the radiance image and temperature of the calibrated blackbody, calculate the quantitative relationship between radiance and energy grayscale, and based on the quantitative relationship and the radiance image of the moving blackbody shutter, convert it into the energy grayscale image of the calibrated blackbody and the energy grayscale image of the moving blackbody shutter. The calibrated blackbody is a temperature calibration reference used for radiation calibration and can be placed in a stable environment such as a laboratory or testing room. Figure 3 As shown;

[0101] Step 3: Based on the reference image, the quantitative relationship, and the grayscale image of the active blackbody shutter energy, calculate and update the gain parameters and bias parameters of the infrared camera for image non-uniformity correction.

[0102] Step 4: Based on the radiance image of the monitored object and the radiance image of the cooperating blackbody, generate the energy grayscale image of the monitored object and the energy grayscale image of the cooperating blackbody through conversion and non-uniformity correction;

[0103] Step 5: Acquire several sets of calibration blackbody energy grayscale images and calibration blackbody temperatures at varying temperatures, statistically analyze the average grayscale value of the calibration blackbody, calculate the first fitting curve, and calculate the correction constant based on the first fitting curve, the cooperative blackbody temperature, and the cooperative blackbody energy grayscale images. Correct the energy grayscale image of the monitored object based on the correction constant.

[0104] Step 6: Based on the calibration blackbody temperature and the calibration blackbody average gray value, calculate the second fitting curve; based on the corrected energy grayscale image of the monitored object and the second fitting curve, calculate the temperature of the monitored object.

[0105] Step 7: Based on the temperature of the monitored object, determine and generate alarm information.

[0106] Through the above steps, the temperature of the monitored object can be intelligently calculated using image vision, and alarm information can be generated to achieve the purpose of monitoring the temperature of targets such as tracks.

[0107] Furthermore, when the infrared camera is in operation, it acquires images of the radiance of the active blackbody shutter by periodically performing instantaneous shutter opening and closing operations.

[0108] Furthermore, such as Figure 4 As shown, step 2 specifically includes the following steps:

[0109] Step 201: Based on the calibrated blackbody temperature, calculate the radiance value L(λ). The specific formula is as follows:

[0110]

[0111] Where c1 represents the first radiation constant, C2 represents the second radiation constant, λ represents the radiation wavelength, n represents the air refractive index, T represents the calibrated blackbody temperature, and λ1 represents light of one wavelength, λ2 represents light of another wavelength.

[0112] Step 202: Based on the radiance value L(λ), calculate the narrowband responsivity Δλ. The specific formula is as follows:

[0113]

[0114] Among them, R T T represents the narrowband responsivity Δλ. int S represents the integration time of the infrared camera. e (λ) represents the number of electrons generated by one pixel of an infrared camera;

[0115] Step 203, based on narrowband responsivity R T Given the radiance value L(λ), calculate the energy grayscale value;

[0116] V=R T L(λ)ΔS / r 2 ;

[0117] Where V represents the energy gray value, ΔS represents the effective area of ​​the calibration blackbody, and r represents the perpendicular distance between the infrared camera and the surface of the calibration blackbody, with the angle between the normal of the calibration blackbody surface and r being 0.

[0118] Step 204: Based on the quantitative relationship between radiance and energy grayscale and the radiance image of the active blackbody shutter, obtain the calibrated blackbody energy grayscale image and the active blackbody shutter energy grayscale image.

[0119] By following the steps above, changes in the external environment can be dynamically perceived, resulting in more accurate output results.

[0120] Furthermore, step 3 specifically includes the following steps:

[0121] Step 301: Extract the grayscale image Φ of the current active blackbody shutter energy. st ;

[0122] Step 302: Using the reference image Φ built into the infrared camera H and the grayscale image Φ of the active blackbody shutter energy st Calculate the gain parameter G i，j (t) and bias parameter O i，j (t), the specific formula can be:

[0123]

[0124]

[0125] Where, x i，j (Φ H ) represents the reference image Φ H Acquire the output value of the infrared focal plane; V H Represents the reference image Φ H Average response:

[0126] x i，j (Φ si ) represents the grayscale image of the active blackbody shutter energy. st Acquire the output value of the infrared focal plane; V s (t) represents the grayscale image of the active blackbody shutter energy Φ. st Average response:

[0127]

[0128] Step 303: Update the gain and bias parameters of the infrared camera. The specific formula can be:

[0129] y i，j (t+n)=G i，j (t)x i，j (t+n)+O i，j (t);

[0130] Among them, y i，j (t+n) represents the energy grayscale output value of the infrared camera, x i，j (t+n) represents the energy grayscale input value of the infrared camera.

[0131] The above steps can be used to correct for non-uniformity in the output.

[0132] Furthermore, step 5 specifically includes the following steps:

[0133] Step 501: Acquire several sets of calibration blackbody energy grayscale images and calibration blackbody temperatures at varying temperatures, and statistically analyze the average grayscale value of the calibration blackbody.

[0134] Step 502: Based on the calibration blackbody temperature and the average gray value of the calibration blackbody, calculate the first fitting curve. The specific formula can be:

[0135] y(T)=A0+A1x+A2x 2 +A3x 3 +A4x 4 +A5x 5 ;

[0136] Where y(T) represents the average gray value, and x represents the temperature value.

[0137] A0, A1, A2, A3, A4, and A5 all represent the coefficients of the polynomial;

[0138] Step 503: Based on the grayscale images of the cooperating blackbody temperature and energy, and the first fitted curve, calculate the grayscale correction constant y0. The specific formula can be:

[0139] y0=y h -y h′ ;

[0140] Among them, y h y represents the actual grayscale value in the cooperating blackbody energy grayscale image. h′ This represents the average gray value of the cooperating blackbody;

[0141] Step 504: Correct the energy grayscale image based on the correction constant y0.

[0142] Further, the specific method of step 501 can be as follows: change the calibration blackbody temperature according to a preset temperature interval, record the current calibration blackbody temperature after the temperature stabilizes, and collect the current calibration blackbody energy grayscale image. Then perform the next set of temperature changes until several sets of data are obtained. Based on the grayscale value of the calibration blackbody energy grayscale image, calculate the average grayscale value of the calibration blackbody.

[0143] Furthermore, the correction method subtracts the correction constant y0 from the gray value of each pixel in the energy grayscale image.

[0144] Through the above steps, the average gray value can be calculated based on the temperature value, and the energy gray value image can be corrected by a correction constant, thus taking into account the influence of environmental radiation and shell radiation, and obtaining a more accurate output result.

[0145] Further, step 6 specifically includes: performing polynomial fitting based on the calibrated blackbody temperature and the average gray value of the calibrated blackbody to calculate the second fitting curve. The specific formula can be:

[0146] C(T) = B0 + B1y + B2y 2 +B3y 3 +B4y 4 +B5y 5 ;

[0147] Where C(T) represents the temperature value, and y represents the grayscale value.

[0148] B0, B1, B2, B3, B4, and B5 all represent the coefficients of the polynomial;

[0149] The temperature of the monitored object is calculated based on the corrected grayscale image of the monitored object and the second fitted curve.

[0150] By following the steps above, the temperature value can be accurately calculated based on the grayscale value.

[0151] Furthermore, the process of judging temperature and generating alarm information includes: generating a warning message when the temperature at a certain location exceeds a first preset range; generating a regular alarm message when the temperature exceeds a second preset range; and generating an emergency alarm message when the temperature exceeds a third preset range.

[0152] Secondly, based on the same inventive concept, embodiments of the present invention also provide an image recognition-based track temperature monitoring system, such as... Figure 5 As shown, it includes a data acquisition module, a data processing module, and a data generation module:

[0153] The data acquisition module includes an image acquisition unit and a temperature acquisition unit:

[0154] The image acquisition unit is used to acquire images of the radiance of the monitored object, images of the active blackbody shutter, images of the cooperative blackbody, and images of the calibrated blackbody.

[0155] The temperature acquisition unit is used to acquire the calibrated blackbody temperature and the cooperative blackbody temperature;

[0156] The data processing module includes an image calibration unit, a parameter correction unit, an image conversion unit, an image correction unit, a temperature calculation unit, and an alarm information unit.

[0157] The image calibration unit calculates the quantitative relationship between radiance and energy grayscale based on the calibrated blackbody radiance image and the calibrated blackbody temperature. Based on the quantitative relationship and the active blackbody shutter radiance image, it converts the calibrated blackbody energy grayscale image and the active blackbody shutter energy grayscale image.

[0158] The parameter correction unit calculates and outputs gain parameters and bias parameters based on the reference image, the quantitative relationship, and the grayscale image of the active blackbody shutter energy, for image non-uniformity correction.

[0159] The image conversion unit generates an energy grayscale image of the monitored object and an energy grayscale image of the cooperating blackbody based on the radiation brightness image of the monitored object and the radiation brightness image of the cooperating blackbody, the gain parameter and the bias parameter, through conversion and non-uniformity correction.

[0160] The image correction unit calculates the average gray value of the calibrated blackbody based on several sets of calibrated blackbody energy grayscale images and calibrated blackbody temperatures, obtains a first fitting curve, calculates a correction constant based on the first fitting curve, the cooperating blackbody temperature and the cooperating blackbody energy grayscale image, and corrects the energy grayscale image of the monitored object based on the correction constant.

[0161] The temperature calculation unit calculates a second fitting curve based on the calibrated blackbody temperature and the average gray value of the calibrated blackbody, and calculates the temperature of the monitored object based on the corrected energy grayscale image of the monitored object and the second fitting curve.

[0162] The alarm information unit is used to determine the temperature of the monitored object and generate alarm information;

[0163] The data generation module is used to output the alarm information to external systems.

[0164] Thirdly, based on the same idea, the present invention also provides an image recognition-based track temperature monitoring device, the device comprising a monitoring point, an infrared camera, a control box, and a power supply device:

[0165] The monitoring points are used to mark the location of the monitored objects, and can be marked by setting temperature recognition targets;

[0166] The infrared camera is installed on the outside of the track and has a built-in temperature sensor to collect data such as the radiance image of the monitored object, the radiance image of the moving blackbody shutter, the radiance image of the cooperating blackbody, and the temperature of the cooperating blackbody.

[0167] The control box is electrically connected to the infrared camera, receives the images and temperatures collected by the infrared camera, processes the images using the above-mentioned image recognition-based track temperature monitoring method, calculates the temperature of the monitored object, determines and generates alarm information and sends it out.

[0168] The power supply equipment is electrically connected to the control box and the infrared camera, and is used to provide power to the control box and the infrared camera.

[0169] Furthermore, the control box is also equipped with a wireless communication device for external wireless transmission and communication, such as a LoRa device, a 2G / 3G / 4G / 5G device, a satellite communication device, etc., which can be flexibly selected according to the communication network conditions of the local area.

[0170] Furthermore, the monitoring points include beam monitoring points, sleeper monitoring points, rail monitoring points, and longitudinal beam monitoring points:

[0171] The beam monitoring points are set at the joints between beams to monitor the temperature of the beam near the joint.

[0172] The steel sleeper monitoring points are set on the surface of the steel sleeper to monitor the temperature of the steel sleeper;

[0173] The rail monitoring points include switch rail monitoring points and stock rail monitoring points: the switch rail monitoring points are located at the end of the switch rail closest to the infrared camera; the stock rail monitoring points are located at the end of the stock rail closest to the infrared camera.

[0174] The monitoring point on the steel longitudinal beam is located at the end of the steel longitudinal beam closest to the infrared camera.

[0175] Example 1:

[0176] like Figure 8 As shown, a track temperature monitoring device based on image recognition specifically includes: a temperature monitoring point 1 for the left track beam; a temperature monitoring point 2 for the right track beam; a temperature monitoring point 3 for the left track sleeper; a temperature monitoring point 4 for the right track sleeper; a temperature monitoring point 5 for the left track switch rail; a temperature monitoring point 6 for the right track switch rail; a temperature monitoring point 7 for the left track main rail; a temperature monitoring point 8 for the right track main rail; a temperature monitoring point 9 for the outer steel longitudinal beam of the left track; a temperature monitoring point 10 for the inner steel longitudinal beam of the left track; a temperature monitoring point 11 for the inner steel longitudinal beam of the right track; a temperature monitoring point 12 for the outer steel longitudinal beam of the right track; a left track infrared camera; a right track infrared camera; a left track infrared camera pole; a right track infrared camera pole; a control box; a power supply device; a left track cable bundle; and a right track cable bundle.

[0177] The location of the main bridge structure of the double-track railway (respectively the left and right tracks) is designated as the monitoring area.

[0178] The left-line infrared camera pole 15 is set on the outside of the left line of the line, and the left-line infrared camera 13 is set on the top of the left-line infrared camera pole 15 for capturing the monitoring area of ​​the left line of the line.

[0179] In a mirror image, the right-line infrared camera pole 16 is positioned outside the right line of the line, and the right-line infrared camera 14 is positioned at the top of the right-line infrared camera pole 16, for capturing images of the monitoring area of ​​the right line of the line.

[0180] All of the aforementioned infrared cameras are equipped with transparent protective covers to prevent the camera lens from being covered by ice, rain, or snow, thus preventing it from losing its shooting function.

[0181] The specific monitoring points are arranged as follows: The left-line beam temperature monitoring point 1 is located at the joint between the two bridges, near the left-line end of the track; the right-line beam temperature monitoring point 2 is located at the joint between the two bridges, near the right-line end of the track; the left-line sleeper temperature monitoring point 3 is located on the upper surface of the left-line sleeper; the right-line sleeper temperature monitoring point 4 is located on the upper surface of the right-line sleeper; the left-line switch rail temperature monitoring point 5 is located at the left-line switch rail, near the left-line infrared camera 13; the right-line switch rail temperature monitoring point 6 is located at the right-line switch rail, near the right-line infrared camera 14; and the left-line main rail temperature monitoring point 7 is located at the left-line main rail, near the left-line infrared camera 14. One end of camera 13; the right track basic rail temperature monitoring point 8 is set at the end of the right track basic rail near the right track infrared camera 14; the left track outer steel longitudinal beam temperature monitoring point 9 is set at the end of the left track outer steel longitudinal beam near the left track infrared camera 13; the left track inner steel longitudinal beam temperature monitoring point 10 is set at the end of the left track inner steel longitudinal beam near the left track infrared camera 13; the right track outer steel longitudinal beam temperature monitoring point 12 is set at the end of the right track outer steel longitudinal beam near the right track infrared camera 14; the right track inner steel longitudinal beam temperature monitoring point 11 is set at the end of the right track inner steel longitudinal beam near the right track infrared camera 14.

[0182] Temperature identification targets can be affixed or sprayed at the aforementioned monitoring points.

[0183] In other embodiments, the location and number of monitoring points can be flexibly set according to actual needs.

[0184] The control box 17 is equipped with a processor, memory, and wireless communication device, used to record, process, analyze, and judge the monitored images according to the method described above, generate alarm information, and transmit it externally. The wireless communication device can be selected from existing communication devices such as LoRa, satellite, 2G, 3G, 4G, and 5G, depending on the on-site network environment. The control box 17 is located next to the left-line infrared camera pole 15 and is electrically connected to the left-line infrared camera 13 via the left-line cable bundle 19, which is laid underground or on the ground. The control box 17 is electrically connected to the right-line infrared camera 14 via the right-line cable bundle 20, which is also laid underground or on the ground. A power supply device 18 is installed on the top of the control box 17, and the top of the power supply device 18 is equipped with a solar panel, which can generate its own electricity using solar energy to provide the necessary power for this device.

[0185] In another implementation, this solution can be implemented using a device, which may include corresponding modules that perform one or more steps in the various embodiments described above. A module may be one or more hardware modules specifically configured to perform the corresponding step, or implemented by a processor configured to perform the corresponding step, or stored in a computer-readable medium for implementation by a processor, or implemented through some combination thereof.

[0186] The processor executes the various methods and processes described above. For example, the method implementations in this scheme can be implemented as software programs tangibly contained in a machine-readable medium, such as memory. In some implementations, part or all of the software program can be loaded and / or installed via memory and / or a communication interface. When the software program is loaded into memory and executed by the processor, one or more steps of the methods described above can be performed. Alternatively, in other implementations, the processor can be configured to execute one of the methods described above by any other suitable means (e.g., by means of firmware).

[0187] This device can be implemented using a bus architecture. A bus architecture can include any number of interconnect buses and bridges, depending on the specific application of the hardware and overall design constraints. The bus connects various circuits, including one or more processors, memory, and / or hardware modules. The bus can also connect various other circuits such as peripherals, voltage regulators, power management circuitry, external antennas, etc.

[0188] Buses can be Industry Standard Architecture (ISA) buses, Peripheral Component Interconnect (PCI) buses, or Extended Industry Standard Component (EISA) buses, etc. Buses can be divided into address buses, data buses, control buses, etc.

[0189] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for monitoring track temperature based on image recognition, characterized in that, include: Step 1: Using an infrared camera and temperature sensor, acquire images of the monitored object's radiance, the moving blackbody shutter's radiance, the cooperating blackbody's radiance, and the cooperating blackbody's temperature. Step 2: Acquire the radiance image and temperature of the calibrated blackbody, and based on the radiance image of the moving blackbody shutter, calculate and convert it to obtain the energy grayscale image of the calibrated blackbody and the energy grayscale image of the moving blackbody shutter; specifically including: Step 201: Calculate the radiance value based on the calibrated blackbody temperature; Step 202: Calculate the narrowband responsivity based on the radiance value; Step 203: Calculate the energy grayscale value based on the narrowband responsivity and radiance values; Step 204: Based on the quantitative relationship between radiance and energy grayscale, the radiance image of the calibrated blackbody and the radiance image of the moving blackbody shutter, obtain the energy grayscale image of the calibrated blackbody and the energy grayscale image of the moving blackbody shutter. Step 3: Based on the quantitative relationship and the grayscale image of the active blackbody shutter energy, calculate and update the gain parameters and bias parameters of the infrared camera; specifically including: Step 301: Extract the grayscale image of the current active blackbody shutter energy; Step 302: Calculate the gain parameters and bias parameters using the reference image and the grayscale image of the active blackbody shutter energy; Step 303: Update the gain and bias parameters of the infrared camera; Step 4: Convert the radiance image of the monitored object and the radiance image of the cooperating blackbody into an energy grayscale image of the monitored object and an energy grayscale image of the cooperating blackbody; Step 5: Acquire several sets of calibration blackbody energy grayscale images and calibration blackbody temperatures at varying temperatures, and calculate grayscale correction constants based on the cooperating blackbody temperatures and energy grayscale images to correct the energy grayscale image of the monitored object; specifically including: Step 501: Acquire several sets of calibration blackbody energy grayscale images and calibration blackbody temperatures at varying temperatures, and statistically analyze the average grayscale value of the calibration blackbody; specifically, this includes: changing the calibration blackbody temperature at preset temperature intervals, recording the current calibration blackbody temperature after the temperature stabilizes, and acquiring the current calibration blackbody energy grayscale image, then performing the next set of temperature changes, until several sets of data are obtained, and statistically calculating the average grayscale value of the calibration blackbody based on the grayscale values ​​of the calibration blackbody energy grayscale images; Step 502: Calculate the first fitting curve based on the calibration blackbody temperature and the average gray value of the calibration blackbody; Step 503: Based on the cooperative blackbody temperature, the grayscale image of the cooperative blackbody energy, and the first fitted curve, calculate the grayscale correction constant; Step 504: Correct the energy grayscale image based on the grayscale correction constant; Step 6: Calculate the temperature of the monitored object based on the calibrated blackbody temperature and the corrected energy grayscale image of the monitored object; specifically, this includes: performing polynomial fitting based on the calibrated blackbody temperature and the average grayscale value of the calibrated blackbody to calculate a second fitting curve; and calculating the temperature of the monitored object based on the corrected energy grayscale image of the monitored object and the second fitting curve. Step 7: Based on the temperature of the monitored object, determine and generate alarm information.

2. The method according to claim 1, characterized in that, The infrared camera acquires images of the radiance of an active blackbody shutter by periodically performing instantaneous shutter opening and closing operations.

3. The method according to claim 1, characterized in that, The correction method in step 504 is to subtract the correction constant from the gray value of each pixel in the energy grayscale image.

4. A system employing the image recognition-based track temperature monitoring method as described in any one of claims 1 to 3, characterized in that, It includes a data acquisition module, a data processing module, and a data generation module: The data acquisition module includes an image acquisition unit and a temperature acquisition unit: The image acquisition unit is used to acquire images of the radiance of the monitored object, images of the active blackbody shutter, images of the cooperative blackbody, and images of the calibrated blackbody. The temperature acquisition unit is used to acquire the calibrated blackbody temperature and the cooperative blackbody temperature; The data processing module includes an image calibration unit, a parameter correction unit, an image conversion unit, an image correction unit, a temperature calculation unit, and an alarm information unit. The image calibration unit calculates the quantitative relationship between radiance and energy grayscale based on the calibrated blackbody radiance image and the calibrated blackbody temperature. Based on the quantitative relationship and the active blackbody shutter radiance image, it converts the calibrated blackbody energy grayscale image and the active blackbody shutter energy grayscale image. The parameter correction unit calculates and outputs gain parameters and bias parameters based on the reference image, the quantitative relationship, and the grayscale image of the active blackbody shutter energy, for image non-uniformity correction. The image conversion unit generates an energy grayscale image of the monitored object and an energy grayscale image of the cooperating blackbody based on the radiation brightness image of the monitored object and the radiation brightness image of the cooperating blackbody, the gain parameter and the bias parameter, through conversion and non-uniformity correction. The image correction unit calculates the average gray value of the calibrated blackbody based on several sets of calibrated blackbody energy grayscale images and calibrated blackbody temperatures, obtains a first fitting curve, calculates a correction constant based on the first fitting curve, the cooperating blackbody temperature and the cooperating blackbody energy grayscale image, and corrects the energy grayscale image of the monitored object based on the correction constant. The temperature calculation unit calculates a second fitting curve based on the calibrated blackbody temperature and the average gray value of the calibrated blackbody, and calculates the temperature of the monitored object based on the corrected energy grayscale image of the monitored object and the second fitting curve. The alarm information unit is used to determine the temperature of the monitored object and generate alarm information; The data generation module is used to output the alarm information to external systems.

5. A track temperature monitoring device based on image recognition, characterized in that, The device includes monitoring points, infrared cameras, a control box, and power supply equipment. The monitoring points are used to mark the location of the monitored objects; The infrared camera is installed on the outside of the track and has a built-in temperature sensor for collecting images of the radiance of the monitored object, images of the radiance of the moving blackbody shutter, images of the radiance of the cooperating blackbody, and the temperature of the cooperating blackbody. The control box is electrically connected to the infrared camera, receives the image and temperature collected by the infrared camera, processes the image and temperature using the method described in any one of claims 1 to 3, calculates the temperature of the monitored object, determines and generates alarm information and sends it out. The power supply equipment is electrically connected to the control box and the infrared camera, and is used to provide power to the control box and the infrared camera.

Images