Method, device, detection vehicle and equipment suitable for locating fault points in a tunnel

Abstract

The application belongs to the technical field of tunnel detection, and discloses a method and device for positioning fault points in a tunnel, a detection vehicle and equipment. A frame image of the tunnel is obtained at a current time. A frame image to be positioned containing a fault point and a target frame image containing a mileage marker are determined. Character information of the mileage marker in the target frame image is recognized, and mileage calibration of the traveling tool at the current time is performed based on the recognized character information. The actual position of the fault point in the frame image to be positioned is positioned based on the character information in the target frame image, the positional relationship between the frame image to be positioned and the target frame image, and the calibrated mileage of the traveling tool at the current time. The application uses the target frame image containing the mileage marker to position the actual position of the frame image to be positioned containing the fault point, thereby obtaining the actual position of the fault point, reducing cumulative errors, and improving the accuracy of fault point positioning.

Description

Technical Field

[0001] This invention relates to the field of tunnel inspection technology, and in particular to a method, apparatus, inspection vehicle and equipment suitable for locating fault points in tunnels. Background Technology

[0002] In real-world environments, some tunnels experience problems after construction due to complex and constantly changing geological conditions, such as lining cracking, segment cracking, misalignment, and tunnel leakage. Furthermore, issues like surrounding soil cavities and overall settlement of the subway tunnel may also arise. Therefore, construction personnel need to regularly inspect tunnel fault points, which requires locating the fault point.

[0003] The existing steps for fault location based on machine vision are as follows:

[0004] Multiple frames of images are acquired within the tunnel. Adjacent frames are compared using features to determine their displacement. This displacement is accumulated and then transferred to the real-world scene to obtain the actual location of the fault point within the tunnel for each frame, thus locating the fault. However, due to various factors within the tunnel (e.g., tunnel surface discontinuities, stations, single-tunnel, double-tunnel configurations), the clarity of frame images changes with the surrounding environment, affecting the accuracy of machine vision-based tunnel inspection vehicle positioning. Furthermore, the accumulated displacement error increases over time, further reducing the accuracy of fault point location.

[0005] Existing methods for locating tunnel inspection vehicles rely on GPS (Global Positioning System) information and wheel rotations to calculate mileage and determine the vehicle's position. However, due to weak GPS signals within tunnels, sometimes even nonexistent, and wheel slippage, accurate positioning becomes impossible. Therefore, machine vision-based positioning solutions for tunnel inspection vehicles have emerged.

[0006] The steps for locating a tunnel inspection vehicle based on machine vision are as follows:

[0007] Two frames of images are acquired, and the displacement difference between the two frames is determined by feature comparison. The actual position of the tunnel inspection vehicle is then calculated. However, due to various factors within the tunnel (such as tunnel surface discontinuities, railway stations, single-tunnel, and double-tunnel configurations), the clarity of the frame images changes with the surrounding environment, affecting the accuracy of machine vision-based tunnel inspection vehicle positioning. Therefore, achieving accurate positioning within the tunnel is a problem that urgently needs to be solved.

[0008] Based on the above analysis, the problems and shortcomings of the existing technology are as follows:

[0009] (1) When the existing technology obtains the actual location of the fault point, the cumulative error is large and the accuracy of fault point location is low.

[0010] (2) In the prior art, the character information recognition in the target frame image of the mileage marker does not calibrate the mileage of the vehicle based on the character information, which makes the positioning accuracy of the vehicle in the tunnel low. Summary of the Invention

[0011] The purpose of this invention is to provide a method, apparatus, detection vehicle and equipment suitable for locating fault points in tunnels, so as to improve the accuracy of fault point location.

[0012] The specific technical solution is as follows:

[0013] This invention provides a method for locating fault points within a tunnel, wherein the tunnel is equipped with mileage markers to indicate the mileage of the tunnel, and the mileage markers are marked with character information. The method for locating fault points within a tunnel includes:

[0014] Frame images of the tunnel are acquired using an image capturing device mounted on the vehicle.

[0015] The frame image determines the frame image to be located containing the fault point and the target frame image containing the mileage marker; the character information of the mileage marker in the target frame image is identified, and the mileage of the vehicle at the current time is calibrated based on the identified character information;

[0016] Based on the character information in the target frame image, the positional relationship between the frame image to be located and the target frame image, and the mileage calibrated by the vehicle at the current time, the actual location of the fault point in the frame image to be located is determined.

[0017] Optionally, the step of locating a target frame image containing mileage markers in the frame images includes:

[0018] A first frame image containing mileage markers is determined from the frame images;

[0019] When there are multiple first frame images, a target frame image is determined from the multiple first frame images. The target frame image is the first frame image whose center point is closest to the center point of the mileage marker.

[0020] The steps for locating the target frame image containing mileage markers in the frame images include:

[0021] In the frame images, identify the target frame image that was captured most recently and whose center point is closest to the center point of the mileage marker.

[0022] Before the step of locating the actual location of the fault point in the tunnel of the current time frame image based on the character information in the target frame image and the positional relationship between the frame image to be located and the target frame image, the method further includes:

[0023] Identify the character information of mileage markers in the target frame image to obtain the identification result;

[0024] The recognition result includes whether the character information of the mileage marker in the target frame image is recognized, and the character information corresponds to the numerical value of its location in the tunnel;

[0025] The step of locating the actual location of the fault point in the frame image to be located based on the character information in the target frame image and the positional relationship between the frame image to be located and the target frame image includes:

[0026] When the recognition result is that the character information in the target frame image is not recognized, a second frame image is determined according to the time sequence of the frame images. The second frame image is a frame image that is different from the mileage markers contained in the target frame image, is the closest to the time of the target frame image, and can recognize the character information.

[0027] When the target frame image contains a fault point in the frame image to be located, the cumulative mileage is calculated based on the frame images captured between the capture time of the second frame image and the capture time of the target frame image.

[0028] Summing the first value with the preset first distance yields the first summation result;

[0029] Wherein, the first value is the value of the position within the tunnel corresponding to the character information of the second frame image;

[0030] Summing the first value with the cumulative mileage yields a second summation result;

[0031] Based on the capture time of the image to be located, the actual location value of the fault point in the frame image to be located is determined from the first summation result and the second summation result.

[0032] The step of determining the actual location of the fault point in the frame image to be located based on the shooting time of the image to be located, in the first summation result and the second summation result, includes:

[0033] When the difference between the first summation result and the second summation result does not exceed the first difference threshold, the difference between the first summation result and the preset second value is determined as the actual location value of the fault point in the frame image to be located.

[0034] When the difference between the first summation result and the second summation result exceeds the first difference threshold, the difference between the second summation result and the preset second value is determined as the actual location value of the fault point in the frame image to be located.

[0035] Optionally, after the step of identifying the character information of mileage markers in the target frame image and obtaining the identification result, the method further includes:

[0036] When there are multiple target frame images, based on the recognition results obtained by recognizing the character information of the mileage markers in each target frame image, the same character information of the same mileage marker and the same number of the same character information are determined.

[0037] When the number of identical character information exceeds a preset threshold, the actual location of the fault point in the frame image to be located is determined based on the number of identical character information exceeding the preset threshold.

[0038] The step of locating the actual location of the fault point in the frame image to be located based on the character information in the target frame image and the positional relationship between the frame image to be located and the target image includes:

[0039] When the recognition result is that the character information is recognized, it is determined whether the difference between the first value and the second value exceeds the second difference threshold.

[0040] Wherein, the first value is the value of the position in the tunnel corresponding to the character information of the second frame image, the second frame image is a frame image that is different from the mileage markers contained in the target frame image, is the most recently captured frame image and can identify character information, and the second value is the value of the position in the tunnel corresponding to the character information in the target frame image.

[0041] When the difference between the first value and the second value does not exceed the second difference threshold, and the target frame image contains the fault point of the frame image to be located, the second value is determined as the actual location value of the fault point in the frame image to be located.

[0042] When the difference between the first value and the second value exceeds the second difference threshold, a second frame image is determined according to the time sequence of the frame images. The second frame image is a frame image that is different from the mileage markers contained in the target frame image, is the closest to the time of the target frame image, and can identify character information.

[0043] When the target frame image contains a fault point in the frame image to be located, the cumulative mileage is calculated based on the frame images between the shooting time of the second frame image and the shooting time of the target frame image;

[0044] The sum of the first value and the cumulative mileage is determined as the actual location value of the fault point in the frame image to be located.

[0045] The step of locating the actual location of the fault point in the frame image to be located based on the character information in the target frame image and the positional relationship between the frame image to be located and the target image includes:

[0046] When the target frame image does not contain the fault point of the frame image to be located, the difference mileage is calculated based on the frame images between the frame image to be located and the target frame image.

[0047] The difference between the first distance and the difference mileage is summed with the first value, and the summation result is determined as the actual location value of the fault point in the frame image to be located.

[0048] The step of calculating the cumulative mileage based on the frame images between the shooting time of the second frame image and the shooting time of the target frame image includes:

[0049] Calculate the actual size of the object space corresponding to each pixel in the historical frame image, wherein the historical frame image is the frame image between the shooting time of the second frame image and the shooting time of the target frame image;

[0050] Using a preset matching algorithm, the homography matrix of adjacent historical frame images is calculated, and the homography matrix represents the magnitude of the change in the adjacent historical frame images;

[0051] Based on the homography matrix, the image displacement of the previous historical frame relative to the next historical frame in adjacent historical frame images is calculated.

[0052] The product of the actual size of the object space corresponding to each pixel and the image displacement is determined as the actual displacement of the previous historical frame image relative to the next historical frame image.

[0053] The actual displacements are accumulated to obtain the cumulative mileage.

[0054] Optionally, the mileage calibration of the vehicle at the current time specifically includes the following steps:

[0055] The image capturing device captures a frame image of the tunnel at the current time, and the image capturing device moves synchronously with the vehicle.

[0056] Locate the target frame image containing mileage markers within the frame images;

[0057] The character information of mileage markers in the target frame image is identified to obtain a recognition result, wherein the recognition result is: whether the character information is recognized.

[0058] When the recognition result indicates that the character information has been recognized, the mileage of the vehicle at the current time is calibrated based on the character information.

[0059] Optionally, after the step of identifying the character information of mileage markers in the target frame image and obtaining the identification result, the method further includes:

[0060] When the recognition result is that the character information is not recognized, acquire historical frame images of the tunnel taken from the last time the mileage of the driving tool was calibrated to the current time.

[0061] Based on the historical frame images, the cumulative mileage of the vehicle is calculated;

[0062] The mileage from the last calibration is summed with the preset first value to obtain the first summation result;

[0063] The mileage from the last calibration is summed with the cumulative mileage to obtain a second summation result;

[0064] The first summation result is compared with the second summation result to calibrate the mileage of the vehicle at the current time;

[0065] The step of comparing the first summation result with the second summation result to calibrate the mileage of the vehicle at the current time includes:

[0066] When the difference between the first summation result and the second summation result does not exceed the first difference threshold, the difference between the first summation result and the preset second value is determined as the mileage of the vehicle at the current time.

[0067] When the difference between the first summation result and the second summation result exceeds the first difference threshold, the difference between the second summation result and the preset second value is determined as the mileage of the vehicle at the current time.

[0068] Optionally, the step of searching for a target frame image containing mileage markers in the frame image includes:

[0069] A first frame image containing mileage markers is determined from the frame images;

[0070] When there are multiple first frame images, a target frame image is determined from the multiple first frame images. The target frame image is the first frame image whose center point is closest to the center point of the mileage marker.

[0071] After the step of recognizing the character information of mileage markers in the target frame image, the method further includes:

[0072] When there are multiple target frame images, based on the recognition results obtained by recognizing the character information of the mileage markers in each target frame image, the same character information and the same number of the same character information are determined.

[0073] The step of calibrating the mileage of the vehicle at the current time based on the character information when the recognition result is that the character information has been recognized includes:

[0074] When the number of identical character information exceeds a preset threshold, the mileage of the vehicle at the current time is calibrated based on the number of identical character information exceeding the preset threshold.

[0075] The character information includes: the distance between the mileage marker and the starting point;

[0076] When the recognition result is that character information has been recognized, the step of calibrating the mileage of the vehicle at the current time based on the character information includes:

[0077] The distance of the mileage marker from the starting point is summed with the initial mileage of the vehicle, and the sum is determined as the mileage of the vehicle at the current time.

[0078] When the recognition result indicates that the character information has been recognized, the step of calibrating the mileage of the vehicle at the current time based on the character information includes:

[0079] When the recognition result is that the character information is recognized, the character information includes: the distance of the mileage marker from the starting point, and whether the difference between the distance in the historical character information and the distance in the character information at the current time exceeds a second difference threshold. The historical character information is the character information of the last mileage calibration.

[0080] When the difference between the distance in the character information at the current time and the distance in the historical character information does not exceed the second difference threshold, the sum of the distance in the character information at the current time and the initial mileage of the vehicle is determined as the mileage of the vehicle at the current time.

[0081] When the difference between the distance in the character information at the current time and the distance in the historical character information exceeds the second difference threshold, acquire historical frame images taken from the last calibration of the vehicle mileage to the current time.

[0082] Calculate the cumulative mileage based on the historical frame images;

[0083] The sum of the accumulated mileage and the distance in the historical character information is determined as the mileage of the vehicle at the current time;

[0084] The step of calculating the cumulative mileage of the vehicle based on the historical frame images includes:

[0085] Calculate the actual size of the object space corresponding to each pixel in the historical frame image;

[0086] Using a preset matching algorithm, the homography matrix of adjacent historical frame images is calculated, and the homography matrix represents the magnitude of the change in the adjacent historical frame images;

[0087] Based on the homography matrix, the image displacement of the next historical frame relative to the previous historical frame is calculated in adjacent historical frame images.

[0088] Based on the actual size of the object space corresponding to each pixel, the actual displacement of the next historical frame image relative to the previous historical frame image is determined.

[0089] The actual displacements are accumulated to obtain the cumulative mileage of the vehicle.

[0090] After the step of calibrating the mileage of the vehicle at the current time based on the character information, the method further includes:

[0091] Based on the calibrated mileage of the vehicle and the tunnel information, the position of the vehicle in the tunnel is determined. The tunnel information includes: the length of the tunnel, the location of the tunnel entrance, the location of the tunnel exit, and the curvature angle of the tunnel at different locations.

[0092] Optionally, the method for locating fault points within a tunnel further includes:

[0093] Acquire or extract frame images taken by a vehicle while it is traveling inside a tunnel;

[0094] The frame image is used to determine the frame image containing the fault point to be located and the target frame image containing mileage markers; the character information of the mileage markers in the target frame image is then identified.

[0095] Based on the character information in the target frame image, the positional relationship between the frame image to be located and the target frame image, and the mileage calibrated by the vehicle at the current time, the actual location of the fault point in the frame image to be located is determined.

[0096] Secondly, the present invention provides a detection vehicle, which includes: a decision-making device connected to an image capturing device, for acquiring frame images of the tunnel captured by the image capturing device at the current time, wherein the image capturing device moves synchronously with the detection vehicle;

[0097] Locate the target frame image containing mileage markers within the frame images;

[0098] The character information of mileage markers in the target frame image is identified to obtain a recognition result, wherein the recognition result is: whether the character information is recognized.

[0099] When the recognition result is that the character information is recognized, the mileage of the detection vehicle at the current time is calibrated based on the character information;

[0100] The determination module is used to determine, from the frame images, the frame image to be located containing the fault point and the target frame image containing the mileage markers;

[0101] The positioning module is used to locate the actual location of the fault point in the frame image to be positioned based on the character information in the target frame image and the positional relationship between the frame image to be positioned and the target frame image.

[0102] Thirdly, the present invention provides a device for locating fault points in tunnels, the device comprising:

[0103] The acquisition module is used to acquire frame images of the tunnel taken by the image acquisition device at the current time. The image acquisition device moves synchronously with the vehicle. Mileage markers are set in the tunnel, and character information is marked on the mileage markers.

[0104] The determination module is used to determine, from the frame images, the frame image to be located containing the fault point and the target frame image containing the mileage markers;

[0105] The positioning module is used to locate the actual location of the fault point in the frame image to be located based on the character information in the target frame image and the positional relationship between the frame image to be located and the target frame image.

[0106] The search module is used to find target frame images containing mileage markers in frame images;

[0107] The recognition module is used to recognize the character information of mileage markers in the target frame image and obtain the recognition result, which is whether the character information is recognized.

[0108] The calibration module is used to calibrate the mileage of the vehicle at the current time based on the character information when the recognition result is that character information has been recognized.

[0109] In a fourth aspect, the present invention provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; the processor is prompted by machine-executable instructions to implement the method for locating fault points in tunnels.

[0110] In a fifth aspect, the present invention provides a computer-readable storage medium storing a computer program, the computer program being executed by a processor of the method for locating fault points in a tunnel.

[0111] Based on the above technical solutions and the technical problems solved, please analyze the advantages and positive effects of the technical solution to be protected by this invention from the following aspects:

[0112] First, addressing the technical problems existing in the prior art and the difficulty in solving them, this paper closely analyzes, in conjunction with the technical solution to be protected by this invention and the results and data obtained during the research and development process, how the technical solution of this invention solves the technical problems, and the inventive technical effects brought about by solving these problems. The specific description is as follows:

[0113] In existing technologies, underground or mountainous tunnels and utility tunnels are not covered by communication networks, and GPS signals are weak, so image capturing devices cannot obtain accurate positioning information for images when taking pictures.

[0114] Existing technologies that rely on the circumference and number of rotations of the wheels (axles) of the driving equipment to measure mileage may have their accuracy affected by factors such as wheel (axle) slippage or lack of air.

[0115] To solve the above problems, the present invention can break free from the limitations of communication networks or GPS and complete high-precision positioning calculations through images.

[0116] This invention can provide high-precision positioning information in real time during image acquisition. Even when real-time computing is not feasible, positioning information can be calculated using image analysis after image acquisition is complete.

[0117] This invention calculates the displacement of the driving equipment using a homography matrix, thereby eliminating the influence of various factors such as the temperature and material of the driving equipment, acceleration, and braking temperature and humidity on the mileage.

[0118] The entire tunnel is divided into several small sections. By identifying the mileage information in the mileage markers and calibrating the position information multiple times, the accuracy of the positioning is further improved.

[0119] Second, considering the technical solution as a whole or from the perspective of the product, the technical effects and advantages of the technical solution to be protected by this invention are specifically described as follows:

[0120] This invention provides a method and apparatus for locating fault points in tunnels. The method involves acquiring frame images of the tunnel; determining a frame image containing the fault point and a target frame image containing mileage markers; and locating the actual position of the fault point in the frame image containing the target frame image based on character information in the target frame image and the positional relationship between the frame image containing the fault point and the target frame image. Compared to existing technologies, mileage markers are more prominent in images and less affected by the environment in tunnels. Furthermore, mileage markers at different locations correspond to different positions. This invention uses a target frame image containing mileage markers to locate the actual position of the frame image containing the fault point, thereby obtaining the actual position of the fault point. This reduces accumulated errors and improves the accuracy of fault point location. Of course, implementing any product or method of this invention does not necessarily require achieving all of the above advantages simultaneously.

[0121] The mileage calibration method and apparatus provided in this invention acquire frame images of a tunnel captured by an image capturing device at the current time, locate a target frame image containing mileage markers within the frame images, identify character information of the mileage markers in the target frame image, obtain a recognition result, and when the recognition result indicates that the character information has been identified, calibrate the mileage of the vehicle at the current time based on the character information. Compared to existing technologies, this invention improves the accuracy of mileage measurement in tunnels by identifying character information in target frame images containing mileage markers with high positioning accuracy and prominent image features, and calibrating the mileage of the vehicle based on the character information. This enhances the accuracy of vehicle positioning in tunnels.

[0122] Third, as supplementary evidence of the inventive step of the claims of this invention, it is also reflected in the following important aspects:

[0123] The technical solution of this invention fills a technological gap in the industry both domestically and internationally:

[0124] This invention does not rely on infrastructure such as GPS or communication networks. Through image analysis, it locates and calibrates fault points within tunnels, achieving high-precision positioning within tunnels and filling a gap in this field both domestically and internationally.

[0125] Does the technical solution of this invention solve a technical problem that people have long desired to solve but have never been able to achieve? This invention provides a practical and effective method and device description for achieving high-precision positioning in underground or mountainous tunnels, pipe corridors and other structures not covered by communication networks by using images acquired by an image acquisition device. It solves the urgent problem of inaccurate positioning in current tunnel construction, inspection, maintenance and repair operations. Attached Figure Description

[0126] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0127] Figure 1 A flowchart illustrating a method for locating fault points within a tunnel, as provided in an embodiment of the present invention;

[0128] Figure 2 A flowchart of step S13 is provided as an embodiment of the present invention;

[0129] Figure 3 A flowchart of another implementation step S13 provided in an embodiment of the present invention;

[0130] Figure 4 A structural diagram of a device for locating fault points in a tunnel, provided by an embodiment of the present invention;

[0131] Figure 5 This is a structural diagram of an electronic device provided in an embodiment of the present invention.

[0132] Figure 6 This is a flowchart of mileage calibration in the method for locating fault points in a tunnel provided in Embodiment 14 of the present invention;

[0133] Figure 7 This is a flowchart of the implementation step S44 provided in Embodiment 16 of the present invention;

[0134] Figure 8 A flowchart for calculating cumulative mileage provided in an embodiment of the present invention;

[0135] Figure 9 This is a structural diagram of a mileage calibration device suitable for tunnels provided in an embodiment of the present invention;

[0136] In the diagram: 41. Acquisition module; 42. Confirmation module; 43. Location module; 44. Search module;

[0137] 45. Identification module; 46. Calibration module; 51. Processor; 52. Communication interface; 53. Memory; 54. Communication bus. Detailed Implementation

[0138] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0139] I. Explanatory and Illustrative Embodiments. To enable those skilled in the art to fully understand how the present invention is specifically implemented, this section provides an explanatory and illustrative description of the embodiments described in the claims.

[0140] This invention provides a method for locating fault points within a tunnel, wherein the tunnel is equipped with mileage markers to indicate the mileage of the tunnel, and the mileage markers are marked with character information. The method for locating fault points within a tunnel includes:

[0141] The image capturing device mounted on the vehicle is used to capture frame images of the tunnel at the current time.

[0142] The frame image determines the frame image to be located containing the fault point and the target frame image containing the mileage marker; the character information of the mileage marker in the target frame image is identified, and the mileage of the vehicle at the current time is calibrated based on the identified character information;

[0143] Based on the character information in the target frame image, the positional relationship between the frame image to be located and the target frame image, and the mileage calibrated by the vehicle at the current time, the actual location of the fault point in the frame image to be located is determined.

[0144] The present invention also provides an inspection vehicle, which includes: a decision-making device connected to an image acquisition device, for acquiring frame images of the tunnel captured by the image acquisition device at the current time, wherein the image acquisition device moves synchronously with the inspection vehicle;

[0145] Locate the target frame image containing mileage markers within the frame images;

[0146] The character information of mileage markers in the target frame image is identified to obtain a recognition result, wherein the recognition result is: whether the character information is recognized.

[0147] When the recognition result is that the character information is recognized, the mileage of the detection vehicle at the current time is calibrated based on the character information;

[0148] The determination module is used to determine, from the frame images, the frame image to be located containing the fault point and the target frame image containing the mileage markers;

[0149] The positioning module is used to locate the actual location of the fault point in the frame image to be positioned based on the character information in the target frame image and the positional relationship between the frame image to be positioned and the target frame image.

[0150] The present invention also provides a device for locating fault points in tunnels, the device comprising:

[0151] The acquisition module is used to acquire frame images of the tunnel taken by the image acquisition device at the current time. The image acquisition device moves synchronously with the vehicle. Mileage markers are set in the tunnel, and character information is marked on the mileage markers.

[0152] The determination module is used to determine, from the frame images, the frame image to be located containing the fault point and the target frame image containing the mileage markers;

[0153] The positioning module is used to locate the actual location of the fault point in the frame image to be located based on the character information in the target frame image and the positional relationship between the frame image to be located and the target frame image.

[0154] The search module is used to find target frame images containing mileage markers in frame images;

[0155] The recognition module is used to recognize the character information of mileage markers in the target frame image and obtain the recognition result, which is whether the character information is recognized.

[0156] The calibration module is used to calibrate the mileage of the vehicle at the current time based on the character information when the recognition result is that character information has been recognized.

[0157] In this embodiment of the invention, the method for locating fault points within a tunnel further includes:

[0158] Acquire or extract frame images taken by a vehicle while it is traveling inside a tunnel;

[0159] The frame image is used to determine the frame image containing the fault point to be located and the target frame image containing mileage markers; the character information of the mileage markers in the target frame image is then identified.

[0160] Based on the character information in the target frame image, the positional relationship between the frame image to be located and the target frame image, and the mileage calibrated by the vehicle at the current time, the actual location of the fault point in the frame image to be located is determined.

[0161] The present invention will be further described below with reference to specific embodiments.

[0162] Example 1

[0163] like Figure 1 As shown, this embodiment of the invention provides a method for locating fault points in tunnels, the method comprising:

[0164] It's understandable that mileage markers are placed in tunnels, and these markers contain textual information. When images of the tunnel are taken, these mileage markers will also be captured. Mileage markers often contain textual information indicating their location within the tunnel or their distance from the starting point. The specific textual information included on each mileage marker varies depending on the actual situation. What is clear is that these mileage markers are precise measurements taken by engineers to determine the tunnel's location, and their position within the tunnel is the most accurate.

[0165] S11, acquire frame images of the tunnel.

[0166] It is understood that the camera's placement on the mobile device or tunnel inspection vehicle can be such that the camera can just capture the mileage markers. Manually acquiring frame images of the tunnel can be done by using a camera to capture images of various parts of the tunnel, using a robot equipped with a camera to capture images of the tunnel, using a device with shooting capabilities, or using a tunnel inspection vehicle with shooting capabilities to capture images of the tunnel. Frame image acquisition is not limited to the above methods, and this invention does not impose any limitations on them.

[0167] S12, determine the frame image to be located containing the fault point and the target frame image containing the mileage marker in the frame image.

[0168] Understandably, if there are cracks or leaks in the tunnel, it is necessary to locate the cracks or leaks, i.e., the location of the fault point. Using the frame image containing the fault point as the frame image to be located, it is necessary to calculate the actual position of the frame image within the tunnel in order to pinpoint the actual location of the fault point. Existing technologies can be used for identifying the frame image to be located, which will not be elaborated upon here.

[0169] That is, by using image recognition technology to perform image recognition on multiple acquired frame images, the frame image to be located containing the fault point and the target frame image of Baoheng mileage marker are identified from the multiple frame images.

[0170] The mileage markers can be 100-meter, 50-meter, or 10-meter markers, depending on the specific mileage markers installed beside the tunnel.

[0171] S13, based on the character information in the target frame image and the positional relationship between the frame image to be located and the target frame image, locate the actual location of the fault point in the frame image to be located.

[0172] The positional relationship refers to the temporal relationship between the capture time of the frame image to be located and the capture time of the target frame image, or it can be the distance relationship between the actual position of the image to be located and the actual position of the target frame image. Character information includes characters corresponding to the location of the mileage marker within the tunnel.

[0173] It is understandable that there is a time interval between the capture time of the frame image to be located and the capture time of the target frame. During this time interval, the camera may capture multiple consecutive frames, and there is a displacement difference between these consecutive frames. The greater the displacement difference, the greater the distance between the actual position of the frame image to be located and the actual position of the target frame image. The actual position of the target frame image can be determined based on the character information of the mileage markers contained in the image.

[0174] It is understandable that the location of the target frame image can be determined by the position of the mileage marker corresponding to the character information in the target frame image within the tunnel. By determining the actual distance between the target frame image and the frame image to be located based on their positional relationship, the actual location of the frame image to be located can be determined, thus revealing the actual location of the fault point within the tunnel.

[0175] For example, if the target frame image is captured at 59 seconds and the frame image to be located is captured at 60 seconds, and the camera captures 5 frames in 1 second, then the distance between the target frame image and the frame image to be located is the actual displacement difference between the two frames multiplied by 6. If the position corresponding to the character in the target frame image is 100 meters away, then the position of the frame image to be located is 6 frames away from 100 meters away.

[0176] Example 2

[0177] As an optional implementation of this invention, determining the target frame image containing mileage markers in S12 above includes the following steps:

[0178] Step 1: Identify the first frame image containing the mileage markers in the frame images;

[0179] It is understood that the features of mileage markers are relatively obvious in images. When searching for target frame images containing mileage markers, feature points of the mileage markers can be extracted. For example, SIFT (Scale Invariant Feature Transform) feature points, corner points, etc., are not limited to these in this invention.

[0180] Step 2: When there are multiple first frame images, determine the target frame image from the multiple first frame images.

[0181] The target frame image is the first frame image whose center point is closest to the center point of the mileage marker.

[0182] It is understandable that, for multiple consecutive frames containing the same mileage marker, the center coordinates (x, y, y) of the mileage marker in each image are calculated. i ,y i ), (i = 1…n). n represents the number of frames captured by the camera that contain the same mileage marker. Then, the distance Dist between the center coordinates of the mileage marker and the center coordinates of the frame image is calculated. i (i = 1…n), select the frame image with the smallest distance as the target frame image.

[0183] This embodiment calculates the distance between the center coordinates of the mileage marker in the frame image and the center coordinates of the frame image, and selects the frame image with the smallest distance as the target frame image, so that the mileage marker is as close to the center of the frame image as possible, thereby improving the accuracy of character information recognition.

[0184] Example 3

[0185] As an optional implementation of this invention, determining the target frame image containing mileage markers in S12 above includes the following steps:

[0186] Identify the target frame image from the frame images that was captured most recently and whose center point is closest to the center point of the mileage marker.

[0187] For example, suppose there are three frame images, A, B, and C. Image A contains a mileage marker image of 100 meters; images B and C contain mileage marker images of 200 meters; image A was captured earlier than images B and C. Among images B and C, image B was captured closest to the capture time of the frame image to be located, so image C is selected as the target frame image.

[0188] It is understandable that, assuming there are multiple frame images containing different mileage markers, selecting the frame image closest to the frame image to be located can reduce accumulated errors and thus improve the accuracy of locating the frame image to be located.

[0189] Example 4

[0190] As an optional implementation of the present invention, after the above-described step S13, the method for locating fault points in a tunnel provided by the present invention further includes: identifying the character information of mileage markers in the target frame image and obtaining the identification result.

[0191] The recognition results include whether the character information of the mileage markers in the target frame image is recognized, and the character information corresponds to the numerical value of the location within the tunnel.

[0192] It is understandable that when recognizing the character information of mileage markers in a target frame image, factors such as image sharpness, image scale, or the position of the mileage marker within the target frame image can affect the character information of the mileage marker. For example, when part of the mileage marker image is in the target frame image, the character information of the target frame image cannot be recognized due to the missing text.

[0193] Example 5

[0194] As an optional implementation method provided in the embodiments of the present invention, such as Figure 2 As shown, step S13 above can be implemented as follows:

[0195] S131, when the recognition result is that no character information in the target frame image is recognized, the second frame image is determined according to the time sequence of the frame images.

[0196] The second frame image is a frame image that contains different mileage markers than the target frame image, is the most recently captured frame image, and has recognizable character information.

[0197] Understandably, if the recognition result shows no character information, it means the location of the mileage marker in the tunnel cannot be determined, and therefore the mileage marker cannot be used to locate the target frame image. In this case, the target frame image needs to be redefined. When redefined, a frame image different from the one containing the current mileage marker can be selected, and this new frame image must contain recognizable character information. Frame images can be stored in a database that records the capture time of each frame image. When a frame image is needed, it can be retrieved from the database. Simultaneously, the position corresponding to the character information of each recognized target frame image is recorded.

[0198] S132, when the target frame image contains a fault point in the frame image to be located, calculate the cumulative mileage based on the frame images captured between the capture time of the second frame image and the capture time of the target frame image.

[0199] It's understandable that if the target frame image contains a fault point in the frame image to be located, knowing the location of the target frame image will lead to the location of the fault point. However, since character recognition in the target frame image may result in errors, it's necessary to further evaluate the accuracy of locating the target frame image based on character information.

[0200] S133, sum the first value with the preset first distance to obtain the first summation result.

[0201] The first value is the value of the position within the tunnel corresponding to the character information of the second frame image.

[0202] It is understandable that the first distance setting increases as the distance between the first and second positions increases. Since the second frame image may contain different mileage markers with different characters, the distance between the second frame image and the target frame image is obtained by acquiring the time length between the frames and then further calculating.

[0203] The first distance is the distance between the two mile markers.

[0204] For example, consider two image frames, A and B, from which character information can be identified. The location corresponding to the character information in image A is 100 meters, and the location corresponding to the character information in image B is 200 meters. If the second image is image B, then the first distance value is 100 meters. If the second image is image A, then the first distance value is 200 meters.

[0205] S134, sum the first value with the cumulative mileage to obtain the second summation result;

[0206] It is understandable that there is a displacement difference between adjacent frame images. Based on the displacement difference between adjacent frame images, the cumulative mileage from the second frame image to the target frame image can be obtained.

[0207] S135, based on the capture time of the image to be located, determine the actual location value of the fault point in the frame image to be located from the first summation result and the second summation result.

[0208] It is understandable that comparing the first summation result with the second summation result can determine the actual location value of the fault point in the frame image to be located. This value can be understood as the displacement value perpendicular to the tunnel cross-section.

[0209] Example 6

[0210] As an optional implementation method provided by the embodiments of the present invention, the above-described step S13 can be implemented as follows:

[0211] Step 1: When the difference between the first summation result and the second summation result does not exceed the first difference threshold, the difference between the first summation result and the preset second value is determined as the actual location value of the fault point in the frame image to be located.

[0212] Step 2: When the difference between the first summation result and the second summation result exceeds the first difference threshold, the difference between the second summation result and the preset second value is determined as the actual location value of the fault point in the frame image to be located.

[0213] The second value is the distance between the target frame image's mileage marker in the tunnel and the camera, i.e., the object distance. The first difference threshold is a value preset based on the tunnel length and practical experience; in practical applications, it can be set to 10 meters.

[0214] It is understood that in frame images containing the same mileage marker, if the first and second frame images selected are different, the distance between the marker and the image will be different. In this embodiment of the invention, each time the frame image to be located is positioned, the distance between the center of the mileage marker in the selected target frame image and the center of the image is the same.

[0215] For example, suppose that when the mileage marker enters the camera's shooting range, the first frame of the image captured by the camera containing the mileage marker is selected as the target frame image; or the nth frame of the image captured by the camera containing the mileage marker is selected as the target frame image. The distance between the center of the mileage marker and the center of the image is different in these two cases, and the object distance of the camera is different, so the value of the second value is different.

[0216] Example 7

[0217] As an optional implementation of this invention, after the step of identifying the character information of mileage markers in the target frame image and obtaining the identification result, the method for locating fault points in tunnels provided in this invention further includes:

[0218] Step 1: When there are multiple target frame images, based on the recognition results obtained by recognizing the character information of the mileage markers in each target frame image, determine the same character information of the same mileage marker and the number of identical characters.

[0219] Step 2: When the number of identical character information exceeds a preset threshold, locate the actual location of the fault point in the frame image to be located based on the number of identical character information exceeding the preset threshold.

[0220] It is understandable that some areas in certain frame images may be blurry. When multiple frame images contain the same mileage marker, the text information in these multiple frame images can be identified; the identical character information of the same mileage marker and the number of identical character information can be determined; and the location of the mileage marker corresponding to the character information with a number exceeding a threshold can be used to locate the actual location of the fault point in the frame image to be located. This can improve the reliability of obtaining the character information of the target frame image.

[0221] Example 8

[0222] As an optional implementation method provided in the embodiments of the present invention, such as Figure 3 As shown, step S13 above can be implemented as follows:

[0223] S31, when the recognition result is that character information has been recognized, determine whether the difference between the first value and the second value exceeds the second difference threshold;

[0224] The first value is the value of the location within the tunnel corresponding to the character information in the second frame image. The second frame image is a frame image that is different from the target frame image in terms of the mileage markers it contains, is the closest to the target frame image in terms of the time it was captured, and whose character information can be identified. The second value is the value of the location within the tunnel corresponding to the character information in the target frame image.

[0225] The second difference threshold is a preset value, which can be 10 meters in practice.

[0226] S32, when the difference between the first value and the second value does not exceed the second difference threshold, and the target frame image contains a fault point in the frame image to be located, the second value is determined as the actual location of the fault point in the frame image to be located.

[0227] S33, when the difference between the first value and the second value exceeds the second difference threshold, determine the second frame image according to the time sequence of the frame images.

[0228] The second frame image is a frame image that contains different mileage markers than the target frame image, is the most recently captured frame image, and has recognizable character information.

[0229] S34, when the target frame image contains a fault point in the frame image to be located, calculate the cumulative mileage based on the frame images between the shooting time of the second frame image and the shooting time of the target frame image;

[0230] S35, the sum of the first value and the cumulative mileage is determined as the actual location value of the fault point in the frame image to be located.

[0231] It is understood that this embodiment compares the value of the position of the character information in the tunnel corresponding to the character information in the second frame image with the value of the position of the character information in the tunnel corresponding to the character information in the target frame image to determine whether the target frame image can be used to locate the frame image to be located, thereby improving the accuracy of locating the fault point.

[0232] Example 9

[0233] As an optional implementation method provided by the embodiments of the present invention, the above-described step S13 can be implemented as follows:

[0234] Step 1: When the target frame image does not contain the fault point of the frame image to be located, calculate the difference mileage based on the frame images between the frame image to be located and the target frame image.

[0235] It's understandable that if the frame image to be located is located between the second frame image and the target frame image, then whether the frame image to be located is located based on the target frame image or the second frame image depends on which frame image the frame image to be located is closest to, thus reducing accumulated errors. If the target frame image and the frame image to be located are closest, and the frame image to be located was captured earlier than the target frame image, then the difference mileage between the target frame image and the frame image to be located is calculated. Then, the first distance, i.e., the distance between the two mileage markers, is subtracted from the difference mileage to obtain the distance between the frame image to be located and the target frame image, thus obtaining the actual location of the fault point in the frame image to be located.

[0236] Step 2: Sum the difference between the first distance and the difference mileage with the first value, and determine the summation result as the actual location of the fault point in the frame image to be located.

[0237] It can be understood that the difference between the first distance and the differential mileage can be used to determine the distance between the fault point in the frame image to be located and the mileage marker in the target frame image. If the location of the frame image to be located in the tunnel is known, the actual location of the frame image to be located in the tunnel can be determined based on the tunnel length, the coordinates of the tunnel entrance and exit, and the location of the mileage marker in the tunnel. This allows for further determination of the location of the fault point in the frame image to be located in the tunnel perpendicular to the circular hollow section.

[0238] Example 10

[0239] As an optional implementation of this invention, the step of calculating the cumulative mileage based on the frame images between the shooting time of the second frame image and the shooting time of the target frame image includes:

[0240] Step 1: Calculate the actual size of the object space corresponding to each pixel in the historical frame image. The historical frame image is the frame image between the shooting time of the second frame image and the shooting time of the target frame image.

[0241] Step 2: Calculate the homography matrix of adjacent historical frame images using a preset matching algorithm;

[0242] The homography matrix represents the magnitude of change between adjacent historical frames.

[0243] Step 3: Based on the homography matrix, calculate the image displacement of the previous historical frame relative to the next historical frame in adjacent historical frame images;

[0244] Step 4: Based on the actual size of the object space corresponding to each pixel, determine the actual displacement of the previous historical frame image relative to the next historical frame image.

[0245] Step 5: Accumulate the actual displacement to obtain the cumulative mileage.

[0246] It can be understood that if the image displacement of the next frame relative to the previous frame is x, and the actual size of the object space corresponding to each pixel is d, then the actual displacement of the next historical frame relative to the previous historical frame is s = d * x.

[0247] Example 11

[0248] As an optional implementation of the present invention, a trained region recognition model can be used to find target frame images containing mileage markers in frame images.

[0249] The region recognition model can be a Cascade R-CNN model. The process of obtaining a trained region recognition model is as follows:

[0250] Step 1: Obtain the sample set, which includes frame images containing mileage markers collected at different locations. Each frame image is a sample.

[0251] Step 2: Label the samples in the sample set, indicating the location of the mileage markers in the tunnel.

[0252] Step 3: Input the labeled samples into the preset network model, use the labeling results as the training target of the Cascade R-CNN model, and iteratively adjust the internal parameters in the network model until the number of iterations or the training target is reached.

[0253] Step 4: Use the network model that has reached the required number of iterations or the training target as the training number for the trained network model.

[0254] It is understood that this embodiment trains a region recognition model using standard samples, and using this region recognition model to recognize frame images can improve the accuracy of the recognition results.

[0255] Example 12

[0256] As an optional implementation of the present invention, a trained character recognition model can be used to recognize character information in a target frame image.

[0257] The character recognition model can be a ResNet model. The process of obtaining a trained character recognition model is as follows:

[0258] Step 1: Collect the different types of characters contained in the mileage marker images and use each character as a training sample.

[0259] Step 2: For each training sample, input the training sample into the preset character recognition model, use the distance information corresponding to the training sample as the training target, and iteratively train the preset character recognition model until the number of iterations is reached to obtain the trained character recognition model.

[0260] It is understood that this embodiment trains a character recognition model by using different types of characters contained in the mileage marker image, and using the character recognition model to recognize the frame image can improve the accuracy of the recognition results.

[0261] Example 13

[0262] As an optional implementation of this invention, before using the trained character recognition model to recognize character information in the target frame image, the method for locating fault points in a tunnel provided by this invention further includes:

[0263] Step 1: Perform threshold segmentation on the mileage marker region in the frame image to obtain a binary image containing characters;

[0264] Step 2: Label the connected components of the binary graph to obtain the connected threshold labeled graph;

[0265] Step 3: Use the trained character recognition model to identify the characters in the connectivity threshold marker map and determine the numerical value that the character represents, which indicates the location of the mileage marker in the tunnel.

[0266] It is understandable that different characters are marked on different locations of mileage markers, and each character has its own meaning. After thresholding the mileage marker area in the frame image, identifying different connected component markers can quickly determine the meaning of each character, that is, the value represented by the character.

[0267] This invention provides a method for locating fault points in tunnels. The method involves acquiring frame images of the tunnel; determining a frame image containing the fault point and a target frame image containing mileage markers; and locating the actual position of the fault point in the frame image containing the target frame image based on character information in the target frame image and the positional relationship between the frame image containing the fault point and the target frame image. Compared to existing technologies, mileage markers are more prominent in images and less affected by the environment in tunnels. Furthermore, mileage markers at different locations correspond to different positions. This invention uses a target frame image containing mileage markers to locate the actual position of the frame image containing the fault point, thereby obtaining the actual position of the fault point. This reduces accumulated errors and improves the accuracy of fault point location.

[0268] like Figure 4 As shown in the figure, an embodiment of the present invention provides a device for locating fault points in a tunnel, the device comprising:

[0269] Acquisition module 41 is used to acquire frame images of the captured tunnel;

[0270] The tunnel is equipped with mileage markers, which are marked with character information.

[0271] The determination module 42 is used to determine, from the frame images, the frame image to be located containing the fault point and the target frame image containing the mileage marker;

[0272] The positioning module 43 is used to locate the actual location of the fault point in the frame image to be located based on the character information in the target frame image and the positional relationship between the frame image to be located and the target frame image.

[0273] Optional, determine the module, specifically used for:

[0274] Identify the first frame image containing the mileage markers within the frame images;

[0275] When there are multiple first frame images, the target frame image is determined from the multiple first frame images. The target frame image is the first frame image whose center point is closest to the center point of the mileage marker.

[0276] Optional, determine the module, specifically used for:

[0277] Identify the target frame image from the frame images that was captured most recently and whose center point is closest to the center point of the mileage marker.

[0278] Optionally, the device for locating fault points in a tunnel provided in this embodiment of the invention further includes: an identification module, used for:

[0279] Identify the character information of mileage markers in the target frame image and obtain the identification results;

[0280] The recognition results include: whether the character information of the mileage markers in the target frame image is recognized, and the value of the character information corresponding to the location in the tunnel;

[0281] Optional, positioning module, specifically used for:

[0282] Based on the character information in the target frame image and the positional relationship between the frame image to be located and the target frame image, the steps for locating the actual location of the fault point in the frame image to be located include:

[0283] When the recognition result is that no character information is recognized in the target frame image, the second frame image is determined according to the time sequence of the frame images. The second frame image is a frame image that is different from the mileage markers contained in the target frame image, is the closest to the time of the target frame image, and can recognize character information.

[0284] When the target frame image contains a fault point in the frame image to be located, the cumulative mileage is calculated based on the frame images captured between the capture time of the second frame image and the capture time of the target frame image.

[0285] Summing the first value with the preset first distance yields the first summation result;

[0286] The first value is the value of the position within the tunnel corresponding to the character information of the second frame image;

[0287] Sum the first value with the cumulative mileage to obtain the second summation result;

[0288] Based on the capture time of the image to be located, the actual location value of the fault point in the frame image to be located is determined from the first summation result and the second summation result.

[0289] Optional, positioning module, specifically used for:

[0290] When the difference between the first summation result and the second summation result does not exceed the first difference threshold, the difference between the first summation result and the preset second value is determined as the actual location value of the fault point in the frame image to be located.

[0291] When the difference between the first summation result and the second summation result exceeds the first difference threshold, the difference between the second summation result and the preset second value is determined as the actual location value of the fault point in the frame image to be located.

[0292] Optional, the recognition module is specifically used for:

[0293] When there are multiple target frame images, based on the recognition results obtained by recognizing the character information of the mileage markers in each target frame image, the same character information of the same mileage marker and the same number of the same character information are determined.

[0294] When the number of identical character information exceeds a preset threshold, the actual location of the fault point in the frame image to be located is determined based on the number of identical character information exceeding the preset threshold.

[0295] Optional, positioning module, specifically used for:

[0296] When the recognition result indicates that character information has been recognized, determine whether the difference between the first value and the second value exceeds the second difference threshold.

[0297] Wherein, the first value is the value of the location in the tunnel corresponding to the character information in the second frame image, the second frame image is a frame image that is different from the target frame image in terms of the mileage markers contained therein, is the closest to the target frame image in terms of the time of the capture, and can identify the character information, and the second value is the value of the location in the tunnel corresponding to the character information in the target frame image.

[0298] When the difference between the first value and the second value does not exceed the second difference threshold, and the target frame image contains the fault point of the frame image to be located, the second value is determined as the actual location value of the fault point in the frame image to be located.

[0299] When the difference between the first value and the second value exceeds the second difference threshold, the second frame image is determined according to the time sequence of the frame images. The second frame image is a frame image that is different from the mileage markers contained in the target frame image, is the closest to the target frame image in terms of shooting time, and can identify character information.

[0300] When the target frame image contains a fault point in the frame image to be located, the cumulative mileage is calculated based on the frame images between the shooting time of the second frame image and the shooting time of the target frame image.

[0301] The sum of the first value and the cumulative mileage is used to determine the actual location of the fault point in the frame image to be located.

[0302] Optional, positioning module, specifically used for:

[0303] When the target frame image does not contain the fault point of the frame image to be located, the difference mileage is calculated based on the frame images between the frame image to be located and the target frame image.

[0304] The difference between the first distance and the difference mileage is summed with the first value, and the summation result is determined as the actual location value of the fault point in the frame image to be located.

[0305] Optional, positioning module, specifically used for:

[0306] Calculate the actual size of the object space corresponding to each pixel in the historical frame image. The historical frame image is the frame image between the shooting time of the second frame image and the shooting time of the target frame image.

[0307] Using a preset matching algorithm, the homography matrix of adjacent historical frame images is calculated. The homography matrix represents the magnitude of the change between adjacent historical frame images.

[0308] Based on the homography matrix, the image displacement of the previous historical frame relative to the next historical frame in adjacent historical frame images is calculated.

[0309] Based on the actual size of the object space corresponding to each pixel, determine the actual displacement of the previous historical frame image relative to the next historical frame image.

[0310] The actual displacements are accumulated to obtain the cumulative mileage.

[0311] This invention provides a device for locating fault points in tunnels. It acquires frame images of the tunnel; determines a frame image containing the fault point and a target frame image containing mileage markers; and locates the actual location of the fault point in the frame image containing the target frame image based on character information in the target frame image and the positional relationship between the frame image containing the fault point and the target frame image. Compared to existing technologies, mileage markers are more prominent in images and less affected by the environment in tunnels. Furthermore, mileage markers at different locations correspond to different positions. This invention uses a target frame image containing mileage markers to locate the actual location of the frame image containing the fault point, thereby obtaining the actual location of the fault point. This reduces accumulated errors and improves the accuracy of fault point location.

[0312] This invention also provides a device for locating fault points, such as... Figure 5 As shown, it includes a processor 51, a communication interface 52, a memory 53, and a communication bus 54, wherein the processor 51, the communication interface 52, and the memory 53 communicate with each other through the communication bus 54.

[0313] Memory 53 is used to store computer programs;

[0314] When processor 51 executes the program stored in memory 53, it performs the following steps:

[0315] Acquire frame images of the tunnel;

[0316] The frame image is used to determine the frame image to be located, which contains the fault point, and the target frame image, which contains mileage markers.

[0317] Based on the character information in the target frame image and the positional relationship between the frame image to be located and the target frame image, the actual location of the fault point in the frame image to be located is determined.

[0318] Optionally, embodiments of the present invention also provide a device for locating fault points, which may further include an image capturing device for capturing frame images of the tunnel.

[0319] Example 14

[0320] like Figure 6 As shown in the figure, an embodiment of the present invention provides a mileage calibration method suitable for tunnels, applied to a vehicle with camera functionality. The method includes:

[0321] S41, acquire a frame image of the tunnel captured by the image capturing device at the current time.

[0322] Mileage markers are installed alongside the tunnel, displaying information including the distance from the starting point. The image acquisition device moves synchronously with the vehicle.

[0323] It is understood that the vehicle can be a car, subway, light rail, train, or other transportation vehicle, or it can be an engineering vehicle used for tunnel inspection, or a trailer attached to an engineering vehicle, equipped with an image acquisition device. This image acquisition device can be a camera, a video camera, or other device with shooting capabilities. The camera angle can capture images of the mileage markers inside the tunnel. The starting point can be at the tunnel entrance, or somewhere some distance from the tunnel entrance; this starting point varies depending on the actual situation. This starting point is the initial position used by engineers to calibrate the distances on the mileage markers.

[0324] Taking engineering vehicles as an example, existing tunnels are equipped with mileage markers at predetermined locations. These markers can be 100-meter markers, 50-meter markers, etc. The mileage markers record the distance from the tunnel entrance to the marker and also indicate the vehicle's direction of travel. Engineers can use these markers to determine the marker's position within the tunnel and its distance from the starting point. Since tunnels are two-way passages, the mileage markers can have both sides. Engineers entering from the tunnel entrance can mark the distance from the entrance on the front and the distance from the exit on the back. Alternatively, the markers can mark the distance from the starting point on both sides, with the markings varying according to the actual situation.

[0325] Mileage markers are signs set by technicians through precise measurements, resulting in high accuracy. In tunnels, mileage markers are visually distinct from their surroundings. The inventors discovered through experiments that mileage markers stand out more prominently in images than surrounding lights or markings on tunnel walls, are less affected by the surrounding environment, and exhibit higher accuracy in machine vision recognition.

[0326] For example, when the engineering vehicle activates its camera function, if the mileage marker is within the vehicle's camera range during its movement, the vehicle can capture a frame image containing the mileage marker; if the mileage marker is not within the vehicle's camera range, the frame image will only contain images of the tunnel's surrounding environment, such as the tunnel walls, tunnel surface, and tunnel lights.

[0327] When the mileage markers are affixed to the tunnel wall, the camera or video camera can be positioned on the side wall of the construction vehicle, facing the tunnel wall. When the mileage markers are placed next to the tunnel wall, with the marker face directly towards the construction vehicle, the camera can be positioned in front of the construction vehicle, or positioned within a 90-degree radius to the left and right of the vehicle's centerline in the direction of travel. The height of the camera should be within a preset range to ensure the mileage markers are within the camera's field of view. This camera can be a movable lens camera, used to rotate and capture images of the mileage markers within a 0-degree radius in the direction of travel and a 180-degree radius in the opposite direction.

[0328] It is understandable that mileage markers are highly distinctive, detailed, and accurately positioned, thus ensuring their reliability and usability. Furthermore, using mileage markers to locate vehicles requires only one image acquisition device, making the solution easy to implement and inexpensive.

[0329] S42, Locate the target frame image containing mileage markers in the frame image.

[0330] The target frame image is a frame image that contains mileage markers.

[0331] It is understood that the features of mileage markers are relatively obvious in the image. When searching for a target frame image containing mileage markers, feature points of the mileage markers can be extracted, such as SIFT (Scale Invariant Feature Transform) feature points and corner points. This invention does not impose any limitations on these features.

[0332] S43, identify the character information of mileage markers in the target frame image and obtain the recognition result.

[0333] The recognition result is whether character information was recognized.

[0334] It is understandable that when recognizing the character information of mileage markers in a target frame image, factors such as image sharpness, image scale, or the position of the mileage marker within the target frame image can affect the recognition of this character information. For example, when part of the mileage marker image is present in the target frame image, the character information cannot be recognized due to the missing text.

[0335] S44, when the recognition result is that character information has been recognized, the mileage of the vehicle at the current time is calibrated based on the character information.

[0336] It is understandable that when the character information is recognized, the distance from the starting point recorded on the mileage marker is compared with the current mileage of the vehicle based on the distance from the starting point in the character information, so that the mileage of the vehicle can be calibrated.

[0337] In this embodiment of the invention, when a vehicle enters a tunnel, a frame image of the tunnel taken at the current time is acquired. A target frame image containing mileage markers is located within this frame image. Character information of the mileage markers in the target frame image is identified, and a recognition result is obtained. When the recognition result indicates that character information has been identified, the mileage of the vehicle at the current time is calibrated based on this character information. Compared to existing technologies, this embodiment of the invention improves the accuracy of mileage tracking in tunnels by identifying character information in target frame images containing mileage markers with high positioning accuracy and prominent image features, and then calibrating the mileage of the vehicle based on this character information. This enhances the accuracy of the vehicle's positioning within the tunnel. Furthermore, this embodiment of the invention can complete mileage calibration without adding additional equipment, resulting in low construction costs.

[0338] Example 15

[0339] As an optional embodiment of the present invention, before step S41 above, the mileage calibration method for tunnels provided by the present invention further includes: calibrating the camera.

[0340] The steps for calibrating a camera are as follows:

[0341] (a) Keep the camera position fixed, place the calibration plate at different positions in the tunnel, collect m frames of images, and detect corner points in the frames. Assume that n corner points can be detected in each frame.

[0342] (b) Iterate through each frame of the image and solve for the homography matrix H. j H j The initial value is used to perform nonlinear optimization on the identity matrix; j represents the sequence number of the frame image.

[0343] (c) Solve for the camera intrinsic parameter matrix K;

[0344] (d) Solve for the camera's extrinsic parameters and calculate R. j and t j .

[0345] Where Rj represents the rotation matrix and tj represents the translation vector.

[0346] (e) Solve for the camera distortion coefficients K1 and K2.

[0347] (f) Parameter optimization: Optimize K and R using nonlinear least squares method. j tj K1 and K2 are used to obtain the camera's intrinsic and extrinsic parameters.

[0348] Example 16

[0349] As an optional embodiment of the present invention, such as Figure 7 As shown, after step S44, the mileage calibration method provided in this embodiment of the invention further includes:

[0350] S51, when the recognition result is that no character information is recognized, acquire historical frame images of the tunnel taken from the last time the driving vehicle mileage was calibrated to the current time.

[0351] It is understandable that if the recognition result is that no character information is recognized, it means that the distance of the mileage marker from the starting point cannot be obtained. Therefore, the mileage marker cannot be used to calibrate the mileage of the vehicle at the current time. In this case, the frame image from the last mileage calibration to the current time is needed to know the distance traveled by the vehicle from the last mileage calibration to the current time.

[0352] S52 calculates the cumulative mileage of the vehicle based on historical frame images.

[0353] It is understandable that when a vehicle is moving, there is a displacement difference between adjacent historical frame images. Based on the displacement difference between adjacent historical frame images, the cumulative mileage of the vehicle from the last calibration mileage to the current time can be obtained.

[0354] S53 sums the mileage from the last calibration with the preset first value to obtain the first summation result.

[0355] The first value is the distance between adjacent mileage markers.

[0356] It's understandable that once the vehicle enters the tunnel, all captured frame images can be stored in a database, which could be an onboard database or a cloud database. The vehicle uploads the captured frame images to the cloud database and records the capture time of each frame image. When a frame image is needed, it can be retrieved from the database. Simultaneously, the time of each mileage calibration is recorded. If the mileage marker recognition fails at the current time, the mileage from the last calibration is added to the distance between the two adjacent mileage markers to determine the vehicle's mileage at the current time.

[0357] S54 sums the mileage from the last calibration with the cumulative mileage to obtain a second summation result.

[0358] It's understandable that summing the mileage from the last calibration with the cumulative mileage yields the estimated mileage of the vehicle based on machine vision and cumulative image displacement. The estimated mileage then needs to be verified.

[0359] S55 compares the first summation result with the second summation result to calibrate the mileage of the vehicle at the current time.

[0360] Example 17

[0361] As an optional embodiment of the present invention, combined with Figure 6 as well as Figure 8 The above S44 is implemented through the following steps:

[0362] Step 1: When the difference between the first summation result and the second summation result does not exceed the first difference threshold, the difference between the first summation result and the preset second value is determined as the mileage of the vehicle at the current time.

[0363] The second value is the distance between the actual location of the mile marker in the tunnel and the camera in the target frame image, i.e., the object distance.

[0364] It is understood that the distance between the target frame images containing the same mileage marker will differ depending on the selected target frame image. In this embodiment of the invention, the distance between the target frame images selected for each mileage calibration is the same; that is, if the selected target frame image is a single image, the position of the mileage marker in the selected target frame image will be the same each time the mileage is calibrated.

[0365] For example, suppose that when a mileage marker is selected to enter the camera's shooting range, the first frame containing the mileage marker is selected as the target frame image, and the nth frame containing the mileage marker is selected as the target frame image. The distance between the mileage marker and the camera is different in these two frames, and the value of the second value is different.

[0366] Step 2: When the difference between the first summation result and the second summation result exceeds the first difference threshold, the difference between the second summation result and the preset second value is determined as the mileage of the vehicle at the current time.

[0367] The first difference threshold is a value preset based on tunnel length and practical experience, and can be set to 10 meters in actual application.

[0368] It's understandable that the camera might have already captured a frame containing the mileage marker before the vehicle even reaches it; the distance between the vehicle and the marker at this point is the second value. The estimated mileage is then verified by comparing the first summation result (the estimated mileage) with the second summation result to determine if the estimated mileage meets the accuracy standard. If so, the difference between the second summation result and the second value is taken to obtain the vehicle's mileage at the current time.

[0369] Example 18

[0370] As an optional embodiment of the present invention, the steps of S42 above include:

[0371] Step 1: Identify the first frame image containing the mileage markers in the frame images;

[0372] Step 2: When there are multiple first frame images, determine the target frame image from the multiple first frame images.

[0373] The target frame image is the first frame image whose center point is closest to the center point of the mileage marker.

[0374] This implementation method can calculate the center coordinates (x, y) of the same mile marker in multiple consecutive frames of images. i ,y i ), (i = 1…n), where n represents the number of frames captured by the camera containing the same mileage marker. Then, the distance Dist between the center coordinates of the mileage marker and the center coordinates of the frame image is calculated. i (i = 1…n), select the frame image with the smallest distance as the target frame image.

[0375] This embodiment calculates the distance between the center coordinates of the mileage marker in the frame image and the center coordinates of the frame image, and selects the frame image with the smallest distance as the target frame image, so that the mileage marker is as close to the center of the frame image as possible, thereby improving the accuracy of character information recognition and mileage calibration.

[0376] Example 19

[0377] As an optional embodiment of the present invention, prior to step S13 above, the mileage calibration method for tunnels provided in this embodiment of the present invention further includes:

[0378] When there are multiple target frame images, based on the recognition results obtained by recognizing the character information of the mileage markers in each target frame image, the same character information and the same number of the same character information are determined.

[0379] It is understandable that when a mileage marker enters the shooting range of the image capturing device, if the vehicle is moving, the image capturing device may capture multiple target frame images containing the mileage marker.

[0380] Furthermore, the steps in S44 above include:

[0381] When the number of identical character messages exceeds a preset threshold, the mileage of the vehicle at the current time is calibrated based on the number of identical character messages exceeding the preset threshold.

[0382] As an optional embodiment of the present invention, the distance of the mileage marker from the starting point is summed with the initial mileage of the vehicle when it enters the tunnel, and the summation result is determined as the mileage of the vehicle at the current time.

[0383] Example 20

[0384] As an optional embodiment of the present invention, step S44 above can be implemented as follows:

[0385] Step 1: When the recognition result is that character information has been recognized, determine whether the difference between the distance in the historical character information and the distance in the current time character information exceeds the second difference threshold.

[0386] The second difference threshold is the product of the difference between the preset standard value and the distance in the historical character information, and the distance in the character information at the current time. The historical character information is the mileage information of the last time the driving tool was calibrated.

[0387] For example, if the distance represented by the mileage marker characters is 300 meters, the distance of the character information of the last mileage calibration is 100 meters, and the preset standard value is 10%, then the value of the second difference threshold is 20 meters. If the distance of the character information of the last mileage calibration is 200 meters, then the value of the second difference threshold is 10 meters.

[0388] Step 2: When the difference between the distance in the current time character information and the distance in the historical character information does not exceed the second difference threshold, the sum of the distance in the current time character information and the initial mileage when the vehicle enters the tunnel is determined as the mileage of the vehicle at the current time.

[0389] It's understandable that if the difference between the distance in the current time's character information and the distance in the historical character information is small, it means the distance in the current time's character information is reasonable and can be used to calibrate the mileage. If the difference between the distance in the current time's character information and the distance in the historical character information is large, it means the distance in the current time's character information is unreasonable, and the current time's character information should be discarded.

[0390] Example 21

[0391] As an optional embodiment of the present invention, after step S44 above, the mileage calibration method provided by the embodiment of the present invention further includes:

[0392] Step 1: When the difference between the distance in the current time character information and the distance in the historical character information exceeds the second difference threshold, acquire historical frame images taken from the last calibration of the driving vehicle mileage to the current time;

[0393] Step 2: Calculate the cumulative mileage based on historical frame images;

[0394] Step 3: The sum of the accumulated mileage and the distance in the historical character information is determined as the mileage of the vehicle at the current time.

[0395] Example 22

[0396] As an optional embodiment of the present invention, such as Figure 8 As shown, the accumulated mileage can be obtained through the following steps:

[0397] S61, calculate the actual size of the object space corresponding to each pixel in the historical frame image.

[0398] S62, using a preset matching algorithm, calculates the homography matrix of adjacent historical frame images.

[0399] The homography matrix represents the magnitude of change between adjacent historical frames.

[0400] S63, based on the homography matrix, calculates the image displacement of the next historical frame relative to the previous historical frame in adjacent historical frame images.

[0401] S64, the product of the actual size of the object space corresponding to each pixel and the image displacement is determined as the actual displacement of the next historical frame image relative to the previous historical frame image.

[0402] It can be understood that if the image displacement of the next frame relative to the previous frame is x, and the actual size of the object space corresponding to each pixel is d, then the actual displacement of the next historical frame relative to the previous historical frame is s = d * x.

[0403] S65 accumulates the actual displacement to determine the cumulative mileage of the vehicle.

[0404] Example 23

[0405] As an optional embodiment of the present invention, after step S44 above, the mileage calibration method for tunnels provided by the present invention further includes: determining the position of the vehicle in the tunnel based on the calibrated mileage of the vehicle and tunnel information.

[0406] The tunnel information includes: the tunnel length, the tunnel entrance location, the tunnel exit location, and the tunnel curvature angle at different locations.

[0407] It's understandable that when a construction vehicle is in a tunnel, the distance traveled within the tunnel can be determined based on the mileage. Knowing this distance, the vehicle's exact location within the tunnel can be determined using tunnel information. For tunnels with significant curvature, the direction of the construction vehicle needs to be determined based on the tunnel's curvature angle, and then the distance traveled can be used to pinpoint the vehicle's exact location within the tunnel.

[0408] Example 24

[0409] like Figure 9 As shown in the figure, an embodiment of the present invention provides a mileage calibration device suitable for tunnels, the device comprising:

[0410] The acquisition module 41 is used to acquire frame images of the tunnel captured by the image capturing device at the current time. The image capturing device moves synchronously with the vehicle. The tunnel is equipped with mileage markers, and the mileage markers are marked with character information.

[0411] The search module 44 is used to search for a target frame image containing mileage markers in the frame image;

[0412] The recognition module 44 is used to recognize the character information of mileage markers in the target frame image and obtain the recognition result, which is whether the character information is recognized.

[0413] The calibration module 46 is used to calibrate the mileage of the vehicle at the current time based on the character information when the recognition result is that character information has been recognized.

[0414] Optionally, the mileage calibration device provided in this embodiment of the invention further includes: a calculation module, used for:

[0415] When the recognition result is that no character information is recognized, acquire historical frame images of the tunnel taken from the last time the driving vehicle mileage was calibrated to the current time.

[0416] Calculate the cumulative mileage of the vehicle based on historical frame images;

[0417] The mileage from the last calibration is summed with the preset first value to obtain the first summation result;

[0418] Sum the mileage from the last calibration with the cumulative mileage to obtain the second summation result.

[0419] Optional, calibration module, specifically used for:

[0420] When the difference between the first summation result and the second summation result does not exceed the first difference threshold, the difference between the first summation result and the preset second value is determined as the mileage of the vehicle at the current time.

[0421] When the difference between the first summation result and the second summation result exceeds the first difference threshold, the difference between the second summation result and the preset second value is determined as the mileage of the vehicle at the current time.

[0422] Optional, the search module, specifically used for:

[0423] Identify the first frame image containing the mileage markers within the frame images;

[0424] When there are multiple first frame images, the target frame image is determined from the multiple first frame images. The target frame image is the first frame image whose center point is closest to the center point of the mileage marker.

[0425] Optionally, the mileage calibration device for tunnels provided in this embodiment of the invention further includes: a first determining module, used for:

[0426] When there are multiple target frame images, based on the recognition results obtained by recognizing the character information of the mileage markers in each target frame image, the same character information and the same number of the same character information are determined.

[0427] The calibration module is specifically used for:

[0428] When the number of identical character messages exceeds a preset threshold, the mileage of the vehicle at the current time is calibrated based on the number of identical character messages exceeding the preset threshold.

[0429] Optional, the character information includes: the distance between the mileage marker and the starting point;

[0430] The calibration module is specifically used for:

[0431] The distance of the mileage marker from the starting point is summed with the initial mileage of the vehicle when it enters the tunnel, and the sum is determined as the mileage of the vehicle at the current time.

[0432] Optional, calibration module, specifically used for:

[0433] When the recognition result is that character information is recognized, the character information includes: the distance of the mileage marker from the starting point, and whether the difference between the distance in the historical character information and the distance in the current time character information exceeds the second difference threshold. The historical character information is the character information of the last mileage calibration.

[0434] When the difference between the distance in the current time character information and the distance in the historical character information does not exceed the second difference threshold, the sum of the distance in the current time character information and the initial mileage of the vehicle is determined as the mileage of the vehicle at the current time.

[0435] Optional, calibration module, specifically used for:

[0436] When the difference between the distance in the current time character information and the distance in the historical character information exceeds the second difference threshold, acquire historical frame images taken from the last calibration of the driving vehicle mileage to the current time.

[0437] Based on historical frame images, the cumulative mileage is calculated; the sum of the cumulative mileage and the distance in the historical character information is determined as the mileage of the vehicle at the current time.

[0438] Optional, calibration module, specifically used for:

[0439] Calculate the actual size of the object space corresponding to each pixel in the historical frame image;

[0440] Using a preset matching algorithm, the homography matrix of adjacent historical frame images is calculated. The homography matrix represents the magnitude of the change between adjacent historical frame images.

[0441] Based on the homography matrix, the image displacement of the next historical frame relative to the previous historical frame is calculated in adjacent historical frame images.

[0442] Based on the actual size of the object space corresponding to each pixel, determine the actual displacement of the next historical frame image relative to the previous historical frame image.

[0443] The actual displacement is accumulated to determine the cumulative mileage of the vehicle.

[0444] Optionally, the mileage calibration device for tunnels provided in this embodiment of the invention further includes: a positioning module, used for:

[0445] Based on the calibrated mileage of the vehicle and tunnel information, the position of the vehicle in the tunnel is determined. The tunnel information includes: the length of the tunnel, the location of the tunnel entrance, the location of the tunnel exit, and the curvature angle of the tunnel at different locations.

[0446] In this embodiment of the invention, when a vehicle enters a tunnel, a frame image of the tunnel taken at the current time is acquired. A target frame image containing mileage markers is located within this frame image. Character information of the mileage markers in the target frame image is identified, and a recognition result is obtained. When the recognition result indicates that character information has been identified, the mileage of the vehicle at the current time is calibrated based on this character information. Compared to existing technologies, this embodiment of the invention improves the accuracy of mileage tracking in tunnels by identifying character information in target frame images containing mileage markers with high positioning accuracy and prominent image features, and then calibrating the mileage of the vehicle based on this character information. This enhances the accuracy of the vehicle's positioning within the tunnel. Furthermore, this embodiment of the invention can complete mileage calibration without adding additional equipment, resulting in low construction costs.

[0447] Example 25

[0448] This invention also provides a device for calibrating mileage, the device comprising: an image acquisition device and a decision-making device.

[0449] The decision-making device is connected to the image acquisition device and is used for:

[0450] The image acquisition device captures a frame image of the tunnel at the current time. The tunnel is equipped with mileage markers, which are marked with character information.

[0451] Locate the target frame image containing mileage markers within the frame image;

[0452] Identify the character information of mileage markers in the target frame image and obtain the identification result, which is: whether the character information was identified.

[0453] When the recognition result is that character information has been recognized, the mileage of the vehicle at the current time is calibrated based on the character information.

[0454] In this embodiment of the invention, when a vehicle enters a tunnel, a frame image of the tunnel taken at the current time is acquired. A target frame image containing mileage markers is located within this frame image. Character information of the mileage markers in the target frame image is identified, and a recognition result is obtained. When the recognition result indicates that character information has been identified, the mileage of the vehicle at the current time is calibrated based on this character information. Compared to existing technologies, this embodiment of the invention improves the accuracy of mileage tracking in tunnels by identifying character information in target frame images containing mileage markers with high positioning accuracy and prominent image features, and then calibrating the mileage of the vehicle based on this character information. This enhances the accuracy of the vehicle's positioning within the tunnel. Furthermore, this embodiment of the invention can complete mileage calibration without adding additional equipment, resulting in low construction costs.

[0455] II. Application Examples. To demonstrate the inventiveness and technical value of the technical solution of this invention, this section provides application examples of the technical solution of the claims on specific products or related technologies.

[0456] The communication bus mentioned in the above-described electronic device of this invention can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the figure, but this does not indicate that there is only one bus or one type of bus.

[0457] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0458] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0459] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0460] The present invention can be applied to a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of any of the above-described methods for locating fault points in tunnels.

[0461] The present invention can also be applied to a computer program product containing instructions that, when run on a computer, cause the computer to execute any of the methods described above for locating fault points in a tunnel.

[0462] III. Evidence of the Relevant Effects of the Embodiments. The embodiments of the present invention have achieved some positive effects during research and development or use, and indeed possess significant advantages compared to existing technologies. The following description, in conjunction with data, charts, and other materials from the experimental process, illustrates these advantages.

[0463] In practical applications, when a vehicle enters a tunnel, an image capturing device is mounted on the vehicle to capture frame images of the tunnel. The capturing frequency of the image capturing device is set to 25 frames per second, meaning that the image capturing device can acquire 25 images of the tunnel per second. The current tunnel length is 100 meters, and a mileage marker is placed every 5 meters inside the tunnel. The vehicle's speed is 10 meters per second, so the vehicle takes 10 seconds to pass through the tunnel. Therefore, the image capturing device will acquire 250 frames of images inside the tunnel. If the fault point in the 132nd frame of the image capturing device is located at the exact center of the image, then by calculation, the mileage corresponding to the fault point at the current time calibration is 50 meters. The positional relationship between the frame image to be located and the target frame image is 132 - 50 / 10 * 25 = 7 frames. Furthermore, since the vehicle travels 10 / 25 meters in one frame, the vehicle travels 2.8 meters in 7 frames. Therefore, the actual location corresponding to the fault point is 52.8 meters.

[0464] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0465] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0466] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments for the device / server / tunnel inspection vehicle / storage medium / computer program are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0467] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A method for locating fault points in tunnels, characterized in that, The tunnel is equipped with mileage markers to indicate the tunnel's mileage. The mileage markers are marked with character information. The method for locating fault points within the tunnel includes: Frame images of the tunnel are acquired using an image capturing device mounted on the vehicle. The frame image is used to determine the frame image containing the fault point to be located and the target frame image containing mileage markers; the character information of the mileage markers in the target frame image is then identified. Based on the character information in the target frame image, the positional relationship between the frame image to be located and the target frame image, and the mileage calibrated by the vehicle at the current time, the actual location of the fault point in the frame image to be located is determined. The steps for locating the target frame image containing mileage markers in the frame images include: A first frame image containing mileage markers is determined from the frame images; When there are multiple first frame images, the target frame image is determined from the multiple first frame images. In the frame images, identify the target frame image that was captured most recently and whose center point is closest to the center point of the mileage marker. Before the step of locating the actual location of the fault point in the tunnel of the current time frame image based on the character information in the target frame image and the positional relationship between the frame image to be located and the target frame image, the method further includes: Identify the character information of mileage markers in the target frame image to obtain the identification result; The recognition result includes whether the character information of the mileage marker in the target frame image is recognized, and the character information corresponds to the numerical value of its location in the tunnel; The step of locating the actual location of the fault point in the frame image to be located based on the character information in the target frame image and the positional relationship between the frame image to be located and the target frame image includes: When the recognition result is that the character information in the target frame image is not recognized, a second frame image is determined according to the time sequence of the frame images. The second frame image is a frame image that is different from the mileage markers contained in the target frame image, is the closest to the time of the target frame image, and can recognize the character information. When the target frame image contains a fault point in the frame image to be located, the cumulative mileage is calculated based on the frame images captured between the capture time of the second frame image and the capture time of the target frame image. Summing the first value with the distance between the two mile markers yields the first summation result; Wherein, the first value is the value of the position within the tunnel corresponding to the character information of the second frame image; Summing the first value with the cumulative mileage yields a second summation result; Based on the capture time of the frame image to be located, the actual location value of the fault point in the frame image to be located is determined from the first summation result and the second summation result. The step of determining the actual location of the fault point in the frame image to be located based on the shooting time of the frame image to be located, from the first summation result and the second summation result, includes: When the difference between the first summation result and the second summation result does not exceed the first difference threshold, the difference between the first summation result and the preset second value is determined as the actual location value of the fault point in the frame image to be located. When the difference between the first summation result and the second summation result exceeds the first difference threshold, the difference between the second summation result and the preset second value is determined as the actual location value of the fault point in the frame image to be located. After the step of identifying the character information of mileage markers in the target frame image and obtaining the identification result, the method further includes: When there are multiple target frame images, based on the recognition results obtained by recognizing the character information of the mileage markers in each target frame image, the same character information of the same mileage marker and the same number of the same character information are determined. When the number of identical character information exceeds a preset threshold, the actual location of the fault point in the frame image to be located is determined based on the number of identical character information exceeding the preset threshold. The step of locating the actual location of the fault point in the frame image to be located based on the character information in the target frame image and the positional relationship between the frame image to be located and the target image includes: When the recognition result is that the character information is recognized, it is determined whether the difference between the first value and the second value exceeds the second difference threshold. Wherein, the first value is the value of the position in the tunnel corresponding to the character information of the second frame image, the second frame image is a frame image that is different from the mileage markers contained in the target frame image, is the most recently captured frame image and can identify character information, and the second value is the value of the position in the tunnel corresponding to the character information in the target frame image. When the difference between the first value and the second value does not exceed the second difference threshold, and the target frame image contains the fault point of the frame image to be located, the second value is determined as the actual location value of the fault point in the frame image to be located. When the difference between the first value and the second value exceeds the second difference threshold, a second frame image is determined according to the time sequence of the frame images. The second frame image is a frame image that is different from the mileage markers contained in the target frame image, is the closest to the time of the target frame image, and can identify character information. When the target frame image contains a fault point in the frame image to be located, the cumulative mileage is calculated based on the frame images between the shooting time of the second frame image and the shooting time of the target frame image; The sum of the first value and the cumulative mileage is determined as the actual location value of the fault point in the frame image to be located. The step of locating the actual location of the fault point in the frame image to be located based on the character information in the target frame image and the positional relationship between the frame image to be located and the target image includes: When the target frame image does not contain the fault point of the frame image to be located, the difference mileage is calculated based on the frame images between the frame image to be located and the target frame image. The difference between the distance between the two mileage markers and the difference mileage is summed with the first value, and the summation result is determined as the actual location value of the fault point in the frame image to be located. The step of calculating the cumulative mileage based on the frame images between the shooting time of the second frame image and the shooting time of the target frame image includes: Calculate the actual size of the object space corresponding to each pixel in the historical frame image, wherein the historical frame image is the frame image between the shooting time of the second frame image and the shooting time of the target frame image; Using a preset matching algorithm, the homography matrix of adjacent historical frame images is calculated, and the homography matrix represents the magnitude of the change in the adjacent historical frame images; Based on the homography matrix, the image displacement of the previous historical frame relative to the next historical frame in adjacent historical frame images is calculated. The product of the actual size of the object space corresponding to each pixel and the image displacement is determined as the actual displacement of the previous historical frame image relative to the next historical frame image. The actual displacements are accumulated to obtain the cumulative mileage.

2. The method for locating fault points in tunnels according to claim 1, characterized in that, The process of calibrating the mileage of the vehicle at the current time specifically includes the following steps: The image capturing device captures a frame image of the tunnel at the current time, and the image capturing device moves synchronously with the vehicle. Locate the target frame image containing mileage markers within the frame images; The character information of mileage markers in the target frame image is identified to obtain a recognition result, wherein the recognition result is: whether the character information is recognized. When the recognition result indicates that the character information has been recognized, the mileage of the vehicle at the current time is calibrated based on the character information.

3. The method for locating fault points in tunnels according to claim 2, characterized in that, After the step of identifying the character information of mileage markers in the target frame image and obtaining the identification result, the method further includes: When the recognition result is that the character information is not recognized, acquire historical frame images of the tunnel taken from the last time the mileage of the driving tool was calibrated to the current time. Based on the historical frame images, the cumulative mileage of the vehicle is calculated; The mileage from the last calibration is summed with the preset first value to obtain the first summation result; The mileage from the last calibration is summed with the cumulative mileage to obtain a second summation result; The first summation result is compared with the second summation result to calibrate the mileage of the vehicle at the current time; The step of comparing the first summation result with the second summation result to calibrate the mileage of the vehicle at the current time includes: When the difference between the first summation result and the second summation result does not exceed the first difference threshold, the difference between the first summation result and the preset second value is determined as the mileage of the vehicle at the current time. When the difference between the first summation result and the second summation result exceeds the first difference threshold, the difference between the second summation result and the preset second value is determined as the mileage of the vehicle at the current time. The step of finding the target frame image containing mileage markers in the frame image includes: A first frame image containing mileage markers is determined from the frame images; When there are multiple first frame images, a target frame image is determined from the multiple first frame images. The target frame image is the first frame image whose center point is closest to the center point of the mileage marker. After the step of recognizing the character information of mileage markers in the target frame image, the method further includes: When there are multiple target frame images, based on the recognition results obtained by recognizing the character information of the mileage markers in each target frame image, the same character information and the same number of the same character information are determined. When the recognition result is that the character information has been recognized, the step of calibrating the mileage of the vehicle at the current time based on the character information includes: When the number of identical character information exceeds a preset threshold, the mileage of the vehicle at the current time is calibrated based on the number of identical character information exceeding the preset threshold. The character information includes: the distance between the mileage marker and the starting point; When the recognition result indicates that the character information has been recognized, the step of calibrating the mileage of the vehicle at the current time based on the character information includes: When the recognition result is that the character information is recognized, the character information includes: the distance of the mileage marker from the starting point, and whether the difference between the distance in the historical character information and the distance in the character information at the current time exceeds a second difference threshold. The historical character information is the character information of the last mileage calibration. When the difference between the distance in the character information at the current time and the distance in the historical character information does not exceed the second difference threshold, the sum of the distance in the character information at the current time and the initial mileage of the vehicle is determined as the mileage of the vehicle at the current time. When the difference between the distance in the character information at the current time and the distance in the historical character information exceeds the second difference threshold, acquire historical frame images taken from the last calibration of the vehicle mileage to the current time. Calculate the cumulative mileage based on the historical frame images; The sum of the accumulated mileage and the distance in the historical character information is determined as the mileage of the vehicle at the current time; The step of calculating the cumulative mileage of the vehicle based on the historical frame images includes: Calculate the actual size of the object space corresponding to each pixel in the historical frame image; Using a preset matching algorithm, the homography matrix of adjacent historical frame images is calculated, and the homography matrix represents the magnitude of the change in the adjacent historical frame images; Based on the homography matrix, the image displacement of the next historical frame relative to the previous historical frame is calculated in adjacent historical frame images. Based on the actual size of the object space corresponding to each pixel, the actual displacement of the next historical frame image relative to the previous historical frame image is determined. The actual displacements are accumulated to obtain the cumulative mileage of the vehicle. After the step of calibrating the mileage of the vehicle at the current time based on the character information, the method further includes: Based on the calibrated mileage of the vehicle and the tunnel information, the position of the vehicle in the tunnel is determined. The tunnel information includes: the length of the tunnel, the location of the tunnel entrance, the location of the tunnel exit, and the curvature angle of the tunnel at different locations.

4. A testing vehicle that applies the method for locating fault points in tunnels as described in any one of claims 1-3, characterized in that, The inspection vehicle includes: a decision-making device connected to an image acquisition device, used to acquire frame images of the tunnel captured by the image acquisition device at the current time, wherein the image acquisition device moves synchronously with the inspection vehicle; Locate the target frame image containing mileage markers within the frame images; The character information of mileage markers in the target frame image is identified to obtain a recognition result, wherein the recognition result is: whether the character information is recognized. When the recognition result is that the character information is recognized, the mileage of the detection vehicle at the current time is calibrated based on the character information; The determination module is used to determine, from the frame images, the frame image to be located containing the fault point and the target frame image containing the mileage markers; The positioning module is used to locate the actual location of the fault point in the frame image to be positioned based on the character information in the target frame image and the positional relationship between the frame image to be positioned and the target frame image.

5. An apparatus for locating fault points in tunnels using the method for locating fault points in tunnels according to any one of claims 1-3, characterized in that, The device for locating fault points inside tunnels includes: The acquisition module is used to acquire frame images of the tunnel taken by the image acquisition device at the current time. The image acquisition device moves synchronously with the vehicle. Mileage markers are set in the tunnel, and character information is marked on the mileage markers. The determination module is used to determine, from the frame images, the frame image to be located containing the fault point and the target frame image containing the mileage markers; The positioning module is used to locate the actual location of the fault point in the frame image to be located based on the character information in the target frame image and the positional relationship between the frame image to be located and the target frame image. The search module is used to find target frame images containing mileage markers in frame images; The recognition module is used to recognize the character information of mileage markers in the target frame image and obtain the recognition result, which is whether the character information is recognized. The calibration module is used to calibrate the mileage of the vehicle at the current time based on the character information when the recognition result is that character information has been recognized.

6. An electronic device, comprising a processor, a communication interface, a memory, and a communication bus, wherein, The processor, the communication interface, and the memory communicate with each other through the communication bus; the processor executes instructions to implement the method for locating fault points in a tunnel as described in any one of claims 1 to 4.

7. A computer-readable storage medium storing a computer program, the computer program being executed by a processor according to any one of claims 1 to 4, the method for locating fault points in a tunnel.

Images