Railway tunnel image processing apparatus, method, device, and storage medium

Abstract

This application relates to a railway tunnel image processing device, method, equipment, and storage medium, belonging to the field of railway tunnel visual inspection technology. The device includes: multiple area array cameras, arranged sequentially on the side surface of the inspection trolley perpendicular to its travel direction, with the total shooting angle of all cameras greater than or equal to 150 degrees. Each area array camera is used to acquire image data of the railway tunnel as the inspection trolley travels back and forth within the tunnel; multiple light sources, spaced apart between any two adjacent area array cameras; and a processor connected to each area array camera, used to acquire image data and perform distortion correction and stitching processing on the image data to obtain a panoramic image of the railway tunnel. This device can improve the accuracy of railway tunnel image processing.

Description

Technical Field

[0001] This application relates to the field of visual inspection technology for railway tunnels, and in particular to a railway tunnel image processing device, method, equipment, and storage medium. Background Technology

[0002] To conduct routine monitoring of railway tunnels, a tunnel moving visual scanning technology has been adopted. This technology uses a tunnel inspection area array camera mounted on a self-moving platform to acquire panoramic images of the tunnel's inner walls. This improves the efficiency of routine monitoring of railway tunnels, and the panoramic images can also be used as high-resolution texture maps for the tunnel's 3D model. However, due to the curvature of the tunnel and the camera's shooting angle, the original images are distorted. Therefore, this tunnel moving visual scanning technology suffers from relatively low processing accuracy. Summary of the Invention

[0003] Therefore, it is necessary to provide a railway tunnel image processing device, method, equipment, and storage medium that can improve processing accuracy in response to the above-mentioned technical problems.

[0004] In a first aspect, this application provides a railway tunnel image processing apparatus, comprising:

[0005] Multiple area array cameras are arranged sequentially on the side surface of the inspection vehicle in a direction perpendicular to the travel direction of the inspection vehicle, and the total shooting angle of all the area array cameras is greater than or equal to 150 degrees. Each area array camera is used to collect image data of the railway tunnel when the inspection vehicle is working back and forth in the railway tunnel.

[0006] Multiple light sources are provided, with each light source spaced apart between any two adjacent area array cameras;

[0007] The processor is connected to each of the area array cameras to acquire the image data and perform distortion correction and stitching processing on the image data to obtain a panoramic image of the railway tunnel.

[0008] In one embodiment, the included angle between any two adjacent area array cameras is the same.

[0009] In one embodiment, in the direction of travel of the inspection vehicle, each of the area array cameras is arranged on the left or right surface of the inspection vehicle, and the shooting faces of each of the area array cameras are different.

[0010] In one embodiment, the processor is further configured to perform feature matching on the image data to determine initial feature points in each of the image data; calculate feature point offsets between adjacent image data based on the initial feature points in each of the image data; and perform distortion correction processing on the image data based on the feature point offsets to obtain corrected image data.

[0011] In one embodiment, the processor is further configured to calculate candidate feature points between adjacent corrected image data; use a robust estimation algorithm to filter out the candidate feature points to obtain target feature points of the corrected image data, and target feature point offsets between adjacent corrected image data; and stitch the corrected image data according to the target feature point offsets to obtain a panoramic image of the railway tunnel.

[0012] In one embodiment, the processor is further configured to stitch the corrected image data according to the target feature point offset, following the stitching order of single camera, adjacent camera, and round-trip images, to obtain a panoramic image of the railway tunnel.

[0013] Secondly, this application provides a railway tunnel image processing method, wherein the processor of the railway tunnel image processing apparatus described in any one of the above embodiments comprises:

[0014] Acquire image data of the railway tunnel collected by each array camera;

[0015] Each of the image data is subjected to distortion correction processing to obtain the corrected image data;

[0016] Determine candidate feature points for the corrected image data;

[0017] A robust estimation algorithm is used to filter out the candidate feature points to obtain the target feature points of the corrected image data, as well as the offset of the target feature points between adjacent corrected image data.

[0018] Based on the offset of the target feature points, the corrected image data is stitched together to obtain a panoramic image of the railway tunnel.

[0019] In one embodiment, the step of performing distortion correction processing on each of the image data to obtain corrected image data includes:

[0020] Feature matching is performed on the image data to determine the initial feature points in each image data.

[0021] Based on the initial feature points in each of the image data, calculate the feature point offset between adjacent image data;

[0022] Based on the distortion correction coefficients corresponding to the offsets of each feature point, the image data is subjected to distortion correction processing to obtain the corrected image data.

[0023] Thirdly, this application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0024] Acquire image data of the railway tunnel collected by each array camera;

[0025] Each of the image data is subjected to distortion correction processing to obtain the corrected image data;

[0026] Determine candidate feature points for the corrected image data;

[0027] A robust estimation algorithm is used to filter out the candidate feature points to obtain the target feature points of the corrected image data, as well as the offset of the target feature points between adjacent corrected image data.

[0028] Based on the offset of the target feature points, the corrected image data is stitched together to obtain a panoramic image of the railway tunnel.

[0029] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0030] Acquire image data of the railway tunnel collected by each array camera;

[0031] Each of the image data is subjected to distortion correction processing to obtain the corrected image data;

[0032] Determine candidate feature points for the corrected image data;

[0033] A robust estimation algorithm is used to filter out the candidate feature points to obtain the target feature points of the corrected image data, as well as the offset of the target feature points between adjacent corrected image data.

[0034] Based on the offset of the target feature points, the corrected image data is stitched together to obtain a panoramic image of the railway tunnel.

[0035] In the aforementioned railway tunnel image processing apparatus, method, device, and storage medium, the railway tunnel image processing apparatus includes multiple area array cameras, multiple light sources, and a processor. The area array cameras are sequentially arranged on the side surface of the inspection trolley, perpendicular to its travel direction, with the total shooting angle of all cameras greater than or equal to 150 degrees. Each area array camera is used to collect image data of the railway tunnel as the inspection trolley travels back and forth within it. Light sources are spaced apart between any two adjacent area array cameras to provide supplementary lighting, improving the uniformity and integrity of the image. The processor is connected to each area array camera to acquire image data and performs distortion correction and stitching processing on the image data to obtain a panoramic image of the railway tunnel. This reduces the distortion of image data caused by the tunnel's curvature and the camera's shooting angle, thereby improving the accuracy of railway tunnel image processing. Attached Figure Description

[0036] To more clearly illustrate the technical solutions in the embodiments of this application or the conventional technology, the drawings used in the description of the embodiments or the conventional technology will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0037] Figure 1 This is a schematic diagram of the structure of a railway tunnel image processing device in one embodiment;

[0038] Figure 2 This is a flowchart illustrating a railway tunnel image processing method in one embodiment;

[0039] Figure 3 This is a schematic diagram of the nonlinear lateral distortion of a tunnel image in one embodiment;

[0040] Figure 4 This is a schematic diagram illustrating the principle of image feature matching in one embodiment;

[0041] Figure 5 A schematic diagram illustrating the principle of calculating the scaling factor for the difference in the lateral offset of pixels in each row in one embodiment;

[0042] Figure 6 This is a schematic diagram comparing image data distortion correction before and after in one embodiment;

[0043] Figure 7 This is a schematic diagram illustrating the effect of a single camera on the stitching result in one embodiment;

[0044] Figure 8 This is a schematic diagram of the uplink and downlink panoramic image stitching results in one embodiment;

[0045] Figure 9 This is a schematic diagram of the panoramic image stitching result of a railway tunnel in one embodiment;

[0046] Figure 10 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0047] To facilitate understanding of this application, a more complete description will be provided below with reference to the accompanying drawings, which illustrate embodiments of the present application. However, the present application can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided so that the disclosure of this application will be thorough and complete.

[0048] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application.

[0049] It is understood that the terms “first,” “second,” etc., used in this application may be used herein to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish one element from another.

[0050] Spatial relation terms such as “below,” “under,” “below,” “under,” “above,” “above,” etc., are used herein to describe the relationship between one element or feature shown in the figure and other elements or features. It should be understood that, in addition to the orientation shown in the figure, spatial relation terms also include different orientations of the device in use and operation. For example, if the device in the figure is flipped, the element or feature described as “below,” “under,” or “below” will be oriented “above” the other element or feature. Therefore, the exemplary terms “below” and “under” can include both above and below orientations. Furthermore, the device may also include other orientations (e.g., rotated 90 degrees or other orientations), and the spatial descriptive terms used herein will be interpreted accordingly.

[0051] It should be noted that when one element is considered to be "connected" to another element, it can be directly connected to the other element or connected to the other element through an intermediary element. Furthermore, in the following embodiments, "connection" should be understood as "electrical connection," "communication connection," etc., if there is transmission of electrical signals or data between the connected objects.

[0052] When used herein, the singular forms of “a,” “an,” and “the” may also include the plural forms unless the context clearly indicates otherwise. It should also be understood that the terms “comprising / including” or “having,” etc., specify the presence of the stated features, wholes, steps, operations, components, parts, or combinations thereof, but do not preclude the possibility of the presence or addition of one or more other features, wholes, steps, operations, components, parts, or combinations thereof. Meanwhile, the term “and / or” as used in this specification includes any and all combinations of the associated listed items.

[0053] As described in the background section, existing tunnel mobile visual scanning technology suffers from low processing accuracy. The inventors have discovered that this problem arises because tunnel mobile visual scanning acquires panoramic images of the tunnel's inner walls using a tunnel inspection area scan camera mounted on a self-moving platform. Area scan cameras have seen rapid development in recent years due to their low cost and high image resolution. The raw image data from the area scan camera can be stitched together to obtain panoramic images of railway tunnels, improving the efficiency of daily railway tunnel monitoring. Furthermore, the panoramic images can be used as high-definition texture maps for the tunnel's 3D model. However, due to the tunnel's curvature and the camera's shooting angle, the acquired raw images are distorted, affecting the accuracy of subsequent image stitching.

[0054] For the reasons mentioned above, the present invention provides a railway tunnel image processing device, which aims to improve processing accuracy.

[0055] In one embodiment, such as Figure 1 As shown, a railway tunnel image processing device is provided, including multiple area array cameras 101, multiple light sources 102 and a processor 103;

[0056] In this system, area array cameras 101 are sequentially arranged on the side surface of the inspection vehicle perpendicular to its travel direction, and the total shooting angle of all area array cameras 101 is greater than or equal to 150 degrees. Each area array camera 101 is used to collect image data of the railway tunnel when the inspection vehicle is working back and forth in the railway tunnel. Each light source 102 is spaced between any two adjacent area array cameras 101. The processor 103 is connected to each area array camera 101 to acquire image data and perform distortion correction and stitching processing on the image data to obtain a panoramic image of the railway tunnel.

[0057] It is understood that the bottom of the inspection trolley is equipped with a traveling mechanism, which is driven by a drive mechanism. The inspection trolley moves along the tunnel track based on the traveling mechanism and stops at the set measuring point. Each area array camera 101 and each light source 102 can be fixed to the bracket first, and then the bracket can be fixed to the inspection trolley. The shape of the bracket is the same as the cross-sectional shape of the railway tunnel, such as an arc, thereby reducing the influence of the tunnel's curvature and the camera's shooting angle, thus reducing the distortion of image data. In addition, if the measuring surface of the inspection trolley itself is arc-shaped, each area array camera 101 and each light source 102 can be directly fixed to the side surface of the inspection trolley. The railway tunnel image processing device also includes a power supply module (not shown in the figure), which is electrically connected to each area array camera 101 and each light source 102.

[0058] It should be noted that each area array camera 101 is positioned on one side of the inspection trolley's operating direction, such as... Figure 1 On the right side surface of the inspection trolley, when the inspection trolley is operating in one direction, the image data collected by each area array camera 101 cannot cover the entire railway tunnel. However, when the inspection trolley is operating in a round trip, it can cover the entire railway tunnel without the need to add more area array cameras 101, thereby reducing image data redundancy, device complexity and cost.

[0059] Preferably, the processor 103 is connected to each area array camera 101 via wireless communication. After the image data is acquired, each area array camera 101 sends the image data to the processor 103. The processor 103 first performs distortion correction processing on the image sequence of a single camera to achieve efficient and accurate nonlinear distortion correction. Then, it stitches the distortion-corrected image data to obtain a panoramic image of the railway tunnel.

[0060] The aforementioned railway tunnel image processing device includes multiple area array cameras 101, multiple light sources 102, and a processor 103. Each area array camera 101 is sequentially arranged on the side surface of the inspection trolley, perpendicular to its travel direction. The total shooting angle of all area array cameras 101 is greater than or equal to 150 degrees. Each area array camera 101 is used to collect image data of the railway tunnel as the inspection trolley travels back and forth within the tunnel. Each light source 102 is spaced between any two adjacent area array cameras 101 to provide supplementary lighting, improving the uniformity and integrity of the image. The processor 103 is connected to each area array camera 101 to acquire image data and perform distortion correction and stitching processing on the image data to obtain a panoramic image of the railway tunnel. This reduces the distortion of image data caused by the tunnel's curvature and the camera's shooting angle, thereby improving the accuracy of railway tunnel image processing.

[0061] In one embodiment, see below. Figure 1Along the travel direction of the inspection vehicle, each area array camera 101 is deployed on the left or right surface of the inspection vehicle, and the shooting faces of each area array camera 101 are different. The included angle between any two adjacent area array cameras 101 is the same.

[0062] It is understandable that, such as Figure 1 Multiple area array cameras 101 are uniformly installed on the right side surface of the inspection vehicle in the direction of travel, arranged laterally perpendicular to the direction of travel. All area array cameras 101 are fixed on the same support structure and are firmly connected to the inspection vehicle by bottom bolts to ensure stability and shock resistance during operation.

[0063] For example, five high-resolution industrial-grade area array cameras 101, numbered CAM1 to CAM5, are arranged sequentially from left to right along the right side surface of the inspection vehicle. The optical axis of each area array camera 101 forms the same horizontal angle with its adjacent camera, such as 30 degrees. That is, the angle between adjacent cameras is constant. For example, CAM2 is deflected by 30 degrees relative to CAM1, CAM3 is deflected by another 30 degrees relative to CAM2, and so on. The overall coverage angle of the shooting angle of each area array camera 101 needs to reach at least 150 degrees, which can completely cover the arc-shaped inner wall of one side of the tunnel, including the arch, side walls and part of the track bed area, effectively avoiding imaging blind spots.

[0064] In addition, the shooting faces of each area array camera 101 are different, that is, each area array camera 101 has an independent viewing direction and is arranged in a fan shape to adapt to the curvature changes of the railway tunnel cross section. This allows the area array cameras 101 at different positions to focus on different areas of the railway tunnel. For example, CAM1 mainly captures the near-rail side wall, CAM3 points to the center of the arch, and CAM5 covers the far side wall and arch shoulder, thereby improving the image resolution and detail reproduction capability of key parts.

[0065] Furthermore, a light source 102 is set between any two adjacent area array cameras 101. The light source 102 shares a power supply system with the cameras, and its switching timing and brightness output are uniformly controlled by the internal industrial control computer of the inspection equipment. This achieves lighting adjustment synchronized with image acquisition, ensuring that high-contrast, low-noise raw image data can still be acquired in low-light environments. During field operations, the railway tunnel image processing device is mounted on a track inspection trolley and travels back and forth inside the railway tunnel. When the inspection trolley moves forward, the area array cameras 101 deployed on the right side complete a continuous scan of the inner wall of the right half of the tunnel; on the return trip, the reverse view information is supplemented, and finally, a complete circular panoramic image is obtained through uplink and downlink data fusion.

[0066] This embodiment achieves large-angle, blind-spot-free imaging coverage of the tunnel interior wall by centrally deploying multiple area array cameras 101 on one side of the inspection vehicle and maintaining an equal included angle and different shooting directions for adjacent area array cameras 101 in a fan-shaped layout. Combined with a dedicated light source 102 and efficient distortion correction and stitching algorithms, it not only ensures image quality and stitching accuracy but also simplifies the hardware structure and reduces system costs. It is suitable for single-sided walking inspection scenarios in double-track railway tunnels and can acquire high-quality panoramic images in a single round trip, meeting the needs of intelligent operation and maintenance of railway infrastructure.

[0067] In one embodiment, the processor 103 is further configured to perform feature matching on the image data to determine initial feature points in each image data; calculate the feature point offset between adjacent image data based on the initial feature points in each image data; and perform distortion correction processing on the image data based on the feature point offsets to obtain corrected image data.

[0068] Feature matching refers to the process of identifying and establishing the correlation between corresponding visual features in two or more images with spatially overlapping areas using algorithms. Initial feature points refer to salient pixels initially extracted from the uncorrected raw image data by feature detection algorithms, which are the first input data in the distortion correction stage. Feature point offset refers to the positional difference between a pair of matching feature points in two images in the pixel coordinate system, usually expressed as a horizontal or vertical displacement value.

[0069] Among them, distortion correction processing refers to the process of performing mathematical modeling and inverse transformation operations on geometric distortions in an image, mainly barrel or pincushion distortion and perspective distortion, to restore the true spatial proportions of the image.

[0070] Understandably, processor 103 is primarily used to find corresponding key structural points, such as crack edges, seam lines, and lamp outlines, between adjacent images, including consecutive frames from the same camera, horizontal images from adjacent cameras, and up / down round-trip images. These serve as the basis for subsequent offset calculations and stitching. Based on the initial feature points in each image data set, it calculates the feature point offset between adjacent image data sets. Specifically, the feature point offset refers to the pixel displacement of the same physical target captured at different times within the same image sequence from the area array camera 101. Then, based on these feature point offsets, it performs distortion correction processing on the image data, such as mathematical modeling and inverse transformation operations, to restore the true spatial proportions of the image data, resulting in corrected image data. This reduces the influence of tunnel curvature and camera shooting angles, thereby reducing image data distortion and further improving the accuracy of railway tunnel image processing.

[0071] In one embodiment, the processor 103 is further configured to calculate candidate feature points between adjacent corrected image data; use a robust estimation algorithm to filter out the candidate feature points to obtain the target feature points of the corrected image data and the target feature point offsets between adjacent corrected image data; and stitch the corrected image data according to the target feature point offsets to obtain a panoramic image of the railway tunnel.

[0072] Candidate feature points can refer to all potential matching points extracted from the corrected image by a feature detection algorithm. These may include mismatches or noise interference points before further screening.

[0073] Robust estimation algorithms can refer to statistical methods that can estimate reliable model parameters from datasets containing a large amount of noise or outliers, with RANSAC (Random Sample Consensus) being a typical example.

[0074] The target feature points refer to the valid feature points that satisfy geometric consistency constraints and are retained after being filtered by a robust estimation algorithm, used for the final image stitching alignment. The target feature point offset refers to the precise displacement vector between adjacent images calculated based on the target feature points, typically expressed as pixel offset values ​​in the horizontal and vertical directions, or as an overall transformation matrix.

[0075] Among them, stitching processing can refer to the process of registering and fusing multiple corrected images with spatial overlapping areas according to their relative positional relationship to generate a continuous and seamless wide panoramic image.

[0076] For example, the processor 103 is also used to calculate candidate feature points between adjacent corrected image data. After completing the distortion correction process, the processor uses the algorithm to extract all feature point sets from adjacent images. Although these candidate feature points are already in a geometrically consistent spatial framework, there may still be erroneous matches caused by texture repetition, weak feature regions, or environmental interference. These erroneous matches need to be further filtered out using a robust estimation algorithm. Then, the robust estimation algorithm is used to filter out the candidate feature points, thereby removing erroneous matches from the candidate feature points, obtaining the target feature points of the corrected image data, and calculating the target feature point offset between adjacent corrected image data. This offset is obtained by calculating the average displacement or fitting the global transformation relationship. Based on the target feature point offset, the corrected image data is stitched together to obtain a panoramic image of the railway tunnel.

[0077] In this embodiment, after distortion correction, candidate feature points between adjacent corrected images are extracted, and a robust estimation algorithm is introduced to effectively filter out mismatched points, significantly improving the accuracy and reliability of feature matching. Due to issues such as texture repetition, uneven illumination, or weak feature regions within tunnels, the original matching results are prone to interference. This embodiment utilizes a robust estimation mechanism to eliminate abnormal matches, retaining target feature points that conform to geometric consistency, thereby obtaining high-precision target feature point offsets. This provides a guarantee for accurate calculation of the relative pose between images. Based on this offset, stitching processing not only achieves seamless alignment between corrected images, reducing ghosting and misalignment during stitching, but also significantly improves the spatial continuity and visual consistency of panoramic images, thereby further improving the accuracy of railway tunnel image processing.

[0078] In one embodiment, the processor 103 is further configured to stitch the corrected image data according to the target feature point offset, in the order of stitching single camera, adjacent camera, and round-trip images, to obtain a panoramic image of the railway tunnel.

[0079] The stitching order refers to the logical sequence followed during the image stitching process, which directly affects stitching efficiency and result consistency.

[0080] For example, the processor 103 is further configured to stitch the corrected image data according to the target feature point offset, following the stitching order of single camera, adjacent camera, and round-trip images, to obtain a panoramic image of the railway tunnel. Understandably, first, time-series stitching within a single camera is performed to ensure image continuity at each viewpoint; then, spatial stitching between adjacent cameras is performed to integrate multiple viewpoints into a cross-sectional panorama; finally, global alignment and fusion of the up-and-down round-trip images are performed to eliminate the influence of minor deviations in the round-trip path. This layered stitching strategy reduces overall computational complexity and improves stitching stability and accuracy, making it particularly suitable for large-scale image processing tasks involving long-distance tunnels.

[0081] In one embodiment, such as Figure 2 As shown, a railway tunnel image processing method is provided, applied to the processor 103 described in any of the above embodiments, the method comprising:

[0082] Step S202: Acquire image data of the railway tunnel collected by each area array camera 101.

[0083] Optionally, the processor 103 acquires image data collected by each area array camera 101 for the railway tunnel, as the data basis for constructing a panoramic image of the railway tunnel.

[0084] Step S204: Perform distortion correction processing on each image data to obtain the corrected image data.

[0085] Optionally, the processor 103 performs distortion correction processing on each image data separately, such as mathematical modeling and inverse transformation operations, to restore the true spatial proportions of the image data and obtain corrected image data. This reduces the influence of the tunnel's curvature and the camera's shooting angle, thereby reducing image data distortion and further improving the accuracy of railway tunnel image processing.

[0086] Step S206: Determine candidate feature points for the corrected image data.

[0087] Optionally, the processor 103 calculates candidate feature points between adjacent corrected image data, and after completing the distortion correction process, extracts the complete set of feature points from adjacent images using the SURF algorithm (Speeded-Up Robust Features).

[0088] Step S208: A robust estimation algorithm is used to filter out candidate feature points to obtain the target feature points of the corrected image data and the offset of the target feature points between adjacent corrected image data.

[0089] Optionally, the processor 103 employs a robust estimation algorithm to screen out candidate feature points, thereby eliminating mismatched points among the candidate feature points, obtaining the target feature points of the corrected image data, and calculating the offset of target feature points between adjacent corrected image data, which is the result of calculating their average displacement or fitting the global transformation relationship.

[0090] Step S210: Based on the offset of the target feature points, the corrected image data is stitched together to obtain a panoramic image of the railway tunnel.

[0091] Optionally, the processor 103 performs stitching processing on the corrected image data according to the target feature point offset, following the stitching order of single camera, adjacent camera, and round-trip images, to obtain a panoramic image of the railway tunnel. It can be understood that the time-series stitching within a single camera is completed first to ensure the continuity of the image under each viewpoint, then the spatial stitching between adjacent cameras is performed to integrate multiple viewpoints to form a cross-sectional panorama, and finally the global alignment and fusion of the up and down round-trip images are performed to eliminate the influence of small deviations in the round-trip path.

[0092] In this embodiment, distortion correction processing is performed on the image data acquired by each area array camera 101, effectively eliminating geometric distortion caused by the curved inner wall of the railway tunnel and the tilt of the camera's viewing angle, restoring the true spatial proportions of the image, and providing a reliable geometric basis for subsequent processing. Subsequently, candidate feature points are extracted from the corrected image, and robust estimation algorithms such as RANSAC are used to eliminate mismatched points caused by texture duplication, illumination changes, or noise interference, significantly improving the accuracy and stability of feature matching. Furthermore, image stitching is completed based on the selected target feature points and their offsets, ensuring seamless alignment between adjacent images and generating a continuous, clear, and misaligned panoramic image of the tunnel, thereby improving the accuracy of image processing for railway tunnels.

[0093] In one embodiment, step S204 performs distortion correction processing on each image data to obtain corrected image data, including:

[0094] Feature matching is performed on the image data to determine the initial feature points in each image data; based on the initial feature points in each image data, the feature point offset between adjacent image data is calculated; based on the distortion correction coefficients corresponding to the feature point offsets, distortion correction processing is performed on the image data to obtain the corrected image data.

[0095] The distortion correction factor can be a proportional factor used to correct the degree of distortion of each row of pixels in an image due to optical system or viewing angle, and is usually expressed as a normalized scaling factor.

[0096] Optionally, the processor 103 performs feature matching on the image data to determine the initial feature points in each image data. Due to the existence of lateral distortion in the image, the field of view of each row of pixels is different, that is, the physical resolution corresponding to each row is different. As the car moves, the rate of change of each row of pixels is also different, that is, the offset of the feature points of each row of pixels is different. Therefore, based on the initial feature points in each image data, the feature point offset between adjacent image data is calculated. This can be done using the pixel feature point offset li, where i is the row number and li can be used to represent the physical resolution corresponding to the i-th row of pixels. After identifying the abnormal matching points, the corresponding feature point offset li of each row of pixels is obtained by fitting a quadratic function, and normalized using li / lmax to obtain the distortion correction coefficient corresponding to each row of pixels. Then, the processor 103 performs distortion correction processing on the image data according to the distortion correction coefficient corresponding to each feature point offset to obtain the corrected image data.

[0097] In this embodiment, due to the arc-shaped structure of the railway tunnel, there is significant lateral geometric distortion when the area array camera 101 captures images. This results in different physical fields of view and resolutions for different rows of pixels, manifested as a non-linear change in the displacement rate of feature points in each row as the inspection trolley moves. Utilizing this phenomenon, with the row number as the independent variable and the feature point offset as the dependent variable, a quadratic function is used for fitting after removing abnormal matching points to accurately characterize the distribution trend of distortion along the vertical direction of the image. Normalization is then used to obtain the distortion correction coefficient for each row. This coefficient reflects the ratio between the actual sampling density and the ideal state of each row, enabling the processor 103 to perform row-by-row adaptive correction on the original image, effectively restoring the spatial consistency of the image, thereby further improving the image processing accuracy of the railway tunnel.

[0098] In one embodiment, another method for processing railway tunnel images is provided, comprising:

[0099] Step 1: Perform feature point matching on adjacent image data from each camera and calculate the feature point offset between adjacent images.

[0100] The cameras of the area array camera inspection device are distributed on the right side of the device, with an angle of 30 degrees between adjacent cameras. The total coverage angle of the device can reach 150 degrees. The light source is installed between each camera, with a unified power supply and unified control by the internal industrial control computer of the device. The device is fixed to the inspection trolley with screws at the bottom.

[0101] Step 2: Remove outlier matching points and use a quadratic polynomial to fit the distribution relationship between the feature point row number and the horizontal offset.

[0102] Step 3: Calculate the lateral offset for each row of pixels based on the function obtained by fitting the quadratic polynomial. Normalize the calculated lateral offset to obtain the distortion correction coefficient for each row of pixels.

[0103] Specifically, when performing image distortion correction, images with low overlap from the same camera are selected for feature matching, such as... Figure 3 As shown, a schematic diagram of the nonlinear lateral distortion of tunnel images is provided, such as... Figure 4 The diagram illustrates the principle of image feature matching. Due to the lateral distortion of the image, the field of view of each row of pixels is different, meaning that the physical resolution of each row is different. As the vehicle moves, the rate of change of each row of pixels is also different, meaning that the offset of the feature points of each row of pixels is different.

[0104] Furthermore, the pixel feature point offset *li*, where *i* is the row number, can be used to represent the physical resolution corresponding to the pixel in the *i*th row. After identifying the outlier matching points, a quadratic function is used to fit the feature point offset *li* corresponding to each row of pixels, and normalization is performed using *li / lmax* to obtain the scaling factor corresponding to each row of pixels, such as... Figure 5 The diagram shown illustrates the principle of calculating the scaling factor based on the difference in the horizontal offset of pixels in each row. Figure 5 (a) is a schematic diagram of the lateral offset of the feature point. Figure 5 (b) is a graph showing the lateral offset after outlier removal. Figure 5 (c) is a graph of the scaling factor corresponding to the normalized pixels.

[0105] Step 4: Based on the calculated distortion correction coefficients, complete the distortion correction for all original images.

[0106] Specifically, by applying this scaling factor, the physical resolution of each row of pixels is unified, thus completing the lateral distortion correction of the image, such as... Figure 6 The image shown is a schematic diagram comparing the image data distortion before and after correction.

[0107] Step 5: Calculate and match the feature points of adjacent images after correction based on the SURF algorithm, remove mismatched points using the RANSAC algorithm, further obtain the offset of the matching points, and obtain the image offset after averaging. Complete the image stitching of a single camera based on the image offset.

[0108] Specifically, after distortion correction, feature points of adjacent images are calculated and matched using the SURF algorithm. Mismatched points are then removed using the RANSAC algorithm. The offsets of these matched points are further calculated, averaged, and then the image offset is obtained. Image stitching is then performed based on these offsets. The stitching process follows the sequence of single-camera stitching, adjacent-camera stitching, and round-trip image stitching. Figure 7 As shown, a schematic diagram illustrating the impact of a single camera on the stitching result is provided, such as... Figure 8 As shown, a schematic diagram of the uplink and downlink panoramic image stitching results is provided, such as... Figure 9 As shown, a schematic diagram of the panoramic image stitching result of a railway tunnel is provided.

[0109] Step 6: Calculate and match feature points of adjacent camera images based on the SURF algorithm, further calculate image offset, and complete image stitching between adjacent cameras based on the image offset between adjacent cameras.

[0110] Step 7: Calculate and match the feature points of the uplink and downlink images based on the SURF algorithm, calculate the image offset, and stitch the uplink and downlink images together according to the offset between them to finally obtain the panoramic image of the tunnel.

[0111] In this embodiment, the present invention provides a method for image acquisition and stitching using a planar array camera in railway tunnels. Its advantages include: the acquisition equipment has a simple structure and is easy to maintain; while ensuring image acquisition quality, it minimizes the number of cameras and does not include additional auxiliary equipment such as laser ranging and positioning, resulting in excellent economic benefits. To reduce the redundancy of the original image data, the cameras of the acquisition equipment are distributed on one side of the working direction, which is more suitable for double-track tunnels, reducing data redundancy and equipment costs. In image data stitching, a feature point recognition-based image distortion correction method is used to address the distortion problem of the original image, improving the accuracy of panoramic tunnel image stitching. The image stitching algorithm is highly efficient. This distortion correction algorithm calculates the difference in offset between pixels in each row based on feature point recognition to obtain a distortion correction model. This process requires only a few sets of images from the same camera, resulting in high computational efficiency and greatly improving the accuracy of image stitching.

[0112] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0113] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 10 As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computational and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage medium. The database stores image data, corrected image data, candidate feature points, target feature points, and target feature point offsets, among other data. The network interface is used for communication with external terminals via a network connection. When executed by the processor, the computer program implements a railway tunnel image processing method.

[0114] Those skilled in the art will understand that Figure 10The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0115] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0116] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0117] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0118] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0119] In the description of this specification, references to terms such as "some embodiments," "other embodiments," and "ideal embodiments" indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative descriptions of the above terms do not necessarily refer to the same embodiments or examples.

[0120] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0121] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A railway tunnel image processing device, characterized in that, include: Multiple area array cameras are arranged sequentially on the side surface of the inspection vehicle in a direction perpendicular to the travel direction of the inspection vehicle, and the total shooting angle of all the area array cameras is greater than or equal to 150 degrees. Each area array camera is used to collect image data of the railway tunnel when the inspection vehicle is working back and forth in the railway tunnel. Multiple light sources are provided, with each light source spaced apart between any two adjacent area array cameras; The processor is connected to each of the area array cameras to acquire the image data and perform distortion correction and stitching processing on the image data to obtain a panoramic image of the railway tunnel.

2. The railway tunnel image processing device according to claim 1, characterized in that, The included angle between any two adjacent area array cameras is the same.

3. The railway tunnel image processing device according to claim 1, characterized in that, In the direction of travel of the inspection vehicle, each of the area array cameras is arranged on the left or right surface of the inspection vehicle, and the shooting face of each of the area array cameras is different.

4. The railway tunnel image processing device according to claim 1, characterized in that, The processor is further configured to perform feature matching on the image data to determine initial feature points in each image data; and to calculate the feature point offset between adjacent image data based on the initial feature points in each image data. Based on the offset of each feature point, the image data is subjected to distortion correction processing to obtain corrected image data.

5. The railway tunnel image processing device according to claim 4, characterized in that, The processor is further configured to calculate candidate feature points between adjacent corrected image data; and to use a robust estimation algorithm to filter out the candidate feature points to obtain the target feature points of the corrected image data, as well as the target feature point offsets between adjacent corrected image data. Based on the offset of the target feature points, the corrected image data is stitched together to obtain a panoramic image of the railway tunnel.

6. The railway tunnel image processing device according to claim 5, characterized in that, The processor is also used to stitch the corrected image data according to the target feature point offset, in the order of stitching single camera, adjacent camera and round-trip images, to obtain a panoramic image of the railway tunnel.

7. A method for processing railway tunnel images, characterized in that, The processor used in the railway tunnel image processing apparatus according to any one of claims 1-6 comprises: Acquire image data of the railway tunnel collected by each array camera; Each of the image data is subjected to distortion correction processing to obtain the corrected image data; Determine candidate feature points for the corrected image data; A robust estimation algorithm is used to filter out the candidate feature points to obtain the target feature points of the corrected image data, as well as the offset of the target feature points between adjacent corrected image data. Based on the offset of the target feature points, the corrected image data is stitched together to obtain a panoramic image of the railway tunnel.

8. The railway tunnel image processing method according to claim 7, characterized in that, The process of performing distortion correction on each of the image data to obtain corrected image data includes: Feature matching is performed on the image data to determine the initial feature points in each image data. Based on the initial feature points in each of the image data, calculate the feature point offset between adjacent image data; Based on the distortion correction coefficients corresponding to the offsets of each feature point, the image data is subjected to distortion correction processing to obtain the corrected image data.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 7 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 7 to 8.

Images