Image-based track surface shooting multi-camera pixel size unification and conversion method

Abstract

The application discloses a kind of based on image's track surface shooting multi-camera pixel size uniformity and conversion method, this method includes the following steps: S1, the image that camera shoots on track is acquired, and the region corresponding to track is detected and extracted;S2, the edge of rail head and rail bottom is acquired;S3, whether image shooting device and track running direction are perpendicular, if perpendicular then enter step S4, otherwise, the image that all camera shoots is corrected;S4, whether there is side angle of camera shooting device, if not enter step S5, otherwise, the image that all camera shoots is corrected;S5, the actual rail bottom size of rail is acquired;S6, the lateral scaling multiple of the image of low-position camera is determined;S7, low-position camera image is scaled according to the lateral scaling multiple determined in lateral direction;S8, after high-position camera image and scaled low-position camera image are spliced, the actual size of detection target in the image that shoots is calculated.The above design can automatically judge whether there is side deviation, pitch and other incorrect installation conditions according to the relationship between rail top and rail bottom in the image that shoots, if there is, can automatically identify and automatically correct the image, improve measurement accuracy.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically, it relates to a method for unifying and converting the pixel size of multiple cameras in orbital plane imaging based on images. Background Technology

[0002] In recent years, with the rapid development and construction of rail transit, more stringent requirements have been placed on the operation and maintenance capabilities of rail transit. The condition of the track lines directly affects the smoothness, safety, and passenger comfort of operating vehicles during normal operation. Therefore, accurately detecting and identifying defects in key track components is of great significance for effectively formulating maintenance strategies and mitigating the further development of defects in these key components.

[0003] With the rapid development of artificial intelligence technology, intelligent operation and maintenance technology based on deep learning is gradually being widely applied in the rail transit field to ensure the safety of rail transit transportation. Among them, image-based track surface defect detection can efficiently, in real time, and online detect and locate track defects, effectively improving detection efficiency and accuracy, reducing costs, and mitigating the impact of adverse factors such as the mental state and professional competence of the detection personnel on the detection results. However, in practical applications, due to the influence of installation location and shooting field of view, a single camera is difficult to complete the image acquisition of the entire track surface, and generally 4 to 5 cameras are needed to complete the task. In order to quantify and analyze the specific size of the defects, it is necessary to convert the pixel size to the actual size. Since the train has real-time and irregular up-and-down movement during operation, it is difficult to determine the specific distance between the camera installation location and the track surface, and these cameras may be located on different installation surfaces. Therefore, it is necessary to determine the pixel relationship between each camera and the conversion relationship between pixel size and actual size in real time during operation. In order to accurately calculate the size of track defects captured by cameras without installing additional equipment, this invention proposes a method for unifying and converting the pixel size of multiple cameras for track surface imaging based on images. Summary of the Invention

[0004] The purpose of this invention is to provide a method for unifying and converting the pixel size of multiple cameras in image-based orbital surface photography, mainly to solve the problem that a single camera is difficult to complete the image acquisition of the entire orbital shooting surface.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] A method for unifying and converting pixel dimensions of multiple cameras captured on an orbital plane based on images, including the following steps:

[0007] S1, acquire the image captured by the camera above the track, and input the image into the convolutional neural network to detect and extract the region corresponding to the track;

[0008] S2, perform edge extraction on the acquired track image region to obtain the edges of the rail head and rail bottom;

[0009] S3, determine whether the image capturing device is perpendicular to the track running direction. If it is perpendicular, proceed to step S4; otherwise, correct the images captured by all cameras.

[0010] S4. Determine if there is a side-tilt angle in the camera shooting device. If not, proceed to step S5. Otherwise, correct the images captured by all cameras.

[0011] S5, determine the rail type based on the top and bottom pixel widths, and obtain the actual bottom dimensions of the rail;

[0012] S6. Determine the horizontal scaling factor of the low-position camera image based on the actual rail base dimensions and camera parameters.

[0013] S7, scales the low-position camera image horizontally according to a determined horizontal scaling factor;

[0014] S8 stitches together the high-position camera image and the scaled low-position camera image and then calculates the actual size of the detected target in the captured image.

[0015] Furthermore, in step S3, the method for determining whether the image capturing device is perpendicular to the direction of track movement is as follows:

[0016] Obtain the X coordinates (x1, x2) of the two vertices above and below the edge of a certain track; if x1 = x2, it indicates that the image capturing device is perpendicular to the track running direction; otherwise, the image capturing device is not perpendicular to the track running direction.

[0017] Furthermore, in step S4, the method for determining whether the camera shooting device has a side-tilt angle is as follows:

[0018] Calculate the pixel distance L between the left edge of the top rail and the left edge of the bottom rail. l,pix ; Calculate the pixel distance L between the right edge of the top rail and the right edge of the bottom rail. r,pix If L l,oix =L r,pix If the angle is 0, it indicates that the camera's shooting device does not have a side-slip angle; otherwise, a side-slip angle exists.

[0019] Furthermore, in step S5, the type of rail is determined based on the rail top pixel width b. pix and track bottom pixel width B pix ratio Obtained by referring to the table.

[0020] Furthermore, in step S6, the method for determining the horizontal scaling factor is as follows:

[0021] S60, using the pixel width B at the bottom of the track. pix And the actual rail base dimension B mm Calculate the conversion relationship α between the unit pixel pitch of the high-position camera and the actual distance. H-pix-mm ;

[0022] S61, utilizing the camera's horizontal resolution parameter w dpi The conversion relationship α between the pixel pitch of a high-position camera and the actual distance. H-pix-mm Calculate the actual field of view w of the high-position camera under the current operating conditions. c1 ;in,

[0023]

[0024] w c1 =w dpi *α H-pix-mm ;

[0025] S62, using the formula for calculating the focal length of a camera objective lens. Formula for estimating the actual working distance of a camera The height h of the high-position camera from the shooting surface is calculated using the formula for calculating the actual working distance of the camera and related parameters. c1 In the formula: f is the focal length of the objective lens; h is the working distance of the camera; w is the actual field of view width of the camera under the current field of view; w cell The width of a camera chip pixel; w dpi The horizontal resolution of the camera;

[0026] S63, calculate the actual field of view of the low-position camera using the distance between the high-position camera and the track surface and the actual field of view of the high-position camera; that is:

[0027]

[0028]

[0029] In the formula, w c2 This refers to the actual field of view of the low-angle camera; w c1 For high-position cameras at height h c1 The actual field of view at that time; h c1 Δh represents the height of the high-position camera from the shooting surface; Δh represents the height difference between the mounting surfaces of the low-position camera and the high-position camera.

[0030] S64, based on the actual field of view width w of the low-position camera c2 and the camera's horizontal resolution parameter w dpi Calculate the conversion relationship α between the unit pixel spacing of the low-position camera and the actual distance. L-pix-mm ;Right now:

[0031]

[0032] S65, utilizing α H-pix-mm and α L-pix-mm The horizontal scaling factor γ of the low-angle camera image is obtained as follows:

[0033]

[0034] Furthermore, in step S8, the actual size of the detected target is determined using the conversion relationship α between image pixels and actual size. H-pix-mm Calculations show that...

[0035] Compared with the prior art, the present invention has the following beneficial effects:

[0036] (1) This invention uses the width ratio of the top and bottom of the rail to automatically determine the rail type, which can meet the needs of different rail types on the same line, or the need to re-enter the rail type after the train changes its running line, and can avoid errors caused by manual input.

[0037] (2) Based on the relationship between the top and bottom of the rail in the captured image, the present invention can automatically determine whether there is incorrect installation such as lateral deviation or pitch. If so, it can automatically identify and correct the image to improve measurement accuracy.

[0038] (3) In the case of multiple cameras not being on the same installation plane due to installation space limitations, the present invention can perform image size conversion in real time and automatically, so that the final stitched image has the same pixel and actual size conversion relationship.

[0039] (4) The present invention can automatically determine the conversion relationship between image pixels and actual size without prior calibration, thus improving the convenience of use. Attached Figure Description

[0040] Figure 1 This is a schematic diagram showing the camera mounting relationship and dimensions in an embodiment of the present invention. Detailed Implementation

[0041] The present invention will be further described below with reference to the accompanying drawings and embodiments. The embodiments of the present invention include, but are not limited to, the following embodiments.

[0042] like Figure 1 As shown, the present invention discloses a method for unifying and converting the pixel size of multiple cameras for orbital plane shooting based on images. The method includes: S1, powering on the camera by powering on the control system; and acquiring the image captured by the camera.

[0043] S2, acquire the image captured by the camera above the track, and input the image into the convolutional neural network to detect and extract the region corresponding to the track;

[0044] S3, perform edge extraction on the acquired track image area to obtain the edges of the rail head and rail bottom;

[0045] S4, determine whether the image capturing device is perpendicular to the track running direction: obtain the X coordinates (x1, x2) of the two vertices above and below a certain track edge line; if x1 = x2, it indicates that the image capturing device is perpendicular to the track running direction and no correction is needed, proceed to step S5; otherwise, the image capturing device is not perpendicular to the track running direction and all images captured by the camera need to be corrected.

[0046] S5, Determine if the camera shooting device has a side-slip angle: Calculate the pixel distance L between the left edge line of the top rail and the left edge line of the bottom rail. l，pix ; Calculate the pixel distance L between the right edge of the top rail and the right edge of the bottom rail. r，pix If L l，pix =L r，pix If the image does not show a side-slip angle, then the image does not need to be corrected, and step S6 can be executed directly; otherwise, if a side-slip angle exists, the images captured by all cameras should be corrected.

[0047] S6, based on the track top pixel width b pix and track bottom pixel width B pix Determine the rail type and obtain the actual rail base dimensions. That is, based on the rail top pixel width b. pix and track bottom pixel width B pix Calculate the width ratio of the rail top to the rail bottom. Using the ratio α, we can look up the existing ratio table of rail head to rail base, as shown in Table 1, to determine the rail type, such as 60 rail; then, based on the rail type, we can retrieve the rail base dimension B. mm .

[0048] Table 1 Parameters of different rail models

[0049] 33# 110 60 0.55 38# 114 68 0.60 43# 114 70 0.61 50# 132 70 0.53 60# 150 73 0.49 75# 150 75 0.50

[0050] S7, using the pixel width B at the bottom of the track. pix And the actual rail base dimension B mm Calculate the conversion relationship α between the unit pixel pitch of the high-position camera and the actual distance. H-pix-mm , unit mm / pix.

[0051] Using the camera's horizontal resolution parameter w dpi The conversion relationship α between the pixel pitch of a high-position camera and the actual distance. H-pix-mmCalculate the actual field of view w of the high-position camera under the current operating conditions. c1 ;in,

[0052]

[0053] w c1 =w dpi *α H-pix-mm ;

[0054] Using the formula for calculating the focal length of a camera objective lens Formula for estimating the actual working distance of a camera The height h of the high-position camera from the shooting surface is calculated using the formula for calculating the actual working distance of the camera and related parameters. c1 In the formula: f is the focal length of the objective lens, mm; h is the working distance of the camera, mm; w is the actual field of view width of the camera under the current field of view, mm; w cell Width of a camera chip pixel, in mm; w dpi This refers to the horizontal resolution of the camera.

[0055] The actual field of view of the low-position camera is calculated using the distance between the high-position camera and the track surface and the actual field of view of the high-position camera; that is:

[0056]

[0057]

[0058] In the formula, w c2 The actual field of view of the low-angle camera is in mm; w c1 For high-position cameras at height h c1 Actual field of view width at that time, mm; h c1 Δh represents the height of the high-position camera from the shooting surface, in mm; Δh represents the height difference between the mounting surfaces of the low-position camera and the high-position camera.

[0059] Based on the actual field of view width of the low-position camera w c2 and the camera's horizontal resolution parameter w dpi Calculate the conversion relationship α between the unit pixel spacing of the low-position camera and the actual distance. L-pix-mm ;Right now:

[0060]

[0061] Using α H-pix-mm and α L-pix-mm The horizontal scaling factor Y of the low-angle camera image is obtained:

[0062]

[0063] S8, scales the low-position camera image horizontally according to a determined horizontal scaling factor;

[0064] S9 stitches the high-position camera image and the scaled low-position camera image together to calculate the actual size of the detected target in the captured image. This is done by using the conversion relationship α between image pixels and actual size. H-pix-mm Calculations show that...

[0065] Through the above design, this invention can automatically determine whether there are incorrect installation conditions such as lateral deviation or pitch based on the relationship between the top and bottom of the rail in the captured image. If such conditions exist, it can automatically identify them and automatically correct the image, thereby improving measurement accuracy. Therefore, compared with the prior art, this invention has outstanding substantive features and significant progress.

[0066] The above embodiments are merely one of the preferred embodiments of the present invention and should not be used to limit the scope of protection of the present invention. Any modifications or refinements made to the main design concept and spirit of the present invention that are not of substantial significance, but solve the same technical problem as the present invention, should be included within the scope of protection of the present invention.

Claims

1. A method for unifying and converting pixel sizes of multiple cameras based on image-based orbital plane photography, characterized in that, Includes the following steps: S1, acquire the image captured by the camera above the track, and input the image into the convolutional neural network to detect and extract the region corresponding to the track; S2, perform edge extraction on the acquired track image region to obtain the edges of the rail head and rail bottom; S3, determine whether the image capturing device is perpendicular to the track running direction. If it is perpendicular, proceed to step S4; otherwise, correct the images captured by all cameras. S4. Determine if there is a side-tilt angle in the camera shooting device. If not, proceed to step S5. Otherwise, correct the images captured by all cameras. S5, determine the rail type based on the top and bottom pixel widths, and obtain the actual bottom dimensions of the rail; S6. Determine the horizontal scaling factor of the low-position camera image based on the actual rail base dimensions and camera parameters. S7, scales the low-position camera image horizontally according to a determined horizontal scaling factor; S8 stitches together the high-position camera image and the scaled low-position camera image and then calculates the actual size of the detected target in the captured image.

2. The method for unifying and converting pixel sizes of multiple cameras based on image-based orbital plane shooting according to claim 1, characterized in that, In step S3, the method for determining whether the image capturing device is perpendicular to the direction of track movement is as follows: Obtain the X coordinates (x1, x2) of the two vertices above and below the edge of a certain track; if x1 = x2, it indicates that the image capturing device is perpendicular to the track running direction; otherwise, the image capturing device is not perpendicular to the track running direction.

3. The method for unifying and converting pixel sizes of multiple cameras based on image-based orbital plane shooting according to claim 2, characterized in that, In step S4, the method for determining whether the camera shooting device has a side-tilt angle is as follows: Calculate the pixel distance L between the left edge of the top rail and the left edge of the bottom rail. l,pix ; Calculate the pixel distance L between the right edge of the top rail and the right edge of the bottom rail. r,pix If L l,pix =L r,pix If the angle is 0, it indicates that the camera's shooting device does not have a side-slip angle; otherwise, a side-slip angle exists.

4. The method for unifying and converting pixel sizes of multiple cameras in image-based orbital plane shooting according to claim 3, characterized in that, In step S5, the rail type is determined based on the rail top pixel width b. pix and track bottom pixel width B pix ratio Obtained by referring to the table.

5. The method for unifying and converting pixel sizes of multiple cameras in image-based orbital plane shooting according to claim 4, characterized in that, In step S6, the method for determining the horizontal scaling factor is as follows: S60, using the pixel width B at the bottom of the track. pix And the actual rail base dimension B mm Calculate the conversion relationship α between the unit pixel pitch of the high-position camera and the actual distance. H-pix-mm ; S61, utilizing the camera's horizontal resolution parameter w dpi The conversion relationship α between the pixel pitch of a high-position camera and the actual distance. H-pix-mm Calculate the actual field of view w of the high-position camera under the current operating conditions. c1 ; in, w c1 =w dpi *a H-pid-mm ; S62, using the formula for calculating the focal length of a camera objective lens. Formula for estimating the actual working distance of a camera The height h of the high-position camera from the shooting surface is calculated using the formula for calculating the actual working distance of the camera and related parameters. c1 In the formula: f is the focal length of the objective lens; h is the working distance of the camera; w is the actual field of view width of the camera under the current field of view; w cell The width of a camera chip pixel; w dpi This refers to the camera's horizontal resolution. S63, calculate the actual field of view of the low-position camera using the distance between the high-position camera and the track surface and the actual field of view of the high-position camera; that is: In the formula, w c2 This refers to the actual field of view of the low-angle camera; w c1 For high-position cameras at height h c1 The actual field of view at that time; h c1 Δh represents the height of the high-position camera from the shooting surface; Δh represents the height difference between the mounting surfaces of the low-position camera and the high-position camera. S64, based on the actual field of view width w of the low-position camera c2 and the camera's horizontal resolution parameter w dpi Calculate the conversion relationship α between the unit pixel pitch of the low-position camera and the actual distance. L-pix-mm ;Right now: S65, utilizing α H-pix-mm and α L-pix-mm The horizontal scaling factor γ of the low-angle camera image is obtained as follows:

6. The method for unifying and converting pixel sizes of multiple cameras in image-based orbital plane shooting according to claim 5, characterized in that, In step S8, the actual size of the detected target is determined using the conversion relationship α between image pixels and actual size. H-pix-mm Calculations show that...

Images