Rail transit vehicle roof three-dimensional modeling system and method based on multi-vision

Abstract

The application discloses a kind of track traffic vehicle roof three-dimensional modeling system and method based on multi-vision, the system includes car number identification module, image acquisition module, image pre-processing module, key position three-dimensional reconstruction module, information storage and display module, database module;Method includes: image acquisition module is collected roof multi-angle image by multi-vision camera, and car number module identifies car number by car number image;The collected image is pre-processed, each camera is calibrated using pinhole model, median filter denoising, improves the contrast of key components;Through stereo matching, corresponding pixel points between images are found, image scene texture features are enhanced, so that stereo matching is more accurate;Merge the three-dimensional point cloud data collected, obtain the dense three-dimensional point cloud data of the scene, establish three-dimensional model.The application improves the accuracy and stability of three-dimensional model display.

Description

Technical Field

[0001] This invention patent relates to the field of multi-view vision 3D modeling, specifically to a multi-view vision-based 3D modeling system and method for the roof of rail transit vehicles. Background Technology

[0002] Since the beginning of the 21st century, rail transit has grown rapidly, with railway operating mileage increasing year by year and annual passenger volume continuously rising. High-speed rail has developed particularly rapidly in recent years. Statistics from 2014 show that over 800 million people chose high-speed rail as their mode of transportation. By the end of 2014, China's high-speed rail operating mileage reached 19,000 kilometers, ranking first in the world. Urban rail transit is also constantly developing, with major cities constructing their own rail transit systems. This rapid progress in rail transit is mainly attributed to the ability of locomotives to maintain continuous high-speed operation, greatly reducing passenger travel time.

[0003] During operation, electric locomotives draw power from the grid by contacting the overhead contact line via the pantograph's contact plate. This contact-based power supply method inevitably leads to electrical and sliding wear on the pantograph's contact plate. If this wear is left unchecked, it can cause the contact line to be pulled apart or even jammed, and in more serious cases, become a direct cause of severe railway accidents. A break in the contact line can also cause the pantograph to break. The locomotive roof also presents many safety hazards. Sometimes, tools are left on the roof due to worker error, and other objects may be present due to birds or other factors. These objects may damage the pantograph or insulators due to inertia when the train starts or during operation, or they may become entangled in cables or transmission lines, rendering the equipment inoperable.

[0004] In 3D reconstruction methods, laser scanning primarily utilizes laser ranging principles to acquire depth information of a target scene. It involves scanning the surface of the target object using laser scanning equipment such as LiDAR (LiDAR) to obtain a 3D point cloud of the target scene. While laser scanning offers advantages such as simple operation and high scanning accuracy, it cannot directly acquire the target's texture information, and laser scanning equipment is generally expensive, resulting in high application costs. Time-of-flight (TOF) methods measure distance by recording the time it takes for a laser pulse to travel from the transmitter to the receiver after reflection from the object. Primarily used in TOF cameras, this method acquires depth images of the target scene and reconstructs 3D point clouds from these images. Although TOF offers advantages such as long detection distance and strong anti-interference capabilities, it suffers from high power consumption, high cost, and relatively low camera resolution. Structured light methods project coded patterns onto the target scene, recovering the 3D information of the target surface from the degree of pattern deformation. Structured light methods offer high accuracy at close range but are susceptible to strong ambient light interference, and their accuracy decreases rapidly with increasing detection distance.

[0005] With the rapid improvement of CCD optical imaging technology, the performance parameters of imaging devices, such as resolution, sensitivity, and spectral response range, have been greatly improved, making passive 3D reconstruction based on stereo vision an important research direction. Summary of the Invention

[0006] The purpose of this invention is to provide an accurate and efficient multi-view vision-based 3D modeling system and method for the roof of rail transit vehicles.

[0007] The technical solution for achieving the objective of this invention is: a multi-view vision-based 3D modeling system for the roof of rail transit vehicles, comprising a vehicle number recognition module, an image acquisition module, an image preprocessing module, a 3D reconstruction module for key parts, an information storage and display module, and a database module, wherein:

[0008] The vehicle number recognition module includes wheel sensors, a PLC, and a vehicle number recognition host. The image acquisition module includes photoelectric sensors, industrial cameras, and an image acquisition system. The wheel sensors trigger the PLC to control the vehicle number recognition host to detect vehicle number information. At the same time, the photoelectric sensors send an electrical signal to the PLC, which controls multiple industrial cameras on the roof to achieve hard triggering. The image acquisition system acquires the original image of the roof. The vehicle number information and the original roof image are stored in the database module.

[0009] The image preprocessing module preprocesses the original image of the roof, uses a pinhole model to calibrate each camera, uses median filtering to denoise the image, enhances the image scene texture, and then uses stereo matching to find corresponding pixels between images.

[0010] The key part 3D reconstruction module performs interpolation matching on each set of matching points based on the output of the image preprocessing module to obtain the final image coordinates of each set of matching points. It merges the point clouds generated by each set of multi-view images to obtain dense 3D point cloud data and builds a 3D model, forming a 3D stereo image and storing it in the database module.

[0011] The information storage and display module processes and displays three-dimensional images to complete the three-dimensional modeling of the roof of rail transit vehicles.

[0012] A method for 3D modeling of the roof of rail transit vehicles based on multi-view vision, the steps of which are as follows:

[0013] Step S0: Set up the vehicle number recognition module, image acquisition module, image preprocessing module, key part 3D reconstruction module, information storage and display module, and database module; the vehicle number recognition module includes wheel sensors, PLC, and vehicle number recognition host, and the image acquisition module includes photoelectric sensors, industrial cameras, and image acquisition system;

[0014] Step S1: The wheel sensor triggers the PLC to control the vehicle number recognition host to detect the vehicle number information. At the same time, the photoelectric sensor sends an electrical signal to the PLC, which controls the four industrial cameras on the roof to achieve hard triggering. The image acquisition system acquires the original image of the roof. The vehicle number information and the original roof image are stored in the database module.

[0015] Step S2: The image preprocessing module preprocesses the original image of the roof, uses a pinhole model to calibrate each camera, uses median filtering to denoise the image, enhances the image scene texture, and then uses stereo matching to find the corresponding pixels between images.

[0016] Step S3: The key part 3D reconstruction module performs interpolation matching on each set of matching points based on the output of the image preprocessing module to obtain the final image coordinates of each set of matching points.

[0017] Step S4: For the image after interpolation matching in step S3, merge the point clouds generated by each group of four-view images to obtain dense three-dimensional point cloud data of the scene, build a three-dimensional model, form a three-dimensional stereo image and store it in the database module.

[0018] Step S5: The information storage and display module processes the 3D image to complete the 3D modeling of the roof of the rail transit vehicle.

[0019] Compared with the prior art, the present invention has the following significant advantages: (1) It obtains three-dimensional stereo images of key components on the roof through a multi-view camera, replacing manual inspection and diagnosis of the roof; (2) It uses an improved interpolation algorithm to preprocess the image before stereo matching, thereby improving the accuracy of stereo matching; (3) It uses scene texture feature enhancement to divide the image into ROI regions, thereby improving the matching efficiency and making the subsequent stereo matching more efficient; (4) It uses point cloud simplification operation to retain a sufficient number of points to represent the shape changes of the region, which greatly improves the effectiveness of the algorithm and the accuracy of three-dimensional modeling compared with traditional point cloud data acquisition. Attached Figure Description

[0020] Figure 1 This is a flowchart of the multi-view vision-based 3D modeling method for the roof of rail transit vehicles according to the present invention.

[0021] Figure 2 This is a flowchart for determining the three-dimensional modeling in this invention.

[0022] Figure 3 This is a block diagram of the multi-view vision-based 3D modeling system for the roof of rail transit vehicles according to the present invention.

[0023] Figure 4This is a hardware diagram of the multi-view vision-based 3D modeling system for the roof of a rail transit vehicle according to the present invention. Detailed Implementation

[0024] This invention discloses a multi-view vision-based 3D modeling system for the roof of rail transit vehicles, comprising a vehicle number recognition module, an image acquisition module, an image preprocessing module, a 3D reconstruction module for key parts, an information storage and display module, and a database module, wherein:

[0025] The vehicle number recognition module includes wheel sensors, a PLC, and a vehicle number recognition host. The image acquisition module includes photoelectric sensors, industrial cameras, and an image acquisition system. The wheel sensors trigger the PLC to control the vehicle number recognition host to detect vehicle number information. At the same time, the photoelectric sensors send an electrical signal to the PLC, which controls multiple industrial cameras on the roof to achieve hard triggering. The image acquisition system acquires the original image of the roof. The vehicle number information and the original roof image are stored in the database module.

[0026] The image preprocessing module preprocesses the original image of the roof, uses a pinhole model to calibrate each camera, uses median filtering to denoise the image, enhances the image scene texture, and then uses stereo matching to find corresponding pixels between images.

[0027] The key part 3D reconstruction module performs interpolation matching on each set of matching points based on the output of the image preprocessing module to obtain the final image coordinates of each set of matching points. It merges the point clouds generated by each set of multi-view images to obtain dense 3D point cloud data and builds a 3D model, forming a 3D stereo image and storing it in the database module.

[0028] The information storage and display module processes and displays three-dimensional images to complete the three-dimensional modeling of the roof of rail transit vehicles.

[0029] Furthermore, the image acquisition module contains four industrial cameras.

[0030] Furthermore, the industrial camera is an acA2400-20gc; the wheel sensor is a magnetic sensor, and the detector of the magnetic sensor is a magnetic probe; the photoelectric sensor is a PZ-G series photoelectric sensor; and the PLC is a Siemens S7-200 PLC.

[0031] A method for 3D modeling of the roof of rail transit vehicles based on multi-view vision, the steps of which are as follows:

[0032] Step S0: Set up the vehicle number recognition module, image acquisition module, image preprocessing module, key part 3D reconstruction module, information storage and display module, and database module; the vehicle number recognition module includes wheel sensors, PLC, and vehicle number recognition host, and the image acquisition module includes photoelectric sensors, industrial cameras, and image acquisition system;

[0033] Step S1: The wheel sensor triggers the PLC to control the vehicle number recognition host to detect the vehicle number information. At the same time, the photoelectric sensor sends an electrical signal to the PLC, which controls the four industrial cameras on the roof to achieve hard triggering. The image acquisition system acquires the original image of the roof. The vehicle number information and the original roof image are stored in the database module.

[0034] Step S2: The image preprocessing module preprocesses the original image of the roof, uses a pinhole model to calibrate each camera, uses median filtering to denoise the image, enhances the image scene texture, and then uses stereo matching to find the corresponding pixels between images.

[0035] Step S3: The key part 3D reconstruction module performs interpolation matching on each set of matching points based on the output of the image preprocessing module to obtain the final image coordinates of each set of matching points.

[0036] Step S4: For the image after interpolation matching in step S3, merge the point clouds generated by each group of four-view images to obtain dense three-dimensional point cloud data of the scene, build a three-dimensional model, form a three-dimensional stereo image and store it in the database module.

[0037] Step S5: The information storage and display module processes the 3D image to complete the 3D modeling of the roof of the rail transit vehicle.

[0038] Furthermore, in step S2, the four-eye camera is calibrated. The linear model camera calibration does not consider the influence of distortion, and directly uses the pinhole model to describe the camera imaging geometry and expresses it using projection equations.

[0039] In the projection equation, m ij This represents the element in the i-th row and j-th column of the projection matrix M, after eliminating the scale factor Z. c The following system of equations is obtained:

[0040]

[0041] Where X w Y w Z w The three axes of the homogeneous coordinate system established for the camera are u and v, which are the horizontal and vertical axes of the imaging plane.

[0042] If m = 1, there are 11 unknowns. For each calibration point with known coordinates, there are the two equations mentioned above. Therefore, the world coordinates and image coordinates of more than 5 calibration points are needed to calculate the camera parameters.

[0043] The calibration of the linear model is as follows: directly obtain the coordinates of the calibration points on the calibration object in the camera image, and combine them with the world coordinates of the calibration points to calculate the parameters of matrix M using the least squares method. The intrinsic and extrinsic parameters of the camera are obtained by decomposing matrix M.

[0044] Furthermore, in step S2, the scene texture is enhanced by selecting a projected black and white checkerboard pattern to increase the texture features of the target, and corner points are extracted as feature points for stereo matching.

[0045] A batch of checkerboard patterns with different corner positions or phases are projected onto the target scene. These checkerboard patterns have phase offsets in the horizontal or vertical directions. In the horizontal direction, starting from the first checkerboard pattern in the upper left corner, each time the pattern is moved to the right by the same displacement Δx, patterns with different phases are obtained. Combining the periodic distribution of squares in the checkerboard pattern and the side length of the squares, a total of M non-repeating patterns are obtained, which is equivalent to subdividing the corner spacing in the horizontal direction by M, resulting in a denser corner distribution.

[0046] Moving the same displacement Δy downwards in the vertical direction each time results in N non-repeating patterns, which is equivalent to subdividing the corner spacing in the vertical direction by N subdivisions.

[0047] If we take Δx = Δy, then M = N, thus generating a checkerboard pattern array containing M × N different checkerboard patterns. These patterns are projected onto the target scene, which is equivalent to enhancing the surface texture features of objects in the scene M × N times. By controlling the size of Δx and Δy, three-dimensional point clouds with different densities can be reconstructed.

[0048] Furthermore, in step S4, the bicubic interpolation method is improved by performing interpolation calculations based on the grayscale values ​​of the surrounding 8 pixels, taking into account the influence of pixel disparity values ​​and the disparity weighting caused by the distance between neighboring pixels. The specific steps are as follows:

[0049] (1) The disparity fill value is calculated using an improved bicubic interpolation method, as shown in the following formula:

[0050]

[0051] Where I(x,y) represents the pixel value of the point to be interpolated, I i (x i ,y i W represents the effective parallax value of the surrounding area. x (I i ) and W x(I i The numbers ) represent the weights of the horizontal and vertical axes, respectively.

[0052] (2) Calculate the weights for the X and Y directions respectively. The formulas for calculating the weights of the horizontal and vertical axes are as follows:

[0053]

[0054] The disparity fill value is calculated using the mean. The variance d of the eight effective disparity values ​​around the pixel to be interpolated is calculated. If the variance d is less than the set threshold, the mean of the eight effective disparity values ​​is used as the disparity fill value; otherwise, bicubic interpolation is used for calculation.

[0055] Furthermore, in step S4, dense 3D point cloud data of the scene is acquired, and the texture features of the target scene are enhanced by projecting a checkerboard pattern, thereby obtaining a dense 3D point cloud of the target scene.

[0056] Before generating the 3D model, a point cloud simplification operation is performed to reduce the number of points, thereby improving the processing speed of subsequent point cloud triangulation and reducing the size of the 3D model.

[0057] A point cloud simplification method based on spatial plane fitting is used. By fitting a spatial plane to the point cloud in a local area, if the distance from the points in the area to the fitted plane is less than a set threshold, the spatial distribution of points in the area is considered to be flat; otherwise, the spatial distribution of points in the area is considered to be turbulent.

[0058] The equation of the space plane is expressed as follows:

[0059] Ax + By + Cz + D = 0

[0060] Among them, parameters A, B, C, and D are constants describing the characteristics of planar space;

[0061] The coordinates of the external measurements of the tetracular system will not be located at (0,0,0), therefore we assume that D≠0. Dividing both sides of the spatial equation by D, we transform it into the following expression:

[0062] ax + by + cz + 1 = 0

[0063] in To fit the plane, we need to solve for the values ​​of a, b, and c, and then use the least squares method to solve the plane equation.

[0064] Let F(x,y,z)=ax+by+cz+1=0. For n points in the region, n≥3, the three-dimensional coordinates of each point are (x,y,z)=ax+by+cz+1=0. i ,y i ,,z i ), i = 1, 2, ..., n, ideally satisfying F(xi ,y i ,,z i Since ) = 0, there is actually a deviation. The sum of squares of the total deviations of all points on the fitted plane equation is:

[0065]

[0066] To minimize S, after completing the spatial plane fitting, calculate the distance from each point to the plane:

[0067]

[0068] Where d is the distance from each point to the plane;

[0069] Preserve the boundary points within the region and the point closest to the centroid of all other points. The characteristic of boundary points is that the polygon formed by connecting adjacent boundary points includes all other points except the boundary points. For regions that need to be simplified in point cloud operations, project all points within the region onto the UL image and find the boundary points within the region and the point closest to the centroid of all other points on the UL image.

[0070] Let the image coordinates of each point in the region be (u i ,v i ), calculate the position of the center of gravity (u0, v0), as shown below:

[0071]

[0072] Where u0 and v0 are the pixel coordinates of the center position, traverse all points and find the point closest to (u0, v0).

[0073] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0074] Example

[0075] This invention relates to a multi-view vision-based 3D modeling system and method for the roof of rail transit vehicles. When a train passes a magnet and photoelectric sensor, the vehicle number detection module and PLC control cameras are triggered to acquire images. Simultaneously, the photoelectric sensor detects the train sending an electrical signal to the PLC, which then controls four cameras on the roof to achieve hard triggering and acquire roof image information. The acquired images are preprocessed: each camera is calibrated using a pinhole model; median filtering is used to denoise the images and improve the contrast of key components; improved interpolation and scene texture feature enhancement are then applied to further process the images; and stereo matching is used to find corresponding pixels between images. The matched images are then used to obtain point cloud data for 3D modeling. Finally, the modeled image is displayed.

[0076] The specific implementation steps of this invention are as follows: Figures 1-4 As shown, it includes the following steps:

[0077] S1: Acquire rooftop image information, trigger the car number recognition module with a magnet to detect which train carriage it is, and at the same time, the photoelectric sensor detects the train sending an electrical signal to the PLC. The PLC controls the four cameras on the rooftop to achieve hard triggering and acquire rooftop image information.

[0078] S2: Preprocess the acquired images, calibrate each camera using a pinhole model, use median filtering to denoise the images, improve the contrast of key components, enhance the features of the image scene texture, and then find the corresponding pixels between images through stereo matching.

[0079] S3: To improve matching accuracy, interpolation matching is performed on each set of matching points to obtain the final image coordinates of each set of matching points;

[0080] S4: For the stereo-matched image, merge the point clouds generated by each group of four-view images to obtain dense 3D point cloud data of the scene and build a 3D model.

[0081] S5: Display and process the 3D modeled image.

[0082] Furthermore, step 1 requires acquiring three types of data: using a train number recognition module to detect the specific train number; and using a magnetic sensor to detect train passing information, triggering the PLC to control the camera to acquire images of the roof.

[0083] Furthermore, in step 1, three types of data need to be acquired, and the train number recognition module is used to detect the specific train number. In order to control the camera to acquire images of the roof, a magnetic sensor is used to detect the train passing information, triggering the PLC to control the camera to acquire images.

[0084] Further, in step 2, the four-lens camera is calibrated. Linear model camera calibration disregards the effects of distortion and directly uses a pinhole model to describe the camera's imaging geometry, which can be represented by a projection equation. In the projection equation, m... ij Represents the element in the i-th row and j-th column of the projection matrix M, eliminating Z. c (Z c Also known as the scaling factor, we obtain the following system of equations:

[0085]

[0086] Where X w Y w Z wThe homogeneous coordinate system established for the camera has three axes, u and v, which are the horizontal and vertical axes of the imaging plane. m = 1 can be specified because this will not affect the imaging relationship expressed by the projection equation. Thus, there are 11 unknowns. For each calibration point with known coordinates, there are two equations. Therefore, the world coordinates and image coordinates of more than 5 calibration points are needed to calculate the camera parameters.

[0087] Calibration of linear models is usually relatively simple. The coordinates of the calibration points on the calibration object in the camera image are directly obtained, and combined with the world coordinates of the calibration points, the parameters of matrix M can be calculated using the least squares method. By decomposing matrix M, the intrinsic and extrinsic parameters of the camera can be obtained.

[0088] Furthermore, in step 2, the scene texture is enhanced by selecting a projected black and white checkerboard pattern to increase the texture features of the target. Its corner points are easy to extract and can be used as feature points for stereo matching.

[0089] A batch of checkerboard patterns containing multiple corner points with different positions (or different phases) is projected onto a target scene. These checkerboard patterns have phase shifts in either the horizontal or vertical direction. In the horizontal direction, starting with the first checkerboard pattern from the upper left, each pattern is moved to the right by the same displacement Δx, thus obtaining patterns with different phases. Considering the periodic distribution of the squares in the checkerboard and the side length of the squares, a total of M non-repeating patterns can be obtained, which is equivalent to subdividing the corner point spacing in the horizontal direction by M, resulting in a denser corner point distribution.

[0090] Moving the same downward displacement Δy in the vertical direction each time results in N unique patterns, equivalent to subdividing the corner spacing in the vertical direction by N. Here, Δx = Δy is typically chosen, so M = N. This generates a checkerboard pattern array containing M × N distinct checkerboard patterns. Projecting these patterns onto the target scene is equivalent to enhancing the surface texture features of objects within the scene M × N times. By controlling the values ​​of Δx and Δy, 3D point clouds with varying densities can be reconstructed.

[0091] Furthermore, in step 4, the bicubic interpolation method is improved by performing interpolation calculations based on the grayscale values ​​of the surrounding 8 pixels, taking into account the influence of pixel disparity values ​​and the disparity weighting caused by the distance between neighboring pixels. The specific steps are as follows:

[0092] (1) The disparity fill value is calculated using an improved bicubic interpolation method. The basic principle is as follows:

[0093]

[0094] Where I(x,y) represents the pixel value of the point to be interpolated, I i (x i ,yi W represents the effective parallax value of the surrounding area. x (I i ) and W x (I i ) represent the weights of the horizontal and vertical axes, respectively.

[0095] (2) The formulas for calculating the weights of the horizontal and vertical axes are as follows:

[0096]

[0097] Weights need to be calculated separately for the X and Y directions. The result of this part using bicubic interpolation is similar to the mean, but the computational cost is significantly different. Therefore, the mean is used to calculate the disparity fill value, reducing computational cost while obtaining a better disparity map. The variance 'd' of the eight effective disparity values ​​surrounding the pixel to be interpolated is calculated. If the variance 'd' is less than a set threshold, the mean of the eight effective disparity values ​​is used as the disparity fill value; otherwise, bicubic interpolation is used.

[0098] Furthermore, in step 5, dense 3D point cloud data of the scene is acquired, and the texture features of the target scene are enhanced by projecting a checkerboard pattern, thereby obtaining the dense 3D point cloud of the target scene.

[0099] Before generating the 3D model, point cloud simplification is performed. Reducing the number of points improves the processing speed of subsequent point cloud triangulation and reduces the size of the 3D model. A point cloud simplification method based on spatial plane fitting is used. This method fits a spatial plane to the point cloud within a local region. If the distance from any point in the region to the fitted plane is less than a certain threshold, the spatial distribution of points in that region is considered flat; otherwise, the spatial distribution is considered to have relatively drastic fluctuations. The spatial plane equation is expressed as follows:

[0100] Ax + By + Cz + D = 0

[0101] Where parameters A, B, C, and D are constants describing the characteristics of the planar space. Generally, the coordinates of the external measurements taken by a four-eye system will not be located at (0,0,0), so D≠0 can be assumed. Dividing both sides of the above spatial equation by D, it can be transformed into the following expression:

[0102] ax + by + cz + 1 = 0

[0103] in To fit the plane, we only need to solve for the values ​​of a, b, and c. Here, we use the least squares method to solve the plane equation. Let F(x,y,z) = ax + by + cz + 1 = 0. For n points (n≥3) in a certain region, the three-dimensional coordinates of each point are (x,y,z) = ax + by + cz + 1 = 0. i ,y i ,,zi (i = 1, 2, ..., n), ideally satisfying F(x) i ,y i ,,z i Since ) = 0, there is actually a deviation. Therefore, the sum of squares of the total deviations of all points on the fitted plane equation is:

[0104]

[0105] To minimize S, after completing the spatial plane fitting, calculate the distance from each point to the plane:

[0106]

[0107] Where d is the distance from each point to the plane. The boundary points within this region, as well as the point closest to the centroid of all other points, are retained. A characteristic of boundary points is that the polygon formed by connecting adjacent boundary points includes all points except the boundary points themselves.

[0108] For regions requiring simplified point cloud operations, project all points within that region onto the UL image. Then, find the boundary points of that region and the points whose centroids are closest to all other points on the UL image. Finding the points closest to the centroids of the original point set is straightforward. Let the image coordinates of each point in that region be (u... i ,v i ), calculate the position of the center of gravity (u0, v0), as shown below:

[0109]

[0110] Where u0 and v0 are the pixel coordinates of the center position, traverse all points and find the point closest to (u0, v0).

[0111] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A 3D modeling system for the roof of a rail transit vehicle based on multi-view vision, characterized in that, It includes a license plate recognition module, an image acquisition module, an image preprocessing module, a 3D reconstruction module for key parts, an information storage and display module, and a database module, among which: The vehicle number recognition module includes wheel sensors, a PLC, and a vehicle number recognition host. The image acquisition module includes photoelectric sensors, industrial cameras, and an image acquisition system. The wheel sensors trigger the PLC to control the vehicle number recognition host to detect vehicle number information. At the same time, the photoelectric sensors send an electrical signal to the PLC, which controls multiple industrial cameras on the roof to achieve hard triggering. The image acquisition system acquires the original image of the roof. The vehicle number information and the original roof image are stored in the database module. The image preprocessing module preprocesses the original image of the roof, uses a pinhole model to calibrate each camera, uses median filtering to denoise the image, enhances the image scene texture, and then uses stereo matching to find corresponding pixels between images. The key part 3D reconstruction module performs interpolation matching on each set of matching points based on the output of the image preprocessing module to obtain the final image coordinates of each set of matching points. It merges the point clouds generated by each set of multi-view images to obtain dense 3D point cloud data and builds a 3D model, forming a 3D stereo image and storing it in the database module. The information storage and display module processes and displays three-dimensional images to complete the three-dimensional modeling of the roof of rail transit vehicles; In the image preprocessing module, scene texture is enhanced by projecting a black and white checkerboard pattern to increase the texture features of the target, and corner points are extracted as feature points for stereo matching. A batch of checkerboard patterns, each with different corner positions or phases, is projected onto a target scene. These checkerboard patterns have phase shifts in either the horizontal or vertical direction. In the horizontal direction, starting with the first checkerboard pattern from the top left, each pattern is moved to the right by the same amount of displacement. Patterns with different phases were obtained. Combining the periodic distribution of squares in the chessboard and the side length of the squares, a total of... A unique pattern is equivalent to adjusting the spacing between corner points in the horizontal direction. The subdivisions show a denser distribution of corner points; Each time, the same amount of downward displacement is moved in the vertical direction. A total of A non-repeating pattern is equivalent to subdividing the vertical corner spacing by N. Pick ,but This produces a checkerboard pattern array containing Each element has a unique checkerboard pattern. These patterns are projected onto the target scene, effectively modifying the surface texture features of objects within the scene. Secondary enhancement, through control and The size of the point cloud is used to reconstruct three-dimensional point clouds with different densities.

2. The multi-view vision-based 3D modeling system for the roof of rail transit vehicles according to claim 1, characterized in that, The image acquisition module contains four industrial cameras.

3. The multi-view vision-based 3D modeling system for the roof of rail transit vehicles according to claim 1, characterized in that, The industrial camera is an ACA2400-20GC; the wheel sensor is a magnetic sensor, and the detector of the magnetic sensor is a magnetic probe; the photoelectric sensor is a PZ-G series photoelectric sensor; and the PLC is a Siemens S7-200 PLC.

4. A method for 3D modeling of the roof of a rail transit vehicle based on multi-view vision, characterized in that, The steps are as follows: Step S0: Set up the vehicle number recognition module, image acquisition module, image preprocessing module, key part 3D reconstruction module, information storage and display module, and database module; the vehicle number recognition module includes wheel sensors, PLC, and vehicle number recognition host, and the image acquisition module includes photoelectric sensors, industrial cameras, and image acquisition system; Step S1: The wheel sensor triggers the PLC to control the vehicle number recognition host to detect the vehicle number information. At the same time, the photoelectric sensor sends an electrical signal to the PLC, which controls the four industrial cameras on the roof to achieve hard triggering. The image acquisition system acquires the original image of the roof. The vehicle number information and the original roof image are stored in the database module. Step S2: The image preprocessing module preprocesses the original image of the roof, uses a pinhole model to calibrate each camera, uses median filtering to denoise the image, enhances the image scene texture, and then uses stereo matching to find the corresponding pixels between images. Step S3: The key part 3D reconstruction module performs interpolation matching on each set of matching points based on the output of the image preprocessing module to obtain the final image coordinates of each set of matching points. Step S4: For the image after interpolation matching in step S3, merge the point clouds generated by each group of four-view images to obtain dense three-dimensional point cloud data of the scene, build a three-dimensional model, form a three-dimensional stereo image and store it in the database module. Step S5: The information storage and display module processes the 3D image to complete the 3D modeling of the roof of the rail transit vehicle; In step S2, scene texture is enhanced by selecting a black and white checkerboard pattern to increase the texture features of the target and extracting corner points as feature points for stereo matching. A batch of checkerboard patterns, each with different corner positions or phases, is projected onto a target scene. These checkerboard patterns have phase shifts in either the horizontal or vertical direction. In the horizontal direction, starting with the first checkerboard pattern from the top left, each pattern is moved to the right by the same amount of displacement. Patterns with different phases were obtained. Combining the periodic distribution of squares in the chessboard and the side length of the squares, a total of... A unique pattern is equivalent to adjusting the spacing between corner points in the horizontal direction. The subdivisions show a denser distribution of corner points; Each time, the same amount of downward displacement is moved in the vertical direction. A total of A unique pattern is equivalent to adjusting the spacing between the corner points in the vertical direction. Subdivision; Pick ,but This produces a checkerboard pattern array containing Each element has a unique checkerboard pattern. These patterns are projected onto the target scene, effectively modifying the surface texture features of objects within the scene. Secondary enhancement, through control and The size of the point cloud is used to reconstruct three-dimensional point clouds with different densities.

5. The method for 3D modeling of the roof of a rail transit vehicle based on multi-view vision according to claim 4, characterized in that, In step S2, the four-eye camera is calibrated. The linear model camera calibration does not consider the influence of distortion. The pinhole model is used to describe the camera imaging geometry and the projection equation is used to represent it. In the projection equation, using Represents the projection matrix The Line number Column elements, eliminate scale factor The following system of equations is obtained: in , , The three axes of the homogeneous coordinate system established for the camera are u and v, which are the horizontal and vertical axes of the imaging plane. Specify Thus, there are 11 unknowns. For each calibration point with known coordinates, there are the two equations mentioned above. Therefore, the world coordinates and image coordinates of more than 5 calibration points are needed to calculate the camera parameters. The calibration of the linear model is as follows: directly obtain the coordinates of the calibration points on the calibration object in the camera image, and combine them with the world coordinates of the calibration points to calculate the matrix using the least squares method. The parameters are obtained by decomposing the matrix. Obtain the camera's internal and external parameters.

6. The method for 3D modeling of the roof of a rail transit vehicle based on multi-view vision according to claim 4, characterized in that, In step S4, the bicubic interpolation method is improved by performing interpolation calculations based on the grayscale values ​​of the surrounding 8 pixels, taking into account the influence of pixel disparity values ​​and the disparity weighting caused by the distance between each neighboring point. The specific steps are as follows: (1) The disparity fill value is calculated using an improved bicubic interpolation method, as shown in the following formula: in, This represents the pixel value of the point to be interpolated. Indicates the effective parallax value of the surrounding area. and These represent the weights of the horizontal and vertical axes, respectively. (2) Calculate the weights for the X and Y directions respectively. The formulas for calculating the weights of the horizontal and vertical axes are as follows: The disparity fill value is calculated using the mean, and the variance of the eight valid disparity values ​​surrounding the pixel to be interpolated is calculated. If variance If the value is less than the set threshold, the average of the eight valid disparity values ​​is used as the disparity fill value; otherwise, bicubic interpolation is used for calculation.

7. The method for 3D modeling of the roof of a rail transit vehicle based on multi-view vision according to claim 4, characterized in that, In step S4, dense 3D point cloud data of the scene is acquired, and the texture features of the target scene are enhanced by projecting a checkerboard pattern, thereby obtaining a dense 3D point cloud of the target scene. Before generating the 3D model, a point cloud simplification operation is performed to reduce the number of points, thereby improving the processing speed of subsequent point cloud triangulation and reducing the size of the 3D model. A point cloud simplification method based on spatial plane fitting is used. By fitting a spatial plane to the point cloud in a local area, if the distance from the points in the area to the fitted plane is less than a set threshold, the spatial distribution of points in the area is considered to be flat; otherwise, the spatial distribution of points in the area is considered to be turbulent. The equation of the space plane is expressed as follows: Among them, parameters It is a constant describing the characteristics of planar space; The coordinate position measured by the four-eye system will not be located at... Therefore, it is believed that Divide both sides of the space equation by . Transform it into the following forms: in , , The fitting plane needs to be solved , and The value of is used to solve the plane equation using the least squares method; set up For the region One point, The three-dimensional coordinates of each point are Ideally, it should meet the following conditions. In reality, there is a bias. The sum of squares of the total biases of all points on the fitted plane equation is: To minimize S, after completing the spatial plane fitting, calculate the distance from each point to the plane: in, The distance from each point to the plane; Preserve the boundary points within the region and the point closest to the centroid of all other points. The characteristic of boundary points is that the polygon formed by connecting adjacent boundary points includes all other points except the boundary points. For regions that need to be simplified in point cloud operations, project all points within the region onto the UL image and find the boundary points within the region and the point closest to the centroid of all other points on the UL image. Let the image coordinates of each point in the region be... Calculate the position of the center of gravity As shown below: in Given the pixel coordinates of the center position, iterate through all points to find the point that is the same as the center position. The closest points.

Images