Image generation method, system and device for subway tunnel clearance, and storage medium

Abstract

Disclosed in the present invention are an image generation method, system and device for subway tunnel clearance, and a storage medium. The method comprises: a LiDAR collecting three-dimensional laser point cloud data along a railway, and an odometer collecting encoder data along the railway; determining current subway tunnel section point cloud data on the basis of a subway clearance detection project, the three-dimensional laser point cloud data and the encoder data; determining a clearance violation area on the basis of the current subway tunnel section point cloud data and clearance; optimizing intrinsic and extrinsic parameters of a camera on the basis of point cloud information corresponding to the clearance violation area, so as to obtain calibrated intrinsic and extrinsic parameters of the camera; and on the basis of the calibrated intrinsic and extrinsic parameters, generating a camera projection map from the current subway tunnel section point cloud data, and constructing an image of subway tunnel clearance on the basis of the camera projection map. The present invention realizes, on the basis of point cloud data and clearance, track clearance violation detection during a high-speed inspection process, thus facilitating inspection and viewing by means of constructing an image of subway tunnel clearance.

Description

Method, system, device and storage medium for generating images of subway tunnel limits Technical Field

[0001] The present invention relates to the field of clearance detection technology, and in particular to a method, system, device and storage medium for generating an image of a subway tunnel clearance. Background Art

[0002] Since the beginning of this century, with the rapid development of China's economy and the acceleration of urbanization, my country's urban rail transit has entered a period of great development.

[0003] With the large number of urban rail transit systems put into operation, factors such as high line traffic density, high maintenance standards, high maintenance workload, and short maintenance time have put forward higher requirements for subway track clearance detection. During operation, traditional track clearance detection methods mainly use contact measurement of simulated clearance frames or manual measurement at certain intervals using a tape measure. These methods have low safety, low operating efficiency, and measurement accuracy that cannot meet the requirements. Therefore, it is very necessary to introduce high-tech, improve detection methods, and improve detection efficiency. Therefore, how to use new measurement technologies to realize the automation of the clearance detection process and quickly output measurement results has become an urgent problem to be solved.

[0004] The above content is only used to assist in understanding the technical solution of the present invention and does not constitute an admission that the above content is prior art.

[0005] Summary of the Invention

[0006] The main purpose of the present invention is to provide a method, system, device and storage medium for generating images of subway tunnel limits, aiming to solve the technical problem of how to use new measurement technology to realize the automation of the limit intrusion detection process and the rapid output of measurement results.

[0007] To achieve the above-mentioned object, the present invention provides a method for generating an image of a subway tunnel limit, the method comprising:

[0008] Collecting three-dimensional laser point cloud data along the railway using a laser radar, and collecting encoder data along the railway using an odometer. The laser radar consists of four mirrors and four lasers.

[0009] Determine the current subway tunnel section point cloud data based on the three-dimensional laser point cloud data and the encoder data based on the subway clearance detection project;

[0010] Determine the corresponding limit according to the mileage information of the current subway tunnel section point cloud data;

[0011] Determine an over-limit area based on the current subway tunnel section point cloud data and the limit;

[0012] Optimizing the internal and external parameters of the camera based on the point cloud information corresponding to the out-of-limit area to obtain calibrated internal and external parameters of the camera;

[0013] A camera projection image is generated from the current subway tunnel section point cloud data according to the calibrated internal and external parameters, and an image of the subway tunnel limit is constructed according to the camera projection image.

[0014] Optionally, the step of optimizing the intrinsic and extrinsic parameters of the camera based on the point cloud information corresponding to the out-of-limit area to obtain calibrated intrinsic and extrinsic parameters of the camera includes:

[0015] Determine the internal and external design parameters of the camera, and determine the design parameters of the built-in laser radar scanner to the camera;

[0016] Projecting the point cloud information corresponding to the over-limit area to a camera coordinate system based on the design parameters to obtain a projection mapping image;

[0017] Obtaining a camera image of the point cloud information corresponding to the out-of-limit area;

[0018] Extracting information of points of the same name from the projection map and the camera image using a maximum mutual information method;

[0019] Searching for corresponding three-dimensional points from the projection map according to the information of the points with the same name;

[0020] The internal and external design parameters of the camera are optimized according to the three-dimensional points and the camera points by using a gradient descent method to obtain calibrated internal and external parameters of the camera.

[0021] Optionally, the step of generating a camera projection image from the current subway tunnel section point cloud data according to the calibrated internal and external parameters includes:

[0022] Projecting the current subway tunnel section point cloud data onto a camera image to obtain a pixel depth value of the camera;

[0023] Taking the point cloud center of the current subway tunnel section point cloud data as the center of the circle;

[0024] A camera projection image is generated according to the circle center and the tunnel cross-section radius through the pixel depth value of the camera.

[0025] Optionally, the step of determining the current subway tunnel section point cloud data based on the three-dimensional laser point cloud data and the encoder data based on the subway clearance detection project includes:

[0026] Setting the point cloud starting mileage and the clearance detection ignore area based on the subway clearance detection project;

[0027] Importing the three-dimensional laser point cloud data and the encoder data according to the point cloud starting mileage and the clearance detection ignored area to determine the subway tunnel section point cloud data at different mileages;

[0028] The current subway tunnel cross-section point cloud data is determined according to the subway tunnel cross-section point cloud data at different mileages.

[0029] Optionally, the step of determining the corresponding limit according to the mileage information of the current subway tunnel section point cloud data includes:

[0030] Searching for the section information of the current inspection location based on the mileage information of the current subway tunnel section point cloud data and the inspection project data corresponding to the subway clearance inspection project;

[0031] The limit data of the current detection location is calculated according to the interval information of the current detection location, and the corresponding limit is determined according to the limit data.

[0032] Optionally, the step of determining the over-limit area based on the current subway tunnel section point cloud data and the limit includes:

[0033] Extracting left track data and right track data from the current subway tunnel section point cloud data;

[0034] Filter the left track data and the right track data according to a distance filtering method to obtain filtered left track data and filtered right track data;

[0035] Convert the filtered left track data and the filtered right track data according to the initial registration value to obtain left track conversion data and right track conversion data;

[0036] Determine a left-right track conversion set according to the left-track conversion data and the right-track conversion data;

[0037] Obtain left-track standard model data and right-track standard model data through standard model data according to the left-track conversion data and the right-track conversion data;

[0038] Determine a left-track and a right-track model set according to the left-track standard model data and the right-track standard model data;

[0039] Performing ICP matching on the left and right track transformation sets and the left and right track model sets to obtain a rotation matrix and an offset matrix;

[0040] The coordinates corresponding to the current subway tunnel section point cloud data are converted into a bounded data coordinate system using the initial registration value, the rotation matrix, and the offset matrix to obtain converted point cloud coordinates;

[0041] An out-of-limit area is determined according to the converted point cloud coordinates and the limit.

[0042] In addition, to achieve the above-mentioned purpose, the present invention further proposes a subway tunnel boundary image generation system, the subway tunnel boundary image generation system comprising:

[0043] An acquisition module, configured to collect three-dimensional laser point cloud data along the railway using a laser radar, and encoder data along the railway using an odometer. The laser radar consists of four mirrors and four lasers.

[0044] a determination module, configured to determine the current subway tunnel section point cloud data based on the three-dimensional laser point cloud data and the encoder data based on the subway clearance detection project;

[0045] The determination module is further configured to determine a corresponding limit based on the mileage information of the current subway tunnel section point cloud data;

[0046] The determination module is further configured to determine an over-limit area based on the current subway tunnel section point cloud data and the limit;

[0047] an optimization module, configured to optimize the intrinsic and extrinsic parameters of the camera based on the point cloud information corresponding to the out-of-limit area to obtain calibrated intrinsic and extrinsic parameters of the camera;

[0048] A generation module is used to generate a camera projection image from the current subway tunnel section point cloud data according to the calibrated internal and external parameters, and to construct an image of the subway tunnel limit according to the camera projection image.

[0049] In addition, to achieve the above-mentioned purpose, the present invention also proposes a subway tunnel limit image generation device, which includes: a memory, a processor, and a subway tunnel limit image generation program stored in the memory and executable on the processor, wherein the subway tunnel limit image generation program is configured to implement the steps of the subway tunnel limit image generation method described above.

[0050] In addition, to achieve the above-mentioned purpose, the present invention also proposes a storage medium, on which a subway tunnel limit image generation program is stored. When the subway tunnel limit image generation program is executed by a processor, the steps of the subway tunnel limit image generation method described above are implemented.

[0051] The present invention first collects three-dimensional laser point cloud data along the railway using a laser radar (LAD), and encoder data along the railway using an odometer. The LIDAR, consisting of four mirrors and four lasers, then determines the current subway tunnel cross-section point cloud data based on the three-dimensional laser point cloud data and encoder data, based on the subway clearance inspection project. The over-limit area is then determined based on the current subway tunnel cross-section point cloud data and the clearance. The camera's internal and external parameters are optimized based on the point cloud information corresponding to the over-limit area to obtain the camera's calibrated internal and external parameters. Finally, a camera projection image is generated based on the calibrated internal and external parameters from the current subway tunnel cross-section point cloud data, and an image of the subway tunnel clearance is constructed based on the camera projection image. Compared to existing methods that use contact measurement with a simulated clearance box or manual measurement at fixed intervals using a tape measure, which suffer from low safety, low efficiency, and insufficient measurement accuracy, the present invention uses point cloud data and clearances to detect track intrusions during high-speed inspections, and then constructs an image of the subway tunnel clearance for easier inspection and review. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] FIG1 is a schematic diagram of the structure of an image generation device for a subway tunnel boundary in a hardware operating environment according to an embodiment of the present invention;

[0053] FIG2 is a flow chart of a first embodiment of a method for generating an image of a subway tunnel boundary according to the present invention;

[0054] FIG3 is a schematic diagram of single-line laser transmission and reception in the first embodiment of the method for generating an image of a subway tunnel boundary according to the present invention;

[0055] FIG4 is a schematic diagram of a laser radar-mounted railcar according to a first embodiment of a method for generating an image of a subway tunnel boundary according to the present invention;

[0056] FIG5 is a schematic diagram showing the positions of the point cloud and the bounding box of the first embodiment of the method for generating an image of a subway tunnel boundary according to the present invention;

[0057] FIG6 is a grayscale image of a first embodiment of a method for generating an image of a subway tunnel boundary according to the present invention;

[0058] FIG7 is a schematic diagram of a camera projection onto a grayscale image according to a first embodiment of the method for generating an image of a subway tunnel boundary according to the present invention;

[0059] FIG8 is a schematic diagram of line information generated by a first embodiment of a method for generating an image of a subway tunnel boundary according to the present invention;

[0060] FIG9 is a structural block diagram of a first embodiment of a system for generating images of subway tunnel boundaries according to the present invention.

[0061] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0062] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0063] 1 , which is a schematic diagram of the structure of an image generation device for a subway tunnel boundary in a hardware operating environment according to an embodiment of the present invention.

[0064] As shown in FIG1 , the image generation device for the subway tunnel boundary may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to implement communication between these components. The user interface 1003 may include a display screen (Display) and an input unit such as a keyboard (Keyboard). Optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a wireless fidelity (Wi-Fi) interface). The memory 1005 may be a high-speed random access memory (RAM) or a stable non-volatile memory (NVM), such as a disk storage. The memory 1005 may also be a storage system independent of the aforementioned processor 1001.

[0065] Those skilled in the art will understand that the structure shown in FIG1 does not constitute a limitation on the image generation device for subway tunnel boundaries, and may include more or fewer components than shown in the figure, or a combination of certain components, or a different arrangement of components.

[0066] As shown in FIG. 1 , the memory 1005 as a storage medium may include an operating system, a network communication module, a user interface module, and a program for generating an image of a subway tunnel boundary.

[0067] In the subway tunnel limit image generation device shown in Figure 1, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the subway tunnel limit image generation device of the present invention can be set in the subway tunnel limit image generation device, and the subway tunnel limit image generation device calls the subway tunnel limit image generation program stored in the memory 1005 through the processor 1001 and executes the subway tunnel limit image generation method provided by the embodiment of the present invention.

[0068] An embodiment of the present invention provides a method for generating an image of a subway tunnel boundary. Referring to FIG. 2 , FIG. 2 is a flow chart of a first embodiment of the method for generating an image of a subway tunnel boundary according to the present invention.

[0069] In this embodiment, the method for generating an image of a subway tunnel boundary includes the following steps:

[0070] Step S10: Collect three-dimensional laser point cloud data along the railway line through a laser radar, and collect encoder data along the railway line through an odometer. The laser radar consists of four mirrors and four lasers.

[0071] It is easy to understand that the execution subject of this embodiment can be a subway tunnel limit image generation system with functions such as data processing, network communication and program running, or other computer equipment with similar functions, etc., and this embodiment is not limited.

[0072] Track intrusion detection equipment primarily consists of acquisition sensors and communication modules, power supply, control and storage, and acquisition and detection software. The acquisition sensors and communication module primarily consists of a camera system, fill light system, odometer, and lidar system; the power supply, control and storage primarily comprises a synchronization controller, workstation host, storage unit, and power conversion module; and the acquisition and detection software primarily comprises a data acquisition module, data fusion module, limit detection module, output and display module, and remote operation software.

[0073] Refer to Figures 3 and 4. Figure 3 is a schematic diagram of the single-line laser transmission and reception of the first embodiment of the subway tunnel clearance image generation method of the present invention. Figure 4 is a schematic diagram of the laser radar mounted on a railcar in the first embodiment of the subway tunnel clearance image generation method of the present invention. In this figure, the laser radar is installed in front of the railcar. If a camera is required, the position of the camera and laser radar must be calibrated in advance for camera and laser radar fusion. Connect the devices and observe whether the laser radar and camera can export point cloud data and camera data in real time.

[0074] It should also be noted that the primary factor limiting track intrusion detection speed is the laser radar's point frequency. Currently, the frequency of laser radars generally hovers around 2 million points, making it difficult to increase detection speed. This embodiment uses a laser radar with an 8 million point frequency, which can meet high-speed inspection requirements. The laser radar's motor speed is 12,000 rpm. Composed of four mirrors and four lasers, the laser radar can scan a single rotation, generating a four-line point cloud and increasing point cloud density.

[0075] It should also be understood that the urban rail transit high-speed clearance detection method includes: Step 1) Installing a LiDAR sensor in front of the railcar. If a camera is required, the camera and LiDAR positions must be calibrated in advance for camera and LiDAR fusion. Step 2) Connecting the devices and observing whether the LiDAR and camera can export point cloud data and camera data in real time.

[0076] It should also be noted that step 3) collects 3D laser point cloud data along the railway using LiDAR, and collects encoder (DMI) data along the railway using odometer. Step 4) obtains the line data and bounding box data corresponding to the currently detected subway tunnel.

[0077] It should be understood that in step 4), different radii of the subway lines correspond to different bounding box data.

[0078] Furthermore, the line data corresponding to the current subway tunnel detection mainly include section, curve type, starting mileage, ending mileage, radius (mm), deviation (left or right), outer widening (mm), inner widening (mm), section (inside or outside the tunnel), and platform type (section, underground island platform or side platform).

[0079] Bounding box data: Provides equipment limit and vehicle limit data corresponding to different curve radii (two-dimensional coordinates corresponding to each vertex of the bounding box), or provides straight segment equipment limit and vehicle limit data as well as outer widening values ​​and inner widening values ​​corresponding to different interval radii.

[0080] Furthermore, the line data is edited to obtain an interval limit file; the limit frame data is edited to obtain a limit frame file; and a subway limit detection project is created based on the interval limit file and the limit frame file.

[0081] In a specific implementation, step 5) edits the collected subway line data and bounding box data, edits the line data tables corresponding to different mileage ranges and the bounding box data tables corresponding to different radii (the two-dimensional coordinates of each vertex of the bounding box).

[0082] Step 5) Edit the route data. Write a line record for each mileage interval according to the route data format provided in step 4), and complete the two route data files for the left and right routes; edit a bounding box data for each radius.

[0083] It should also be understood that, in step 6), a subway clearance inspection project is created, with one inspection project for each subway line.

[0084] The creation of the subway clearance project in step 6) is mainly to associate and manage the edited line data and clearance data.

[0085] In this embodiment, the method of creating the subway clearance detection project in step 6) is to add the left and right line data of the subway track line respectively, and the data format is the format after editing in step 5).

[0086] Furthermore, adding device limits is divided into two cases:

[0087] 1. The radius of all curves along the entire line is the same equipment limit: In this case, two equipment limits, one inside the tunnel and one outside the tunnel, need to be added. If there is no equipment outside the tunnel, the equipment limits inside and outside the tunnel are the same.

[0088] 2. The entire line has multiple equipment limits according to the curve radius: In this case, all provided equipment limits need to be added. If there is no equipment limit outside the tunnel, the equipment limit outside the tunnel is consistent with the equipment limit inside the tunnel.

[0089] Adding subway equipment limits requires three parameters: start radius, end radius, and limit file. If the current line equipment provides adding subway straight segment limits, radius 150 limits, radius 400 limits, radius 650 limits, radius 1000 limits, radius 1200 limits, radius 2000 limits, and radius 3000 limits, then the addition method is:

[0090] 1. Radius 0-1 straight line segment limit

[0091] 2. Radius 2-399 radius 150 limit

[0092] 3. Radius 400-649, radius 400 limit

[0093] 4. Radius 650-999, radius 650 limit

[0094] 5. Radius 1000-1199, radius 1000 limit

[0095] 6. Radius 1200-1999 radius 1200 limit

[0096] 7. Radius 2000-2998, radius 2000 limit

[0097] 8. Radius 2999-3000 radius 3000 limit

[0098] Add vehicle limits, which are divided into vehicle limits inside the tunnel and vehicle limits outside the tunnel. If there is no outside the tunnel, the vehicle limits inside the tunnel will replace the vehicle limits outside the tunnel.

[0099] Add track type, currently there are two main types: 60 tracks and 50 tracks. The file here is the 50 track or 60 track contour vertex coordinates (2D data)

[0100] Based on the data selected by the user, an inspection project is automatically created. This project manages all mileage data and limit frame data of the line, and serves as the basis for engineering management of line limit inspection and automatic mileage switching.

[0101] Step S20: determining the current subway tunnel section point cloud data based on the subway clearance detection project according to the three-dimensional laser point cloud data and the encoder data.

[0102] Furthermore, the starting mileage of the point cloud and the area ignored for the limit detection are set based on the subway limit detection project; the three-dimensional laser point cloud data and encoder data are imported according to the starting mileage of the point cloud and the area ignored for the limit detection to determine the point cloud data of the subway tunnel section at different mileages; the current subway tunnel section point cloud data is determined according to the point cloud data of the subway tunnel section at different mileages.

[0103] In this embodiment, step 7) opens the inspection project, sets relevant input parameters, imports subway tunnel laser point cloud data, and displays subway tunnel cross-section point cloud data at different mileages in real time. Step 8) sets the starting mileage of the point cloud, sets the ignored area for clearance detection, and performs initial track matching. The starting mileage of the point cloud is the actual mileage of the subway tunnel at the start of the inspection; the ignored area for clearance detection is the location where the inspection process does not require output of the limit violation, such as the contact network and third rail; the initial track matching provides basic alignment parameter values ​​for track alignment during the automatic tunnel violation detection process. After the initial matching, the offset data of the point cloud in the X and Y directions relative to the standard rail head are dx and dy, respectively.

[0104] Step S30: determining a corresponding limit according to the mileage information of the current subway tunnel section point cloud data.

[0105] Calculate the mileage information of the current subway tunnel section point cloud data, and determine the corresponding limit based on the mileage information of the current subway tunnel section point cloud data.

[0106] The mileage information of the current subway tunnel section point cloud data is calculated using a mileage formula.

[0107] Mileage formula: M=M b +k*||(pp b )|| / n*C

[0108] Where, M is the mileage information of the current subway tunnel section point cloud data, M b is the actual mileage at the detection starting position, k is the mileage direction of the track intrusion detection equipment, p is the number of encoder pulses at the current detection position, and p b is the number of encoder pulses at the starting mileage, n is the number of encoder pulses when the track intrusion detection device wheel rotates one circle, and C is the outer contour circumference of the track intrusion detection device wheel.

[0109] The processing method for determining the corresponding limit based on the mileage information of the current subway tunnel section point cloud data is to search the interval information of the current detection location based on the mileage information of the current subway tunnel section point cloud data and the inspection project data corresponding to the subway limit inspection project; calculate the limit data of the current detection location based on the interval information of the current detection location, and determine the corresponding limit based on the limit data.

[0110] In a specific implementation, step 9) automatically calculates the mileage information at the current mileage and automatically switches the limit corresponding to the current mileage.

[0111] To automatically switch the limit corresponding to the current mileage, the relevant parameters set in step 7) are as follows:

[0112] 1. Whether the direction of the mobile measurement system is consistent with the direction of the line data;

[0113] 2. Side platform position value;

[0114] 3. The mileage direction of the mobile measurement system (long mileage or short mileage).

[0115] The specific implementation process of step 9) is as follows:

[0116] Step 9.1 Calculate the mileage information M of the current tunnel section point cloud data. The calculation formula is as follows: M=M b +k*||(pp b )|| / n*C

[0117] The parameters are described as follows:

[0118] M is the mileage information of the current subway tunnel section point cloud data, M b is the actual mileage at the detection starting position, k is the mileage direction of the track intrusion detection device, when the track intrusion detection device is traveling in the direction of large mileage, k = 1; when the mobile measurement system is traveling in the direction of small mileage, k = -1, p is the number of encoder pulses at the current detection position, p b is the number of encoder pulses at the starting mileage, n is the number of encoder pulses when the track intrusion detection device wheel rotates one circle, and C is the outer contour circumference of the track intrusion detection device wheel.

[0119] Step 9.2 calculates the mileage data M of the current detection location according to step 9.1 and the detection engineering data opened in step 7), and automatically calculates and finds the interval information (direction, radius r, and platform type) of the current detection location and the vertex set S1 (x i ,y i), i∈(1, n), n is the number of vertices of the bounding box; the vertex set S2(x i ,y i ), i∈(1, n), and obtain the device limit data vertex set S3(x i ,y i ),i∈(1,n).

[0120] Step 9.3: Calculate the limit data at the current detection position using the data obtained in step 9.2. The calculation can be divided into two cases:

[0121] If the section information of the current detection location is a side platform or an underground island platform, the calculation method is as follows:

[0122] If it is a side platform, and the parameter setting is that the side platform is on the left side of the moving measurement system, then the left side of the current limit data is the vertex set S1(x i ,y i ) i The set of points S3(x i ,y i ), where i∈(1, n), and the right side is the vertex set S2(x i ,y i ) i The set of points S4(x i ,y i ), then the point set S4(x i ,y i )∈(S4∪S3) is the limit data at the current detection position.

[0123] If it is a side platform, and the parameter setting is that the side platform is on the right side of the moving measurement system, then the left side of the current limit data is the vertex set S2(x i ,y i ) i The set of points S3(x i ,y i ), where i∈(1, n), and the right side is the vertex set S1(x i ,y i ) i The set of points S4(x i ,y i ), then the point set S4(x i ,y i )∈(S4∪S3) is the limit data at the current detection position.

[0124] If it is an underground island platform and the parameter setting is that the side station is on the left side of the moving measurement system's direction of travel, then the left side of the current limit data is the vertex set S2(x i ,y i ) i The set of points S3(x i ,y i ), where i∈(1, n), and the right side is the vertex set S1(x i ,y i ) i The set of points S4(x i ,y i ), then the point set S4(x i ,y i )∈(S4∪S3) is the limit data at the current detection position.

[0125] If it is a side station, and the parameter setting is that the side station is on the right side of the moving measurement system's direction of travel, then the left side of the current limit data is the vertex set S1(x i ,y i ) i The set of points S3(x i ,y i ), where i∈(1, n), and the right side is the vertex set S2(x i ,y i ) i The set of points S4(x i ,y i ), then the point set S4(x i ,y i )∈(S4∪S3) is the limit data at the current detection position.

[0126] If the section information of the current detection location is a side platform or an underground island platform, the calculation method is as follows:

[0127] If the direction of the mobile measurement system is consistent with the direction of the line data, and the current deflection direction is left, then the current limit data is the vertex set S3(x i ,y i ), where i∈(1,n).

[0128] If the moving direction of the mobile measurement system is consistent with the direction of the line data, and the current deflection direction is right, then the vertex set S3(x i ,y i ) i =x i *-1, the new set S4(x i ,y i ) is the current limit data. Where i∈(1,n).

[0129] If the moving direction of the mobile measurement system is opposite to the line data direction, and the current deflection direction is left, then the current limit data is the vertex set S3(x i ,y i ), where i∈(1,n).

[0130] If the moving direction of the mobile measurement system is opposite to the direction of the line data, and the current deflection direction is right, then the vertex set S3(x i ,y i ) i =x i *-1, the new set S4(x i ,y i ) is the current limit data. Where i∈(1,n).

[0131] In step 9.4, the limit data calculated in step 9.3 is displayed in real time on the software display interface. The limit is then obtained based on the limit data.

[0132] Step S40: determining an over-limit area based on the current subway tunnel section point cloud data and the limit.

[0133] Further, the left track data and the right track data are extracted from the current subway tunnel section point cloud data; the left track data and the right track data are filtered according to the distance filtering method to obtain filtered left track data and filtered right track data; the filtered left track data and the filtered right track data are respectively converted according to the initial alignment value to obtain left track conversion data and right track conversion data; the left and right track conversion sets are determined according to the left track conversion data and the right track conversion data; the left track standard model data and the right track standard model data are obtained through the standard model data according to the left track conversion data and the right track conversion data; the left and right track model sets are determined according to the left track standard model data and the right track standard model data; ICP matching is performed on the left and right track conversion sets and the left and right track model sets to obtain a rotation matrix and an offset matrix; the coordinates corresponding to the current subway tunnel section point cloud data are converted to the limit data coordinate system through the initial alignment value, the rotation matrix and the offset matrix to obtain the converted point cloud coordinates; the track intrusion detection is implemented according to the converted point cloud coordinates and the limit, and the over-limit area is determined.

[0134] In this embodiment, step 10) automatically aligns the track in real time to unify the subway tunnel section point cloud data and the current limit data coordinate system.

[0135] The specific implementation steps of real-time automatic track registration are as follows:

[0136] Step 10.1 Get the current detection section point cloud data P1(x i ,y i), where i∈(1, n), according to the structural design parameters of the mobile measurement system, the left track data L(x i ,y i ), where i∈(1, n1) and the right track data R(x i ,y i ), where i∈(1,n2).

[0137] Step 10.2: Filter the left track data L and the right track data R according to the distance filtering method. Set the filtering distance to 0.01 and the filtering threshold to 5. Iterate and calculate the distance between each point and itself and the 100 points around it. If there are less than 5 points with a distance less than 0.01, the point is removed to obtain the filtered left track data L1(x i ,y i ), where i∈(1, n3) and the filtered right track data R1(x i ,y i ), where i∈(1,n4).

[0138] Step 10.3: The filtered left track data L1(x i ,y i ) and the filtered right track data R1(x i ,y i ) to convert: x i =x i +dx y i =y i +dy

[0139] Get the converted left track data (left track conversion data) L2(x i ,y i )i∈(1,n3) and the converted right track data (right track conversion data) R2(x i ,y i )i∈(1,n4).

[0140] Step 10.4 Read the standard model data M(x i ,y i ), copy the model data and place it symmetrically according to the center C (0, 0) and the standard track gauge to obtain the standard model data M for the left and right tracks L (x i ,y i ) and M R (x i ,y i ), where i∈(1,n5).

[0141] Step 10.5 obtains the left track conversion data L2(x i ,yi )i∈(1,n3) and right track conversion data R2(x i ,y1)i∈(1,n4) of the left and right orbit transformation set G(x i ,y i )∈(L2∪R2)i∈(1,n6), and at the same time obtain the standard model data M for the left and right tracks in step 10.4 L (x i ,y i )i∈(1,n5) and M R (x i ,y i )i∈(1,n5) of the left and right track model set G1(x i ,y i )∈(M L ∪M R )i∈(1,n7).

[0142] Step 10.6: The point set G(x i ,y i )i∈(1,n6) and the point set G1(x i ,y i )i∈(1,n7) performs ICP matching, and the specific algorithm implementation is as follows:

[0143] ① Select point set G from the target point set G i ∈G

[0144] ② Find the corresponding point set Q in the source point set G1 i ∈G1, such that ||G i -Q i ||=min;

[0145] ③Calculate the rotation matrix R and translation matrix T to minimize the error function;

[0146] ④To G i Use the rotation matrix R and translation matrix T obtained in the previous step to perform rotation and translation transformations to obtain a new corresponding point set G i '={G i '=R*G i +T,G i ∈G};

[0147] ⑤Calculate G i ' and the corresponding point set Q i The average distance:

[0148] ⑥ If d is less than a given threshold or greater than the preset maximum number of iterations, stop the iterative calculation. Otherwise, return to step ② until the convergence condition is met.

[0149] ⑦ Get the final rotation matrix R and offset matrix T: T=[t x t y ] T

[0150] Step 10.7 uses the initial registration data calculated in step 8) and the rotation matrix and offset matrix calculated in step 10.6 to obtain the current detection section point cloud data P1 (x i ,y i ), the coordinates of i∈(1,n) are transformed into the bounded data coordinate system, unifying the coordinate systems of the bounded data and the point cloud data.

[0151] Refer to Figure 5, which is a schematic diagram of the positions of the point cloud and the limit box of the first embodiment of the image generation method for the subway tunnel limit of the present invention. It is detected whether point cloud data appears in the limit. If so, it proves that the limit is violated. If not, it proves that the limit is not violated.

[0152] In the specific implementation, step 11) limit detection searches for point cloud data within the limit box. The point cloud data is represented as subway limit violation data, and its maximum violation distance, number of violation positions, and minimum height of violation positions are calculated.

[0153] The specific implementation steps of limit detection are as follows:

[0154] Step 11.1 Obtain the current detected subway tunnel section point cloud data P(x i ,y i ),i∈(1,n), where n is the number of points in the point cloud data.

[0155] Step 11.2: Convert the coordinates of the point cloud data according to step 10).

[0156] Step 11.3: Calculate the set of points Q within the bounding box data using the point-in-polygon algorithm. i (x j ,y j ), i∈(1, n), j∈(1, m n ), where n represents the number of intrusions, m n Represents the number of points at each intrusion position, if the point set Q i If it is empty, it means there is no intrusion at this mileage, and the process returns to step 9) to continue detecting the intrusion information of the subway tunnel section at the next mileage. If there is intrusion, the process proceeds to the next step.

[0157] Step 11.4 From each point set Q i (x j ,y j ), i∈(1, n), j∈(1, mn ) Find the limit point, if all or some of the points in the point set have x j ≤0, then get the x in the point set j The largest point P1(x, y); if the x of all points in the point set j ≥0, then get the point x in the set j The smallest point P2(x, y).

[0158] Step 11.5 If the point calculated in step 11.4 is point P1 (x, y), then search and calculate the distance D1 from P1 to the bounding box from the left half of the bounding box; if the point calculated in step 11.4 is point P2 (x, y), then search and calculate the distance D2 from P2 to the bounding box from the right half of the bounding box, and D1 or D2 is the maximum intrusion distance of the intrusion position.

[0159] Step 11.6 Refer to steps 11.4-11.5 to calculate the set Q of all points i (x j ,y j ), i∈(1, n), j∈(1, m n )’s maximum intrusion distance.

[0160] Step S50: Optimizing the intrinsic and extrinsic parameters of the camera based on the point cloud information corresponding to the out-of-limit area to obtain calibrated intrinsic and extrinsic parameters of the camera.

[0161] Furthermore, the internal and external parameters of the camera are optimized based on the point cloud information corresponding to the out-of-limit area. The processing method for obtaining the calibrated internal and external parameters of the camera is to determine the internal and external design parameters of the camera, and determine the design parameters of the lidar built-in scanner to the camera; based on the design parameters, the point cloud information corresponding to the out-of-limit area is projected to the camera coordinate system to obtain a projection mapping diagram; a camera diagram of the point cloud information corresponding to the out-of-limit area is obtained; information of points of the same name is extracted from the projection mapping diagram and the camera diagram by the maximum mutual information method; corresponding three-dimensional points are searched from the projection mapping diagram according to the information of points of the same name; the internal and external design parameters of the camera are optimized by the gradient descent method according to the three-dimensional points and the camera points to obtain the calibrated internal and external parameters of the camera.

[0162] In a specific implementation, in step 12), the bounded detection area can project the camera image to obtain a projection image to assist in viewing the results.

[0163] The steps for projecting the camera onto the point cloud in step 12 are as follows:

[0164] Step 12.1: For the over-limit area, extract the relative information of the camera and point cloud information based on the mileage information and perform calibration.

[0165] Step S60: generating a camera projection image from the current subway tunnel cross-section point cloud data according to the calibrated internal and external parameters, and constructing an image of the subway tunnel boundary according to the camera projection image.

[0166] Furthermore, a processing method for generating a camera projection image from the current subway tunnel section point cloud data according to the calibrated internal and external parameters is to project the current subway tunnel section point cloud data onto a camera image to obtain the pixel depth value of the camera; the point cloud center of the current subway tunnel section point cloud data is used as the center of the circle; and the camera projection image is generated according to the pixel depth value of the camera based on the center of the circle and the radius of the tunnel section.

[0167] In the specific implementation, in step 12.2, to facilitate viewing of the camera and point cloud information, the point cloud information is converted to a grayscale image, and the camera is projected onto the grayscale image. Referring to Figures 6 and 7 , Figure 6 shows a grayscale image of the first embodiment of the method for generating an image of a subway tunnel boundary according to the present invention, and Figure 7 shows a schematic diagram of the camera projected onto the grayscale image according to the first embodiment of the method for generating an image of a subway tunnel boundary according to the present invention.

[0168] It should also be noted that when the track intrusion detection result is an intrusion, the image result corresponding to the mileage information of the current subway tunnel section point cloud data is obtained, and the subway intrusion data within the limit range is determined based on the converted point cloud coordinates; the maximum intrusion distance, the number of intrusion positions and the minimum height information of the intrusion positions are determined based on the subway intrusion data; the current subway tunnel limit detection report is generated based on the maximum intrusion distance, the number of intrusion positions and the minimum height information of the intrusion positions; when the track intrusion detection is completed, the current subway tunnel limit detection report and the image result are output, refer to Figure 8, Figure 8 is a schematic diagram of the line information generated by the first embodiment of the image generation method for the subway tunnel limit of the present invention.

[0169] Step 13) After the clearance detection is completed, the subway tunnel clearance detection report and image results are automatically output.

[0170] Step 14) Manual spot checks and verification of clearance inspection results. Currently, manual spot checks are conducted for areas with large clearance distances or a large number of clearance violations. The main existing manual inspection method is to use the DJJ-8 laser catenary detector, a railway-specific laser rangefinder with an inspection accuracy of less than 1mm. The final clearance inspection accuracy of this method is verified to be 3mm.

[0171] In this embodiment, three-dimensional laser point cloud data along the railway line is first collected using a laser radar (LAD), and encoder data along the railway line is collected using an odometer. The LIDAR, which consists of four mirrors and four lasers, then determines the current subway tunnel cross-section point cloud data based on the three-dimensional laser point cloud data and encoder data, based on the subway clearance inspection project. The overrun area is then determined based on the current subway tunnel cross-section point cloud data and the clearance. Based on the point cloud information corresponding to the overrun area, the camera's intrinsic and extrinsic parameters are optimized to obtain calibrated intrinsic and extrinsic parameters. Finally, a camera projection image is generated from the calibrated intrinsic and extrinsic parameters, and an image of the subway tunnel clearance is constructed based on the camera projection image. Compared to existing methods that use contact measurement with simulated clearance boxes or manual measurement at fixed intervals using a tape measure, which suffer from low safety, inefficiency, and insufficient measurement accuracy, this embodiment uses point cloud data and clearances to detect track intrusions during high-speed inspections. Furthermore, by constructing an image of the subway tunnel clearance, it facilitates inspection and review.

[0172] 9 , which is a structural block diagram of a first embodiment of a system for generating images of subway tunnel boundaries according to the present invention.

[0173] As shown in FIG9 , the subway tunnel boundary image generation system proposed in an embodiment of the present invention includes:

[0174] The acquisition module 9001 is used to collect three-dimensional laser point cloud data along the railway through a laser radar, and to collect encoder data along the railway through an odometer. The laser radar consists of four mirrors and four lasers.

[0175] The determination module 9002 is used to determine the current subway tunnel section point cloud data based on the subway clearance detection project according to the three-dimensional laser point cloud data and the encoder data.

[0176] The determination module 9002 is further configured to determine a corresponding limit based on the mileage information of the current subway tunnel section point cloud data.

[0177] The determination module 9002 is further configured to determine an over-limit area based on the current subway tunnel section point cloud data and the limit.

[0178] The optimization module 9003 is used to optimize the intrinsic and extrinsic parameters of the camera based on the point cloud information corresponding to the out-of-limit area to obtain the calibrated intrinsic and extrinsic parameters of the camera.

[0179] The optimization module 9003 is also used to determine the internal and external design parameters of the camera, and determine the design parameters of the built-in scanner of the laser radar to the camera; project the point cloud information corresponding to the out-of-limit area to the camera coordinate system based on the design parameters to obtain a projection mapping diagram; obtain a camera image of the point cloud information corresponding to the out-of-limit area; extract the same-name point information based on the projection mapping diagram and the camera image through the maximum mutual information method; search for the corresponding three-dimensional point from the projection mapping diagram based on the same-name point information; optimize the internal and external design parameters of the camera through the gradient descent method based on the three-dimensional points and the camera points to obtain the calibrated internal and external parameters of the camera.

[0180] The generating module 9004 is used to generate a camera projection image from the current subway tunnel section point cloud data according to the calibrated internal and external parameters, and construct an image of the subway tunnel limit according to the camera projection image.

[0181] The generation module 9004 is also used to project the current subway tunnel section point cloud data onto a camera image to obtain the pixel depth value of the camera; use the point cloud center of the current subway tunnel section point cloud data as the center of the circle; and generate a camera projection image based on the pixel depth value of the camera according to the center of the circle and the tunnel section radius.

[0182] In this embodiment, three-dimensional laser point cloud data along the railway line is first collected using a laser radar (LAD), and encoder data along the railway line is collected using an odometer. The LIDAR, which consists of four mirrors and four lasers, then determines the current subway tunnel cross-section point cloud data based on the three-dimensional laser point cloud data and encoder data, based on the subway clearance inspection project. The overrun area is then determined based on the current subway tunnel cross-section point cloud data and the clearance. Based on the point cloud information corresponding to the overrun area, the camera's intrinsic and extrinsic parameters are optimized to obtain calibrated intrinsic and extrinsic parameters. Finally, a camera projection image is generated from the calibrated intrinsic and extrinsic parameters, and an image of the subway tunnel clearance is constructed based on the camera projection image. Compared to existing methods that use contact measurement with simulated clearance boxes or manual measurement at fixed intervals using a tape measure, which suffer from low safety, inefficiency, and insufficient measurement accuracy, this embodiment uses point cloud data and clearances to detect track intrusions during high-speed inspections. Furthermore, by constructing an image of the subway tunnel clearance, it facilitates inspection and review.

[0183] Other embodiments or specific implementations of the subway tunnel boundary image generation system of the present invention can refer to the above-mentioned method embodiments and will not be repeated here.

[0184] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or system comprising a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or system. In the absence of further limitations, an element defined by the phrase "comprising a..." does not preclude the presence of other identical elements in the process, method, article, or system comprising the element.

[0185] The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

[0186] Through the description of the above embodiments, those skilled in the art can clearly understand that the above embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better embodiment. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as read-only memory / random access memory, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0187] The above are only preferred embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention description and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. An image generation method for the clearance of a subway tunnel, characterized in that The method for generating an image of the subway tunnel clearance includes the following steps: Collect three-dimensional laser point cloud data along the railway through a lidar, and collect encoder data along the railway through an odometer. The lidar consists of four mirrors and four lasers; Based on the subway clearance detection project, determine the current subway tunnel cross-section point cloud data according to the three-dimensional laser point cloud data and the encoder data; Determine the corresponding clearance according to the mileage information of the current subway tunnel cross-section point cloud data; Determine the over-limit area according to the current subway tunnel cross-section point cloud data and the clearance; Optimize the internal and external parameters of the camera based on the point cloud information corresponding to the over-limit area to obtain the calibrated internal and external parameters of the camera; Generate a camera projection map from the current subway tunnel cross-section point cloud data according to the calibrated internal and external parameters, and construct an image of the subway tunnel clearance according to the camera projection map.

2. The method according to claim 1, wherein, The step of optimizing the internal and external parameters of the camera based on the point cloud information corresponding to the over-limit area to obtain the calibrated internal and external parameters of the camera includes: Determine the internal and external design parameters of the camera, and determine the design parameters from the lidar internal scanner to the camera; Project the point cloud information corresponding to the over-limit area onto the camera coordinate system based on the design parameters to obtain a projection mapping map; Obtain the camera image of the point cloud information corresponding to the over-limit area; Extract the homologous point information according to the projection mapping map and the camera image by the maximum mutual information method; Find the corresponding three-dimensional points from the projection mapping map according to the homologous point information; Optimize the internal and external design parameters of the camera by the gradient descent method according to the three-dimensional points and the camera points to obtain the calibrated internal and external parameters of the camera.

3. The method according to claim 2, wherein The step of generating a camera projection map from the current subway tunnel cross-section point cloud data according to the calibrated internal and external parameters includes: Project the current subway tunnel cross-section point cloud data onto the camera image to obtain the pixel depth value of the camera; Take the point cloud center of the current subway tunnel cross-section point cloud data as the center of the circle; Generate a camera projection map according to the center of the circle and the tunnel cross-section radius through the pixel depth value of the camera.

4. The method according to claim 1, characterized in that The step of determining the current subway tunnel cross-section point cloud data based on the subway clearance detection project according to the three-dimensional laser point cloud data and the encoder data includes: Set the point cloud starting mileage and the clearance detection ignored area based on the subway clearance detection project; Import the three-dimensional laser point cloud data and the encoder data according to the point cloud starting mileage and the clearance detection ignored area to determine the subway tunnel cross-section point cloud data at different mileages; Determine the current subway tunnel cross-section point cloud data according to the subway tunnel cross-section point cloud data at different mileages.

5. The method according to claim 4, characterized in that The step of determining the corresponding clearance according to the mileage information of the current subway tunnel cross-section point cloud data includes: Search for the interval information at the current detection location according to the mileage information of the current subway tunnel cross-section point cloud data and the detection project data corresponding to the subway clearance detection project; Calculate the clearance data at the current detection location according to the interval information at the current detection location, and Determine the corresponding clearance according to the clearance data.

6. The method according to claim 5, wherein The step of determining the over-limit area according to the current subway tunnel cross-section point cloud data and the boundary includes: Extracting left rail data and right rail data from the current subway tunnel cross-section point cloud data; Filtering the left rail data and the right rail data according to a distance filtering method to obtain filtered left rail data and filtered right rail data; Converting the filtered left rail data and the filtered right rail data respectively according to the initial registration value to obtain left rail conversion data and right rail conversion data; Determining a left and right track conversion set according to the left rail conversion data and the right rail conversion data; Obtaining left rail standard model data and right rail standard model data from the standard model data according to the left rail conversion data and the right rail conversion data; Determining a left and right rail model set according to the left rail standard model data and the right rail standard model data; Performing ICP matching on the left and right track conversion set and the left and right rail model set to obtain a rotation matrix and an offset matrix; Converting the coordinates corresponding to the current subway tunnel cross-section point cloud data to the boundary data coordinate system through the initial registration value, the rotation matrix and the offset matrix to obtain the converted point cloud coordinates; Determining the over-limit area according to the converted point cloud coordinates and the boundary.

7. An image generation system for the clearance of a subway tunnel, characterized in that, The image generation system for the subway tunnel boundary includes: An acquisition module, configured to collect three-dimensional laser point cloud data along the railway through a lidar, and collect encoder data of the railway along the line through an odometer, where the lidar consists of four mirrors and four lasers; A determination module, configured to determine the current subway tunnel cross-section point cloud data based on the subway boundary detection project according to the three-dimensional laser point cloud data and the encoder data; The determination module is further configured to determine the corresponding boundary according to the mileage information of the current subway tunnel cross-section point cloud data; The determination module is further configured to determine the over-limit area according to the current subway tunnel cross-section point cloud data and the boundary; An optimization module, configured to optimize the internal and external parameters of the camera based on the point cloud information corresponding to the over-limit area to obtain the calibrated internal and external parameters of the camera; A generation module, configured to generate a camera projection map from the current subway tunnel cross-section point cloud data according to the calibrated internal and external parameters, and construct an image of the subway tunnel boundary according to the camera projection map. The device includes: a memory, a processor, and an image generation program for the subway tunnel boundary stored on the memory and executable on the processor, where the image generation program for the subway tunnel boundary is configured to implement the steps of the image generation method for the subway tunnel boundary according to any one of claims 1 to 6.

8. An image generation device for the clearance limit of a subway tunnel, characterized in that, The storage medium stores an image generation program for the subway tunnel boundary, and when the image generation program for the subway tunnel boundary is executed by a processor, it implements the steps of the image generation method for the subway tunnel boundary according to any one of claims 1 to 6.

9. A storage medium, characterized in that, ​

Images