Local flatness detection device and detection method using machine vision

Abstract

The invention provides a local flatness detection device and detection method using machine vision, and the method comprises the steps: enabling a vision sensing module to be aligned with a to-be-detected tunnel local lining surface, collecting the point cloud data of the to-be-detected tunnel local lining surface, and enabling an inclination angle sensing module to collect the angle deviation between a point cloud collection center and a tunnel hole wall; marking a detection range on a color image formed by the point cloud data by using a detection range frame guide algorithm to obtain a rectangular point cloud preprocessing region; assigning the coordinates of the point cloud data in the rectangular point cloud preprocessing area to obtain a point cloud array; correcting the point cloud array by using the angle deviation measured by the inclination angle sensing module; and fitting the corrected point cloud data, and calculating a flatness index by using a fitting line. According to the method, the point cloud data deviation caused by the angle inclination of the scanner can be corrected, so that an accurate flatness degree result is obtained. The handheld detection device does not need to be installed on original equipment and is suitable for all tunnel scenes.

Description

Technical Field

[0001] The present invention relates to the technical field of tunnel construction, and in particular to a local flatness detection device and a detection method using machine vision. Background Art

[0002] The primary lining support and secondary lining support of the tunnel are key steps in the tunnel forming operation and tunnel maintenance process. In order to detect the lining quality, the tunnel lining flatness detection is a necessary detection step after the lining operation. The standard requires that the flatness be checked by a ruler, and the allowable deviation of the surface flatness (the height difference between the highest point and the lowest point) is 5cm for the side wall and 7cm for the arch. At present, the flatness of the tunnel lining is mostly checked by a ruler, that is, a level ruler is placed close to the shotcrete surface, and the highest and lowest points are measured with a ruler based on the level ruler. After measurement, the difference is considered to be the flatness. This method requires 2-3 people to cooperate. At the same time, the position of the shotcrete bridge is narrow, and it is easy to be dangerous when multiple people work. In addition, the subjective factors of manual inspection with a ruler are large, and the detection accuracy is low.

[0003] In recent years, with the in-depth research on the flatness detection of tunnel lining support, relevant methods for visual detection of tunnel flatness such as laser radar and structured light camera have gradually emerged. This method uses the whole ring tunnel lining feature point cloud obtained by vision to calculate the height difference and obtain the flatness index. This method first installs the visual sensor on the tunnel ring track, obtains the feature point cloud of the tunnel lining support through rotational motion, and then performs point cloud computing. This method can calculate the flatness of the tunnel lining support, but it does not meet the requirements of the on-site flatness detection implementation standard. The tunnel construction site requires the use of a steel ruler to conduct random sampling of the local flatness of the tunnel lining. Therefore, the calculation of the feature point cloud of the whole arch tunnel lining requires the processing of a lot of data, resulting in a huge amount of data storage and calculation. At the same time, it does not have a portable function and is difficult to meet daily use needs. At the same time, the currently published detection method does not consider the angle problem between the acquisition center line of the point cloud and the tunnel wall during the visual acquisition process, which will cause the calculated flatness index to have a large error. Therefore, it is necessary to design a local flatness detection device using machine vision and a detection method that can take into account the deviation angle error.

[0004] The invention patent with application number 202311647731.0 discloses a method and system for detecting the flatness of the initial support of a tunnel, which relates to the technical field of tunnel engineering equipment, including constructing a complete initial support flatness detection system, using a 3D laser scanner to scan the inside of the tunnel, and setting up a data acquisition module, a data processing module and a flatness calculation module inside the tunnel. When in use, a complete initial support flatness detection system is constructed, and the inside of the tunnel is scanned by using a 3D laser scanner, and the scanning device is controlled by rotation and movement to obtain all-round point cloud data inside the tunnel, and the flatness of the initial support of the tunnel is detected and analyzed, and the structural integrity and reliability of the initial support are comprehensively evaluated, which not only improves the overall detection efficiency and detection accuracy, but also facilitates real-time data management and storage, provides data reference for similar detection in the future, improves the intelligent level of initial support flatness detection management, and meets the needs of normal tunnel construction. However, the above detection method takes into account the scanning angle tilt caused by machine deviation and human behavior deviation, resulting in a large calculation error in the flatness detection, and thus cannot truly feedback the flatness of the tunnel lining surface. Summary of the invention

[0005] In view of the technical problems in the prior art that the amount of data for flatness detection using vision is extremely large and the detection error is extremely large, the present invention proposes a local flatness detection device and detection method using machine vision, which uses an inclination sensor to read the angle value between the point cloud collection centerline and the tunnel wall when collecting point clouds, and uses the angle deviation to correct the point cloud data, which can meet the requirements of accurate calculation of flatness.

[0006] In order to achieve the above object, the technical solution of the present invention is implemented as follows: a local flatness detection method using machine vision, the steps of which are as follows:

[0007] Step 1: Aim the visual sensor module at the local lining surface of the tunnel to be inspected and collect point cloud data of the local lining surface of the tunnel to be inspected, and the inclination sensor module collects the angle deviation between the point cloud collection center and the tunnel wall;

[0008] Step 2: Use the detection range frame guidance algorithm to mark the detection range on the color image formed by the point cloud data to obtain a rectangular point cloud preprocessing area;

[0009] Step 3: Assign coordinates of the point cloud data in the rectangular point cloud preprocessing area to obtain a point cloud array; correct the point cloud array using the angle deviation measured by the inclination sensor module; fit the corrected point cloud data and calculate the flatness index using the fitting line.

[0010] Preferably, in the step 2, point cloud data of the selected flatness detection area is screened. Taking the 1m ruler method as an example, a detection range box guidance algorithm is used to define a 1m*1m rectangular point cloud preprocessing area in the collected point cloud data with the center of the visual sensing module as the center.

[0011] Preferably, the detection range frame guidance algorithm: points E, F, G on the local lining surface of the tunnel to be inspected are imaged as points E', F', G' on the CCD of the visual sensing module through the lens of the visual sensing module respectively; G is a point on the optical axis of the visual sensing module; F is the center point of the target area, and E is the calibration point;

[0012] Image F'G' is the image of object FG on the visual sensor module. When the area range of object FG is known, the image height of image F'G' corresponding to object FG is obtained. The number of occupied pixels is obtained according to the image height and pixel size. The border of the occupied pixels is drawn by creating a new layer, and the layer is superimposed on the color image formed by the point cloud to obtain the detection area.

[0013] Preferably, the border size of the detection area is:

[0014]

[0015] Among them, size pixel is the pixel size, FG is the size of the calculation area, F′G′ is the size of the image of the calculation area, d is the distance between the object and the lens of the visual sensor module, and f is the focal length of the lens;

[0016] Border size num pixel is a function of the distance d, based on the known size of the object EG and the number of pixels num occupied by the image height E'G' on the CCD of the vision sensor module pixel (E'G'), calculate the pixel size pixel ;and

[0017]

[0018] Image height E'G' is the size of the image of object EG formed on CCD through the lens of the visual sensor module;

[0019] Combine the following two formulas to calculate the border size num pixel for

[0020]

[0021] Preferably, the method for correcting the point cloud array using the angle deviation measured by the inclination sensor module is to use a random selection algorithm to take a 1m straight line in the selected rectangular point cloud preprocessing area, take the point cloud data covered by the 1m straight line, assign the point cloud data coordinates, and establish a point cloud array f(x, y, z). The point cloud array f(x, y, z) is corrected using the angle deviation a between the point cloud acquisition center and the tunnel wall to obtain the positive vertical incidence point cloud array: cosa dot product f(x, y, z).

[0022] Preferably, the method for calculating the flatness index is: using the positive vertical incidence point cloud array to establish a point cloud fitting baseline, and traversing the height difference of each point in the positive vertical incidence point cloud array with respect to the fitting baseline, defining points above the baseline as positive and points below the fitting line as negative, taking the maximum value D1 of the positive array and the minimum value D2 of the negative array, calculating the distance D=D1-D2, and defining the straight line spacing L between the maximum and minimum points at the same time, then the flatness index calculation result is P1=D / L.

[0023] Preferably, the residual of the normal vertical incident point cloud array and the point cloud array f(x, y, z) is calculated to obtain the difference ε in the depth direction. z , establish the residual matrix M = [x, y, z, ε z ], segment the residual data in the residual matrix M, take the point cloud that satisfies the 1m straight line, and get the new set M 1 ; for the set M 1 Perform grid filtering; calculate the average value of the point cloud in the grid and obtain the set M 2 , calculate the two-dimensional array M 2 The highest point is recorded as contact point A; calculate the lines connecting contact point A and other points, and find the point corresponding to the line with the lowest slope, recorded as contact point B; calculate the distance D from the point within line segment AB to straight line AB, as well as the distance L between contact points A and B; calculate the flatness formula P=D / L.

[0024] Preferably, the visual sensing module performs feature recognition on the area where the local lining surface of the tunnel to be inspected is located and automatically generates point cloud data;

[0025] Assign color values ​​to the point cloud data to obtain a color image of the point cloud, annotate the detection range in the color image, define the range size independently, and adaptively display the size of the detection frame according to the distance between the acquisition point and the surface to be measured;

[0026] The flatness detection area is repeatedly detected N times using a random selection algorithm to obtain a flatness index array [P] of the area, and the flatness index array [P] is averaged to obtain the flatness index P of the detection area;

[0027] If the flatness exceeds the national standard, an alarm will be issued, and the detection time, location, and detection personnel will be recorded. A flatness detection record sheet will be automatically generated and can be printed or archived.

[0028] A local flatness detection device using machine vision comprises a shell, wherein a visual sensing module, an inclination sensing module, a data storage and calculation module and a display and interaction module are arranged in the shell, the centers of the inclination sensing module and the visual sensing module are on the same horizontal line, the visual sensing module and the inclination sensing module are both connected to the data storage and calculation module, the data storage and calculation module is connected to the display and interaction module, and the data storage and calculation module is connected to a PLC controller of a tunnel boring machine; the visual sensing module realizes lining surface feature recognition and point cloud data generation, the inclination sensing module collects angle deviation data between the point cloud collection center and the tunnel wall, and transmits the angle deviation data to the data storage and calculation module to perform flatness calculation correction.

[0029] Preferably, a filter module is provided at the front side of the visual sensor module, the filter module is fixed in front of the housing, and the center of the filter module is on the same horizontal line as the center of the lens of the visual sensor module.

[0030] Preferably, the shell includes a second shell and a third shell, the third shell is provided with a display and interaction module, the third shell is provided with a data storage and computing module, the lower part of the third shell is connected to the second shell, the second shell is provided with a visual sensing module and a tilt sensing module, the visual sensing module is arranged on the front side of the second shell, and the filter module is fixed on the front side of the second shell.

[0031] Preferably, a first shell is fixed to the lower part of the second shell, a battery module is arranged in the first shell, and the battery module is respectively connected to the data storage and calculation module, the display and interaction module, the visual sensor module and the tilt sensor module;

[0032] A printing module is fixed at the lower part of the second housing, and the printing module is connected to the data storage and computing module;

[0033] The first shell and the second shell are connected by a snap-fit ​​and slot structure, and the second shell and the third shell are fixedly connected by bolts;

[0034] The visual sensor module uses a binocular structured light camera, and the binocular structured light camera is installed in the second housing by bolts.

[0035] Compared with the prior art, the present invention has the following beneficial effects: the use of the inclination sensor is for the correction of the point cloud data. The present invention is a local flatness detection device that meets the use requirements of the construction site, that is, the construction personnel hold the detection device to shoot the detection point cloud of a certain collection position that needs to be checked, and install an inclination sensor or gyroscope on the detection device to collect the absolute three-dimensional angle of the current collection position relative to the vertical standing of the person in real time, including the three directions of xyz, and then use the algorithm to correct the point cloud data, which can meet the precise calculation of flatness. The present invention can correct the deviation of the point cloud data caused by the tilt of the scanner angle, and then obtain an accurate flatness result. At the same time, the handheld detection device of the present invention does not need to be installed on the original equipment and is suitable for all tunnel scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0037] Figure 1 It is a stereogram of the present invention.

[0038] Figure 2 It is a schematic diagram of the internal structure of the present invention.

[0039] Figure 3 Flow chart of the detection method of the present invention.

[0040] Figure 4 Schematic diagram of the object-image relationship in the detection range frame guidance algorithm of the present invention.

[0041] Figure 5 Schematic diagram of the detection range frame of the present invention.

[0042] Figure 6 Schematic diagram of point cloud correction according to the present invention.

[0043] Figure 7 This is a schematic diagram of calculating the flatness index according to the present invention.

[0044] In the figure, 1 is a visual sensing module, 2 is a data storage and computing module, 3 is a display and interaction module, 4 is a printing module, 5 is a power module, 6 is a first shell, 7 is a tilt sensing module, 8 is a second shell, 9 is a third shell, 10 is a filter module, 11 is a point cloud, 12 is a point cloud array set, 13 is a point cloud fitting baseline, 14 is a tilt coordinate system, 15 is a corrected vertical coordinate system, and 16 is a tilt angle a read by the tilt sensing module. DETAILED DESCRIPTION

[0045] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0046] Example 1

[0047] like Figure 1 and Figure 2 As shown, a local flatness detection device using machine vision includes a housing, the housing includes a first housing 6, a second housing 8 and a third housing 9, the third housing 9 is provided with a display and interaction module 3, the third housing 9 is provided with a data storage and calculation module 2, the data storage and calculation module 2 is connected to the display and interaction module 3, the lower part of the third housing 9 is fixed with a second housing 8, the second housing 8 is provided with a visual sensor module 1 and a tilt sensor module 7, the visual sensor module 1 is arranged at the front end of the second housing 8, and the front end of the visual sensor module 1 is provided with a filter module 10. The visual sensor module 1 and the tilt sensor module 7 are both connected to the data storage and calculation module 2. The lower part of the second housing 8 is fixed with a first housing 6, the first housing 6 is provided with a battery module 5, the battery module 5 is respectively connected to the data storage and calculation module 2, the display and interaction module 3, the visual sensor module 1 and the tilt sensor module 7, and provides power support. The data storage and calculation module 2 is connected to the PLC controller of the tunneling machine equipment. The visual sensing module 1 realizes lining surface feature recognition and point cloud data generation; the filter module 10 realizes other visible light filtering in the tunnel environment to reduce the noise and errors generated by the point cloud data. The data storage and calculation module 2 realizes the storage and analysis of point cloud data, and can interact with the PLC of the tunnel boring machine equipment to obtain tunneling data such as mileage. The inclination sensor module 7 realizes the collection of angle deviation data between the point cloud acquisition center and the tunnel wall, and transmits the angle deviation data to the data storage and calculation module 2 for flatness calculation correction. The power supply module 5 realizes the power supply to each module of the system; the display and interaction module 3 displays the data calculation, and can design the host computer according to user requirements, such as the layout of the overall page, the setting of operation buttons, the setting of display content, the color of the host computer interface, etc.

[0048] like Figure 2 As shown, the first shell 6 is used to encapsulate the battery module 5 and can serve as the handle of the entire device. The second shell 8 is used to encapsulate the visual sensor module, the filter module and the tilt sensor module. The third shell 9 is used to encapsulate the data storage and computing module 2 and the display and interaction module 3. The lower part of the second shell 8 is fixed with a printing module 4, which is connected to the data storage and computing module 2 to realize printing and archiving.

[0049] The first shell 6 and the second shell 8 are connected by a buckle slot structure, which is convenient for disassembly and maintenance; the second shell 8 and the third shell 9 are connected by bolts, and the connection can be more tightly connected and can ensure sealing. The display and interaction module 3 is a liquid crystal display screen, which can display the point cloud data collected by the visual sensor module 1 and the angle deviation between the point cloud collection center and the tunnel wall collected by the inclination sensor module 7 in real time, and the flatness index obtained. At the same time, the interface setting of the host computer can be adjusted according to the user's choice, and then displayed on the liquid crystal display screen. The data storage and calculation module 2 specifically includes a single-chip microcomputer, which is provided with a memory, the memory is used for storing the collected data, and the single-chip microcomputer is used for processing the point cloud data. The visual sensor module 1 can use a binocular structured light camera, which can be installed in the second shell 8 by bolts. The binocular structured light camera can generate point cloud data by projecting structured light to the target surface and collecting the reflected structured light. Feature recognition is to compile these point cloud data. The tilt sensor module 7 is a tilt sensor. The installation position of the tilt sensor needs to be kept level with the installation baseline of the visual sensor module 1. At the same time, the tilt sensor needs to be calibrated when the whole machine leaves the factory, so that the horizontal baseline of the tilt sensor is parallel to the horizontal plane. It can be used directly without further calibration. The filter module 10 can select a filter with a corresponding filtering effect according to the use environment, so that the collected point cloud data will not generate noise due to ambient light. The center of the filter module 10 is on the same horizontal line as the acquisition head of the visual sensor module 1, and does not affect the data acquisition of the visual sensor module 1. The printing module 4 is a portable printer, which can print the bom generated by the collected data through a wireless connection device.

[0050] Because the local flatness detection device of the present invention is manually operated, when the instrument is held in hand, the angle of the visual sensor module 8 relative to the target detection surface is different each time, so the point cloud data collected after shooting is distorted to a certain extent, and it is necessary to perform algorithmic correction through the angle deviation data collected by the inclination sensor module 7. The method of using the local flatness detection device of the present invention is as follows: first, the detection personnel holds the first shell 6, aligns the visual sensing module 1 with the local lining surface of the tunnel to be inspected, and clicks the "Start Detection" button on the display and interaction module 3. The system automatically starts the visual sensing module 1, performs feature recognition on the area and automatically generates point cloud data, which is transmitted to the data storage and calculation module 2 for temporary storage. At the same time, the data storage and calculation module 2 reads the angle deviation between the point cloud acquisition center collected by the inclination sensor module 7 and the tunnel wall. The data storage and calculation module 2 corrects the point cloud data according to the angle deviation, and calculates the flatness index using the corrected point cloud array, and displays the flatness index on the data storage and calculation module 2. If the flatness index exceeds the national standard, the data storage and calculation module 2 will issue an alarm prompt, and at the same time record the detection time, location, and detection personnel, and automatically generate a flatness detection record sheet. You can choose to print or archive, and the printing module 4 will print the content to be printed.

[0051] The local flatness detection device of the present invention has the advantages of portable operation, rapid detection, and consideration of angle deviation, and can solve the problems of large data volume and large detection error in the prior art of flatness detection using vision.

[0052] Example 2

[0053] like Figure 3 As shown, a local flatness detection method using machine vision is shown, and the steps of performing local flatness detection using the local flatness detection device using machine vision in Example 1 are as follows:

[0054] Step 1: Aim the local flatness detection device at the local lining surface of the tunnel to be inspected, the visual sensor module 1 collects the point cloud data of the local lining surface of the tunnel to be inspected, and the inclination sensor module collects the angle deviation between the point cloud collection center and the tunnel wall.

[0055] When performing local flatness detection operations, first aim the local flatness detection device at the local lining surface of the tunnel to be inspected, click the "Start Detection" button on the display and interaction module 3, and the system automatically starts the visual sensing module, performs feature recognition on the area and automatically generates point cloud data, which is transmitted to the data storage and calculation module for temporary storage. At the same time, the data storage and calculation module reads the angle deviation between the point cloud collection center collected by the inclination sensor module and the tunnel wall.

[0056] Step 2: Use the detection range box to guide the algorithm to mark the detection range on the color image formed by the point cloud data to obtain a rectangular point cloud preprocessing area.

[0057] Point cloud data of the selected flatness detection area is screened. According to the current national standard requirements, the local flatness detection range can be defined. This embodiment takes the 1m ruler method as an example, and uses the detection range box to guide the algorithm to define a 1m*1m rectangular point cloud preprocessing area in the collected point cloud data with the center of the visual sensor module as the center.

[0058] When the detection device is used on site, because the field of view of the visual sensor module is large, the visual field of the operator's handheld instrument display and the interactive module 3 is also large. At the same time, the local range for calculating the flatness according to the national standard is small. Therefore, the present invention provides a detection range frame guidance function. The binocular module of the binocular structured light camera can automatically collect color values ​​with the point cloud data, that is, the black and white point cloud data can be assigned to a color point cloud, and the detection range is marked in the collected color image. At the same time, the range size can be defined independently, and the size of the detection frame can be adaptively displayed according to the distance from the collection point to the surface to be measured. For example, if the detection range is also defined as 1m*1m, the detection frame is displayed larger when the distance is close, and smaller when the distance is far. The field of view size can automatically adapt to 1m*1m according to the distance. In short, the detection range is guaranteed to be 1m*1m.

[0059] like Figure 4 As shown in the figure, the detection range frame guidance algorithm is implemented as follows: E, F, G on the local lining surface of the tunnel to be inspected are imaged at E', F', G' on the CCD of the visual sensor module after passing through the lens of the filter module. G is a point on the CCD optical axis; F is the center point of the target area. If a 1m*1m area is to be displayed, FG = 0.5m; E is a calibration point used to calculate the pixel size.

[0060] Image F'G' is the image of the object FG formed on the CCD through the lens. d is the distance of the object from the camera, which can be calculated through the depth map. The calculation method is to take the average depth value of the pixels in the center area of ​​the origin. It is a parameter of the camera and can be read directly. f is the focal length of the lens, which is a known value. When the object area range FG is known, the image height F'G' can be obtained. According to the image height F'G' and the pixel size, the number of occupied pixels can be obtained. By creating a new layer to draw the border of the occupied pixels, and superimposing the layer on the RGB image, that is, the color image, the representation area can be obtained.

[0061]

[0062] Among them, OG is the distance from the lens to the real target surface, OG′ is the distance from the lens to the CCD charge coupled device, and the pixel size is sizepixel Related to CCD, fixed value, size pixel The data can be obtained through CCD parameters or calibration calculation. f is the focal length of the lens, which is a fixed value. FG is the size of the calculation area, which is a fixed value according to the setting. The final frame size num pixel is a function of the object distance d:

[0063]

[0064] Const indicates a semantic constraint in the compiled language, and the compiler will enforce this constraint. pixel , calculate the pixel size size based on the known object EG size and the number of pixels occupied by the image height E'G' on the CCD numpixel(E'G') pixel E'G' is the image of the object EG formed on the CCD through the lens. The relationship between the object image and the pixel size is shown below. The frame size num pixel It can be calculated on the CCD:

[0065]

[0066] Combining the following two formulas, we can calculate the border size num pixel :

[0067]

[0068] available

[0069]

[0070] The point cloud area size required for shooting is 1m*1m. If the target is too close during use, that is, the distance d is too small, the image of the object exceeds the CCD frame size, and it can be marked as a range frame of one color. If the target is too far during use, that is, the distance d is large, the full frame size of the RGB area will be larger than the 1m*1m area, and it can be marked as a range frame of another color. The operator can judge whether the current detection distance is appropriate and whether the detection result is scientific based on the color of the detection frame.

[0071] like Figure 5 As shown, the detection range box obtained in the point cloud 11 is a rectangular box, and the point cloud data within the rectangular box is a point cloud array set 12.

[0072] Step 3: Assign coordinates of the point cloud data in the rectangular point cloud preprocessing area to obtain a point cloud array; correct the point cloud array using the angle deviation measured by the inclination sensor module; fit the corrected point cloud data and calculate the flatness index using the fitting line.

[0073] In the selected rectangular point cloud preprocessing area, a 1m straight line is taken using a random selection algorithm, and the point cloud data covered by the 1m straight line is taken and randomly selected within a certain range. The random selection algorithm can completely simulate the method of calculating flatness on site. The coordinates of this part of the point cloud data are assigned according to the coordinate system of the local flatness detection device, and the point cloud array f(x, y, z) is established. The point cloud array f(x, y, z) is corrected using the angle deviation a between the point cloud acquisition center and the tunnel wall to obtain the cosa dot product f(x, y, z) of the positive vertical incidence point cloud array, and the point cloud fitting baseline 13 is established using the positive vertical incidence point cloud array, as shown in FIG. Figure 6 As shown, the height difference of each point in the array relative to the fitting line is traversed, and the points above the baseline are defined as positive, and those below the fitting line are defined as negative. The maximum value D1 of the positive array and the minimum value D2 of the negative array are taken, and the distance D = D1-D2 is calculated. At the same time, the straight line spacing between the maximum value point and the minimum value point is defined as L. Then the calculation result of the flatness index of the array is P1 = D / L, that is, P1 = (D1-D2) / L. The specific implementation method of point cloud correction is:

[0074] 1). According to the camera coordinate system, point cloud values ​​are assigned to the features of the surface to be measured, that is, the original point cloud data set, i.e., the point cloud array f(x, y, z), can be obtained.

[0075] 2). Use the least squares method to fit the plane of the point cloud. The fitting equation is as follows:

[0076] f(x,y,z)=a*x+b*y+c*z+d

[0077] Among them, a, b, c, and d represent the coefficients of the fitting plane equation respectively.

[0078] 3) Obtain the data set of the original point cloud through the least squares point cloud assignment representation method In the point cloud set, a, b, and c are data features, and are also the coefficients of xyz of each point, that is, the normal vectors in each direction.

[0079] 4). The deflection angle of the point cloud in three directions collected by the three-axis tilt sensor module is the angle between the normal vectors in each direction, that is, Figure 6 In the example, [a], [a] = [a x 、a y 、a z ]. The coordinate conversion between the inclined coordinate system 14 and the corrected vertical coordinate system 15 is achieved through the deflection angle.

[0080] 5). Data collection of original point cloud Perform point cloud correction to obtain a new point cloud dataset

[0081] 6). That is, the plane fitting reference plane equation after angle deflection correction is:

[0082] f(x,y,z)=a*cos a x *x+b*cos a y *y+c*cos a z *z+d

[0083] 7) Calculate the residual of the reference plane equation, and obtain the z-axis residual, that is, the difference ε in the depth direction, based on the reference plane equation f(x, y, z) and the original data z' value z =z′-z.

[0084] 8). Establish the residual matrix M = [x, y, z, ε z ], M is the basic parameter for calculating flatness.

[0085] 9). Split the residual data in the residual matrix M, take the points that satisfy x∈(-A,A),y∈(-B,B), that is, take a certain strip within the camera's field of view, that is, a 1m straight line for point cloud computing, A and B are constants, and get a new set M 1 ;

[0086] 10). For set M 1 Grid filtering, the filter grid size is δx, and the number of grids is m; the number of grids can be actively set according to the calculation configuration to improve the calculation efficiency.

[0087] 11) After calculating the average value of the point clouds in the grid, the set M is obtained again 2 , set M 2 is a two-dimensional array;

[0088] 12). Calculate the two-dimensional array M 2 The highest point is recorded as contact A. Figure 7 shown.

[0089] 13). Calculate the lines connecting the highest point and other points, find the point corresponding to the line with the lowest slope, and record it as contact point B.

[0090] 14). Calculate the distance D between the point within line segment AB and AB, and the distance L between the contact points A and B.

[0091] 15). Calculate the flatness according to the national standard method: P=D / L.

[0092] The flatness detection area is repeated N times (N≥3) using a random selection algorithm to obtain the flatness index array [P] of the area, and the flatness index array [P] is averaged to obtain the flatness index P of the detection area, and displayed in the display and interaction module 3. The average algorithm can make the calculated flatness index data more scientific. If the flatness exceeds the national standard, an alarm will be issued, and the detection time, location, and detection personnel will be recorded at the same time, and a flatness detection record sheet will be automatically generated, which can be printed or archived.

[0093] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A local flatness detection method using machine vision, characterized in that: The steps are as follows: Step 1: Aim the visual sensor module at the local lining surface of the tunnel to be inspected and collect point cloud data of the local lining surface of the tunnel to be inspected, and the inclination sensor module collects the angle deviation between the point cloud collection center and the tunnel wall; Step 2: Use the detection range frame guidance algorithm to mark the detection range on the color image formed by the point cloud data to obtain a rectangular point cloud preprocessing area; Step 3: Assign coordinates of the point cloud data in the rectangular point cloud preprocessing area to obtain a point cloud array; correct the point cloud array using the angle deviation measured by the inclination sensor module; fit the corrected point cloud data and calculate the flatness index using the fitting line.

2. The local flatness detection method using machine vision according to claim 1, characterized in that: In the step 2, point cloud data of the selected flatness detection area is screened. Taking the 1m ruler method as an example, a detection range box guidance algorithm is used to define a 1m*1m rectangular point cloud preprocessing area in the collected point cloud data with the center of the visual sensor module as the center.

3. The local flatness detection method using machine vision according to claim 2, characterized in that: The detection range frame guidance algorithm: points E, F, and G on the local lining surface of the tunnel to be inspected are imaged as points E', F', and G' on the CCD of the visual sensing module through the lens of the visual sensing module respectively; G is a point on the optical axis of the visual sensing module; F is the center point of the target area, and E is the calibration point; Image F'G' is the image of object FG on the visual sensor module. When the area range of object FG is known, the image height of image F'G' corresponding to object FG is obtained. The number of occupied pixels is obtained according to the image height and pixel size. The border of the occupied pixels is drawn by creating a new layer, and the layer is superimposed on the color image formed by the point cloud to obtain the detection area.

4. The local flatness detection method using machine vision according to claim 3, characterized in that: The border size of the detection area is: Among them, size pixel is the pixel size, FG is the size of the calculation area, F′G′ is the size of the image of the calculation area, d is the distance between the object and the lens of the visual sensor module, and f is the focal length of the lens; Border size num pixel is a function of the distance d, based on the known size of the object EG and the number of pixels num occupied by the image height E'G' on the CCD of the vision sensor module pixel (E'G'), calculate the pixel size pixel ;and Image height E'G' is the size of the image of object EG formed on CCD through the lens of the visual sensor module; Combine the following two formulas to calculate the border size num pixel for 5. The local flatness detection method using machine vision according to any one of claims 1 to 4, characterized in that: The method for correcting the point cloud array using the angle deviation measured by the inclination sensor module is as follows: in the selected rectangular point cloud preprocessing area, a 1m straight line is taken using a random selection algorithm, and the point cloud data covered by the 1m straight line is taken, and the point cloud data coordinates are assigned to establish a point cloud array f(x, y, z), and the point cloud array f(x, y, z) is corrected using the angle deviation a between the point cloud acquisition center and the tunnel wall to obtain the positive vertical incidence point cloud array: cosa dot product f(x, y, z).

6. The local flatness detection method using machine vision according to claim 5, characterized in that: The method for calculating the flatness index is as follows: using the positive vertical incidence point cloud array to establish a point cloud fitting baseline, and traversing the height difference of each point in the positive vertical incidence point cloud array with respect to the fitting baseline, defining points above the baseline as positive and points below the fitting line as negative, taking the maximum value D1 of the positive array and the minimum value D2 of the negative array, calculating the distance D=D1-D2, and defining the straight line spacing L between the maximum and minimum points at the same time, then the flatness index calculation result is P1=D / L.

7. The local flatness detection method using machine vision according to claim 6, characterized in that: The residual of the positive vertical incident point cloud array and the point cloud array f(x, y, z) is calculated to obtain the difference ε in the depth direction z , establish the residual matrix M = [x, y, z, ε z ], segment the residual data in the residual matrix M, take the point cloud that satisfies the 1m straight line, and obtain a new set M1; perform grid filtering on the set M1; calculate the average value of the point cloud in the grid to obtain the set M2, calculate the highest point of the two-dimensional array M2, and record it as contact point A; calculate the line between contact point A and other points, find the point corresponding to the line with the lowest slope, and record it as contact point B; calculate the distance D between the point between line segment AB and line AB, and the distance L between contact points A and B; The formula for calculating flatness is P=D / L.

8. The local flatness detection method using machine vision according to claim 6 or 7, characterized in that: The visual sensing module performs feature recognition on the area where the local lining surface of the tunnel to be inspected is located and automatically generates point cloud data; Assign color values ​​to the point cloud data to obtain a color image of the point cloud, annotate the detection range in the color image, define the range size independently, and adaptively display the size of the detection frame according to the distance between the acquisition point and the surface to be measured; The flatness detection area is repeatedly detected N times using a random selection algorithm to obtain a flatness index array [P] of the area, and the flatness index array [P] is averaged to obtain the flatness index P of the detection area; If the flatness exceeds the national standard, an alarm will be issued, and the detection time, location, and detection personnel will be recorded. A flatness detection record sheet will be automatically generated and can be printed or archived.

9. A detection device using the local flatness detection method using machine vision according to any one of claims 1 to 8, characterized in that: The invention comprises a shell, wherein a visual sensing module (1), a tilt sensing module (7), a data storage and calculation module (2) and a display and interaction module (3) are arranged in the shell, the center of the tilt sensing module (7) and the visual sensing module (1) are on the same horizontal line, the visual sensing module (1) and the tilt sensing module (7) are both connected to the data storage and calculation module (2), the data storage and calculation module (2) is connected to the display and interaction module (3), and the data storage and calculation module (2) is connected to a PLC controller of a tunnel boring machine; the visual sensing module (1) realizes lining surface feature recognition and point cloud data generation, the tilt sensing module (7) collects angle deviation data between the point cloud collection center and the tunnel wall, and transmits the angle deviation data to the data storage and calculation module (2) to perform flatness calculation correction.

10. The detection device according to claim 9, characterized in that: A filter module (10) is provided on the front side of the visual sensor module (1), the filter module (10) is fixed in front of the housing, and the center of the filter module (10) is on the same horizontal line as the center of the lens of the visual sensor module (1).

11. The detection device according to claim 10, characterized in that: The shell comprises a second shell (8) and a third shell (9); the third shell (9) is provided with a display and interaction module (3); the third shell (9) is provided with a data storage and calculation module (2); the lower part of the third shell (9) is connected to the second shell (8); the second shell (8) is provided with a visual sensor module (1) and a tilt sensor module (7); the visual sensor module (1) is arranged on the front side of the second shell (8); and the filter module (10) is fixed on the front side of the second shell (8).

12. The detection device according to claim 11, characterized in that: A first shell (6) is fixed to the lower part of the second shell (8), a battery module (5) is arranged in the first shell (6), and the battery module (5) is respectively connected to the data storage and calculation module (2), the display and interaction module (3), the visual sensor module (1) and the tilt sensor module (7); A printing module (4) is fixed to the lower part of the second housing (8), and the printing module (4) is connected to the data storage and computing module (2); The first shell (6) and the second shell (8) are snap-fitted together by means of a snap-fit ​​slot structure, and the second shell (8) and the third shell (9) are fixedly connected by means of bolts; The visual sensor module (1) uses a binocular structured light camera, which is installed in the second housing (8) by means of bolts.

Images