Depth vision measurement system for automatically grabbing and positioning shield segment and measurement method thereof

Abstract

The depth vision measurement system for automatic grabbing and positioning of the shield segment comprises a vision detection module and a data processing module, the vision detection module comprises a vision camera with an adjustable position posture, and the data processing module is used for receiving and processing an image shot by the vision camera; the visual detection module further comprises a structural grating which is fixed relative to the position of a vacuum chuck on the segment assembly machine, a plurality of grating lines which are arranged at intervals can be projected on the shield segment by utilizing the structural grating, and the grating lines can generate angular points with suddenly changed brightness at the edge of the shield segment; the data processing module is used for fitting edge lines of the shield segment according to the positions of all angular points in the image, and the edge lines are used for segmenting the image, so that invalid features in the image can be filtered out, external environment interference can be eliminated, and the recognition efficiency and accuracy are greatly improved; the depth vision measurement method for automatic grabbing and positioning of the shield segment is simple in step, high in accuracy and easy to implement.

Description

Technical Field

[0001] The invention belongs to the technical field of shield segment assembly, and in particular relates to a depth vision measurement system and a measurement method thereof for automatic grasping and positioning of shield segments. Background Art

[0002] As we all know, in tunnel construction, shield segments are usually assembled into rings to form stable supports. With the diversified development of tunnel construction, the construction of large-diameter and long-distance tunnels has gradually increased. As the diameter of the tunnel increases, the size and mass of the shield segments used also increase accordingly. Before grasping and positioning, this type of shield segments need to be detected in space. The traditional detection method is manual visual inspection, that is, relying on manual control of the remote control for operation, and achieving grasping and positioning through multiple attempts. This solution has problems such as high labor intensity, low efficiency, limited accuracy, and high safety risks, and has been gradually abandoned.

[0003] There are also methods that use machine vision technology to identify segment feature identification, analyze and construct the spatial posture of the shield segment to be grasped, and then realize the automatic grasping and positioning of the shield segment; for example, Chinese patent CN108104834B discloses a method and system for automatic assembly of shield segments. This scheme completes the adjustment of the automatic assembly action of the shield segment according to the image feature information and the motion logic of the assembly machine equipment; for example, Chinese patent CN107464262B discloses a vacuum suction cup positioning and grasping system based on evolutionary image processing, which grasps and positions the shield segment through ultrasonic sensors and cameras to realize the automatic grasping operation of the shield segment; for example, patent CN113107550A discloses a method for automatic assembly of shield segments based on machine vision, which detects the target feature angle of the segment The segment assembly points are calculated, and the movement speed of the assembly machine is controlled in a closed loop through a visual servo control strategy, thereby realizing an automated segment assembly process. For example, Chinese patent CN117231264A discloses a segment position and posture recognition and grasping method based on depth image stereo vision, which uses a depth vision sensor to obtain segment depth information, calculates the position and posture of the segment to be grasped relative to the sensor, solves the motion estimation and motion parameters of the segment to be assembled, and automatically grasps the segment. However, the above schemes do not take into account the complexity of the environment during shield construction, including poor visual detection environment, low ambient illumination, easy contamination of the segment surface, uneven lighting, and the problem that the visual sensor is easily collided in other construction links, causing it to deviate from the installation position, thereby affecting the detection results. Summary of the invention

[0004] The purpose of the present invention is to overcome one or more shortcomings in the prior art and to provide a depth vision measurement system and a measurement method for automatic grasping and positioning of shield segments.

[0005] In order to achieve the above-mentioned purpose, the product in the technical solution provided by the present invention is a depth vision measurement system for automatic grasping and positioning of shield segments, comprising:

[0006] A visual inspection module, the visual inspection module comprising a visual camera whose field of view covers the shield segment, and the position and posture of the visual camera are adjustable;

[0007] A data processing module, the data processing module is used to receive and process the images taken by the visual camera;

[0008] The visual inspection module further comprises a structured grating installed on the segment assembly machine, wherein the relative position of the structured grating and the vacuum suction cup on the segment assembly machine is kept fixed, and when the vacuum suction cup grabs and positions the shield segment, the structured grating projects a plurality of spaced grating lines on the shield segment, and the grating lines generate corner points with sudden brightness changes at the edge of the shield segment;

[0009] The data processing module fits the edge line of the shield segment according to the positions of all the corner points in the image, and uses the edge line to segment the image to filter out invalid features in the image.

[0010] Preferably, the visual detection module also includes an adjustment mechanism for adjusting the position and posture of the visual camera. The adjustment mechanism is data-connected with the data processing module. When the data processing module detects that the spacing between adjacent grating lines in the image is not equal, the data processing module sends an adjustment signal to the adjustment mechanism. The adjustment mechanism adjusts the position and posture of the camera, so that the visual camera re-takes the image and sends it to the data processing module until the spacing between adjacent grating lines in the image is equal.

[0011] Preferably, the data processing module includes a visual system control and image processing unit, a host computer, and a programmable controller. The visual system control and image processing unit is data-connected with the visual camera and is used to receive the image taken by the visual camera. The host computer is data-connected with the visual system control and image processing unit and the programmable controller, and the programmable controller is data-connected with the execution module of the segment assembly machine.

[0012] The method in the technical solution provided by the present invention is a depth vision measurement method for automatic grasping and positioning of shield segments, which is implemented by using any one of the above-mentioned depth vision measurement systems for automatic grasping and positioning of shield segments, and is characterized in that the depth vision measurement method for automatic grasping and positioning of shield segments comprises the following steps:

[0013] a. Adjust the position and posture of the visual camera so that its field of view can cover the shield segment;

[0014] b. Turning on the structured grating to project a plurality of spaced grating lines on the shield segment;

[0015] c. Taking a photo through the visual camera and sending the image taken to the data processing module;

[0016] d. The data processing module fits the edge line of the shield segment according to the corner points where the brightness of the grating line in the image changes suddenly;

[0017] e. The data processing module divides the image using the edge line to obtain a segment area corresponding to the shield segment and a non-segment area outside the shield segment;

[0018] f. The data processing module calculates the spatial position of the shield segment according to the segment area, providing a basis for the automatic grasping and positioning of the segment assembly machine.

[0019] Preferably, the fitting in step d includes two steps: locating the center of the corner point and constructing a fitting straight line;

[0020] The positioning of the corner point center is achieved by:

[0021] S1. Denoise and binarize the original image of the grating line to obtain a binary image;

[0022] S2. Traverse the binary image to obtain the center contour area point, and cycle close to the center point;

[0023] S3. Calculate the distance from the center of the circle (x m ,y m )(m=1,2,...) nearest point(x i ,y i ) The distance is r m ,in,

[0024]

[0025] S4. Move the center of the circle (x m ,y m ) so that it coincides with the point (x i ,y i ) increases in the straight line direction, that is,

[0026]

[0027] S5. Perform a loop from step S3 to S4, comparing the current distance with the distance of the previous loop in each loop. m+1 <r m When the loop is exited, the required circle radius rm can be obtained. +1 , the center of the circle is (xm+1 ,y m+1 );

[0028] The construction of the fitting straight line is achieved by performing linear fitting on the centers of circles of multiple corner points to obtain the edge line.

[0029] Further preferably, the segmentation in step e is performed by extending the two end points of the edge line so as to intersect with the border of the image to obtain a fitting segmentation line, and the image is segmented into the pipe segment area and the non-pipe segment area according to the fitting segmentation line.

[0030] Preferably, the depth vision measurement method for automatic grasping and positioning of shield segments also includes a step x of correcting the position and posture of the visual camera.

[0031] Further preferably, the step x is: taking a photo through the visual camera, and sending the image obtained by taking the photo to the data processing module; if the data processing module detects that the spacings between adjacent grating lines in the image are not equal, the data processing module sends an adjustment signal to the adjustment mechanism of the visual camera, and the adjustment mechanism adjusts the position and posture of the camera. After the adjustment, the visual camera re-takes the image and sends it to the data processing module until the data processing module detects that the spacings between adjacent grating lines in the image are equal.

[0032] Further preferably, the step x is located between the step e and the step f, or, the step x is located between the step b and the step c.

[0033] Preferably, step b is placed before step a.

[0034] Due to the application of the above technical solution, the present invention has the following advantages compared with the prior art:

[0035] The present invention provides a deep vision measurement system for automatic grasping and positioning of shield segments, comprising a vision detection module and a data processing module. The vision detection module comprises a vision camera with adjustable position and posture. The data processing module is used to receive and process images taken by the vision camera. By making the vision detection module also include a structured grating whose relative position with a vacuum suction cup on a segment assembly machine is kept fixed, the structured grating can be used to project a plurality of spaced grating lines on the shield segment, and the grating lines will produce corner points with sudden brightness changes at the edges of the shield segment. The data processing module then fits the edge lines of the shield segment according to the positions of all corner points in the image, and uses the edge lines to segment the image, thereby filtering out invalid features in the image, eliminating external environmental interference, and greatly improving recognition efficiency and accuracy. The deep vision measurement method for automatic grasping and positioning of shield segments provided by the present invention has simple steps, high accuracy, and is easy to implement. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is a structural block diagram of the depth vision measurement system for automatic grasping and positioning of shield segments in the present invention.

[0037] Figure 2 It is a schematic diagram of the change of grating lines after the visual camera is offset in the present invention.

[0038] Figure 3 right Figure 2 The intervals between the raster lines are marked.

[0039] Figure 4 yes Figure 2 Detection results of grating lines after the mid-vision camera is offset.

[0040] Figure 5 It is a flow chart of the depth vision measurement method for automatic grasping and positioning of shield segments in the present invention. DETAILED DESCRIPTION

[0041] like Figures 1 to 4 As shown, the depth vision measurement system for automatic grasping and positioning of shield segments provided by the present invention includes: a vision detection module and a data processing module, the vision detection module includes a vision camera whose field of view covers the shield segments, and the position and posture of the vision camera are adjustable; the data processing module is used to receive and process images taken by the vision camera; the vision detection module also includes a structured grating installed on the segment assembly machine, the relative position of the structured grating and the vacuum suction cup on the segment assembly machine remains fixed, when the vacuum suction cup grasps and positions the shield segment, the structured grating projects a plurality of spaced grating lines on the shield segment, and the grating lines produce corner points with sudden brightness changes at the edge of the shield segment; the data processing module fits the edge line of the shield segment according to the positions of all corner points in the image, and uses the edge line to segment the image to filter out invalid features in the image.

[0042] The advantage of this setting is that the grating lines projected by the structured grating can be used as a reference to form corner points with sudden changes in brightness at the edge of the shield segment. The data processing module can then fit the edge lines of the shield segment according to the positions of all corner points in the image, and use the edge lines to segment the image. This can filter out invalid features in the image, eliminate external environmental interference, and greatly improve recognition efficiency and accuracy.

[0043] Preferably, the visual detection module also includes an adjustment mechanism for adjusting the position and posture of the visual camera. The adjustment mechanism is data-connected to the data processing module. When the data processing module detects that the spacing between adjacent grating lines in the image is not equal, the data processing module sends an adjustment signal to the adjustment mechanism. The adjustment mechanism adjusts the position and posture of the camera, so that the visual camera re-takes the image and sends it to the data processing module until the spacing between adjacent grating lines in the image is equal.

[0044] The advantage of this setting is that the grating lines projected by the structured grating can be used as a reference to determine the position offset of the visual camera and make corresponding adjustments to it, thereby ensuring that the visual camera is in the correct position and posture when subsequent images are taken, avoiding erroneous identification and measurement caused by changes in the position of the visual camera itself, reducing the safety risks of automated grasping and positioning operations, ensuring that the automatic recognition and measurement data of the shield segments are reliable, and making the automatic grasping and positioning process of the shield segments safe and stable.

[0045] To facilitate the linkage control, in this embodiment, the data processing module includes a visual system control and image processing unit, a host computer, and a programmable controller. The visual system control and image processing unit is data-connected with the visual camera to receive images taken by the visual camera. The host computer is data-connected with the visual system control and image processing unit and the programmable controller. The programmable controller is data-connected with the execution module of the segment assembly machine to facilitate the linkage between the depth vision measurement system for automatic grasping and positioning of the shield segments and the segment assembly machine. Furthermore, the execution module of the segment assembly machine includes an assembling machine actuator and input and output devices. The input device is used to send control information from the assembling machine actuator to the programmable controller, and the output device is used to receive control signals sent by the programmable controller and transmit them to the assembling machine actuator.

[0046] like Figure 5 As shown, the present invention also provides a depth vision measurement method for automatic grasping and positioning of shield segments, which is implemented by using the above-mentioned depth vision measurement system for automatic grasping and positioning of shield segments. The measurement method includes the following steps:

[0047] a. Adjust the position and posture of the visual camera so that its field of view can cover the shield segment;

[0048] b. Turn on the structured grating to project a number of spaced grating lines on the shield segment;

[0049] c. Take pictures with a visual camera and send the pictures to a data processing module;

[0050] d. The data processing module fits the edge line of the shield segment according to the corner points where the brightness of the grating lines in the image changes suddenly;

[0051] e. The data processing module uses the edge line to segment the image to obtain the segment area corresponding to the shield segment and the non-segment area outside the shield segment;

[0052] f. The data processing module calculates the spatial posture of the shield segment according to the segment area, providing a basis for the automatic grasping and positioning of the segment assembly machine.

[0053] Preferably, the fitting in step d includes two steps: locating the center of the corner point and constructing a fitting straight line;

[0054] The center of the corner point is located in the following way:

[0055] S1. Perform noise reduction and binarization processing on the original image of the grating line to obtain a binary image;

[0056] S2. traverse the binary image to obtain the center contour area point, and cyclically approach the center point;

[0057] S3. Calculate the distance from the center of the circle (x m ,y m )(m=1,2,...) nearest point(x i ,y i ) The distance is r m ,in,

[0058]

[0059] S4. Move the center of the circle (x m ,y m ) so that it coincides with the point (x i ,y i ) increases in the straight line direction, that is,

[0060]

[0061] S5. Perform a loop from step S3 to S4, comparing the current distance with the distance of the previous loop in each loop. m+1 <r m When the loop is exited, the required circle radius rm can be obtained. +1 , the center of the circle is (x m+1 ,y m+1 );

[0062] The fitting straight line is constructed by linearly fitting the centers of multiple corner points to obtain the edge line.

[0063] Further preferably, the segmentation in step e is performed by extending the two end points of the edge line so as to intersect with the border of the image to obtain a fitting segmentation line, and the image is segmented into a pipe segment area and a non-pipe segment area according to the fitting segmentation line.

[0064] Preferably, the depth vision measurement method for automatic grasping and positioning of shield segments also includes a step x of correcting the position and posture of the visual camera.

[0065] Further preferably, step x is: taking a picture through a visual camera, and sending the image obtained by taking the picture to a data processing module; if the data processing module detects that the spacing between adjacent grating lines in the image is not equal, the data processing module sends an adjustment signal to the adjustment mechanism of the visual camera, and the adjustment mechanism adjusts the position and posture of the camera. After the adjustment, the visual camera re-takes the image and sends it to the data processing module until the data processing module detects that the spacing between adjacent grating lines in the image is equal.

[0066] Further preferably, step x is located between step e and step f.

[0067] The depth vision measurement method for automatic grasping and positioning of shield segments provided by the present invention has simple steps, high accuracy and is easy to implement.

[0068] The present invention provides a depth vision measurement method and a measurement method for automatic grasping and positioning of shield segments. Through a visual detection module composed of a structured grating and a visual camera, all suspected feature targets within the visual camera field of view can be segmented using the depth vision technology, and then multiple parallel and spaced grating lines are projected to the edge of the segment through the structured grating. The grating lines form a cross-sectional fold line at the edge of the shield segment, and bright light spots (corner points) are generated at the fold line. The corner points generated by the intersection of all grating lines and the edge of the segment are extracted, and the corner points are fitted into a straight line. The image taken by the camera can be segmented into two parts through the straight line: one part is the surface of the segment, and the other part is outside the surface of the segment; in this way, the suspected feature targets outside the surface of the segment can be filtered out, which greatly increases the anti-interference ability of the visual system and improves the visual detection performance. At the same time, the laser corner points formed by the structured grating projected to the edge of the segment can also verify whether the position of the system sensor has changed, ensuring the safety of construction automation operations.

[0069] At the same time, it is also possible to determine whether the current position and posture of the visual camera needs to be adjusted by checking whether the spacing between adjacent grating lines in the visual camera image is equal, thereby ensuring the stability of the visual measurement results.

[0070] Specifically, Figure 3 and 4 As shown in the figure, the grating line spacing data is analyzed. If the installation position of the visual camera is offset from the initial calibration position, the grating line spacing pixels will change accordingly. When the visual camera position is tilted to the left, the grating line spacing horizontal pixel deviation occurs by 40dpi, and the vertical pixel deviation is not obvious; when the visual camera position is tilted to the right, the grating line spacing horizontal pixel deviation occurs by -34dpi, and the vertical pixel deviation is not obvious; when the visual camera position is tilted forward, the grating line spacing horizontal pixel deviation is not obvious, the vertical pixel deviation occurs by -43dpi, and the horizontal pixel deviation is not obvious; when the visual camera position is tilted backward, the grating line spacing horizontal pixel deviation is not obvious, and the vertical pixel deviation occurs by 32dpi.

[0071] The displacement change of the visual camera can be clearly observed from the above-mentioned changes in the grating line interval pixels. By adjusting the position of the visual camera mounting bracket according to the changes, the identification and measurement safety during the automatic grasping and positioning of the shield segment can be effectively guaranteed.

[0072] It should be noted that, in the present invention, the equal spacing between adjacent grating lines means that the spacing is equal within a set accuracy range, not absolutely equal.

[0073] The above embodiments are only for illustrating the technical concept and features of the present invention, and their purpose is to enable people familiar with this technology to understand the contents of the present invention and implement them accordingly. They cannot be used to limit the protection scope of the present invention. Any equivalent changes or modifications made according to the spirit of the present invention should be included in the protection scope of the present invention.

Claims

1. Depth vision measurement system for automatic grasping and positioning of shield segments, including: A visual inspection module, the visual inspection module comprising a visual camera whose field of view covers the shield segment, and the position and posture of the visual camera are adjustable; A data processing module, the data processing module is used to receive and process the images taken by the visual camera; Features: The visual inspection module further comprises a structured grating installed on the segment assembly machine, wherein the relative position of the structured grating and the vacuum suction cup on the segment assembly machine is kept fixed, and when the vacuum suction cup grabs and positions the shield segment, the structured grating projects a plurality of spaced grating lines on the shield segment, and the grating lines generate corner points with sudden brightness changes at the edge of the shield segment; The data processing module fits the edge line of the shield segment according to the positions of all the corner points in the image, and uses the edge line to segment the image to filter out invalid features in the image.

2. The depth vision measurement system for automatic grasping and positioning of shield segments according to claim 1 is characterized in that: The visual detection module also includes an adjustment mechanism for adjusting the position and posture of the visual camera. The adjustment mechanism is data-connected with the data processing module. When the data processing module detects that the spacing between adjacent grating lines in the image is not equal, the data processing module sends an adjustment signal to the adjustment mechanism. The adjustment mechanism adjusts the position and posture of the camera so that the visual camera retakes the image and sends it to the data processing module until the spacing between adjacent grating lines in the image is equal.

3. The depth vision measurement system for automatic grasping and positioning of shield segments according to claim 1 is characterized by: The data processing module includes a visual system control and image processing unit, a host computer, and a programmable controller. The visual system control and image processing unit is data-connected with the visual camera and is used to receive the image taken by the visual camera. The host computer is data-connected with the visual system control and image processing unit and the programmable controller. The programmable controller is data-connected with the execution module of the segment assembly machine.

4. A depth vision measurement method for automatic grasping and positioning of shield segments, implemented by a depth vision measurement system for automatic grasping and positioning of shield segments as claimed in any one of claims 1 to 3, characterized in that: The depth vision measurement method for automatic grasping and positioning of shield segments comprises the following steps: a. Adjust the position and posture of the visual camera so that its field of view can cover the shield segment; b. Turning on the structured grating to project a number of spaced grating lines on the shield segment; c. Taking a photo through the visual camera and sending the image taken to the data processing module; d. The data processing module fits the edge line of the shield segment according to the corner points where the brightness of the grating line in the image changes suddenly; e. The data processing module divides the image using the edge line to obtain a segment area corresponding to the shield segment and a non-segment area outside the shield segment; f. The data processing module calculates the spatial position of the shield segment according to the segment area, providing a basis for the automatic grasping and positioning of the segment assembly machine.

5. The depth vision measurement method for automatic grasping and positioning of shield segments according to claim 4 is characterized in that: The depth vision measurement method for automatic grasping and positioning of shield segments also includes a step x of correcting the position and posture of the vision camera.

6. The depth vision measurement method for automatic grasping and positioning of shield segments according to claim 5 is characterized in that: The step x is: taking a photo through the visual camera, and sending the image obtained by taking the photo to the data processing module. If the data processing module detects that the spacing between adjacent grating lines in the image is not equal, the data processing module sends an adjustment signal to the adjustment mechanism of the visual camera, and the adjustment mechanism adjusts the position and posture of the camera. After the adjustment, the visual camera re-takes the image and sends it to the data processing module until the data processing module detects that the spacing between adjacent grating lines in the image is equal.

Images