A method for detecting the edge - sticking state of a road roller based on line laser and high - definition camera

Abstract

The present invention relates to a method for detecting the edge - sticking state of a road roller based on line laser and high - definition camera, comprising the following steps: obtaining the front - side and rear - side photos of the ground where the road roller is located collected by the high - definition camera for the first time; obtaining the front - side and rear - side photos collected again by the high - definition camera and having overlap between the photos; obtaining the image from a top - down perspective and the correlation between pixel points and the real - plane distance; splicing the images from the top - down perspective to obtain the spliced image, identifying the intersection points of the multiple laser lines, and connecting the intersection points to obtain a straight line; identifying the edge of the curb that the road roller is close to and obtaining the edge line by using a fitting method; calculating the pixel - point interval between the straight line and the edge line, and inversely calculating the real - plane distance corresponding to the pixel - point interval according to the correlation to obtain the edge - sticking state of the road roller. Compared with the prior art, the present invention has the advantages of realizing the automatic detection and accurate discrimination of the edge - sticking state of the road roller, etc.

Description

Technical Field

[0001] The present invention relates to the technical field of road engineering, and particularly to a method for detecting the edge - sticking state of a roller based on line laser and high - definition camera. Background Technique

[0002] When a roller compacts road materials, it usually needs to perform edge - sticking compaction operations along the edge of the curbstone. During the edge - sticking compaction operation, it is necessary to ensure that the working wheel of the roller is as close as possible to the edge of the road to achieve a better compaction degree, and at the same time, it is necessary to avoid the working wheel of the roller rolling onto the curbstone and damaging the curb facilities. Therefore, it is necessary to quickly and accurately detect the edge - sticking state during the compaction operation.

[0003] Traditional methods for detecting the edge - sticking state are mostly based on visual inspection, that is, the operator judges the edge - sticking state by visually observing the contact state between the working wheel of the roller and the curb. This method requires high requirements for the operator and has insufficient accuracy. There are also some technical solutions that obtain image information of the scene around the roller through video technology to assist the operator in discrimination, but still do not achieve automated detection of the edge - sticking state.

[0004] Patent CN113467443B discloses a roller edge - sticking control device, a roller, a group of rollers and a control method, which relates to the technical field of vehicle control. It installs a light - emitting device that emits visible light beams in the roller edge - sticking control device, and makes it project a visible line segment on the working surface in front of or behind the roller; the visible line segment is on the same straight line as the longitudinal boundary of the roller; the driver controls the roller to operate according to the positional relationship between the visible line segment and other reference objects on the working surface.

[0005] The above application uses visible light beams to assist the driver in discriminating the edge - sticking state, but it only describes at the equipment level and does not propose an intelligent detection method, and still requires the operator to observe and discriminate. Summary of the Invention

[0006] The purpose of the present invention is to provide a method for detecting the edge - sticking state of a roller based on line laser and high - definition camera, which can accurately and automatically identify edge - sticking compaction.

[0007] The purpose of the present invention can be achieved by the following technical solutions:

[0008] A method for detecting the edge - sticking state of a roller based on line laser and high - definition camera includes the following steps:

[0009] Obtain the front - side and rear - side photos of the ground where the roller is located collected by the high - definition camera for the first time;

[0010] Obtain the front - side photos and rear - side photos collected again by the high - definition camera, and there is an overlap between the photos. Among them, the front - side photos and rear - side photos obtained again are obtained after adjusting the high - definition camera with multiple laser lines;

[0011] Based on the front - side photos and rear - side photos obtained initially and again, calibrate the high - definition camera to obtain an image from a top - down perspective and the correlation between pixel points and the real - plane distance;

[0012] Stitch the images from the top - down perspective to obtain a stitched image, identify the intersection points of the multiple laser lines, and connect the intersection points to obtain a straight line;

[0013] Based on the stitched image, identify the edge of the road curb that the road roller is close to, and use a fitting method to obtain an edge line;

[0014] Calculate the pixel - point interval between the straight line and the edge line, and inversely calculate the real - plane distance corresponding to the pixel - point interval according to the correlation to obtain the edge - adhering state of the road roller.

[0015] Further, there are three laser lines, which are respectively shot at the ground in front of the road roller, the ground behind the road roller, and the ground on the side of the road roller. The laser lines shown on the side ground intersect with the laser lines on the front ground and the rear ground respectively, and the intersection angle is a right angle.

[0016] Further, when calibrating, the black - and - white grid calibration method is used, and the size of a single grid in the black - and - white grid does not exceed 20 cm.

[0017] Further, the sift feature extraction method or the hog feature extraction method is used for stitching.

[0018] Further, the specific steps of identifying the intersection points of the multiple laser lines include:

[0019] Pre - process the stitched image to obtain a filtered image;

[0020] Use the threshold segmentation method to process the filtered image to obtain a segmented image;

[0021] Use an identification method to identify multiple laser lines in the segmented image;

[0022] According to the coordinate arrays of the laser lines, calculate the overlapping pixel points, and use the overlapping pixel points as the intersection points of the laser lines.

[0023] Further, the pre - processing includes one or more of Gaussian filtering and wavelet filtering.

[0024] Further, the threshold segmentation method includes the fixed global threshold method, the histogram global threshold method, and the local threshold method.

[0025] Further, the recognition method includes one or more of the gray center of gravity method, the Steger algorithm, and the deep convolutional neural network.

[0026] Further, the fitting method includes the straight line fitting method and the curve fitting method.

[0027] Further, one or more of the deep convolutional neural network and the Hough transform are used to recognize the edge of the curb that the roller is close to.

[0028] Compared with the prior art, the present invention has the following beneficial effects:

[0029] (1) The present invention initially takes photos of the front side and the rear side of the ground where the roller is located through a high-definition camera, adjusts the high-definition camera according to the laser line to obtain photos of the front side and the rear side with overlap, and performs a series of calculations such as calibration, laser line recognition, intersection point recognition, and distance discrimination on the obtained photos, realizing the automatic edge attachment detection of the roller.

[0030] (2) The present invention simply installs equipment on the roller by means of the laser line and the high-definition camera. The implementation process is simple, and the edge attachment rolling efficiency can be significantly improved through the automatic detection method.

[0031] (3) By processing the obtained photos, the present invention can accurately calculate the distance between the working wheel edge and the curb. The calculation result can provide reliable data support for the high-precision control of the edge attachment rolling of the roller.

[0032] (4) In the calculation processes such as intersection point recognition and edge line recognition, the present invention provides multiple methods, which can be selected according to actual needs and has good practicability.

[0033] (5) By using black and white grids for calibration, compared with other complex calibration methods, this method is simple and easy to implement, has low cost, and the set grid size does not exceed 20 cm, which can reduce the complexity of data processing, improve the measurement accuracy, thereby realizing the accurate correction and measurement of the image, and improving the accuracy of the edge attachment detection of the roller.

[0034] (6) By shooting three laser lines towards the front ground, the rear ground, and the side ground of the roller and controlling the emission angles of the three laser lines, the laser lines on the side ground intersect with the laser lines on the front ground and the rear ground, and the intersection angle is a right angle. The high-definition camera adjusts the angle according to the three laser lines, and can collect photos with large overlap, providing a good basis for subsequent photo processing, thereby improving the accuracy of the edge attachment detection. Description of the Drawings

[0035] Figure 1 This is the flowchart of the method of the present invention;

[0036] Figure 2 This is the layout position of the high-definition camera and the laser emitter in the embodiment of the present invention;

[0037] Figure 3 This is the top view of the ground laser line in the embodiment of the present invention;

[0038] Figure 4 This is the flowchart of identifying the intersection points S1 and S2 of the laser lines to obtain a straight line in the embodiment of the present invention. Detailed implementation manners

[0039] The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. This embodiment is implemented on the premise of the technical solution of the present invention, and gives detailed implementation manners and specific operation processes, but the protection scope of the present invention is not limited to the following embodiments.

[0040] This embodiment provides a method for detecting the edge-attaching state of a roller based on line laser and high-definition camera, as Figure 1 shown, the method includes the following steps:

[0041] Step 1, obtain the front-side photo and rear-side photo of the ground where the roller is located collected by the high-definition camera for the first time.

[0042] Install a line laser emitter (laser1, laser2, laser3) in front of, behind and on the side of the roller respectively, and install two high-definition cameras (camera1, camera2) on the top of the roller, and shoot towards the front side and the rear side respectively to obtain images image1 and image2. The three line laser emitters and the two high-definition cameras are connected to the central control device, and the layout position is as Figure 2 shown. Among them, to ensure high-speed data acquisition in the mobile operation state, the acquisition frame rate of the high-definition camera is not less than 40fps. At the same time, the installed high-definition camera, line laser emitter and central control device are connected to the internal circuit of the roller for power supply. This process involves simple equipment installation on the roller, and the operation is simple and convenient.

[0043] Start the roller and adjust the working wheels of the roller to the straight-ahead direction. After the three line laser emitters are powered on, they respectively emit laser lines to the ground in front of, behind and on the side of the roller, and the top view of the laser lines is as Figure 3 shown. At the same time, the high-definition camera starts to collect the front-side and rear-side photos of the ground for the first time.

[0044] Step 2: Obtain the front-side and rear-side photos collected again by the high-definition camera, and there is an overlap between the photos. The front-side and rear-side photos obtained again are obtained after adjusting the high-definition camera with multiple laser lines.

[0045] First, adjust the emission angles of the three laser emitters so that the laser lines shown on the front ground and the rear ground are parallel to the wheel axle of the roller working wheel, the laser line shown on the side ground is perpendicular to the wheel axle of the roller working wheel and tangent to the edge of the working wheel. The widths of the laser lines in the front and the rear should be kept the same, and the laser line on the side ground should intersect with the laser lines in the front and the rear.

[0046] Adjust the shooting angles of the two high-definition cameras so that the intersection point of the laser lines on the front ground and the side ground appears within the field of view of the front-side camera, the intersection point of the cross lines of the rear ground and the side ground appears within the field of view of the rear-side camera, and there is a large overlap of the side ground photographed by image1 and image2.

[0047] Step 3: Calibrate the high-definition camera based on the front-side and rear-side photos obtained initially and again, and obtain the image from a top-down perspective and the correlation between pixel points and the real plane distance.

[0048] Calibrate the two high-definition cameras using black and white grids, convert image1 and image2 into images upimage1 and upimage2 from a top-down perspective, and obtain the correlation between the pixel points of the top-down perspective pictures and the real plane distance. In this embodiment, the size of a single grid in the black and white grids used in the calibration process does not exceed 20 cm.

[0049] In this step, based on the obtained photos, detect the grid points on the black and white grid calibration board, extract the pixel coordinates, and extract parameters such as the internal and external parameters of the two cameras. According to the internal and external parameters of the camera, correct the images of image1 and image2 and convert them into images upimage1 and upimage2 from a top-down perspective. Thus, according to the calibration results and the size of the black and white grid calibration board, establish the correlation between pixel points and the real plane distance. Compared with other complex calibration methods, using black and white grids for calibration is not only simple and easy to implement, low in cost, but also can accurately correct and measure the images, improving the accuracy of the edge detection of the roller.

[0050] Step 4: Stitch the images from the top-down perspective to obtain the stitched image, and identify the intersection points of the multiple laser lines, and connect the intersection points to obtain a straight line.

[0051] Stitch upimage1 and upimage2 to obtain the stitched image pasteimage, identify the two laser line intersection points S1 and S2, and connect S1 and S2 to obtain the straight line L1.

[0052] As Figure 4 shown, the steps to identify the two laser line intersection points S1 and S2 are specifically as follows:

[0053] Step 41: Preprocess the stitched image pasteimage to eliminate the noise signals in the image. The preprocessing method can adopt one or more of Gaussian filtering and wavelet filtering to obtain the filtered image pasteimage_denoised.

[0054] Step 42: Process the filtered image pasteimage_denoised using the threshold segmentation method to increase the contrast between the laser line and the background. The threshold segmentation method can adopt one of the fixed global threshold method, histogram global threshold method, and local threshold method to obtain the segmented image pasteimage_seg.

[0055] Step 43: Identify multiple laser lines in the image segmented by pasteimage_seg, including the laser line laserline1 on the front ground, the laser line laserline2 on the side ground, and the laser line laserline3 on the rear ground. Each laser line is output in the format of a pixel point coordinate array [u, v]. The identification method can adopt one or more of the gray center of gravity method, Steger algorithm, deep convolutional neural network, etc.

[0056] Step 44: According to the coordinate arrays of the laser lines laserline1 and laserline2, calculate their overlapping pixel points, and use the central coordinates [ucenter1, vcenter1] of the overlapping pixel points as the laser intersection point S1; similarly, calculate the coordinates [ucenter1, vcenter1] of the laser intersection point S2 according to the coordinate arrays of laserline2 and laserline3.

[0057] As a preferred technical solution, in this step, upimage1 and upimage2 can be stitched using methods such as sift feature extraction and hog feature extraction. After stitching, the laser line on the side ground should be continuous and have no obvious dislocation.

[0058] Step 5: Based on the stitched image, identify the edge of the road curb that the roller is close to, and use the fitting method to obtain the edge line.

[0059] When the road roller is close to the road edge, identify the edge of the road edge in the spliced image pasteimage, and fit it with a straight line or a curve to obtain the edge line L2.

[0060] As a preferred technical solution, in this step, the identification of the road edge in pasteimage can adopt one or more of deep convolutional neural network and hough transform.

[0061] Step 6: Calculate the pixel point interval between the straight line and the edge line, and inversely calculate the real plane distance corresponding to the pixel point interval according to the association relationship to obtain the edge-attached state of the road roller.

[0062] Calculate the pixel point interval pixdist between L2 and L1, and inversely calculate the real plane distance corresponding to pixdist according to the association relationship between the pixel point and the real plane distance in S5, which is the edge-attached state of the road roller.

[0063] During the calculation of pixdist, for each pixel point coordinate [u, v] in L1, draw a perpendicular line L1' to L1, and calculate the intersection coordinate [u', v'] of L1' and L2. According to each pixel point in L1 and the corresponding intersection coordinate, calculate its Euclidean distance to form an array pixdist of pixel point intervals. Then, according to the association relationship between the pixel point and the real plane distance in S5, inversely calculate the real plane distance array dist corresponding to pixdist, which is the edge-attached state of the road roller. According to the obtained dist array, further calculate its minimum value min(dist), which is marked as the minimum edge-attached distance. The road roller operator can control the road roller according to the minimum edge-attached distance.

[0064] The above-mentioned series of calculations such as calibration, laser line identification, intersection identification, and distance discrimination of the obtained photos realize the automatic edge-attached detection of the road roller. A variety of methods are adopted in the calculation process to realize the accurate calculation of the distance between the working wheel edge and the road edge. The calculation results can provide reliable data support for the high-precision control of the edge-attached rolling of the road roller, and there are also many choices for a variety of methods, which has good practicability for the actual application scenario.

[0065] When the above functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs.

[0066] Those skilled in the art should understand that the embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memories, CD-ROMs, optical memories, etc.) that contain computer-usable program codes. The solutions in the embodiments of the present invention can be implemented in various computer languages. For example, object-oriented programming languages such as Java and interpreted scripting languages such as JavaScript.

[0067] The present invention is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to the embodiments of the present invention. It should be understood that each flow and / or block in the flowchart and / or block diagram, as well as the combination of flows and / or blocks in the flowchart and / or block diagram, can be realized by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for realizing the functions specified in Figure 1 one or more flows and / or blocks Figure 1 one or more blocks.

[0068] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device, and the instruction device realizes the functions specified in Figure 1 one or more flows and / or blocks Figure 1 one or more blocks.

[0069] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus, so that a series of operation steps are executed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions for implementing the functions specified in one process or a plurality of processes and / or blocks Figure 1 one process or a plurality of processes and / or blocks Figure 1 in one block or a plurality of blocks.

[0070] Although the preferred embodiments of the present invention have been described, additional changes and modifications can be made by those skilled in the art once they learn of the basic inventive concept. Therefore, the appended claims are intended to be construed to cover the preferred embodiments as well as all changes and modifications falling within the scope of the present invention.

[0071] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention is also intended to include these modifications and variations.

Claims

1. A method for detecting the edge - sticking state of a road roller based on line laser and high - definition camera, characterized in that, It includes the following steps: Obtain the front-side and rear-side photos of the ground where the roller is located collected by the high-definition camera for the first time; Obtain the front-side and rear-side photos collected again by the high-definition camera with overlapping photos, where the front-side and rear-side photos obtained again are obtained after adjusting the high-definition camera with multiple laser lines; Based on the front-side and rear-side photos obtained for the first time and again, calibrate the high-definition camera to obtain an image from a top-down perspective and the correlation between pixel points and the real plane distance; Stitch the images from the top-down perspective to obtain a stitched image, identify the intersection points of the multiple laser lines, and connect the intersection points to obtain a straight line; The specific steps for identifying the intersection points of the multiple laser lines include: Preprocess the stitched image to obtain a filtered image; Process the filtered image using the threshold segmentation method to obtain a segmented image; Use an identification method to identify multiple laser lines in the segmented image; According to the coordinate arrays of the laser lines, calculate the overlapping pixel points, and use the overlapping pixel points as the intersection points of the laser lines; Based on the stitched image, identify the edge of the road curb that the roller is close to, and use a fitting method to obtain an edge line; Calculate the pixel point interval between the straight line and the edge line, and inversely calculate the real plane distance corresponding to the pixel point interval according to the correlation to obtain the edge-attachment state of the roller.

2. The method for detecting the edge - sticking state of a road roller based on line laser and high - definition camera according to claim 1, wherein, There are three laser lines, which are respectively projected onto the ground in front of the roller, the ground behind the roller, and the ground on the side of the roller. The laser lines shown on the side ground intersect with the laser lines on the front ground and the rear ground respectively, and the intersection angles are right angles.

3. The method for detecting the edge attachment state of a roller based on line laser and high-definition camera according to claim 1, characterized in that, When calibrating, use the black and white grid calibration method, and the size of a single grid in the black and white grid does not exceed 20 cm.

4. The method for detecting the edge - sticking state of a roller based on line laser and high - definition camera according to claim 1, characterized in that, Use the sift feature extraction method or the hog feature extraction method for stitching.

5. The edge attachment state detection method of a road roller based on line laser and high-definition camera according to claim 1, wherein, The above-mentioned preprocessing includes one or more of Gaussian filtering and wavelet filtering.

6. The method for detecting the edge - sticking state of a road roller based on line laser and high - definition camera according to claim 1, wherein, The above-mentioned threshold segmentation method includes the fixed global threshold method, the histogram global threshold method, and the local threshold method.

7. A method for detecting the edge - adhering state of a roller based on line laser and high - definition camera according to claim 1, characterized in that, The above-mentioned identification method includes one or more of the gray center of gravity method, the Steger algorithm, and the deep convolutional neural network.

8. A method for detecting the edge - sticking state of a road roller based on line laser and high - definition camera according to claim 1, wherein, The above-mentioned fitting method includes the straight line fitting method and the curve fitting method.

9. A method for detecting the edge - sticking state of a roller based on line laser and high - definition camera according to claim 1, characterized in that, Use one or more of the deep convolutional neural network and the Hough transform to identify the edge of the road curb that the roller is close to.

Images