Colorless Difference Intelligent Patching System and Method for Road Surface Cracks Based on Image Recognition

Abstract

The present invention discloses a colorless difference intelligent repair system and method for pavement cracks based on image recognition, including: a chassis unit, an asphalt pavement grooving unit, an asphalt pavement repair material mixing unit, and a control unit; the asphalt pavement grooving unit and the asphalt pavement repair material mixing unit are respectively fixed on the chassis unit, and the control unit is respectively connected to the asphalt pavement grooving unit and the asphalt pavement repair material mixing unit. The control unit recognizes cracks based on the surface image information of the asphalt pavement, fits the crack function curve according to the distribution of crack pixel pairs; and segments and plans the grooving trajectory and grooving depth of the crack groove according to the curvature of the fitted curve. The present invention integrates functions such as crack recognition, determination of grooving trajectory, and calculation of the dosage of repair materials, and can automatically control the cutting tool to perform grooving cutting according to the determined grooving trajectory and depth, realizing the intelligent recognition and repair of asphalt pavement cracks.

Description

Technical Field

[0001] The present invention relates to the technical field of road maintenance, and particularly to a colorless intelligent repair system and method for road surface cracks based on image recognition. Background Technique

[0002] The statements in this part only provide background technical information related to the present invention and do not necessarily constitute prior art.

[0003] Affected by climate, humidity and traffic load, irregular cracks will appear on asphalt pavements, thereby reducing the service life and comfort of asphalt pavements. Therefore, timely maintenance of road surface cracks is required.

[0004] Currently, the asphalt pavement crack repair technology mainly relies on manual slotting repair. However, the slotting construction causes great damage to the structure of the asphalt pavement, and the degree of automation is low. At the same time, there will be a large color difference between the repaired part of the asphalt pavement and the original pavement.

[0005] In addition, the asphalt pavement repair methods disclosed in the prior art often only rely on the mechanical structure of the repair equipment and lack intelligent algorithms. Although the prior art has disclosed the application of neural network algorithms to the intelligent repair of asphalt pavements, there are still problems of low recognition accuracy and inability to solve the problem of color deviation between the repaired pavement and the original pavement. Summary of the Invention

[0006] To solve the above problems, the present invention proposes a colorless intelligent repair system and method for road surface cracks based on image recognition. The precise segmentation of asphalt pavement cracks is achieved through an instance segmentation algorithm, and the crack slots are cut according to the shape of the cracks, reducing the damage to the road surface structure by the slotting repair method; the color of the asphalt cold mix is adjusted using staining pigments to make the color of the repaired road surface area consistent with that of the original road surface area.

[0007] In some embodiments, the following technical solutions are adopted:

[0008] A colorless intelligent repair system for road surface cracks based on image recognition includes: a chassis unit, an asphalt pavement slotting unit, an asphalt pavement repair material mixing unit, and a control unit; the asphalt pavement slotting unit and the asphalt pavement repair material mixing unit are respectively fixed on the chassis unit, and the control unit is respectively connected to the asphalt pavement slotting unit and the asphalt pavement repair material mixing unit;

[0009] Among them, the asphalt pavement slotting unit includes: a cross slide and a cross rail arranged below the chassis unit, and the cross slide can move along the cross rail; a camera module and a hydraulic lifting module are respectively arranged on the cross slide, and the hydraulic lifting module is connected to the cutting module and can drive the cutting module to move up and down;

[0010] The control unit identifies cracks based on the surface image information of the asphalt pavement, fits the crack function curve according to the distribution of crack pixel points, and plans the groove track and groove depth of the crack groove in sections according to the curvature of the fitting curve.

[0011] Optionally, the asphalt pavement repair material mixing unit includes: a mixing barrel, a mixing tool and a material delivery pipe arranged above the base frame unit; the mixing tool can rotate in the mixing barrel under the drive of the motor, and the material delivery pipe is connected to the mixing barrel for delivering materials to the outside.

[0012] Optionally, the control unit receives the asphalt pavement surface image information acquired by the camera module and performs asphalt pavement crack image recognition; the specific process is as follows:

[0013] Preprocessing the received asphalt pavement surface image, inputting the preprocessed image into a trained crack recognition model to obtain a crack recognition result;

[0014] The training process of the crack recognition model is as follows:

[0015] A simple crack dataset and a complex crack dataset are constructed respectively. The crack recognition model is first trained using the simple crack dataset, and then the complex crack dataset is used to train the crack recognition model.

[0016] Optionally, the control unit fits the crack function curve according to the distribution of crack pixel pairs, specifically:

[0017] The coordinate pairs of the detected crack pixel points are extracted from the image, and the bifurcated cracks are divided into multiple small segments with the crack bifurcation as the node;

[0018] The least square method is used to preliminarily fit the crack function curve of each small section of asphalt pavement, traverse all pixel pairs, and eliminate outliers;

[0019] The support vector regression algorithm is used to refit the asphalt pavement crack function curve of each small section, and the support vector regression algorithm boundary zone is used as the dividing line between the asphalt pavement cracks and the rest of the area.

[0020] Optionally, the slotting trajectory and slotting depth of the crack slot are planned in sections according to the curvature of the fitting curve, specifically:

[0021] The curvature of each point on the regression curve of the support vector regression algorithm is calculated, and the point in the curve whose curvature is greater than the set threshold is set as the inflection point;

[0022] The cracks in the asphalt pavement are divided into several small segments using the inflection point as the dividing line, and multiple crack areas are obtained;

[0023] Draw the minimum bounding rectangle of each crack area as the contour of the crack slot;

[0024] Determine the conversion ratio between the pixel size and the actual size, and judge the damage degree of the road surface crack to the structural layer according to the width of the fitting curve regression band in the support vector regression algorithm, and then recommend the grooving depth of the crack slot to the user.

[0025] Optionally, the control unit drives the cross slide and the hydraulic lifting module of the asphalt pavement grooving unit to move according to the grooving track and the grooving depth;

[0026] During the driving process, unify the coordinates of the cutting tool in the cutting module and the camera in the camera module:

[0027]

[0028] Among them, T ab represents the coordinate transformation matrix; d1 represents the distance between the cutting tool and the camera, the lengths of the cutting tool and the camera are l1 and l2 respectively, and the length of the hydraulic rod in the hydraulic lifting module is l3. The moving amounts of the cutting tool along the x, y, and z axes in the base coordinate system are d x 、d y 、d z .

[0029] Optionally, the control unit stores a corresponding table of pixel values of the asphalt pavement repair material in the RGB space under different dosages of colored dyes;

[0030] The control unit calculates the average pixel value of the road surface in the current area, and determines the dosages of the colored dye and the asphalt pavement repair material according to the average pixel value of the road surface in the current area, the grooving area and the depth.

[0031] In some other embodiments, the following technical solution is adopted:

[0032] An intelligent colorless repair method for road surface cracks based on image recognition, including:

[0033] Obtain the surface image information of the asphalt pavement and perform preprocessing;

[0034] Based on the image information, use the trained crack recognition model to obtain the crack recognition result;

[0035] Based on the crack recognition result, fit the crack function curve according to the distribution of crack pixel point pairs; segmentally plan the grooving track and the grooving depth of the crack slot according to the curvature of the fitting curve;

[0036] According to the grooving track and the grooving depth, drive the cross slide and the hydraulic lifting module of the asphalt pavement grooving unit to move, so as to drive the cutting tool to groove and cut along the grooving track and the grooving depth;

[0037] Calculate the average value of the pavement pixels in the current area, and determine the dosages of the colored dye and the asphalt pavement repair material according to the average value of the pavement pixels in the current area, the grooving area and the depth; mix the colored dye and the asphalt pavement repair material according to the dosages to achieve colorless repair of the asphalt pavement.

[0038] Optionally, calculate the average value of the pavement pixels in the current area, and determine the dosages of the colored dye and the asphalt pavement repair material according to the average value of the pavement pixels in the current area, the grooving area and the depth. The specific process is as follows:

[0039] Remove the crack area segmented from the asphalt pavement surface image, and calculate the average value of the pixels in the remaining area;

[0040] Take the average of the average values of the pixels calculated from multiple frames of images to obtain the final average value of the pavement pixels;

[0041] Perform linear interpolation operations on the asphalt pavement surface image in the three RGB channels respectively, calculate the blending amounts of the three colored dyes, and calculate the final blending ratio of the colored dyes by using the method of weighted average; calculate the dosage of the asphalt repair material based on the grooving width and length of the crack groove, and then determine the dosage of the colored dye according to the blending ratio.

[0042] Optionally, segment the grooving trajectory and depth of the crack groove according to the curvature of the fitting curve in sections, specifically:

[0043] Calculate the curvature of each point on the regression curve of the support vector regression algorithm, and set the points with curvature greater than the set threshold in the curve as inflection points;

[0044] Take the inflection points as the dividing lines to divide the asphalt pavement cracks into several small sections to obtain multiple crack areas;

[0045] Draw the minimum circumscribed rectangle of each crack area as the contour of the crack groove;

[0046] Determine the conversion ratio between the pixel size and the actual size, and judge the damage degree of the pavement crack to the structural layer according to the width of the regression band of the fitting curve in the support vector regression algorithm, and then recommend the grooving depth of the crack groove to the user.

[0047] Compared with the prior art, the beneficial effects of the present invention are:

[0048] (1) The colorless intelligent repair system for pavement cracks based on image recognition of the present invention integrates functions such as crack recognition, grooving trajectory determination, and dosage calculation of repair materials, and can automatically control the cutting tool to perform grooving cutting according to the determined grooving trajectory and depth, realizing intelligent recognition and repair of asphalt pavement cracks.

[0049] (2) The present invention cuts the crack groove according to the shape of the crack, reducing the damage to the road surface structure by the grooving repair method and reducing the amount of repair materials used.

[0050] (3) Based on the area and depth of the groove, by automatically identifying the pixels of the road surface, the present invention automatically determines the doping ratio and dosage of the repair material and the colored dye, enabling the color of the repaired road surface area to be consistent with that of the original road surface area and achieving colorless difference automatic repair.

[0051] Other features and advantages of the present invention will be partially given in the following description, partially become obvious from the following description, or be understood through the practice of this aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 It is a schematic structural diagram of a colorless difference intelligent repair system for road surface cracks based on image recognition in an embodiment of the present invention;

[0053] Figure 2 It is a front view of a colorless difference intelligent repair system for road surface cracks based on image recognition in an embodiment of the present invention;

[0054] Figure 3 is Figure 1 A - A cross-sectional view in

[0055] Figure 4 is Figure 1 B - B cross-sectional view in

[0056] Figure 5 It is a schematic diagram of the positional relationship between the cutting tool and the camera in an embodiment of the present invention;

[0057] Figure 6 It is an example segmentation sample of an asphalt road surface in an embodiment of the present invention;

[0058] Among them, 1. Universal wheel; 2. Camera; 3. Cutting tool; 4. Cutting tool motor; 5. Cross slide; 6. Cross slide rail; 7. Stirring barrel; 8. Stirrer motor; 9. Trolley operating rod; 10. Trolley chassis; 11. Stirrer tool; 12. Stirring barrel discharge valve; 13. Trolley handle; 14. Stirrer wire; 15. Power supply; 16. Feeding pipe; 17. Feeding nozzle; 18. Baffle; 19. Trolley support; 20. Hydraulic rod; 21. Stirring barrel base; 22. Stirring barrel lid; 23. Stirring barrel wire; 24. Controller; 25. Human - machine interaction panel. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0059] It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used in the present invention have the same meaning as commonly understood by those of ordinary skill in the technical field to which this application belongs.

[0060] It should be noted that the terms used herein are for the purpose of describing specific embodiments only and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular forms are also intended to include the plural forms. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they specify the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0061] Example 1

[0062] In one or more embodiments, a colorless intelligent pavement crack repair system based on image recognition is disclosed, including: a chassis unit, an asphalt pavement grooving unit, an asphalt pavement repair material mixing unit, and a control unit; the asphalt pavement grooving unit and the asphalt pavement repair material mixing unit are respectively fixed on the chassis unit, and the control unit is respectively connected to the asphalt pavement grooving unit and the asphalt pavement repair material mixing unit;

[0063] Among them, the control unit identifies cracks based on the image information of the asphalt pavement surface, fits the crack function curve according to the distribution of crack pixel pairs; and plans the grooving trajectory and grooving depth of the crack groove in segments according to the curvature of the fitted curve.

[0064] Specifically, in combination with Figure 1 and Figure 2 , the system structure of this embodiment is specifically as follows:

[0065] The chassis unit includes a trolley chassis 10, universal wheels 1, trolley struts 19, a trolley handle 13, a trolley operating rod 9, a braking device, etc. The trolley base 10 is connected to the trolley struts 19 below for installing the asphalt pavement grooving unit and supporting the trolley chassis; the universal wheels 1 are installed below the trolley struts 19 to facilitate the movement and transportation of the equipment; the trolley handle 13 and the trolley operating rod 9 are installed above the trolley chassis 10 for holding the trolley operating rod 9 or the trolley handle 13 to push the trolley to move; the braking device is placed below the universal wheels during use for fixing the chassis unit.

[0066] The asphalt pavement grooving unit includes: a cross slide 5 and a cross rail 6 provided below the chassis unit, and the cross slide can move along the cross rail; a camera module and a hydraulic lifting module are respectively provided on the cross slide, and the hydraulic lifting module is connected to the cutting module and can drive the cutting module to move up and down.

[0067] In this embodiment, in combination withFigure 3 and Figure 4 As shown in the structural schematic diagram of the cross slide in Figure 4 , the camera module mainly includes a camera 2, and the cutting module mainly includes a cutting tool 3 and a cutting tool motor 4; the hydraulic lifting module is a hydraulic rod 20, and the baffle 18 is installed on the trolley support 19 for installing the cross slide rail 6 and protecting the precision components of the asphalt pavement grooving system; the cross slide rail 6 is installed on the baffle 18 for pulling the cross slide to move in the horizontal direction; the cross slide 5 is installed on the cross slide rail 6 for pulling the camera 2 and the hydraulic rod 20 to move in the horizontal direction; the hydraulic rod 20 is installed below the cross slide 5, with a camera 3 installed on its left side and a cutting tool motor 4 and a cutting tool 3 installed below it for controlling the up and down movement of the cutting tool. At the same time, the cutting tool motor 4 and the cutting tool 3 can also move horizontally along with the hydraulic rod 20. The camera 2 is installed on the left side of the hydraulic rod 20 and does not move up and down with the hydraulic rod. It is used to collect the surface information of the asphalt pavement, perform asphalt pavement crack image recognition, and plan the grooving path of the asphalt pavement crack based on the current pixel value of the asphalt pavement; the cutting tool motor 4 and the cutting tool 3 are installed below the hydraulic rod 20. The cutting tool motor 4 is used to drive the cutting tool 3, and the cutting tool 3 is used to groove the asphalt pavement crack area.

[0068] The asphalt pavement repair material mixing unit includes a mixing barrel 7, a mixer motor 8, mixer cutters 11, a mixing barrel discharge valve 12, mixer wires 14, a material conveying pipe 16, a material delivery nozzle 17, a mixing barrel base 21, a mixing barrel lid 22, a mixing barrel wire 23, etc. The mixing barrel base 21 is welded above the bottom frame 10 of the handcart for fixing the mixing barrel 10; the mixing barrel 7 is detachably nested above the mixing barrel base 21. The mixing barrel 7 has a heating function and is used for placing, heating, and mixing asphalt pavement repair materials and colored dyes; the mixing barrel discharge valve 12 is installed at the lower left of the mixing barrel 7 for connecting the material conveying pipe 16 and controlling whether the asphalt pavement repair material can flow out from this port; the material conveying pipe 16 is detachably connected to the mixing barrel discharge valve 12 for conveying the asphalt pavement repair material; the material delivery nozzle 17 is detachably installed on the material conveying pipe 16. By replacing the material delivery nozzles with different opening sizes, the flow rate of the asphalt pavement repair material is controlled, which is easy to finely control the filling effect of the asphalt pavement repair material on the crack groove and reduce or avoid voids; the mixing barrel lid 22 is detachably placed on the mixing barrel 7 for supporting the mixer motor 8 and the mixer cutters 11 and for preventing the asphalt pavement repair material from splashing out; the mixer motor 8 is welded above the mixing barrel lid 22 for driving the mixer cutters 11; the mixer cutters 11 are connected below the mixing barrel lid 22 for mixing the mixture of the asphalt pavement repair material and the colored dye. The mixing barrel wire 23 is detachably connected to the lower right of the mixing barrel 7 for supplying power to heat the asphalt pavement repair material in the mixing barrel 7. The asphalt pavement repair material mixing system is designed with detachable components, which is convenient for removing the mixing barrel 7 when the fluidity of the asphalt repair material is poor, manually taking out the materials therein, and filling the crack groove, and is also convenient for cleaning the mixing barrel 7 and the mixer cutters 11.

[0069] The control unit includes a power supply 15, a controller 24, a human-machine interaction panel 25, etc. The power supply 15 uses multiple lead-acid batteries or lithium batteries connected in series to form a power supply system and is placed on the bottom frame 10 of the handcart to supply power to the cutting tool motor 3, the cross slide 5, the mixing barrel 7, and the mixer motor 8; the controller 24 processes the image data collected by the imaging device, drives the automatic operation of the slide system, and gives the mixing ratio of the asphalt pavement repair material and the colored dye. The human-machine interaction panel 25 is installed above the controller 24 for giving the visual segmentation results of asphalt pavement crack examples, the grooving trajectory of the grooving system, the current pixel value of the asphalt pavement, and the mixing dosage of the asphalt pavement repair material and the colored dye, and accepting the adjustment of various specific parameters, such as grooving depth, mixing time, heating temperature, and time, etc.

[0070] In this embodiment, the surface image information of the asphalt pavement is obtained through the imaging module, and the control unit receives the surface image information of the asphalt pavement obtained by the imaging module for preprocessing. The preprocessed image is input into the trained crack recognition model to obtain the crack recognition result.

[0071] As a specific implementation, the crack recognition model of this embodiment is based on the Detectron2 algorithm. Detectron2 is a deep learning toolkit for object detection and instance segmentation developed by Facebook AI Research (FAIR).

[0072] S11: Construct a dataset for asphalt pavement image recognition and precise segmentation. The specific construction process is as follows:

[0073] S11-1: Select suitable weather and sections of asphalt pavement with more cracks, and use drone equipment to collect asphalt pavement image data.

[0074] S11-2: Use image processing algorithms to preprocess the collected images, including but not limited to image filtering, image enhancement, image cutting, image filling, and image rotation. The processed images should be converted into sub-images of size 640×640.

[0075] S11-3: Check and screen the cut images, and select the images with complete cracks and clear shooting. Divide the selected cracks into two groups, one group for simple cracks and one group for complex cracks, and label them separately. In this embodiment, simple cracks refer to horizontal or vertical cracks without bifurcations, or regular reticular cracks; complex cracks refer to cracks with bifurcations, or oblique cracks, or multiple cracks intertwined but the crack shapes do not form reticular cracks.

[0076] S11-4: Use the image annotation tool labelme to annotate it, and invite an experienced inspector to check all the annotation results.

[0077] S11-5: Use the label conversion program to convert the labelme labels into the format of the coco dataset, and divide the dataset into a training set, a validation set, and a test set, and assemble them into three independent JSON files, train.json, vaild.json, and test.json.

[0078] S21: Train the crack recognition model based on the constructed dataset. The specific training process is as follows:

[0079] S12-1: Configure the deep learning environment and install Detectron2 in the library function. Use the simple crack dataset to train the model for 200 rounds to guide the model to learn the main features of asphalt pavement cracks. Adopt an early stopping strategy. Calculate the loss function value and prediction accuracy of the model on the simple crack validation set for each round. When the calculated loss function value and prediction accuracy are within a certain threshold, stop the training of the model.

[0080] S12-2: Train the model for 100 rounds using the complex crack database to improve the accuracy of the model in identifying complex asphalt pavement cracks. Also adopt the early stopping strategy. Calculate the loss function value and prediction accuracy of the model on the complex crack validation set for each round. When the calculated loss function value and prediction accuracy are within a certain threshold, stop the training of the model.

[0081] S12-3: Evaluate the training effect of the model on the test set to determine whether the model meets the requirements for accurate segmentation of asphalt pavement cracks.

[0082] S13: Embed the trained algorithm into the controller. The specific process is as follows:

[0083] S13-1: Select Raspberry Pi as the hardware platform of the embedded controller and install the running environment on the embedded system, including the required ubuntu operating system, Python and related libraries, ROS operating system, and detectron2 algorithm.

[0084] S13-2: Export the trained detectron2 algorithm as an ONNX model and store the model in the controller. Implement model inference in the controller, including loading the exported Detectron2 model and setting up a real-time detection loop. Use the model for real-time image detection, capture the video stream of the camera, pass the image to the model for inference, and process the detection results.

[0085] S13-3: Control the actuator or device according to the detection results. This involves driving the motor, performing actions, or sending signals to achieve the required control tasks. And set up monitoring and logging to track the system performance and problems, and perform regular maintenance to ensure the reliability of the system.

[0086] In this embodiment, the control unit identifies cracks based on the asphalt pavement surface image information, fits the crack function curve according to the distribution of crack pixel pairs; segments and plans the grooving trajectory and grooving depth of the crack groove according to the curvature of the fitted curve; the control unit drives the cross slide and hydraulic lifting module of the asphalt pavement grooving unit to move according to the grooving trajectory and grooving depth.

[0087] As a specific implementation scheme, the method for fitting the crack function curve according to the distribution of crack pixel pairs is as follows:

[0088] S21-1: Use the deep learning algorithm in the controller to detect asphalt pavement cracks. The specific detection process is as Figure 6 shown in (a)-(c) below. Extract the coordinate pairs of the detected crack pixels from the image, and take the crack bifurcation as the node to segment the complex crack into multiple simple small segments.

[0089] S21-2: Use the least squares method to preliminarily fit the asphalt pavement crack function curve of each small segment, traverse all pixel pairs, and eliminate the outliers among them. The least squares method can remove the identified noise points. During actual identification, some noise points may be identified, such as some scattered points far from the crack. These need to be removed before actual modeling. The purpose of the preliminary fitting in this embodiment is to remove the points that are too far from the fitting curve.

[0090] S21-3: Use the support vector regression algorithm to refit the asphalt pavement crack function curve of each small segment, and use the boundary band of the support vector regression algorithm as the dividing line between the asphalt pavement crack and the rest of the area. The support vector regression algorithm will give a boundary band, that is, fit a curve, and then there will be an upper and lower limit. The data within the upper and lower lines is retained, and the data outside is removed as outliers.

[0091] The method for segmentally planning the grooving trajectory of the crack groove according to the curvature of the fitting curve is as follows:

[0092] S22-1: Calculate the curvature of each point on the regression curve of the support vector regression algorithm, and set the points with curvature greater than a certain threshold in the curve as inflection points. Use the inflection points as the dividing lines to divide the asphalt pavement crack into several relatively straight small segments, and draw the minimum circumscribed rectangle of each crack segment area as the contour of the crack groove. In this embodiment, the grooving path is designed according to the shape and trend of the crack, saving the amount of asphalt pavement repair material and reducing the damage to the asphalt pavement structure.

[0093] S22-2: Determine the conversion ratio between the pixel size and the actual size. According to the width of the regression band of the fitting curve in the support vector regression algorithm, judge the damage degree of the pavement crack to the structural layer, and then recommend the grooving depth of the crack groove to the user. The user can select the grooving depth according to the actual situation to reduce the damage to the asphalt pavement structure. In this embodiment, the pixel size is the number of pixel points occupied by the crack, and the actual size is the actual width and length of the crack.

[0094] The method for driving the cross slide 5, the pressure rod 20, and the grooving motor 4 to groove according to the planned grooving trajectory and the grooving depth specified by the user, combined with Figure 5 , is as follows:

[0095] S23-1: It is necessary to unify the coordinates of the cutting tool 3 and the camera 2. The grooving system is built based on a three-degree-of-freedom rectangular coordinate system. The z-axis of the cutting tool 3 is rigidly connected to the cross slide 5. During grooving operation, the lengths of the camera 2 and the cutting tool 3 are l1 and l2 respectively, the length of the hydraulic rod 20 is l3, and the distance between the camera 2 and the cutting tool 3 is d1. The camera 2 and the cutting tool 3 are rigidly connected in the xy-axis direction to establish the end coordinate system O of the cutting tool.b -x b y t z b , the camera coordinate system O a -x a y a z a , assuming that the moving amounts of the cutting tool along the x, y, and z axes in the base coordinate system are d x , d y , d z , the translational transformation relationship between the camera coordinate system and the end coordinate system of the cutting tool can be obtained.

[0096]

[0097] Among them, Tab is the coordinate transformation matrix, O b = T ab ·O a .,, O b represents the coordinate of the cutting tool, and O a represents the coordinate of the camera.

[0098] S23-2: Use a PID controller to manage the position and speed of the cross slide and the hydraulic rod, and adjust the output signal according to feedback signals such as the sensor position to maintain the required position and speed. The cross slide is driven by a motor, and the hydraulic rod is driven by a hydraulic system. The advantage of doing this is that when performing grooving work, the height of the cutting tool remains unchanged in the vertical direction during each grooving operation, while it needs to move frequently in the horizontal direction. The motor drive can control the position of the cutting tool more precisely, and the hydraulic rod has a longer service life while meeting the movement requirements in the vertical direction.

[0099] In this embodiment, the control unit stores a corresponding table of the pixel values of the asphalt pavement repair material in the RGB space under different dosages of colored dyes; the control unit calculates the average pixel value of the road surface in the current area, and determines the dosages of the colored dye and the asphalt pavement repair material based on the average pixel value of the road surface in the current area, the grooving area, and the depth.

[0100] As a specific implementation, the corresponding table of the pixel values of the asphalt pavement repair material in the RGB space under different dosages of colored dyes is specifically:

[0101] In the range of 1 - 7%, ten different dosages of pigments, namely 0%, 0.75%, 1.5%, 2.25%, 3%, 3.75%, 4.5%, 5.25%, 6%, 6.75%, are selected with an equal difference gradient of 0.75%. Using light-colored asphalt, aggregates, and mineral powder as raw materials, the above ten different pigment dosages are respectively incorporated to replace part of the mineral powder to prepare the mixture; a color difference meter is used to determine the pixel values of the color of the asphalt mixture in the RGB space under different pigment dosages.

[0102] Calculate the average value of the pavement pixels in the current area. The specific process is as follows:

[0103] Take the image within the image recognition label box as the region of interest, perform image preprocessing on the acquired image, which is achieved by methods such as image segmentation and edge detection. Remove the segmented crack region, calculate the average value of the pixels in the remaining region, and for multiple frames of images in the collected video, calculate the average value of the pixels of multiple frames as the final calculation result.

[0104] Determine the dosages of the colored dye and the asphalt pavement repair material based on the average value of the pavement pixels in the current area, the grooving area, and the depth. The specific process is as follows:

[0105] S3-1: Perform linear interpolation operations on the color image in the three RGB channels respectively, calculate the blending amounts of the three colored dyes, and use the weighted average method to calculate the final blending ratio of the colored dyes.

[0106] S3-2: According to the selected crack groove cutting depth by the user, the grooving width and length of the planned crack groove, calculate the dosage of the asphalt repair material, and then determine the dosage of the colored dye according to the blending ratio, and feedback it to the user through the human-computer interaction interface 25.

[0107] S3-3: Put the asphalt repair material and the colored dye into the mixing barrel 7 in proportion, and start the mixer to ensure that the asphalt and the aggregate are fully mixed.

[0108] The pavement crack colorless difference intelligent repair system based on image recognition in this embodiment integrates functions such as crack recognition, grooving trajectory determination, and calculation of the dosage of repair materials, and can automatically control the cutting tool to perform grooving cutting according to the determined grooving trajectory and depth, realizing the intelligent recognition and repair of asphalt pavement cracks.

[0109] Embodiment 2

[0110] In one or more embodiments, an image recognition-based pavement crack colorless difference intelligent repair method is disclosed, including:

[0111] Obtain the surface image information of the asphalt pavement and perform preprocessing;

[0112] Based on the image information, use the trained crack recognition model to obtain the crack recognition result;

[0113] Based on the crack recognition result, according to the distribution of the crack pixel point pairs, fit the crack function curve; segmentally plan the grooving trajectory and grooving depth of the crack groove according to the curvature of the fitted curve;

[0114] Drive the cross slide and hydraulic lifting module of the asphalt pavement grooving unit to move according to the grooving track and grooving depth, so as to drive the cutting tool to perform grooving cutting along the grooving track and grooving depth;

[0115] Calculate the average pixel value of the road surface in the current area, and determine the dosages of the colored dye and asphalt pavement repair material according to the average pixel value of the road surface in the current area, the grooving area and depth; mix the colored dye and asphalt pavement repair material according to the dosages to achieve colorless difference repair of the asphalt pavement.

[0116] The specific implementation manner of the above process is the same as that in Embodiment 1 and will not be elaborated here.

[0117] As a more specific implementation scheme, after using an intelligent inspection device or other methods to determine the specific position of the asphalt pavement crack, use a carrier vehicle to bring the intelligent asphalt pavement crack grooving and repairing device to the asphalt pavement maintenance construction site, clean the asphalt pavement with cracks, and remove the sundries and dust on the road surface. And take corresponding safety measures, set warning signs near the construction area to ensure the safety of the working area.

[0118] Move the device to the asphalt pavement crack, use the braking facility to fix the device under the universal wheels, start the device to identify the asphalt pavement crack, plan the movement path of the cutting tool 3, feedback the recommended grooving depth of the crack groove to the user, and perform the crack groove grooving operation according to the user's selection. At the same time, on the human-machine interaction interface 25, the user can adjust the movement path of the cutting tool by adjusting the curve curvature threshold. When grooving the asphalt pavement crack, pay attention to spraying atomized water on the grooving part to reduce the dust generated by the asphalt pavement operation.

[0119] After cutting is completed, move the device away from the asphalt pavement crack, clean the remaining asphalt fragments, clean the gravel and waste residues in and around the crack groove, there should be no mud and other sundries in the pothole, and the repaired crack groove should have neat cut edges, and the removal of waste residues should reach the solid and firm surface.

[0120] According to the recommended dosages of the colored dye and asphalt repair material given by the human-machine interaction interface 25, place the materials in the mixing barrel 7, drive the heating device built in the mixing barrel 7 to heat the materials, and drive the agitator motor 8 to stir the materials.

[0121] Control the feeding pipe 16 to fill the asphalt repair material into the crack groove, and use tools such as shovels and scrapers to evenly distribute the asphalt pavement repair material in the construction area. Use a compactor or roller to compact the cold patch material to ensure that the asphalt cold patch material is fully bonded to the surrounding road surface. Use road maintenance equipment to trim the construction area to make it flush with the surrounding road surface.

[0122] Clean up the construction site to ensure that no asphalt pavement crack repair materials or tools are left on the road. Maintain and clean the mixer and construction equipment to ensure their performance and lifespan during subsequent use.

[0123] Although the specific implementation manners of the present invention have been described in conjunction with the accompanying drawings, it is not a limitation on the protection scope of the present invention. Those skilled in the art should understand that various modifications or deformations that can be made without creative efforts on the basis of the technical solution of the present invention are still within the protection scope of the present invention.

Claims

1. An intelligent colorless crack repair system for road surfaces based on image recognition, characterized in that, Including: A chassis unit, an asphalt pavement grooving unit, an asphalt pavement repair material mixing unit, and a control unit; The asphalt pavement grooving unit and the asphalt pavement repair material mixing unit are respectively fixed on the chassis unit, and the control unit is respectively connected to the asphalt pavement grooving unit and the asphalt pavement repair material mixing unit; Among them, the asphalt pavement grooving unit includes: a cross slide and a cross rail arranged below the chassis unit, and the cross slide can move along the cross rail; a camera module and a hydraulic lifting module are respectively arranged on the cross slide, and the hydraulic lifting module is connected to a cutting module and can drive the cutting module to move up and down; The control unit identifies cracks based on the asphalt pavement surface image information, fits a crack function curve according to the distribution of crack pixel pairs; segments and plans the grooving trajectory and grooving depth of the crack groove according to the curvature of the fitted curve; the control unit calculates the average pixel value of the road surface in the current area, and determines the dosage of the colored dye and the asphalt pavement repair material according to the average pixel value of the road surface in the current area, the grooving area and depth; Among them, segmenting and planning the grooving trajectory and grooving depth of the crack groove according to the curvature of the fitted curve specifically is: Calculating the curvature of each point on the regression curve of the support vector regression algorithm, and setting the points with curvature greater than the set threshold in the curve as inflection points; Dividing the asphalt pavement crack into several small segments with the inflection point as the demarcation line to obtain multiple crack regions; Drawing the minimum circumscribed rectangle of each crack region as the outline of the crack groove; Determining the conversion ratio between the pixel size and the actual size, and judging the damage degree of the road surface crack to the structural layer according to the width of the regression band of the fitted curve in the support vector regression algorithm, and then recommending the grooving depth of the crack groove to the user; The determination process of the average pixel value of the road surface in the current area is: removing the crack region segmented from the asphalt pavement surface image, and calculating the average pixel value of the remaining region; taking the average of the average pixel values calculated from multiple frames of images to obtain the final average pixel value of the road surface.

2. The colorless difference intelligent repair system for road surface cracks based on image recognition according to claim 1, characterized in that, The asphalt pavement repair material mixing unit includes: a mixing barrel, mixing cutters and a feeding pipe arranged above the chassis unit; the mixing cutters can rotate in the mixing barrel driven by a motor, and the feeding pipe is connected to the mixing barrel for feeding outwards.

3. The colorless difference intelligent repair system for road surface cracks based on image recognition according to claim 1, characterized in that, The control unit receives the asphalt pavement surface image information acquired by the camera module and conducts asphalt pavement crack image recognition; the specific process is as follows: Preprocessing the received asphalt pavement surface image, and inputting the preprocessed image into the trained crack recognition model to obtain a crack recognition result; Among them, the training process of the crack recognition model is: Respectively constructing a simple crack data set and a complex crack data set, first training the crack recognition model with the simple crack data set, and then training the crack recognition model with the complex crack data set.

4. The colorless intelligent repair system for road cracks based on image recognition according to claim 1, characterized in that, The control unit fits a crack function curve according to the distribution of crack pixel pairs, specifically: Extracting the coordinate pairs of the detected crack pixels from the image, and taking the crack bifurcation as a node to divide the bifurcated crack into multiple small segments; Using the least squares method to preliminarily fit the asphalt pavement crack function curve of each small segment, traversing all pixel pairs, and removing the outliers among them; Use the support vector regression algorithm to refit the asphalt pavement crack function curve of each small segment, and use the boundary band of the support vector regression algorithm as the dividing line between the asphalt pavement crack and the rest of the area.

5. The colorless difference intelligent repair system for road surface cracks based on image recognition according to claim 1, characterized in that, The control unit drives the cross slide and the hydraulic lifting module of the asphalt pavement grooving unit to move according to the grooving track and the grooving depth. During the driving process, the coordinates of the cutting tool in the cutting module and the camera in the camera module are unified. Among them, represents the coordinate transformation matrix; Indicates the distance between the cutting tool and the camera. The lengths of the cutting tool and the camera are respectively and , and the length of the hydraulic rod in the hydraulic lifting module is . The cutting tool moves along the axes in the base coordinate system, and the movement amounts are respectively , , .

6. The colorless difference intelligent repair system for road surface cracks based on image recognition according to claim 5, wherein The control unit stores the pixel value correspondence table of the asphalt pavement repair material in the RGB space under different colored dye dosages.

7. An intelligent colorless crack repair method for road surfaces based on image recognition, characterized in that, It includes: Obtain the surface image information of the asphalt pavement and perform preprocessing. Based on the image information, use the trained crack recognition model to obtain the crack recognition result. Based on the crack recognition result, according to the distribution of the crack pixel point pairs, fit the crack function curve; segmentally plan the grooving track and the grooving depth of the crack groove according to the curvature of the fitted curve. According to the grooving track and the grooving depth, drive the cross slide and the hydraulic lifting module of the asphalt pavement grooving unit to move, so as to drive the cutting tool to groove and cut along the grooving track and the grooving depth. Calculate the average value of the pavement pixels in the current area, and determine the dosages of the colored dye and the asphalt pavement repair material according to the average value of the pavement pixels in the current area, the grooving area and the depth; mix the colored dye and the asphalt pavement repair material according to the dosages to achieve colorless repair of the asphalt pavement. Among them, segmentally planning the grooving track and the grooving depth of the crack groove according to the curvature of the fitted curve is specifically: Calculate the curvature of each point on the regression curve of the support vector regression algorithm, and set the points with curvature greater than the set threshold in the curve as inflection points. Use the inflection points as the dividing lines to divide the asphalt pavement crack into several small segments to obtain multiple crack areas. Draw the minimum circumscribed rectangle of each crack area as the outline of the crack groove. Determine the conversion ratio between the pixel size and the actual size, and judge the damage degree of the pavement crack to the structural layer according to the width of the regression band of the fitted curve in the support vector regression algorithm, and then recommend the grooving depth of the crack groove to the user. The determination process of the average value of the pavement pixels in the current area is: remove the crack area segmented from the asphalt pavement surface image, and calculate the average value of the pixels in the remaining area. Take the average of the pixel averages calculated from multiple frames of images to obtain the final average value of the pavement pixels.

8. The colorless difference intelligent repair method for road surface cracks based on image recognition according to claim 7, characterized in that Determine the dosages of the colored dye and the asphalt pavement repair material according to the average value of the pavement pixels in the current area, the grooving area and the depth. The specific process is: Perform linear interpolation operations on the asphalt pavement surface image in the three RGB channels respectively, calculate the dosages of the three colored dyes, and use the weighted average method to calculate the final mixing ratio of the colored dyes; calculate the dosage of the asphalt repair material based on the grooving width and length of the crack groove, and then determine the dosage of the colored dye according to the mixing ratio.

Images