Intelligent crack pouring method, device, equipment, storage medium and product

Abstract

The application discloses an intelligent crack pouring method, device, equipment, storage medium and product, and relates to the technical field of highway construction.The intelligent crack pouring method comprises the following steps: if there is road surface damage in an asphalt pavement, and based on image data of the asphalt pavement, it is determined that the damage type is a target type of existing cracks, a skeleton line at the cracks is identified in the image data, wherein the target type is a transverse crack and a longitudinal crack, wherein the target type is a transverse crack and a longitudinal crack; a trajectory planning scheme of a mechanical arm moving from a starting point of the skeleton line to an end point is calculated; and the mechanical arm is controlled to pour cracks based on the path trajectory planning scheme until the cracks are closed. The application realizes crack pouring through an automatic process, does not need to rely on manual operation of construction personnel, and avoids the problems of low construction efficiency and fluctuation of construction quality.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of highway construction, and in particular to an intelligent crack pouring method, device, equipment, storage medium and product. BACKGROUND

[0002] In related technologies, the routine maintenance of highway pavement is mainly the crack pouring construction of pavement cracks by the cooperative work of construction personnel of maintenance teams and crack pouring equipment. Although mechanization is realized to different degrees and labor intensity is reduced in the construction process, the crack pouring equipment relies on manual operation of construction personnel, resulting in low construction efficiency and large fluctuation of construction quality. SUMMARY

[0003] The main purpose of the present application is to provide an intelligent crack pouring method, device, equipment, storage medium and product, aiming to solve the technical problem that the crack pouring equipment relies on manual operation of construction personnel, resulting in low construction efficiency and large fluctuation of construction quality.

[0004] To achieve the above-mentioned purpose, the present application provides an intelligent crack pouring method, which comprises:

[0005] If the asphalt pavement exists pavement damage, and the damage type is determined to be the target type of existing cracks based on the image data of the asphalt pavement, the skeleton line at the crack is identified in the image data, wherein the target type is a transverse crack and a longitudinal crack, and wherein the target type is a transverse crack and a longitudinal crack.

[0006] A trajectory planning scheme of the mechanical arm moving from the starting point to the ending point of the skeleton line is calculated.

[0007] The crack pouring of the mechanical arm based on the path trajectory planning scheme is controlled until the crack is closed.

[0008] In an embodiment, the step of identifying the skeleton line at the crack in the image data of the asphalt pavement comprises:

[0009] The image data is converted into a binary image.

[0010] Each foreground pixel in the binary image is sequentially traversed, the foreground pixels satisfying the preset deletion condition are deleted, until the first foreground pixel remaining after the deletion does not satisfy the preset deletion condition, and a first skeleton line is determined based on the first foreground pixel, wherein the foreground pixel is a pixel of a crack area.

[0011] Each foreground pixel in the binary image is sequentially traversed, the foreground pixels are deleted with a preset template until the second foreground pixel remaining does not exist matching the preset template, and a second skeleton line is determined based on the second foreground pixel.

[0012] calculating a deviation between the first bone line and the second bone line, and if the deviation is not more than a threshold, averaging the first bone line and the second bone line as the bone line at the crack.

[0013] In an embodiment, the step of calculating the trajectory planning scheme of the robot arm moving from the start point to the end point of the bone line comprises:

[0014] generating a spatial path based on the bone line;

[0015] calculating a joint space position of the robot arm at each path point in the spatial path;

[0016] calculating a reaching time of the robot arm to reach each path point;

[0017] determining a trajectory planning scheme based on the spatial path, the joint space position and the reaching time.

[0018] In an embodiment, the step of calculating the reaching time to reach each path point comprises:

[0019] determining a tangent direction of the bone line at each path point;

[0020] determining a normal direction perpendicular to the tangent direction;

[0021] calculating a crack distance in the normal direction based on the path point;

[0022] determining a crack filling speed corresponding to the crack distance;

[0023] determining the reaching time of each path point based on the crack filling speed.

[0024] In an embodiment, the step of determining the crack filling speed corresponding to the crack distance comprises:

[0025] judging whether the crack distance is greater than a distance threshold;

[0026] if yes, determining the crack filling speed corresponding to the crack distance based on a preset speed mapping relationship;

[0027] if no, taking a fixed crack filling speed as the crack filling speed corresponding to the crack distance.

[0028] In an embodiment, the robot arm is integrated with an air nozzle and an asphalt nozzle, and the step of controlling the robot arm to fill the crack based on the path trajectory planning scheme until the crack is closed comprises:

[0029] Control the mechanical arm to move based on the path trajectory planning scheme, and clean the crack by spraying compressed air through the air nozzle, and seal the crack by spraying through the asphalt nozzle.

[0030] Effectively judge the pavement image of the sealed crack, and if the crack does not exist in the pavement image, determine that the crack is closed.

[0031] In addition, to achieve the above-mentioned purpose, the present application also provides an intelligent sealing device, which comprises:

[0032] The recognition module is configured to identify the skeleton line at the crack in the image data if the asphalt pavement has pavement damage and the damage type is determined to be the target type of existing cracks based on the image data of the asphalt pavement, wherein the target type is transverse cracks and longitudinal cracks.

[0033] The calculation module is configured to calculate a trajectory planning scheme for the mechanical arm to move from the starting point to the end point of the skeleton line.

[0034] The control module is configured to control the mechanical arm to seal the crack based on the path trajectory planning scheme until the crack is closed.

[0035] In addition, to achieve the above-mentioned purpose, the present application also provides an intelligent sealing device, which comprises: a memory, a processor and a computer program stored on the memory and executable on the processor, the computer program being configured to implement the steps of the intelligent sealing method as described above.

[0036] In addition, to achieve the above-mentioned purpose, the present application also provides a storage medium, which is a computer readable storage medium, and the storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the intelligent sealing method as described above.

[0037] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which comprises a computer program, and the computer program is executed by a processor to implement the steps of the intelligent sealing method as described above.

[0038] The one or more technical solutions provided by the present application have at least the following technical effects:

[0039] Compared with the related art, the daily maintenance of the highway pavement is mainly the joint filling construction of the pavement cracks by the construction personnel of the maintenance team and the joint filling equipment. Although the mechanical operation is realized to a certain extent and the labor intensity is reduced during the construction process, the joint filling equipment relies on the manual operation of the construction personnel, resulting in low construction efficiency and large fluctuation of construction quality. Compared with the related art, if the asphalt pavement is damaged and the damage type is determined to be the target type of existing cracks based on the image data of the asphalt pavement, the skeleton line at the crack is identified in the image data, wherein the target type is a transverse crack and a longitudinal crack; a path trajectory planning scheme of the mechanical arm moving from the starting point of the skeleton line to the end point is calculated; and the mechanical arm is controlled to fill the crack based on the path trajectory planning scheme until the crack is closed. When the damage type of the asphalt pavement is a transverse crack or a longitudinal crack, the skeleton line at the crack is automatically identified in the image data of the asphalt pavement, the path trajectory planning scheme from the starting point to the end point of the skeleton line is calculated, and the mechanical arm is automatically controlled to fill the crack based on the path trajectory planning scheme until the crack is closed. The joint filling is realized through the automatic process, the manual operation of the construction personnel is not needed, and the problems of low construction efficiency and large fluctuation of construction quality are avoided. BRIEF DESCRIPTION OF DRAWINGS

[0040] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and serve to explain the principles of the present application together with the specification.

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the accompanying drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, those skilled in the art can obtain other drawings according to these drawings without any creative effort.

[0042] Figure 1 A flowchart is provided for the first embodiment of the intelligent joint filling method of the present application;

[0043] Figure 2 A joint filling control flowchart is provided for the intelligent joint filling method of the present application;

[0044] Figure 3 A flowchart is provided for the second embodiment of the intelligent joint filling method of the present application;

[0045] Figure 4 A joint space position determination diagram is provided for the intelligent joint filling method of the present application;

[0046] Figure 5 A joint filling diagram is provided for the intelligent joint filling method of the present application;

[0047] Figure 6 A module structure diagram of the intelligent crack pouring device in the embodiment of the present application is shown in FIG. 1.

[0048] Figure 7 A device structure diagram of a hardware running environment involved in the intelligent crack pouring method in the embodiment of the present application is shown in FIG. 2.

[0049] The object implementation, functional features and advantages of the present application will be further described with reference to the accompanying drawings in conjunction with the embodiments. DETAILED DESCRIPTION

[0050] It should be understood that the specific embodiments described herein are merely intended to explain the technical solutions of the present application, and are not intended to limit the present application.

[0051] In order to better understand the technical solutions of the present application, the specific embodiments will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0052] The main solution of the embodiment of the present application is: if there is damage to the asphalt pavement, and based on the image data of the asphalt pavement, it is determined that the damage type is the target type of existing cracks, the skeleton line at the crack is identified in the image data, wherein the target type is transverse cracks and longitudinal cracks, wherein the target type is transverse cracks and longitudinal cracks; a path trajectory planning scheme is calculated for the mechanical arm to move from the starting point of the skeleton line to the end point; the mechanical arm is controlled to pour cracks based on the path trajectory planning scheme until the cracks are closed.

[0053] In the related art, routine maintenance of highway pavement is mainly the cooperative work of construction personnel of maintenance team and crack pouring equipment for crack pouring construction of pavement cracks. Although mechanization is realized to some extent and labor intensity is reduced during the construction process, the crack pouring equipment and manual operation of the construction personnel result in low construction efficiency and large fluctuation of construction quality.

[0054] When the damage type of the asphalt pavement is transverse cracks or longitudinal cracks, the present application automatically identifies the skeleton line at the crack in the image data of the asphalt pavement, calculates the path trajectory planning scheme from the starting point to the end point of the skeleton line, and automatically controls the mechanical arm to pour cracks based on the path trajectory planning scheme until the cracks are closed. The present application realizes crack pouring through an automatic process, does not need to rely on manual operation of the construction personnel, and avoids the problems of low construction efficiency and large fluctuation of construction quality.

[0055] It should be noted that the execution subject of the present embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device, an intelligent crack pouring device, etc. capable of realizing the above functions. The present embodiment and each of the following embodiments will be described below taking the intelligent crack pouring device as an example.

[0056] Based on this, the intelligent crack pouring method provided in the embodiments of the present application comprises the following steps: Figure 1 , Figure 1 The flowchart of the first embodiment of the intelligent crack pouring method of the present application is shown in FIG. 1.

[0057] In the embodiment, the intelligent crack pouring method comprises steps S10-S40:

[0058] In step S10, if the asphalt pavement has pavement damage, and it is determined that the damage type is the target type of existing cracks based on the image data of the asphalt pavement, the skeleton line at the crack is identified in the image data, wherein the target type is a transverse crack and a longitudinal crack, and wherein the target type is a transverse crack and a longitudinal crack.

[0059] It should be noted that the execution subject of the embodiment is an intelligent crack pouring device. The front of the intelligent crack pouring device is provided with a vertically downward line scan camera, which is used to collect image data of pavement damage. The asphalt pavement damage includes 11 types: alligator cracking, block cracking, transverse cracking, longitudinal cracking, settlement, rutting, wave heave, pothole, loose, oil bleeding and repair. Different treatment methods are taken for different types of pavement diseases: for pavement damage such as alligator cracking, block cracking, loose, pothole and settlement, special maintenance treatment is needed; for transverse cracks and longitudinal cracks, crack pouring construction and maintenance are carried out to seal the cracks in time and prevent the pavement structure disease from further developing and deteriorating. The image data is obtained by the line scan camera or the area array camera, which is used for AI recognition and analysis. The skeleton line refers to the center line extracted from the crack area, which retains the topological structure (such as length, direction, branch, etc.) of the crack, and is used for path planning. The intelligent crack pouring device first identifies the image data through image recognition (such as an AI model), judges whether the pavement has damage, further judges whether the damage belongs to the target type (i.e. transverse cracks and longitudinal cracks) which can be directly processed, extracts the image instances of transverse cracks and longitudinal cracks, and further extracts the skeleton line in the image instance, that is, the center line or skeleton of the crack.

[0060] In step S20, a path trajectory planning scheme is calculated for the mechanical arm to move from the starting point to the ending point of the skeleton line.

[0061] It can be understood that the mechanical arm is a multi-joint industrial robot with multiple degrees of freedom (usually 6 axes), which can be positioned and oriented in three-dimensional space. The crack skeleton of the intelligent crack pouring device becomes a time-joint angle curve, that is, a path trajectory planning scheme, which enables the mechanical arm to smoothly, on time and without impact from the starting point to the ending point, and completes automatic crack pouring.

[0062] Step S30, control the mechanical arm to seal the crack based on the path trajectory planning scheme until the crack is closed.

[0063] It should be noted that the intelligent crack sealing device controls the mechanical arm to spray glue accurately while walking at a constant speed based on the path trajectory planning scheme, and the camera checks whether the crack is completely filled in real time, and stops when it is filled, and completes automatic crack sealing.

[0064] In a possible implementation, the step of identifying the skeleton line at the crack in the image data of the asphalt pavement comprises:

[0065] Converting the image data into a binary image;

[0066] It can be understood that the binary image is a grayscale image, which only contains 0 and 1 (or 0 and 255) images, and white (0) represents a crack, and black (1) represents a background, which provides an input for subsequent thinning / skeleton extraction. The intelligent crack sealing device revalues each pixel as 0 or 255 (or 0 / 1) according to the grayscale threshold value through an image processing algorithm, that is, automatically changes the crack to white and the background to black according to the grayscale threshold value, to obtain a binary image with only black and white, which facilitates subsequent extraction of the crack skeleton.

[0067] Iterating each foreground pixel in the binary image in sequence, deleting the foreground pixel that meets the preset deletion condition until the first foreground pixel remaining after the deletion does not meet the preset deletion condition, and determining a first skeleton line based on the first foreground pixel, wherein the foreground pixel is a pixel in a crack region;

[0068] It should be noted that the foreground pixel is a pixel with a value of 1 (white) in the binary image, which corresponds to a crack entity. The intelligent crack sealing device iterates all pixels that are not 0 in the binary image through the Zhang-Suen algorithm. When judging whether to delete or retain each pixel (P1), the values of the 8 neighbor pixels (P2, P3, P4, P5, P6, P7, P8) around the pixel need to be observed, wherein the order of P2 to P8 is specified by the algorithm, and whether the point needs to be deleted is determined according to the preset deletion condition. The single-pixel white line after iteration is the first skeleton line of the crack.

[0069] Iterating each foreground pixel in the binary image in sequence, deleting the foreground pixel that meets the preset deletion condition until the first foreground pixel remaining after the deletion does not meet the preset deletion condition, and determining a first skeleton line based on the first foreground pixel, wherein the foreground pixel is a pixel in a crack region;

[0070] It can be understood that the 256 8-neighborhood patterns are divided into two categories in advance, and the binary schematic string of the values corresponding to the neighbor pixels arranged in a certain order in the template to be deleted is determined, that is, the preset template. The intelligent crack pouring device determines the skeleton line through the Guo-Hall algorithm, adopts a serial iteration mode, and needs to process the unprocessed pixel points each time. When the algorithm is processing, the pixel value is updated according to the template matching result of the current pixel neighborhood. For example, for each pixel to be processed, its neighborhood includes 8 directions (such as p2 to p9), the pixel value combination (such as p9|p2<<1|p3<<2, etc.) of these directions is calculated, and whether to keep the current pixel is determined according to the matching result.

[0071] The deviation between the first skeleton line and the second skeleton line is calculated, and if the deviation does not exceed a threshold value, the skeleton line obtained by averaging the first skeleton line and the second skeleton line is taken as the skeleton line at the crack.

[0072] It should be noted that the threshold value is the maximum average difference of two lines allowed. The intelligent crack pouring device compares and analyzes the two calculations of extracting the skeleton line, and if the deviation is less than the threshold value, the average processing is performed to obtain the skeleton line of the crack; if the deviation exceeds the threshold value, a warning is issued, and the skeleton line of the crack is determined by a person.

[0073] In a possible implementation, step S30 includes:

[0074] The mechanical arm is controlled to move based on the path trajectory planning scheme, compressed air is sprayed through the air nozzle to clean the crack, and the asphalt nozzle is used to pour crack to the cleaned crack, so as to obtain the crack after pouring.

[0075] It can be understood that the air nozzle is a high-pressure air nozzle installed at the end of the mechanical arm, with a pressure of 0.6-0.8 MPa, which is used for dust removal, moisture removal, and debris removal. The asphalt nozzle is an electric heating nozzle, with a temperature of 180-200 ℃, and an internal flow valve or gear pump, which can accurately control the amount of glue. The intelligent crack pouring device controls the mechanical arm to execute the planning curve of the path trajectory planning scheme, the end of the mechanical arm is integrated with a high-pressure air nozzle (usually 0.6-0.8 MPa), and the air nozzle blows along the crack to remove dust, debris, and moisture, so as to ensure that the glue is firmly bonded to the wall. The heating asphalt / sealant nozzle on the same end (or parallel head) sprays glue immediately after air cleaning, with a glue temperature of 180-200 ℃, and a flow rate linked to the moving speed, to ensure that the crack is filled with glue. After the glue cools down, a black elastic sealing band is formed, and the crack is completely sealed. The subsequent rear camera can be used to verify the sealing rate again, as shown in FIG. 6. Figure 2 , Figure 2 A crack pouring control flowchart is provided.

[0076] Effectively judging a road surface image of the crack after the crack is grouted, if the crack does not exist in the road surface image, it is determined that the crack is closed.

[0077] It should be noted that the intelligent crack grouting device uses a rear line scan / area array camera to take a high-resolution road surface photo immediately after crack grouting is completed. The AI segmentation model or traditional image processing automatically detects whether there is a crack pixel in the photo. If the crack pixel area output by the algorithm is less than a preset threshold (for example, 5% of the original area), it is considered that the crack has been completely covered by the glue.

[0078] In this embodiment, the image algorithm (such as deep learning CNN and morphological analysis) is used to accurately distinguish the transverse crack and the longitudinal crack, and identify the skeleton line of the crack. The skeleton line represents the geometric center of the crack, effectively eliminates the interference caused by the change of the crack width, and provides a high-precision and low-noise guide line for subsequent path planning.

[0079] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as the above embodiment one can refer to the above introduction, and the subsequent will not be described in detail. On this basis, please refer to Figure 3 , step S20, the intelligent crack grouting method further comprises steps S01-S04:

[0080] Step S01, generating a spatial path based on the skeleton line;

[0081] It should be noted that the spatial path is a set of geometric ordered points that the mechanical arm needs to pass through from the starting point to the ending point, and does not contain time information. The intelligent crack grouting device converts the single-pixel coordinates in the skeleton line in the geodetic coordinate system into the Cartesian target list in the base coordinate system of the mechanical arm, that is, the spatial path.

[0082] Specifically, the intelligent crack grouting device uses a high-precision Beidou positioning system to obtain the geodetic ellipsoid coordinates of the image data. The four corner point coordinates (x1, y1, z1), (x2, y2, z2), (x3, y3, z3), and (x4, y4, z4) of each image data.

[0083] Step S02, calculating the joint space position of the mechanical arm at each path point in the spatial path;

[0084] It can be understood that the joint space position is a vector of four joint angles. The intelligent crack grouting device inversely kinematically solves the joint space position of each path point, and refers to Figure 4 , Figure 4 A joint space position determination diagram is provided.

[0085] Further, referring to Figure 5 , Figure 5 A crack grouting schematic diagram is provided, Figure 5The middle part shows that after the robot arm reaches a new path point, the joint in the robot arm will be rotated according to the joint space position corresponding to the path point, so that the robot arm control is directed to a different coordinate system.

[0086] Step S03, the arrival time of the robot arm reaching each path point is calculated;

[0087] It should be noted that the intelligent crack pouring device calculates the accurate time for the robot arm to reach each path point from the starting point. By precisely controlling the arrival time of each path point, complete crack pouring of cracks of different widths can be achieved.

[0088] Step S04, based on the space path, the joint space position and the arrival time, a path trajectory planning scheme is determined.

[0089] It can be understood that the intelligent crack pouring device assigns time information, including speed, acceleration, etc., to the path based on path planning, obtains a path trajectory planning scheme, and enables the robot arm to complete movement according to the predetermined time and movement characteristics.

[0090] In a possible implementation, step S02 includes:

[0091] The tangent direction of the bone line at each path point is determined;

[0092] It should be noted that the intelligent crack pouring device determines the local trend of the bone line at each path point on the bone line, indicating the "extension direction" of the crack.

[0093] The normal direction perpendicular to the tangent direction is determined;

[0094] It can be understood that the intelligent crack pouring device determines a vector perpendicular to the tangent direction, pointing to the edges of the crack on both sides, reflecting the "lateral expansion direction" of the crack.

[0095] The crack distance in the normal direction is calculated based on the path point;

[0096] It should be noted that the intelligent crack pouring device takes the Euclidean distance from one edge of the crack to the other edge in the normal direction as the crack distance.

[0097] The crack pouring speed corresponding to the crack distance is determined;

[0098] It can be understood that the intelligent crack pouring device converts the real-time measured crack width (i.e. "crack distance") into the movement speed of the robot arm, i.e. the crack pouring speed.

[0099] Based on the crack pouring speed, the arrival time of each path point is determined.

[0100] It should be noted that the intelligent crack pouring device uses the distance between adjacent path points and the corresponding speed to integrate and calculate the segmented motion time, and accumulates the absolute arrival time of each point to form an accurate time axis.

[0101] Further, for a wider crack, a single straight pouring line may cause material to accumulate in the middle and not fully bond with the two side walls of the crack, so the mechanical arm of the intelligent crack pouring device will have a high-frequency, small-amplitude periodic swing motion in the vertical direction (Z-axis) while moving in the crack direction (X-axis), which makes the nozzle swing left and right in the wide crack, ensuring that the crack pouring material can be evenly applied and pressed into the two side walls of the crack, significantly improving the filling fullness and bonding force.

[0102] Specifically, the mechanical arm of the intelligent crack pouring device is driven in the Z-axis with a sine wave or a triangular wave function:

[0103]

[0104] where A is the amplitude, usually half of the crack width, f is the frequency, determined according to the crack pouring speed, and t is the time.

[0105] The intelligent crack pouring device can also be provided with a front sensor for real-time detection of the current crack width, and the amplitude can be dynamically adjusted according to the crack width. The wider the crack, the larger the amplitude, to expand the material coating coverage; the narrower the crack, the smaller the amplitude, until zero, smoothly transitioning back to the straight pouring mode.

[0106] Specifically, the amplitude calculation formula is:

[0107]

[0108] where, is the current oscillation amplitude (one-sided swing distance), i.e. the nozzle swing range is [-A(t), +A(t)], is the real-time detected crack width, k is the filling coefficient, and the recommended value range is 0.4≤k≤0.6, which can be adjusted according to the material shrinkage rate: materials with high shrinkage rate (such as some polyurethane) can slightly increase k to 0.55.

[0109] Further, the advancing speed V of the mechanical arm is cooperatively planned with the oscillation frequency f. To ensure the coating effect of oscillation, the advancing distance of the mechanical arm in each oscillation period (i.e. V / f) needs to be less than the nozzle diameter to ensure continuous overlapping coverage of the material and avoid missing points. When the oscillation mode is started, the intelligent pouring device sends a signal to the asphalt pumping system to moderately increase the output flow of the material to match the increased filling volume requirement due to the swing, ensuring sufficient material supply and effectively guiding the material into the crack and expelling the air inside, overcoming the defects of "hollow" or "incomplete filling" that may occur in simple gravity pouring. The amplitude and frequency of the entire oscillation process can be dynamically adjusted according to the real-time characteristics of the crack, realizing the leap from "open-loop execution" to "closed-loop adaptation" and embodying high intelligence.

[0110] In a feasible implementation, the step of determining the pouring speed corresponding to the crack distance comprises:

[0111] determining whether the crack distance is greater than a distance threshold;

[0112] It can be understood that the intelligent pouring device measures the actual width of the crack through the set front sensor, sets a distance threshold according to the road maintenance standard, for example, 5 mm, and compares the measured crack width with the preset distance threshold.

[0113] If greater, based on the preset speed mapping relationship, the pouring speed corresponding to the crack distance is determined;

[0114] It should be noted that the intelligent pouring device pre-establishes a mapping relationship table or function between the crack width and the pouring speed, for example, the larger the crack width, the slower the required pouring speed to ensure sufficient filling of the material. If the crack width is greater than the threshold, the corresponding pouring speed is found or calculated according to the mapping relationship.

[0115] If less than or equal to, a fixed pouring speed is used as the pouring speed corresponding to the crack distance.

[0116] It can be understood that the intelligent pouring device pre-sets a fixed pouring speed for processing cracks with a width less than or equal to the threshold. This speed is usually pre-determined according to the material properties and equipment capacity. For cracks with a width less than or equal to the threshold, this fixed pouring speed is directly applied.

[0117] In this embodiment, through Z-axis swing, the material is "brushed" into the two side walls of the crack under high pressure, forming a three-dimensional, mechanically interlocked strong bond, greatly extending the service life of the repaired road surface.

[0118] It should be noted that the above examples are only used for understanding the present application and do not constitute a limitation on the intelligent crack pouring method of the present application. More forms of simple changes based on this technical concept are within the protection scope of the present application.

[0119] The present application also provides an intelligent crack pouring device, please refer to Figure 6 , the intelligent crack pouring device comprises:

[0120] The recognition module 10 is used for identifying the skeleton line at the crack in the image data if the asphalt pavement exists pavement damage and the damage type is determined to be the target type of existing cracks based on the image data of the asphalt pavement, wherein the target type is a transverse crack and a longitudinal crack, and wherein the target type is a transverse crack and a longitudinal crack.

[0121] The calculation module 20 is used for calculating the trajectory planning scheme of the mechanical arm moving from the starting point to the end point of the skeleton line.

[0122] The control module 30 is used for controlling the mechanical arm to pour cracks based on the path trajectory planning scheme until the cracks are closed.

[0123] Optionally, the recognition module comprises:

[0124] The reservation sub-module converts the image data into a binary image; sequentially traverses each foreground pixel in the binary image, deletes the foreground pixel that meets the preset deletion condition, until the first foreground pixel reserved after the deletion does not meet the preset deletion condition, determines a first skeleton line based on the first foreground pixel, wherein the foreground pixel is a pixel of a crack area; sequentially traverses each foreground pixel in the binary image, deletes the foreground pixel that matches the preset template, until the second foreground pixel reserved does not exist the preset template, determines a second skeleton line based on the second foreground pixel; calculates the deviation between the first skeleton line and the second skeleton line, if the deviation does not exceed a threshold, the skeleton line after averaging the first skeleton line and the second skeleton line is taken as the skeleton line at the crack.

[0125] Optionally, the calculation module comprises:

[0126] The determination sub-module is used for generating a spatial path based on the skeleton line; calculating the joint space position of each path point of the mechanical arm in the spatial path; calculating the arrival time of the mechanical arm to each path point; determining a trajectory planning scheme based on the spatial path, the joint space position and the arrival time.

[0127] Optionally, the determination sub-module comprises:

[0128] The arrival time determination unit is configured to determine a tangent direction of the skeleton line at each path point; determine a normal direction perpendicular to the tangent direction; calculate a crack distance in the normal direction based on the path point; determine a crack pouring speed corresponding to the crack distance; and determine an arrival time of each path point based on the crack pouring speed.

[0129] Optionally, the arrival time determination unit comprises:

[0130] The judgment subunit is configured to judge whether the crack distance is greater than a distance threshold value; if yes, determine a crack pouring speed corresponding to the crack distance based on a preset speed mapping relationship; and if no, take a fixed crack pouring speed as the crack pouring speed corresponding to the crack distance.

[0131] Optionally, the control module comprises:

[0132] The crack pouring sub-module is configured to control the mechanical arm to move based on the path trajectory planning scheme, spray compressed air to the crack through the air nozzle to clean the crack, pour crack sealing material to the cleaned crack through the asphalt nozzle to obtain the crack after pouring, and judge the effect of the road surface image of the crack after pouring; if the crack does not exist in the road surface image, it is determined that the crack is closed.

[0133] The intelligent crack pouring device provided in the application can solve the technical problem of intelligent crack pouring by using the intelligent crack pouring method in the above embodiments. Compared with the prior art, the intelligent crack pouring device provided in the application has the same beneficial effects as the intelligent crack pouring method provided in the above embodiments, and other technical features in the intelligent crack pouring device are the same as the features disclosed in the above embodiments, which will not be repeated here.

[0134] The application provides an intelligent crack pouring device, which comprises at least one processor and a memory in communication connection with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the intelligent crack pouring method in the above embodiment one.

[0135] The following refers to Figure 7The diagram illustrates a structural schematic suitable for implementing the intelligent crack sealing device of the embodiments of this application. The intelligent crack sealing device in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, tablets, digital broadcast receivers, PDAs (Personal Digital Assistants), PMPs (Portable Media Players), vehicle terminals (e.g., vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 7 The intelligent crack sealing device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0136] like Figure 7 As shown, the intelligent crack sealing device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the intelligent crack sealing device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touch screen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. The communication device 1009 allows the smart crack sealing device to communicate wirelessly or wiredly with other devices to exchange data. Although the figures show smart crack sealing devices with various systems, it should be understood that it is not required to implement or possess all of the systems shown. More or fewer systems may be implemented alternatively.

[0137] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product comprising a computer program carried on a computer readable medium, the computer program containing program codes for executing the method shown in the flowchart. In such embodiments, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiments disclosed in the present application are executed.

[0138] The intelligent crack pouring device provided by the present application adopts the intelligent crack pouring method in the above embodiments, and can solve the technical problem of intelligent crack pouring. Compared with the prior art, the intelligent crack pouring device provided by the present application has the same beneficial effects as the intelligent crack pouring method provided by the above embodiments, and other technical features in the intelligent crack pouring device are the same as the features disclosed in the previous embodiment method, which will not be repeated here.

[0139] It should be understood that various parts of the present application can be realized by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0140] The above is merely specific implementation of the present application, but the protection scope of the present application is not limited thereto, any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

[0141] The present application provides a computer readable storage medium having stored thereon computer readable program instructions (i.e. computer program) for executing the intelligent crack pouring method in the above embodiments.

[0142] The computer readable storage medium provided in the application may, for example, be a U disk, but is not limited to an electric, magnetic, optical, electromagnetic, infrared, or semiconductor system, system, or device, or any combination of the above. More specific examples of the computer readable storage medium can include, but are not limited to, an electric connection with one or more conductive wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the embodiment, the computer readable storage medium can be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system, or device. The program code contained on the computer readable storage medium can be transmitted by any suitable medium, including but not limited to an electric wire, an optical cable, an RF (Radio Frequency), and the like, or any suitable combination of the above.

[0143] The computer readable storage medium described above can be contained in the intelligent crack pouring device, or can exist separately without being assembled into the intelligent crack pouring device.

[0144] The computer readable storage medium described above carries one or more programs, which, when executed by the intelligent crack pouring device, cause the intelligent crack pouring device to: if there is a road damage on the asphalt pavement, and based on the image data of the asphalt pavement, it is determined that the damage type is the target type of existing cracks, identify the skeleton line at the crack in the image data, wherein the target type is a transverse crack and a longitudinal crack, wherein the target type is a transverse crack and a longitudinal crack; calculate a trajectory planning scheme of the mechanical arm moving from the starting point of the skeleton line to the end point; control the mechanical arm to pour cracks based on the path trajectory planning scheme until the crack is closed.

[0145] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0146] The flow diagrams and the block diagrams in the drawings are illustrations of architectures, functionalities, and operations of possible implementations of systems, methods, and computer program products according to various embodiments of the present application. In this regard, each block in the flow diagrams or block diagrams can represent a module, a segment, or a portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks may

[0147] The modules involved in the embodiments of the present application can be implemented in the form of software or in the form of hardware. In some cases, the name of the module does not constitute a limitation on the module itself.

[0148] The readable storage medium provided by the present application is a computer readable storage medium, which stores computer readable program instructions (i.e., a computer program) for executing the above intelligent pouring joint method, and can solve the technical problem of intelligent pouring joint. Compared with the prior art, the computer readable storage medium provided by the present application has the same beneficial effects as the intelligent pouring joint method provided by the above embodiments, which will not be described here.

[0149] The application further provides a computer program product comprising a computer program which, when executed by a processor, implements the steps of the intelligent crack pouring method as described above.

[0150] The computer program product provided by the application can solve the technical problem of intelligent crack pouring. Compared with the prior art, the computer program product provided by the application has the same beneficial effects as the intelligent crack pouring method provided by the above-mentioned embodiments, and will not be described here.

[0151] The above only describes some embodiments of the application, and does not limit the protection scope of the application. Any equivalent structure transformation made by using the content of the application specification and drawings, or direct / indirect application in other related technical fields is included in the protection scope of the application.

Claims

1. A smart crack pouring method characterized by, The intelligent crack pouring method comprises: If the asphalt pavement has pavement damage, and based on the image data of the asphalt pavement, it is determined that the damage type is the target type of existing cracks, a skeleton line at the cracks in the image data is identified, wherein the target type is transverse cracks and longitudinal cracks; A path trajectory planning scheme of the mechanical arm moving from the starting point to the ending point of the skeleton line is calculated; The mechanical arm is controlled to pour cracks based on the path trajectory planning scheme until the cracks are closed; The step of calculating the path trajectory planning scheme of the mechanical arm moving from the starting point to the ending point of the skeleton line comprises: Based on the skeleton line, a space path is generated; The joint space position of the mechanical arm at each path point in the space path is calculated; The arrival time of the mechanical arm at each path point is calculated; Based on the space path, the joint space position and the arrival time, a path trajectory planning scheme is determined; The step of calculating the arrival time of the mechanical arm at each path point comprises: The tangent direction of the skeleton line at each path point is determined; The normal direction perpendicular to the tangent direction is determined; The crack distance in the normal direction is calculated based on the path point as a reference; The crack pouring speed corresponding to the crack distance is determined; Based on the crack pouring speed, the arrival time of each path point is determined; The step of determining the crack pouring speed corresponding to the crack distance comprises: It is judged whether the crack distance is greater than a distance threshold value; If yes, the crack pouring speed corresponding to the crack distance is determined based on a preset speed mapping relationship; If no, a fixed crack pouring speed is taken as the crack pouring speed corresponding to the crack distance.

2. The intelligent crack pouring method of claim 1, wherein, The step of identifying the skeleton line at the cracks in the image data of the asphalt pavement comprises: The image data is converted into a binary image; Each foreground pixel in the binary image is sequentially traversed, and the foreground pixel satisfying a preset deletion condition is deleted until the first foreground pixel remaining after deletion does not satisfy the preset deletion condition, and a first skeleton line is determined based on the first foreground pixel, wherein the foreground pixel is a pixel of a crack region; Each foreground pixel in the binary image is sequentially traversed, and the foreground pixel matching a preset template is deleted until the second foreground pixel remaining does not have the matching preset template, and a second skeleton line is determined based on the second foreground pixel; The deviation between the first skeleton line and the second skeleton line is calculated, and if the deviation does not exceed a threshold value, the skeleton line averaged from the first skeleton line and the second skeleton line is taken as the skeleton line at the cracks.

3. The intelligent crack pouring method of claim 1, wherein, The mechanical arm is integrated with an air nozzle and an asphalt nozzle, and the step of controlling the mechanical arm to pour cracks based on the path trajectory planning scheme until the cracks are closed comprises: The mechanical arm is controlled to move based on the path trajectory planning scheme, and compressed air is sprayed out through the air nozzle to clean the cracks, and the cleaned cracks are poured through the asphalt nozzle to obtain the cracks after pouring. An effect judgment is performed on a pavement image of the crack after the crack is sealed, and if the crack does not exist in the pavement image, it is determined that the crack is closed.

4. A smart crack pouring device, characterized in that, The device comprises: The identification module is configured to, if the asphalt pavement has a pavement damage and a damage type is determined to be a target type of existing cracks based on image data of the asphalt pavement, identify a skeleton line at the cracks in the image data, wherein the target type is a transverse crack and a longitudinal crack. The calculation module is configured to calculate a path trajectory planning scheme of the mechanical arm from a starting point to an ending point of the skeleton line. The control module is configured to control the mechanical arm to seal the cracks based on the path trajectory planning scheme until the cracks are closed. The calculation module comprises: The determination submodule is configured to generate a spatial path based on the skeleton line, calculate a joint space position of the mechanical arm at each path point in the spatial path, calculate an arrival time of the mechanical arm at each path point, and determine a trajectory planning scheme based on the spatial path, the joint space position, and the arrival time. The determination submodule comprises: The arrival time determination unit is configured to determine a tangent direction of the skeleton line at each path point, determine a normal direction perpendicular to the tangent direction, calculate a crack distance in the normal direction based on the path point, determine a sealing speed corresponding to the crack distance, and determine an arrival time of each path point based on the sealing speed. The arrival time determination unit comprises: The judgment subunit is configured to judge whether the crack distance is greater than a distance threshold, determine a sealing speed corresponding to the crack distance based on a preset speed mapping relationship if the crack distance is greater than the distance threshold, or use a fixed sealing speed as the sealing speed corresponding to the crack distance if the crack distance is less than or equal to the distance threshold.

5. A smart crack pouring device, characterized in that, The device comprises a memory, a processor, and a computer program stored on the memory and executable on the processor, and the computer program is configured to implement the steps of the intelligent sealing method according to any one of claims 1 to 3.

6. A storage medium, characterized by The storage medium is a computer readable storage medium, and the storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the intelligent sealing method according to any one of claims 1 to 3.

7. A computer program product, characterised in that, The computer program product comprises a computer program, and the computer program is executed by a processor to implement the steps of the intelligent sealing method according to any one of claims 1 to 3.

Images