Road pavement disease detection method and equipment

Abstract

The invention discloses a road pavement disease detection method and equipment. The method comprises the following steps: acquiring a pavement image obtained by shooting a to-be-detected road; determining at least one piece of target feature data of a disease target block included in the road surface image according to the pixel values of the pixel points included in the road surface image; wherein one disease target block comprises N continuous disease pixel points, and the pixel values of the disease pixel points are within a set pixel value range; applying the at least one target feature data and the weight corresponding to the at least one target feature data to obtain a disease probability; and determining the type of the disease included in the pavement image according to the relationship between the disease probability and a set disease probability threshold. And the precision of detecting the damage of the road surface is improved.

Description

Technical Field

[0001] The present application relates to the technical field of fault detection, and in particular to a method and device for detecting road pavement defects. Background Art

[0002] During normal road operation, affected by factors such as weather, vehicle overload and long-term operation, road pavement defects such as bumps and pits appear, affecting road operation efficiency and service quality, and even threatening the safety of people and vehicles. This problem is more obvious on highways. Therefore, how to detect road pavement defects with high precision is particularly important.

[0003] Among the related technologies, one method is manual inspection, which uses patrolling, taking photos, and uploading records to detect diseases. This method has high accuracy but high labor costs. Another method is to use professional equipment to collect and analyze road data. The detection and maintenance costs of professional equipment are relatively poor. Another method is to collect video data of road scenes in real time for analysis. However, due to the interference of complex light and shadow environments, and the fact that the target features of diseases are not significant, the detection accuracy is low, and diseases with different morphologies cannot be accurately identified. Summary of the invention

[0004] In an exemplary embodiment of the present application, a method and device for detecting road pavement defects are provided to improve the accuracy of road pavement defect detection.

[0005] According to a first aspect of an exemplary embodiment, a method for detecting road pavement defects is provided, the method comprising at least the following steps:

[0006] Acquire a road surface image obtained by photographing the road to be tested;

[0007] Determine at least one target feature data of a defect target block included in the road surface image according to the pixel values ​​of the pixel points included in the road surface image; wherein one defect target block includes N consecutive defect pixel points, and the pixel values ​​of the defect pixel points are within a set pixel value range;

[0008] Applying at least one target characteristic data and a weight corresponding to each of the at least one target characteristic data to obtain a disease probability;

[0009] The type of defects included in the road surface image is determined based on the relationship between the defect probability and a set defect probability threshold.

[0010] According to a second aspect of an exemplary embodiment, there is provided a road pavement disease detection device, the detection device comprising a data transmission unit and a processor;

[0011] The data transfer unit is configured to perform:

[0012] Acquire a road surface image obtained by photographing the road to be tested;

[0013] The processor is configured to execute:

[0014] Determine at least one target feature data of a defect target block included in the road surface image according to the pixel values ​​of the pixel points included in the road surface image; wherein one defect target block includes N consecutive defect pixel points, and the pixel values ​​of the defect pixel points are within a set pixel value range;

[0015] Applying at least one target characteristic data and a weight corresponding to each of the at least one target characteristic data to obtain a disease probability;

[0016] The type of defects included in the road surface image is determined based on the relationship between the defect probability and a set defect probability threshold.

[0017] According to a third aspect of an exemplary embodiment, there is provided a device for detecting road pavement defects, comprising:

[0018] An image acquisition unit is used to: acquire a road surface image obtained by photographing the road to be tested;

[0019] The data processing unit is used to determine at least one target feature data of a defect target block included in the road surface image according to the pixel values ​​of the pixel points included in the road surface image; wherein one defect target block includes N consecutive defect pixel points, and the pixel values ​​of the defect pixel points are within a set pixel value range;

[0020] The data processing unit is further used to: apply the at least one target characteristic data and the weight corresponding to each of the at least one target characteristic data to obtain the disease probability;

[0021] The disease detection unit is used to determine the type of disease included in the road surface image according to the relationship between the disease probability and the set disease probability threshold.

[0022] According to a fourth aspect of an exemplary implementation, a computer storage medium is provided, in which computer program instructions are stored. When the instructions are executed on a computer, the computer executes the road pavement disease detection method as described in the first aspect.

[0023] In the embodiment of the present application, a road surface image is obtained by photographing the road to be tested. Since the characteristics of the defect target are not obvious, the concept of a defect target block is introduced. A defect target block includes N consecutive defect pixels, and the pixel value of each defect pixel is within a set pixel value range. Therefore, according to the pixel values ​​of the pixels included in the road surface image, at least one target feature data of the defect target block included in the road surface image is determined. Then, the at least one target feature data and the weight corresponding to each of the at least one target feature data are applied to obtain the defect probability. Finally, according to the relationship between the defect probability and the set defect probability threshold, the type of defect included in the road surface image is determined. Such a design fully combines the characteristics of road defects and improves the accuracy of road pavement defect detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0025] Figure 1 A schematic diagram of a road pavement disease provided by an embodiment of the present application is exemplarily shown;

[0026] Figure 2 A flowchart of a road pavement disease detection method provided by an embodiment of the present application is exemplarily shown;

[0027] Figure 3 A schematic diagram of a three-scale attention module provided in an embodiment of the present application is exemplarily shown;

[0028] Figure 4 The schematic diagram of the structure of a first image detection network model provided by an embodiment of the present application is exemplarily shown;

[0029] Figure 5 A flowchart of a method for determining a disease level provided in an embodiment of the present application is exemplarily shown;

[0030] Figure 6 A flowchart of a method for determining a size condition provided by an embodiment of the present application is exemplarily shown;

[0031] Figure 7 A schematic diagram of a crack disease detection area provided by an embodiment of the present application is exemplarily shown;

[0032] Figure 8 A complete schematic diagram of a road pavement disease detection method provided by an embodiment of the present application is exemplarily shown;

[0033] Fig. 9 The schematic diagram of the structure of a road pavement disease detection device provided in an embodiment of the present application is exemplarily shown;

[0034] Fig.10 The schematic diagram of the structure of a road pavement defect detection device provided in an embodiment of the present application is exemplified. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0036] During normal road operation, affected by factors such as weather, vehicle overload and long-term operation, road pavement defects such as bumps and pits appear, affecting road operation efficiency and service quality, and even threatening the safety of people and vehicles. This problem is more obvious on highways. Therefore, how to detect road pavement defects with high precision is particularly important.

[0037] Among the related technologies, one method is manual inspection, which uses patrolling, taking photos, and uploading records to detect diseases. This method has high accuracy but high labor costs. Another method is to use professional equipment to collect and analyze road data. The detection and maintenance costs of professional equipment are relatively poor. Another method is to collect video data of road scenes in real time for analysis. However, due to the interference of complex light and shadow environments, and the fact that the target features of diseases are not significant, the detection accuracy is low, and diseases with different morphologies cannot be accurately identified.

[0038] To this end, the embodiment of the present application provides a method for detecting road pavement defects, in which a road surface image can be segmented into different defect target blocks, at least one target feature data used to describe each defect target block is calculated, and then the defect probability is calculated based on at least the target feature data, and then the type of defect included in the road surface image is determined based on the relationship between the defect probability and a set defect probability threshold. Such a design improves the accuracy of road pavement defect detection.

[0039] After introducing the design ideas of the embodiments of the present application, the following briefly introduces the application scenarios to which the technical solutions of the embodiments of the present application can be applied. It should be noted that the application scenarios introduced below are only used to illustrate the embodiments of the present application and are not limited. In specific implementation, the technical solutions provided by the embodiments of the present application can be flexibly applied according to actual needs.

[0040] First, the classification of road defects is illustrated by examples. Taking asphalt pavement as an example, the types of defects are mainly divided into cracks, looseness, deformation and others.

[0041] (1) Crack diseases can be divided into tortoise cracks, block cracks, longitudinal cracks, transverse cracks, etc. according to the direction and cause of the cracks.

[0042] (2) Looseness disease is a disease that occurs frequently in the rainy season and mainly includes looseness and pitting.

[0043] Due to the insufficient water stability of asphalt pavement, the road surface will gradually become rough, loose, and even have potholes, peeling, and gnawing edges.

[0044] (3) Deformation damage mainly includes subsidence, rutting and wave bulge.

[0045] (4) Other types of diseases: In addition to the three types of diseases, namely cracks, looseness and deformation, asphalt pavement will also produce some other types of diseases, mainly including oil spills, repairs, etc.

[0046] refer to Figure 1 , shows a schematic diagram of a road pavement disease, where: Figure 1 Several types of defects are shown, 11 is a bulge, 12 is a crack, and 13 is a pothole. The bulge 11 and the pothole 12 are usually curved surfaces, and the crack 13 is usually a line.

[0047] To further illustrate the technical solution provided by the embodiment of the present application, this is described in detail below in conjunction with the accompanying drawings and specific implementation methods. Although the embodiment of the present application provides the method operation steps shown in the following embodiments or drawings, more or fewer operation steps may be included in the method based on routine or no creative labor. In the steps where there is no necessary causal relationship logically, the execution order of these steps is not limited to the execution order provided in the embodiment of the present application.

[0048] Combine the following Figure 1 The application scenario shown is shown in Figure 2 , which is a flow chart of a method for detecting road pavement defects. The method can be performed by a road pavement defect detection device, which can be an electronic device or a server with data processing functions, etc. Figure 2 , the technical solution provided in the embodiments of the present application is described.

[0049] S201: The detection device obtains a road surface image obtained by photographing the road to be tested.

[0050] S202: The detection device determines at least one target feature data of a defect target block included in the road surface image according to pixel values ​​of pixel points included in the road surface image.

[0051] Among them, a defect target block includes N continuous defect pixels, and the pixel values ​​of the defect pixels are within the set pixel value range.

[0052] S203: The detection device applies at least one target feature data and a weight corresponding to each of the at least one target feature data to obtain a disease probability.

[0053] S204: The detection device determines the type of defects included in the road surface image according to the relationship between the defect probability and a set defect probability threshold.

[0054] In the embodiment of the present application, a road surface image is obtained by photographing the road to be tested. Since the characteristics of the defect target are not obvious, the concept of a defect target block is introduced. A defect target block includes N consecutive defect pixels, and the pixel value of each defect pixel is within a set pixel value range. Therefore, according to the pixel values ​​of the pixels included in the road surface image, at least one target feature data of the defect target block included in the road surface image is determined. Then, the at least one target feature data and the weight corresponding to each of the at least one target feature data are applied to obtain the defect probability. Finally, according to the relationship between the defect probability and the set defect probability threshold, the type of defect included in the road surface image is determined. Such a design fully combines the characteristics of road defects and improves the accuracy of road pavement defect detection.

[0055] In S201, the detection device may communicate with the road monitoring device to obtain a road surface image obtained by shooting the road to be tested, or may process the obtained road surface video to obtain a road surface image. Alternatively, the real-time video stream of the road surface may be obtained through a vehicle-mounted monitoring camera, and the image may be obtained frame by frame and decoded, and re-encoded into a road surface image in a set format.

[0056] In S202, due to the fact that the characteristics of the disease target are not obvious, in the embodiment of the present application, the concept of a disease target block is introduced. A disease target block includes N consecutive diseased pixels, and the pixel value of each diseased pixel is within a set pixel value range. If the road surface image is an RGB image, the set pixel value range can be determined according to the characteristics of the disease; if the road surface image is binarized, the pixel value of 1 is a diseased pixel, and the pixel value of 0 is a non-disease pixel on the road surface.

[0057] Therefore, in the embodiment of the present application, at least one target feature data of the defect target block included in the road surface image is determined according to the pixel value of the pixel point included in the road surface image. Optionally, a road surface image may include multiple defect target blocks, and the multiple defect target blocks correspond to a set of target feature data, and a set of target feature data includes at least one target feature data.

[0058] Among them, at least one target feature data includes one or more of the following data: the number M of diseased target blocks, the lateral position X of the diseased target blocks in the road surface image, the longitudinal position Y of the diseased target blocks in the road surface image, the disease feature density ρ and the average area S of the diseased target blocks.

[0059] For example, a rectangular coordinate system can be established with the lower left corner of the road surface image as the origin. The number of defect target blocks M formed by connecting adjacent defect pixel points that can be connected continuously is counted, and discontinuous defect pixel points are not connected. The number of defect pixel points with the same ordinate in each defect target block is counted as X1, X2, ..., X M The average value of is taken as the lateral position X of the diseased target block in the road surface image. The number of diseased pixels with the same horizontal coordinates in each diseased target block is counted, Y1, Y2, ..., Y M The average value of is taken as the longitudinal position Y of the defect target block in the road surface image. The sum of the number of defect feature pixels in each defect target block and the ratio of the sum of the number of all pixels in each defect target block are counted as the defect feature density ρ. The number of defect pixels in all defect target blocks is counted, and the average area S is calculated.

[0060] In S203, the number M of the diseased target blocks, the lateral position X of the diseased target blocks in the road surface image, the longitudinal position Y of the diseased target blocks in the road surface image, the diseased feature density P and the average area S of the diseased target blocks are normalized, and the disease probability is obtained by applying at least one target feature data after the normalization process and the weight corresponding to each of the at least one target feature data. The weight corresponding to each of the target feature data can be set according to the importance of the feature.

[0061] For example, the normalized target feature data is represented by a feature vector V, V = [m, x, y, ρ0, s], m is the result of M normalization, x is the result of X normalization, y is the result of Y normalization, ρ0 is the result of ρ normalization, and s is the result of S normalization. The weight vector is W = [w1, w2, w3, w4, w5], so the disease probability P = V × W is obtained.

[0062] In S204, the detection device determines the type of damage included in the road surface image according to the relationship between the damage probability and the set damage probability threshold, which can be illustrated by the following situations:

[0063] The first case: if the disease probability is greater than or equal to the disease probability threshold, the type of the disease is determined to be a crack, where the disease probability threshold can be 0.6.

[0064] The first case: if the disease probability is less than the disease probability threshold, then determine whether the feature density is greater than the feature density threshold. If so, determine the type of disease as blistering, otherwise determine the type of disease as pockmarking. The feature density threshold can be 0.9.

[0065] In the above embodiment, the process of obtaining the type of damage included in the road surface image based on the road surface image can be implemented through a network model. Specifically, the road surface image is input into the first image detection network model to obtain the type of damage included in the road surface image.

[0066] Optionally, in an embodiment of the present application, a three-scale attention neural network model combined with a feature fusion network is proposed, called a first image detection network model. A road surface image can be input into the first image detection network model, and the feature extraction network in the first image detection network model can be applied to identify at least one disease feature in the road surface image.

[0067] Among them, the attention sampling mechanism is like the human visual nervous system, which can actively capture useful information from a large amount of target information, learn useful features, and suppress useless features. By learning the feature network and using the convolutional neural network with the attention mechanism, as the network goes deeper, the attention module will adaptively learn to extract useful information from the image. In view of the defects of the YOLO detection algorithm in road disease detection, the embodiment of the present application constructs a three-scale attention (triple-attention, TA) module to perform three-times enhanced extraction of feature channels, so as to achieve the purpose of strengthening the features of the diseased target block and suppressing interference features.

[0068] Exemplarily, the first image detection network model is a yolov detection network model, and the yolov detection network includes a darknet53 feature extractor; the darknet53 feature extractor includes a three-scale attention module, wherein the three-scale attention module includes a standard attention module, a first attention module and a second attention module, and the first attention module and the second attention module are respectively connected in parallel with the standard attention module; the first attention module is used to enhance the features of the diseased target block, and the second attention module is used to suppress interference features.

[0069] Figure 3 A schematic diagram of a three-scale attention module is shown, where 31 is a standard attention module, 32 is a first attention module, and 32 is a second attention module. Activation function 1 can use the relu activation function, activation function 2 can use the sigmoid activation function, and activation function 3 is the leaky relu activation function.

[0070] exist Figure 3In the diagram, adding 1×1 pixel and 3×3 pixel convolutions before the global average pooling can achieve cross-channel information integration, which is combined with the standard attention module to form a three-scale attention module, which strengthens the spatial connection of road disease images at different scales and different forms, and enhances the ability to extract multi-scale features. Finally, an attention branch with activation function 3 is added on the basis of the three-scale, which retains the gradient information in the original network to the greatest extent, and forms a three-scale attention network module while increasing the performance of multi-form disease detection.

[0071] The first attention module includes the first convolution layer of 1×1 pixel, the second convolution layer of 3×3 pixel, the global average pooling layer, the fully connected layer and the activation function layer; the first convolution layer and the second convolution layer are used to extract the target features of the road surface image under different disease forms. With this design, the first attention module can enhance the features of the disease target block, and the second attention module can suppress the interference features.

[0072] In summary, in the embodiment of the present application, the YOLOV algorithm includes two parts: a cspdarknet53 feature extractor and a multi-scale feature fusion. Among them, the darknet53 feature extractor uses a series of 1×1 pixel and 3×3 pixel convolutions, which are composed of 5 residual structures. The convolution layer in the residual structure adopts a skip-layer connection method to form a residual network. After multiple convolution operations, the image of 416×416 pixel×3 channels can be output as an image of 13×13 pixel×1024 channels. The feature fusion part of the YOLOV algorithm uses the principle of feature pyramid network and adopts a top-down approach to fuse the high-level prediction layer with the low-level layer, strengthens the transmission of feature flow, and then predicts and outputs feature maps of 3 different scales.

[0073] Figure 4 A structural diagram of a first image detection network model provided for an embodiment of the present application, wherein 41 is a three-scale attention module. Due to the presence of road disease targets of different forms and small sizes in actual application scenarios, the difficulty of disease identification is greatly increased. Considering the characteristics of multi-morphological disease identification, it is necessary to obtain more morphological feature information. On this basis, a three-scale attention TA module is designed and embedded in the cspdarknet-53 backbone network. Since the first attention module and the second attention module are added on the basis of the standard attention module, the weights of the three attention modules can be redistributed, and the distribution principle can be related to the pre-set recognition accuracy. In the embodiment of the present application, it can be evenly distributed, and the feature channel weight redistribution calibration can be performed to suppress the interference features in the image. At the same time, the feature fusion structure is used to improve the accuracy of disease morphology detection.

[0074] Figure 4 In the figure, only one attention residual network is shown, but multiple ones can be used in practice; DBL includes convolutional layers, batch normalization layers, and activation functions. The overall network design process is as follows: In the cspdarknet-53 network, the designed TA module is embedded in the adjusted residual structure, the weights of the channel relationship in the feature extraction network are redistributed, the spatial connection of road diseases at different scales is enhanced, and the learning and expression capabilities of the model are enhanced; combined with the feature fusion structure of the original yolov, a top-down and bottom-up fusion structure is adopted, and a skip-layer connection method is added to avoid adding additional parameters while improving the ability to extract feature information of multi-morphological disease changes.

[0075] In a specific example, yolov in the embodiment of the present application may be yolov3 or yolov5.

[0076] Since there are many types of defects, in order to show the defects to maintenance personnel more intuitively and clearly, after determining the type of defects included in the road surface image, the level of the defects can also be determined, which is used to describe the severity of the defects. This design can also obtain the severity of the defects more intuitively.

[0077] Optional, Figure 5 A flow chart for determining the disease level provided in an embodiment of the present application.

[0078] S500: The detection device determines the size of the disease according to the size of the disease target block.

[0079] S501: When the detection device determines that the disease is a blister or a pockmark according to the type of the disease, determine whether the size of the disease is greater than a set size. If so, execute S502; otherwise, execute S503.

[0080] Among them, the hug or pockmark disease has a certain size, and optionally, the size is set to 60*60px, where px is the pixel size.

[0081] S502: The detection device determines that the grade of the disease is a grade one disease.

[0082] S503: The detection equipment determines that the grade of the disease is a grade 2 disease;

[0083] S504: When the detection device determines that the defect is a crack according to the type of the defect, determine whether the size of the defect meets the size condition, and if so, execute S502, otherwise execute S503.

[0084] Among them, crack diseases usually include multiple pavement points, each of which can be called a detection point, and each disease detection point is a disease pixel point.

[0085] Optionally, in an embodiment of the present application, the severity of the primary disease is greater than the severity of the secondary disease.

[0086] Such a design can determine the level of the disease according to the type of disease, which is intuitive and accurate.

[0087] It should be noted that there is no obvious order relationship between the processes of S501, S502 and S503 and the processes of S504, S502 and S503. These are just examples for illustration and do not constitute a specific limitation.

[0088] Optional, reference Figure 6 , the size conditions are determined as follows:

[0089] S601: If the type of the defect is a crack, the detection device obtains at least one defect pixel point.

[0090] For example, if the defect is a crack-type defect, the number of defect detection points to be selected is determined based on the requirements of detection accuracy.

[0091] S602: The detection device determines the horizontal width and the vertical width of the defective area including the at least one defective pixel point according to the position of the at least one defective pixel point.

[0092] For example, the coordinates of the defective pixel are (x, y), and the minimum value x of the horizontal coordinates of multiple defective pixels is recorded respectively. min , the maximum value x of the horizontal axis max , the minimum value of the vertical axis y min and the maximum value y of the ordinate max In this example, the horizontal width of the detection area is |x max -x min |, the vertical width is |y max -y min |.

[0093] Assuming that the number of selected defective pixel points is Z, the horizontal width and vertical width of the defective area including the Z defective pixel points can be calculated. Figure 7 is a schematic diagram of a diseased area of ​​a crack disease in an embodiment of the present application, Figure 7 The number of defective pixels shown in is for example only.

[0094] S603: The detection device determines that the size condition is: the sum of the horizontal width and the vertical width is greater than the first set width, or either the horizontal width or the vertical width is greater than the second set width.

[0095] The first setting width is 100px, and the second setting width is 60px.

[0096] Such a design is more comprehensive and accurate than directly determining the disease level based on the arc length of the crack.

[0097] In order to make the technical solution of this application more perfect, Figure 8 A complete schematic diagram of a method for detecting road pavement defects provided in an embodiment of the present application, wherein the detection device takes an electronic device with data processing function as an example.

[0098] S800: The road monitoring device sends the road surface image obtained by photographing the road surface to be tested to the electronic device.

[0099] S801: The electronic device uses the first image detection network model to determine at least one target feature data of a defect target block included in the road surface image according to the pixel values ​​of the pixel points included in the road surface image; applies the at least one target feature data and the weight corresponding to each of the at least one target feature data to obtain a defect probability; and determines the type of defects included in the road surface image according to the relationship between the defect probability and a set defect probability threshold.

[0100] The implementation method of this step can refer to steps S202-S204, which will not be repeated here.

[0101] S802: The electronic device determines the level of the damage according to the type and size of the damage included in the road surface image.

[0102] The implementation method of this step may refer to steps S501-S504, which will not be described in detail here.

[0103] S803: The electronic device sends the disease level to the control device.

[0104] The control device may be a handheld terminal of a maintenance person, or a computer installed in a maintenance studio.

[0105] In order to solve the problem of insufficient accuracy in road disease detection under complex light interference and multiple disease morphological differences, the embodiments of the present application improve the YOLO detection algorithm based on the attention mechanism and the feature fusion structure. By constructing a three-scale attention module and embedding the YOLO feature extraction network, the feature channel is semantically enhanced three times, which effectively improves the feature extraction capability of the algorithm, extracts and identifies key features of road diseases, and realizes high-precision disease recognition applications.

[0106] like Fig. 9 As shown, based on the same inventive concept, an embodiment of the present application provides a road pavement disease detection device, including an image acquisition unit 91, a data processing unit 92 and a disease detection unit 93.

[0107] The image acquisition unit 91 is used to: acquire a road surface image obtained by photographing the road to be tested;

[0108] The data processing unit 92 is used to determine at least one target feature data of a defect target block included in the road surface image according to the pixel values ​​of the pixel points included in the road surface image; wherein a defect target block includes N consecutive defect pixel points, and the pixel values ​​of the defect pixel points are within a set pixel value range;

[0109] The data processing unit 92 is further used to: apply at least one target feature data and a weight corresponding to each of the at least one target feature data to obtain a disease probability;

[0110] The defect detection unit 93 is used to determine the type of defects included in the road surface image according to the relationship between the defect probability and the set defect probability threshold.

[0111] In a possible implementation, the at least one target characteristic data includes one or more of the following data:

[0112] The number of diseased target blocks, the lateral position of the diseased target blocks in the road surface image, the longitudinal position of the diseased target blocks in the road surface image, the disease feature density and the average area of ​​the diseased target blocks.

[0113] In a possible implementation, the lateral position of the defect target block in the road surface image is: the average value of the number of defect pixel points with the same ordinate in each defect target block;

[0114] The longitudinal position of the defect target block in the road surface image is: the average value of the number of defect pixel points with the same horizontal coordinate in each defect target block;

[0115] The disease feature density is: the ratio of the sum of the number of disease feature pixels in each disease target block to the sum of the number of all pixels in each disease target block.

[0116] In a possible implementation, the disease detection unit 93 is specifically used for:

[0117] If the disease probability is greater than or equal to the disease probability threshold, the type of the disease is determined to be crack;

[0118] If the disease probability is less than the disease probability threshold, it is determined whether the feature density is greater than the feature density threshold. If so, the type of the disease is determined to be a bag, otherwise the type of the disease is determined to be a pockmark.

[0119] In a possible implementation manner, the damage detection unit 93 is further configured to determine the type of damage included in the road surface image by:

[0120] The road surface image is input into the first image detection network model to obtain the type of damage included in the road surface image.

[0121] In a possible implementation, the first image detection network model is a yolov detection network model, and the yolov detection network includes a darknet53 feature extractor;

[0122] The darknet53 feature extractor includes a three-scale attention module, wherein the three-scale attention module includes a standard attention module, a first attention module and a second attention module, and the first attention module and the second attention module are respectively connected in parallel with the standard attention module; the first attention module is used to enhance the features of the diseased target block, and the second attention module is used to suppress interference features.

[0123] In a possible implementation, the first attention module includes a first convolution layer of 1×1 pixel, a second convolution layer of 3×3 pixel, a global average pooling layer, a fully connected layer, and an activation function layer; the first convolution layer and the second convolution layer are respectively used to extract features of the diseased target blocks of the road surface image.

[0124] In a possible implementation, a level determination unit is further included, which is used to, after determining the type of damage included in the road surface image:

[0125] Determine the size of the disease according to the size of the disease target block;

[0126] If the type of the disease is blistering or pockmarking, when the size of the disease is larger than the set size, the disease level is determined to be a first-level disease, otherwise the disease level is determined to be a second-level disease;

[0127] If the type of the disease is crack, then when the size of the disease meets the size condition, the grade of the disease is determined to be a first-level disease, otherwise the grade of the disease is determined to be a second-level disease;

[0128] The severity of primary disease is greater than that of secondary disease.

[0129] In a possible implementation manner, the level determination unit is specifically configured to:

[0130] If the type of the defect is a crack, obtaining at least one defect pixel point;

[0131] Determine, according to the position of at least one defective pixel, a lateral width and a longitudinal width of a defective area including at least one defective pixel;

[0132] The size condition includes: the sum of the horizontal width and the vertical width is greater than the first set width, or either the horizontal width or the vertical width is greater than the second set width.

[0133] Since the device is the device in the method in the embodiment of the present application, and the principle of solving the problem by the device is similar to that of the method, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0134] like Fig.10 As shown, based on the same inventive concept, an embodiment of the present application provides a road pavement disease detection device, which includes a data transmission unit 1001 and a processor 1002.

[0135] The data transmission unit 1001 is configured to execute:

[0136] Acquire a road surface image obtained by photographing the road to be tested;

[0137] The processor 1002 is configured to execute:

[0138] Determine at least one target feature data of a defect target block included in the road surface image according to the pixel values ​​of the pixel points included in the road surface image; wherein one defect target block includes N consecutive defect pixel points, and the pixel values ​​of the defect pixel points are within a set pixel value range;

[0139] Applying at least one target characteristic data and a weight corresponding to each of the at least one target characteristic data to obtain a disease probability;

[0140] The type of defects included in the road surface image is determined based on the relationship between the defect probability and a set defect probability threshold.

[0141] In a possible implementation, the at least one target characteristic data includes one or more of the following data:

[0142] The number of diseased target blocks, the lateral position of the diseased target blocks in the road surface image, the longitudinal position of the diseased target blocks in the road surface image, the disease feature density and the average area of ​​the diseased target blocks.

[0143] In a possible implementation, the lateral position of the defect target block in the road surface image is: the average value of the number of defect pixel points with the same ordinate in each defect target block;

[0144] The longitudinal position of the defect target block in the road surface image is: the average value of the number of defect pixel points with the same horizontal coordinate in each defect target block;

[0145] The disease feature density is: the ratio of the sum of the number of disease feature pixels in each disease target block to the sum of the number of all pixels in each disease target block.

[0146] In one possible implementation, the processor 1002 is specifically configured to execute;

[0147] If the disease probability is greater than or equal to the disease probability threshold, the type of the disease is determined to be crack;

[0148] If the disease probability is less than the disease probability threshold, it is determined whether the feature density is greater than the feature density threshold. If so, the type of the disease is determined to be a bag, otherwise the type of the disease is determined to be a pockmark.

[0149] In a possible implementation, the processor 1002 is specifically configured to execute: determining the type of damage included in the road surface image by:

[0150] The road surface image is input into the first image detection network model to obtain the type of damage included in the road surface image.

[0151] In a possible implementation, the first image detection network model is a yolov detection network model, and the yolov detection network includes a darknet53 feature extractor;

[0152] The darknet53 feature extractor includes a three-scale attention module, wherein the three-scale attention module includes a standard attention module, a first attention module and a second attention module, and the first attention module and the second attention module are respectively connected in parallel with the standard attention module; the first attention module is used to enhance the features of the diseased target block, and the second attention module is used to suppress interference features.

[0153] In a possible implementation, the first attention module includes a first convolution layer of 1×1 pixel, a second convolution layer of 3×3 pixel, a global average pooling layer, a fully connected layer, and an activation function layer; the first convolution layer and the second convolution layer are respectively used to extract features of the diseased target blocks of the road surface image.

[0154] In a possible implementation, after determining the type of damage included in the road surface image, the processor 1002 is further configured to execute:

[0155] Determine the size of the disease according to the size of the disease target block;

[0156] If the type of the disease is blistering or pockmarking, when the size of the disease is larger than the set size, the disease level is determined to be a first-level disease, otherwise the disease level is determined to be a second-level disease;

[0157] If the type of the disease is crack, then when the size of the disease meets the size condition, the grade of the disease is determined to be a first-level disease, otherwise the grade of the disease is determined to be a second-level disease;

[0158] The severity of primary disease is greater than that of secondary disease.

[0159] In a possible implementation, the processor 1002 is specifically configured to execute:

[0160] If the type of the defect is a crack, obtaining at least one defect pixel point;

[0161] Determine, according to the position of at least one defective pixel, a lateral width and a longitudinal width of a defective area including at least one defective pixel;

[0162] The size condition includes: the sum of the horizontal width and the vertical width is greater than the first set width, or either the horizontal width or the vertical width is greater than the second set width.

[0163] The embodiment of the present application also provides a computer storage medium, in which computer program instructions are stored. When the instructions are executed on a computer, the computer executes the steps of the above-mentioned road pavement disease detection method.

[0164] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0165] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0166] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0167] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0168] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for detecting road pavement defects, characterized in that: include: Acquire a road surface image obtained by photographing the road to be tested; Determine at least one target feature data of a defect target block included in the road surface image according to the pixel values ​​of the pixel points included in the road surface image; wherein a defect target block includes N consecutive defect pixel points, and the pixel values ​​of the defect pixel points are within a set pixel value range; Applying the at least one target characteristic data and the weights corresponding to the at least one target characteristic data to obtain a disease probability; The type of the damage included in the road surface image is determined according to the relationship between the damage probability and a set damage probability threshold.

2. The method according to claim 1, characterized in that The at least one target characteristic data includes one or more of the following data: The number of the diseased target blocks, the lateral position of the diseased target blocks in the road surface image, the longitudinal position of the diseased target blocks in the road surface image, the disease feature density and the average area of ​​the diseased target blocks.

3. The method according to claim 2, characterized in that The lateral position of the defect target block in the road surface image is: the average value of the number of defect pixel points with the same ordinate in each defect target block; The longitudinal position of the defect target block in the road surface image is: the average value of the number of defect pixel points with the same horizontal coordinate in each defect target block; The disease feature density is: the ratio of the sum of the number of disease feature pixels in each disease target block to the sum of the number of all pixels in each disease target block.

4. The method according to any one of claims 1 to 3, characterized in that: Determining the type of damage included in the road surface image according to the relationship between the damage probability and a set damage probability threshold; If the disease probability is greater than or equal to the disease probability threshold, determining that the type of the disease is a crack; If the disease probability is less than the disease probability threshold, it is determined whether the feature density is greater than the feature density threshold. If so, the type of the disease is determined to be a bag; otherwise, the type of the disease is determined to be a pockmark.

5. The method according to any one of claims 1 to 3, characterized in that: The type of damage included in the road surface image is determined by: The road surface image is input into a first image detection network model to obtain the type of damage included in the road surface image.

6. The method according to claim 5, characterized in that The first image detection network model is a yolov detection network model, and the yolov detection network includes a darknet53 feature extractor; The darknet53 feature extractor includes a three-scale attention module, wherein the three-scale attention module includes a standard attention module, a first attention module and a second attention module, the first attention module and the second attention module are respectively connected in parallel with the standard attention module; the first attention module is used to enhance the features of the diseased target block, and the second attention module is used to suppress interference features.

7. The method according to claim 6, characterized in that The first attention module includes a first convolution layer of 1×1 pixel, a second convolution layer of 3×3 pixel, a global average pooling layer, a fully connected layer and an activation function layer; the first convolution layer and the second convolution layer are respectively used to extract features of the diseased target blocks of the road surface image.

8. The method according to any one of claims 1 to 3 or 6 to 7, characterized in that: After determining the type of damage included in the road surface image, the method further includes: Determining the size of the disease according to the size of the disease target block; If the type of the disease is blistering or pockmarking, then when the size of the disease is greater than the set size, the grade of the disease is determined to be a first-level disease, otherwise the grade of the disease is determined to be a second-level disease; If the type of the disease is a crack, then when the size of the disease meets the size condition, the grade of the disease is determined to be a first-level disease, otherwise the grade of the disease is determined to be a second-level disease; The severity of the primary disease is greater than the severity of the secondary disease.

9. The method according to claim 8, characterized in that The method further comprises: If the type of the defect is a crack, obtaining at least one defect pixel point; Determine, according to the position of the at least one defective pixel, a lateral width and a longitudinal width of a defective area including the at least one defective pixel; The size condition includes: the sum of the lateral width and the longitudinal width is greater than a first set width, or either the lateral width or the longitudinal width is greater than a second set width.

10. A road surface disease detection device, characterized in that: including a data transmission unit and a processor; The data transmission unit is configured to execute: Acquire a road surface image obtained by photographing the road to be tested; The processor is configured to perform: Determine at least one target feature data of a defect target block included in the road surface image according to the pixel values ​​of the pixel points included in the road surface image; wherein a defect target block includes N consecutive defect pixel points, and the pixel values ​​of the defect pixel points are within a set pixel value range; Applying the at least one target characteristic data and the weights corresponding to the at least one target characteristic data to obtain a disease probability; The type of the damage included in the road surface image is determined according to the relationship between the damage probability and a set damage probability threshold.

Images