Road surface disease detection method and device, electronic equipment and storage medium

Abstract

The application provides a road surface disease detection method and device, an electronic device and a storage medium, wherein the method comprises: acquiring road surface image information to be analyzed; inputting the road surface image information into a first target model for processing to obtain disease mask information output by the first target model, wherein the disease mask information carries disease positioning information and a disease type; processing the disease positioning information to obtain depth information; and inputting the depth information and the disease type into a second target model to obtain road surface disease detection information of the road surface image information. Through the application, the problems of high missing detection rate, low efficiency, and inability to provide comprehensive road surface information to ensure vehicle traffic safety in the related art are solved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method and apparatus for detecting road surface defects, electronic equipment, and storage medium. Background Technology

[0002] With the rapid development of the economy and society, the public road network is constantly being improved. How to quickly and intelligently identify, detect, maintain, and manage road surface defects without closing roads has become an important task in the road construction field. Currently, traditional methods for detecting road surface defects still mainly rely on manual visual inspection and measurement, resulting in high missed detection rates, low efficiency, and significant safety hazards for inspectors taking photos amidst traffic.

[0003] In recent years, with the rapid development of deep learning technology and the widespread use of IoT devices, more and more deep learning technologies have been applied in the field of road inspection. Deep learning technology can leverage its advantage of learning representations from data to improve the accuracy of road defect identification, and can significantly save road maintenance workers time in taking timely and effective repair measures.

[0004] Existing deep learning methods, such as YOLOv5 (a single-stage object detection algorithm), Faster R-CNN (a fast object detection algorithm), adversarial learning, and semantic segmentation, all have their own shortcomings. For example, object detection algorithms like YOLOv5 and Faster R-CNN cannot accurately extract the contours of road defects, thus limiting their application value. Adversarial learning algorithms are complex, have a narrow application range, and cannot effectively guarantee the accuracy of the generated lossless road surface images. Semantic segmentation algorithms require detection and localization before semantic segmentation, necessitating multiple deep learning models. Furthermore, the detection results can significantly impact semantic segmentation, allowing only local features to be utilized, which is detrimental to classification results.

[0005] Therefore, how to quickly and comprehensively determine road surface-related defects based on deep learning technology and improve vehicle traffic safety has become an urgent problem to be solved in this field. Summary of the Invention

[0006] This application provides a method and apparatus for detecting road surface defects, an electronic device, and a storage medium, to at least solve the problems of high missed detection rate and low efficiency in road surface defect detection in related technologies, which cannot provide comprehensive road surface information to ensure vehicle traffic safety.

[0007] According to one aspect of the embodiments of this application, a method for detecting pavement defects is provided, the method comprising:

[0008] Obtain the road surface image information to be analyzed;

[0009] The road surface image information is input into a first target model for processing to obtain the disease mask information output by the first target model, wherein the disease mask information carries disease location information and disease type;

[0010] The disease location information is processed to obtain depth information;

[0011] The depth information and the type of damage are input into the second target model to obtain the road damage detection information of the road image information.

[0012] According to another aspect of the embodiments of this application, a pavement distress detection device is also provided, the device comprising:

[0013] The first acquisition module is used to acquire road surface image information to be analyzed;

[0014] The first processing module is used to input the road surface image information into the first target model for processing to obtain the disease mask information output by the first target model, wherein the disease mask information carries disease location information and disease type.

[0015] The second processing module is used to process the disease location information to obtain depth information;

[0016] The first input module is used to input the depth information and the type of damage into the second target model to obtain the road damage detection information of the road image information.

[0017] Optionally, the first processing module includes:

[0018] The input unit is used to input the road surface image information into the first target model, and to determine whether each pixel in the road surface image information is a disease area through convolution, and to obtain the disease type of each pixel.

[0019] The generation unit is used to connect pixels of the same type of disease to generate the disease mask information.

[0020] Optionally, the second processing module includes:

[0021] The acquisition unit is used to acquire the original image and depth point cloud image corresponding to the disease location information;

[0022] The first determining unit is used to determine the first pixel position corresponding to the disease location information based on the original image, and to determine the second pixel position corresponding to the disease location information based on the depth point cloud image.

[0023] A fusion unit is used to fuse the features of the first pixel position and the second pixel position to obtain at least two key point information;

[0024] The second determining unit is used to determine the depth information based on the key point information.

[0025] Optionally, the second determining unit includes:

[0026] The first acquisition submodule is used to acquire the physical coordinates of each key point in the physical coordinate system;

[0027] A generation submodule is used to generate the depth information using the physical coordinates.

[0028] Optionally, the module obtained includes:

[0029] The third determining unit is used to determine the target physical size output type based on the target disease type when it is determined that the disease type input to the second target model is the target disease type.

[0030] The processing unit is used to process the physical coordinates corresponding to the depth information using a preset calculation scheme to obtain the physical dimensions corresponding to the depth information;

[0031] The unit is used to output the target physical size output type and the physical size as the output result to obtain the pavement defect detection information.

[0032] Optionally, the third determining unit includes:

[0033] The second acquisition submodule is used to acquire the target mapping table in the second target model, wherein the target mapping table stores the mapping relationship between each disease type and the physical size output type;

[0034] The determination submodule is used to determine the target physical size output type based on the target mapping table and the target disease type.

[0035] Optionally, the device further includes:

[0036] The second acquisition module is used to acquire the road surface image information at a preset frequency before the road surface image information is input into the first target model for processing.

[0037] The preprocessing module is used to perform image preprocessing on the road surface image information;

[0038] The second input module is used to input the preprocessed road surface image information into the first target model for processing.

[0039] According to another aspect of the embodiments of this application, an electronic device is also provided, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; wherein the memory is used to store a computer program; and the processor is used to execute the method steps of any of the above embodiments by running the computer program stored in the memory.

[0040] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, wherein a computer program is stored therein, wherein the computer program is configured to execute the method steps of any of the above embodiments when it is run.

[0041] In this embodiment, road surface image information to be analyzed is acquired; the road surface image information is input into a first target model for processing to obtain the defect mask information output by the first target model, wherein the defect mask information carries defect location information and defect type; the defect location information is processed to obtain depth information; the depth information and defect type are input into a second target model to obtain road surface defect detection information of the road surface image information. Because this embodiment combines the road surface defect type obtained from the first target model with the depth information of defect location to obtain the associated information of road surface defects, it enables intelligent identification and work measurement of road surface defects, improving maintenance efficiency while reducing costs. It effectively enhances the reliability, comprehensiveness, and accuracy of road surface defect detection information acquisition, ensuring vehicle traffic safety, and thus solves the problems of high false negative rates and low efficiency in road surface defect detection in related technologies, which fail to provide comprehensive road surface information to ensure vehicle traffic safety. Attached Figure Description

[0042] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0043] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, those skilled in the art can obtain other drawings based on these drawings without creative effort.

[0044] Figure 1 This is a schematic flowchart of an optional pavement distress detection method according to an embodiment of this application;

[0045] Figure 2 This is a schematic diagram of the overall process of an optional pavement distress detection method according to an embodiment of this application;

[0046] Figure 3This is a structural block diagram of an optional road surface defect detection device according to an embodiment of this application;

[0047] Figure 4 This is a structural block diagram of an optional electronic device according to an embodiment of this application. Detailed Implementation

[0048] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0049] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0050] In recent years, with the rapid development of deep learning technology and the widespread use of IoT devices, more and more deep learning technologies have been applied in the field of road detection.

[0051] Existing deep learning methods include: 1) Faster R-CNN: By improving the model's feature extraction network, a target detection model suitable for 3D pavement defect identification on highways is constructed; 2) Based on the existing mature YOLOv5 model, a target detection model suitable for highway pavement defect identification is constructed by improving the model's feature extraction network and redesigning the model's anchor boxes; 3) Adversarial learning: Acquire the pavement image to be identified; generate a non-destructive pavement image based on the adversarial learning method; compare the non-destructive pavement image with the pavement image to be identified, determine the image difference regions, and determine the defect identification result of the pavement image to be identified based on the characteristics of the image difference regions; 4) Semantic segmentation after detection and localization: First, detect and acquire the pavement defect image to be analyzed; input the pavement defect image into a hybrid neural network model to obtain the defect localization information and defect mask information output by the hybrid neural network model.

[0052] However, the target detection algorithms YOLOv5 and Faster R-CNN cannot accurately extract the contours of road defects, thus limiting their application value. Adversarial learning algorithms are complex, have a narrow application range, and cannot effectively guarantee the accuracy of the generated lossless road surface images. Semantic segmentation algorithms require detection and localization before semantic segmentation, necessitating multiple deep learning models, and the detection results significantly impact semantic segmentation, relying only on local features and hindering classification results. Therefore, to address the series of problems existing in current deep learning algorithms during road defect detection and information acquisition, this application proposes a road defect detection method, such as... Figure 1 As shown, this method can be applied to the autonomous driving domain control or computing platform on the vehicle side, or to a computing platform set up on the roadside. The method includes:

[0053] Step S101: Obtain the road surface image information to be analyzed;

[0054] Step S102: Input the road surface image information into the first target model for processing to obtain the disease mask information output by the first target model. The disease mask information carries the disease location information and the disease type.

[0055] Step S103: Process the disease location information to obtain depth information;

[0056] Step S104: Input the depth information and the type of damage into the second target model to obtain the road damage detection information of the road image information.

[0057] Optionally, firstly, the road surface defects involved in the embodiments of this application mainly include cracks, fissures, potholes, depressions, protrusions, etc. on the road surface. In autonomous driving applications in scenarios such as port parks, mountainous mining areas, etc., in addition to detecting pedestrians, vehicles, and the surrounding environment, vehicles also need to pay attention to the road surface conditions. Based on the road surface protrusions or depressions, they need to determine whether the route is feasible and whether there is a risk of passing through. At this time, it is necessary to detect defects and obtain related information from the current road surface information.

[0058] Furthermore, taking the vehicle-side autonomous driving domain control or computing platform as the execution entity as an example, the following explanation is provided: The vehicle can first acquire the road surface image information to be analyzed, and then input this road surface image information into the first target model for processing. This first target model can be a pre-trained semantic segmentation model. This semantic segmentation model consists entirely of convolutional layers and has no fully connected layers, thus achieving pixel-level classification. Each pixel is assigned to a specific category, and the entire image is segmented into different regions. The input road surface image information can be RGB three-channel image data, and the output is the road surface defect classification label for each pixel. This classification label represents the defect type to which each image pixel belongs. Since the convolutional layers of the semantic segmentation model can effectively capture local features in the image and nest many such images hierarchically, connecting adjacent points of image pixels with the same defect type generates connected components, thus obtaining the defect mask information.

[0059] Therefore, as long as the road surface image information is input into the first target model for processing, the defect mask information can be obtained.

[0060] It is understood that the formation of disease mask information is not only related to the type of disease, but also requires obtaining the disease location of all image pixels of a certain disease type. Therefore, the disease mask information carries both disease type and disease location information.

[0061] Then, target algorithms, such as those based on binocular cameras, depth estimation models, lidar, and geometric calibration, can be used to process the disease location information in the disease mask information to obtain the real physical coordinates of each pixel in the image in the relative coordinate system, thus obtaining the depth information.

[0062] Then, by combining depth information and disease type, pavement disease detection information output by the second target model is obtained. The pavement disease detection information includes at least one of the following: actual length information of pavement disease, average width information of pavement disease, and area information of pavement disease.

[0063] In this embodiment, road surface image information to be analyzed is acquired; the road surface image information is input into a first target model for processing to obtain the defect mask information output by the first target model, wherein the defect mask information carries defect location information and defect type; the defect location information is processed to obtain depth information; the depth information and defect type are input into a second target model to obtain road surface defect detection information of the road surface image information. Because this embodiment combines the road surface defect type obtained from the first target model with the depth information of defect location to obtain the associated information of road surface defects, it enables intelligent identification and work measurement of road surface defects, improving maintenance efficiency while reducing costs. It effectively enhances the reliability, comprehensiveness, and accuracy of road surface defect detection information acquisition, ensuring vehicle traffic safety, and thus solves the problems of high false negative rates and low efficiency in road surface defect detection in related technologies, which fail to provide comprehensive road surface information to ensure vehicle traffic safety.

[0064] As an optional embodiment, road surface image information is input into a first target model for processing, and the resulting defect mask information output by the first target model includes:

[0065] The road surface image information is input into the first target model. The convolution method is used to determine whether each pixel in the road surface image information is a disease area and to obtain the disease type of each pixel.

[0066] Connect pixels with the same type of disease to generate disease mask information.

[0067] Optionally, the target pavement defect image is input into a target semantic segmentation network model. Through convolution, each pixel in the pavement image is determined to be a defect region, i.e., whether each pixel matches a feature of a set of preset defect types. Pixels of the same defect type are then connected to obtain a defect mask with the same size as the entire or a local portion of the pavement image, and a value of 0 or 1. Each pixel position has only two values: 0 and 1. 0 indicates that the pixel belongs to the background class, and 1 indicates that the pixel belongs to the target defect type (here, the target defect type is any one of a set of preset defect types, such as crack defect).

[0068] It should be noted that when connecting pixels with the same type of disease, the pixels can be filtered based on the actual scenario requirements and confidence settings. For example, if a pixel's pixel count is less than a preset pixel value (such as 30*30 pixels), then the pixel can be discarded directly.

[0069] As an optional embodiment, the disease location information is processed to obtain depth information, including:

[0070] Obtain the original image and depth point cloud image corresponding to the disease location information;

[0071] The first pixel position corresponding to the disease location information is determined based on the original image, and the second pixel position corresponding to the disease location information is determined based on the depth point cloud image.

[0072] By fusing features from the first pixel position and the second pixel position, at least two key point information points are obtained;

[0073] Depth information is determined based on key point information.

[0074] Optionally, a binocular camera can be used to acquire the original image and depth point cloud image corresponding to the defect location information, thereby obtaining the correspondence between the original image and the depth point cloud image. The first pixel position corresponding to the defect location information is determined based on the original image, and the second pixel position corresponding to the defect location information is determined based on the depth point cloud image. By fusing the first pixel position and the second pixel position, multiple key point information of the defect on the current road surface can be determined. Based on this key point information, its physical coordinate value in the physical coordinate system can be obtained, and then these physical coordinate values ​​are used as the depth information reflecting the road surface defect.

[0075] In this embodiment of the application, after obtaining depth information, it is possible to intelligently identify and statistically analyze road surface defects such as cracks, potholes, and protrusions without closing the road, thereby improving the accuracy of defect judgment and increasing work efficiency.

[0076] As an optional embodiment, the depth information and the type of damage are input into the second target model to obtain pavement damage detection information from the pavement image information, including:

[0077] If the disease type of the input second target model is determined to be the target disease type, the target physical size output type is determined according to the target disease type;

[0078] The physical coordinates corresponding to the depth information are processed using a preset calculation scheme to obtain the physical dimensions corresponding to the depth information;

[0079] By using the target physical dimension output type and physical dimension as the output result, pavement distress detection information is obtained.

[0080] Optionally, after the second target model obtains the disease type, it will determine the corresponding physical size output type based on the current disease type. Further, if the current disease type is determined to be the target disease type of crack disease, the corresponding target physical size output type should be the length type. If the current disease type is determined to be the target disease type of pit disease, the corresponding target physical size output type should be the area type.

[0081] Then, using a preset calculation scheme, such as addition and subtraction methods, the physical coordinates corresponding to the depth information are processed to obtain the physical dimensions corresponding to the depth information, namely, the actual length value, the actual area value, or the actual height value, etc. Taking the calculation of the actual length value as an example, the starting point and ending point of the coordinates can be determined based on the depth information. Subtracting the starting point coordinate value from the ending point coordinate value yields the actual length value, which is the physical dimension.

[0082] Finally, the current target physical dimension output type and corresponding physical dimension are used as the final output result, outputting pavement distress detection information, such as the actual length of pavement distress (x cm) and the area of ​​pavement distress (y cm). 2 Information such as the height of road surface defects (zcm).

[0083] In this embodiment of the application, by estimating the physical dimensions of the road surface, it is determined whether the vehicle can pass through, avoiding dangerous road sections such as depressions or protrusions that may affect the vehicle's progress, and providing real-time assistance to engineering vehicles to safely perform autonomous driving or path planning in mountainous areas, mining areas, ports and other scenarios.

[0084] As an optional embodiment, determining the target physical size output type based on the target disease type includes:

[0085] Obtain the target mapping table in the second target model, where the target mapping table stores the mapping relationship between each disease type and the physical size output type;

[0086] Based on the target mapping table and the target disease type, determine the target physical size output type.

[0087] Optionally, in this embodiment of the application, a target mapping table may be provided in the second target model. The target mapping table mainly stores the mapping relationship between each type of disease and each type of physical size output type. For example, the physical size output type corresponding to crack disease is the length type, and the physical size output type corresponding to pit disease is the area type, etc.

[0088] Based on the target mapping table and the current target disease type, the current target physical size output type can be determined.

[0089] As an optional embodiment, before inputting the road surface image information into the first target model for processing, the method further includes:

[0090] Obtain road surface image information according to a preset frequency;

[0091] Image preprocessing is performed on road surface image information;

[0092] The preprocessed road surface image information is input into the first target model for further processing.

[0093] Optionally, road surface image information can be acquired from the vehicle or roadside at a preset frequency, such as by interval frame, by time, or by travel distance. Then, the road surface image information is preprocessed, including but not limited to image scaling, rotation, cropping, and brightness adjustment. After that, the preprocessed road surface image information is input into the first target model for further processing.

[0094] As an alternative embodiment, such as Figure 2 As shown, Figure 2 This is a schematic diagram of the overall process of an optional pavement distress detection method according to an embodiment of this application. The specific process is as follows:

[0095] Images are acquired from the vehicle or roadside.

[0096] The image is input into the semantic segmentation model and the deep information model;

[0097] The semantic segmentation model infers and obtains road surface mask information, including the type and location of defects;

[0098] Depth information models acquire image depth information. These models can be geometric algorithms, stereo cameras, depth estimation models, etc.

[0099] By combining the defect mask information with the depth information, the pavement defect detection information output by the geometric algorithm model is obtained; wherein, the pavement defect detection information includes at least one of the following: actual length information of pavement defects, average width information of pavement defects, and area information of pavement defects.

[0100] The road surface defect detection methods described in the above embodiments can be applied to autonomous driving, road obstacle detection, route planning, and other fields. They can be used for defect detection and road condition detection of dirt roads, asphalt roads, and cement roads in the maintenance field. They can also be extended to road surface and facility maintenance in engineering vehicle application scenarios such as docks and ports. The equipment that performs the road surface defect detection methods described in the above embodiments can be deployed on products such as vehicle-mounted intelligent equipment, automatic defect identification integrated machines, and intelligent maintenance service cloud.

[0101] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0102] According to another aspect of the embodiments of this application, a pavement defect detection device for implementing the above-described pavement defect detection method is also provided. Figure 3This is a structural block diagram of an optional pavement distress detection device according to an embodiment of this application, such as... Figure 3 As shown, the device may include:

[0103] The first acquisition module 301 is used to acquire road surface image information to be analyzed;

[0104] The first processing module 302 is used to input road surface image information into the first target model for processing to obtain the disease mask information output by the first target model, wherein the disease mask information carries disease location information and disease type.

[0105] The second processing module 303 is used to process the disease location information to obtain depth information;

[0106] The first input module 304 is used to input depth information and disease type into the second target model to obtain road surface disease detection information from the road surface image information.

[0107] It should be noted that the first acquisition module 301 in this embodiment can be used to execute the above step S101, the first processing module 302 in this embodiment can be used to execute the above step S102, the second processing module 303 in this embodiment can be used to execute the above step S103, and the first input module 304 in this embodiment can be used to execute the above step S104.

[0108] By combining the pavement distress types obtained from the first target model with the depth information of distress location through the above modules, the associated information of pavement distress can be obtained. This enables intelligent identification and measurement of pavement distress, improving maintenance efficiency while reducing costs. It also effectively enhances the reliability, comprehensiveness and accuracy of pavement distress detection information, ensuring vehicle traffic safety. This solves the problems of high false negative rate and low efficiency in pavement distress detection in related technologies, which cannot provide comprehensive pavement information to ensure vehicle traffic safety.

[0109] As an optional embodiment, the first processing module includes:

[0110] The input unit is used to input road surface image information into the first target model, and to determine whether each pixel in the road surface image information is a disease area through convolution, and to obtain the disease type of each pixel.

[0111] The generation unit is used to connect pixels of the same type of disease to generate disease mask information.

[0112] As an optional embodiment, the second processing module includes:

[0113] The acquisition unit is used to acquire the original image and depth point cloud image corresponding to the disease location information;

[0114] The first determining unit is used to determine the first pixel position corresponding to the disease location information based on the original image, and to determine the second pixel position corresponding to the disease location information based on the depth point cloud image.

[0115] The fusion unit is used to fuse the features of the first pixel position and the second pixel position to obtain at least two key point information.

[0116] The second determining unit is used to determine depth information based on key point information.

[0117] As an optional embodiment, the second determining unit includes:

[0118] The first acquisition submodule is used to acquire the physical coordinates of each key point in the physical coordinate system.

[0119] The generation submodule is used to generate depth information using physical coordinates.

[0120] As an optional embodiment, the obtained module includes:

[0121] The third determining unit is used to determine the target physical size output type based on the target disease type when the disease type of the input second target model is determined to be the target disease type.

[0122] The processing unit is used to process the physical coordinates corresponding to the depth information using a preset calculation scheme to obtain the physical dimensions corresponding to the depth information.

[0123] The unit is used to output the target physical size output type and physical size as the output result to obtain pavement defect detection information.

[0124] As an optional embodiment, the third determining unit includes:

[0125] The second acquisition submodule is used to acquire the target mapping table in the second target model, wherein the target mapping table stores the mapping relationship between each disease type and the physical size output type;

[0126] The determination submodule is used to determine the target physical size output type based on the target mapping table and the target disease type.

[0127] As an optional embodiment, the device further includes:

[0128] The second acquisition module is used to acquire road surface image information at a preset frequency before inputting the road surface image information into the first target model for processing.

[0129] The preprocessing module is used to preprocess road surface image information;

[0130] The second input module is used to input the preprocessed road surface image information into the first target model for processing.

[0131] According to another aspect of the embodiments of this application, an electronic device for implementing the above-described pavement defect detection method is also provided. The electronic device may be a server, a terminal, or a combination thereof.

[0132] Figure 4 This is a structural block diagram of an optional electronic device according to an embodiment of this application, such as... Figure 4 As shown, it includes a processor 401, a communication interface 402, a memory 403, and a communication bus 404. The processor 401, communication interface 402, and memory 403 communicate with each other via the communication bus 404.

[0133] Memory 403 is used to store computer programs;

[0134] When processor 401 executes a computer program stored in memory 403, it performs the following steps:

[0135] Obtain the road surface image information to be analyzed;

[0136] The road surface image information is input into the first target model for processing to obtain the disease mask information output by the first target model. The disease mask information carries the disease location information and disease type.

[0137] The disease location information is processed to obtain depth information;

[0138] By inputting depth information and disease type into the second target model, pavement disease detection information from pavement image information is obtained.

[0139] Optionally, in this embodiment, the communication bus can be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0140] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0141] The memory may include RAM, or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0142] As an example, such as Figure 4 As shown, the memory 403 may include, but is not limited to, the first acquisition module 301, the first processing module 302, the second processing module 303, and the first input module 304 of the road surface defect detection device. Furthermore, it may include, but is not limited to, other module units of the road surface defect detection device, which will not be elaborated upon in this example.

[0143] The processors mentioned above can be general-purpose processors, including but not limited to: CPU (Central Processing Unit), NP (Network Processor), etc.; they can also be DSP (Digital Signal Processor), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0144] In addition, the aforementioned electronic device also includes a display for showing the results of the road surface defect detection method.

[0145] Optionally, specific examples in this embodiment can refer to the examples described in the above embodiments, and will not be repeated here.

[0146] Those skilled in the art will understand that Figure 4 The structure shown is for illustrative purposes only. The equipment used to implement the above-mentioned road surface defect detection method can be a terminal device, such as a smartphone (e.g., Android phone, iOS phone), tablet computer, handheld computer, mobile internet device (MID), PAD, etc. Figure 4 This does not limit the structure of the aforementioned electronic devices. For example, the terminal device may also include components that are more advanced than those described above. Figure 4 The more or fewer components shown (such as network interfaces, display devices, etc.), or having the same Figure 4 The different configurations shown.

[0147] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, ROM, RAM, disk or optical disk, etc.

[0148] According to another aspect of the embodiments of this application, a storage medium is also provided. Optionally, in this embodiment, the storage medium can be used to execute program code for a pavement distress detection method.

[0149] Optionally, in this embodiment, the storage medium may be located on at least one of the network devices in the network shown in the above embodiment.

[0150] Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps:

[0151] Obtain the road surface image information to be analyzed;

[0152] The road surface image information is input into the first target model for processing to obtain the disease mask information output by the first target model. The disease mask information carries the disease location information and disease type.

[0153] The disease location information is processed to obtain depth information;

[0154] By inputting depth information and disease type into the second target model, pavement disease detection information from pavement image information is obtained.

[0155] Optionally, specific examples in this embodiment can refer to the examples described in the above embodiments, and will not be repeated in this embodiment.

[0156] Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, ROMs, RAMs, portable hard drives, magnetic disks, or optical disks.

[0157] According to another aspect of the embodiments of this application, a computer program product or computer program is also provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium; a processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, causing the computer device to perform the road surface defect detection method steps in any of the above embodiments.

[0158] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0159] If the integrated units in the above embodiments are implemented as software functional units and sold or used as independent products, they can be stored in the aforementioned computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause one or more computer devices (which may be personal computers, servers, or network devices, etc.) to execute all or part of the steps of the road surface distress detection methods of the various embodiments of this application.

[0160] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0161] In the several embodiments provided in this application, it should be understood that the disclosed client can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or the indirect coupling or communication connection of units or modules may be electrical or other forms.

[0162] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the solution provided in this embodiment, depending on actual needs.

[0163] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0164] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for detecting pavement defects, characterized in that, The method includes: Obtain the road surface image information to be analyzed; The road surface image information is input into a first target model for processing to obtain the disease mask information output by the first target model, wherein the disease mask information carries disease location information and disease type; The disease location information is processed to obtain depth information; The process of processing the disease location information to obtain depth information includes: Obtain the original image and depth point cloud image corresponding to the disease location information; The first pixel position corresponding to the disease location information is determined based on the original image, and the second pixel position corresponding to the disease location information is determined based on the depth point cloud image. The first pixel position and the second pixel position are fused to obtain at least two key point information; Obtain the physical coordinates of each key point in the physical coordinate system; The depth information is generated using the physical coordinates; The depth information and the type of damage are input into the second target model to obtain the road damage detection information of the road image information.

2. The method according to claim 1, characterized in that, The step of inputting the road surface image information into the first target model for processing to obtain the defect mask information output by the first target model includes: The road surface image information is input into the first target model, and each pixel in the road surface image information is determined one by one by convolution to determine whether it is a disease area and to obtain the disease type of each pixel. Connect pixels of the same type of lesion to generate the lesion mask information.

3. The method according to claim 1, characterized in that, The process of inputting the depth information and the type of damage into the second target model to obtain the road surface damage detection information from the road surface image information includes: If the disease type input to the second target model is determined to be the target disease type, the target physical size output type is determined according to the target disease type; The physical coordinates corresponding to the depth information are processed using a preset calculation scheme to obtain the physical dimensions corresponding to the depth information; The target physical size output type and the physical size are used as output results to obtain the pavement distress detection information.

4. The method according to claim 3, characterized in that, The step of determining the target physical size output type based on the target disease type includes: Obtain the target mapping table in the second target model, wherein the target mapping table stores the mapping relationship between each of the disease types and the physical size output type; The target physical size output type is determined based on the target mapping table and the target disease type.

5. The method according to any one of claims 1 to 4, characterized in that, Before inputting the road surface image information into the first target model for processing, the method further includes: The road surface image information is acquired at a preset frequency; The road surface image information is preprocessed. The preprocessed road surface image information is input into the first target model for further processing.

6. A pavement defect detection device, characterized in that, The device includes: The first acquisition module is used to acquire road surface image information to be analyzed; The first processing module is used to input the road surface image information into the first target model for processing to obtain the disease mask information output by the first target model, wherein the disease mask information carries disease location information and disease type. The second processing module is used to process the disease location information to obtain depth information; wherein, processing the disease location information to obtain depth information includes: Obtain the original image and depth point cloud image corresponding to the disease location information; The first pixel position corresponding to the disease location information is determined based on the original image, and the second pixel position corresponding to the disease location information is determined based on the depth point cloud image. The first pixel position and the second pixel position are fused to obtain at least two key point information; Obtain the physical coordinates of each key point in the physical coordinate system; The depth information is generated using the physical coordinates; The first input module is used to input the depth information and the type of damage into the second target model to obtain the road damage detection information of the road image information.

7. An electronic device, comprising a processor, a communication interface, a memory, and a communication bus, wherein, The processor, the communication interface, and the memory communicate with each other via the communication bus, characterized in that... The memory is used to store computer programs; The processor is configured to perform the method steps of any one of claims 1 to 5 by running the computer program stored in the memory.

8. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of the method described in any one of claims 1 to 5.

Images