Road disease detection method, device and computer equipment

Abstract

The present application relates to a road disease detection method, device, and computer device. By inputting the received image to be measured into a trained and complete road disease detection model, an object detection result of the road disease is obtained. Among them, the road disease detection model is used to preliminarily detect each target object in the image to be measured to obtain a preliminary detection result. The target objects include road diseases, shadows, and shadow-related objects. Based on the preliminary detection result of the shadow and / or the shadow-related object, the preliminary detection result of the road disease is corrected to obtain an object detection result, which solves the influence of shadows on the road surface on the accuracy of road disease detection and improves the road disease detection effect.

Description

Technical Field

[0001] The present application relates to the technical field of intelligent transportation, and particularly to a method and device for detecting road diseases and a computer device. Background Art

[0002] Road diseases such as cracks and potholes seriously affect the service life and safety of roads. Conducting road maintenance in the initial stage of road diseases is more economical than large-scale repairs in the later stage. Conventional road disease detection is mostly completed by regular inspections by staff, but this method consumes a large amount of manpower. With the development of the field of intelligent transportation, especially the development of intelligent monitoring devices, detecting road diseases through intelligent algorithms has gradually become the main method.

[0003] Existing intelligent algorithms mostly detect road diseases through semantic segmentation. However, general semantic segmentation methods are easily affected by the shadows formed by lamp posts and trees on both sides of the road, resulting in false detections and poor road disease detection effects. Summary of the Invention

[0004] In this embodiment, a method and device for detecting road diseases and a computer device are provided to solve the problem that the road disease detection effect is poor due to the influence of road surface shadows in related technologies.

[0005] In a first aspect, in this embodiment, a method for detecting road diseases is provided. The method includes:

[0006] Input the received image to be detected into a trained road disease detection model to obtain the target detection result of road diseases;

[0007] Wherein, the road disease detection model is used to preliminarily detect each target object in the image to be detected to obtain a preliminary detection result; the target objects include road diseases, shadows, and shadow-related objects; and based on the preliminary detection result of the shadow and / or the shadow-related object, the preliminary detection result of the road disease is corrected to obtain the target detection result.

[0008] In some of these embodiments, correcting the preliminary detection result of the road disease based on the preliminary detection result of the shadow and / or the shadow-related object to obtain the target detection result includes:

[0009] Correcting the preliminary detection result of the road disease based on the preliminary detection result of the shadow to obtain the target detection result;

[0010] Or, correcting the preliminary detection result of the road disease based on the preliminary detection result of the shadow-related object to obtain the target detection result;

[0011] Alternatively, based on the preliminary detection results of the shadow and the shadow-related object, the preliminary detection results of the road disease are double-corrected to obtain the target detection result.

[0012] In some embodiments, the road disease detection model includes a multi-task instance segmentation module and a segmentation result extraction module;

[0013] The multi-task instance segmentation module is used to identify and segment each target object in the to-be-detected image, and obtain a first feature corresponding to the shadow-related object, a second feature corresponding to the shadow, and a third feature corresponding to the road disease;

[0014] The segmentation result extraction module is used to obtain a first mask based on the first feature and a second mask based on the second feature;

[0015] The preliminary detection results include the first feature, the second feature, the third feature, the first mask, and the second mask.

[0016] In some embodiments, the road disease detection model further includes: a shadow correction module and an associated object suppression module;

[0017] The shadow correction module is used to perform shadow correction on the third feature based on the second mask to obtain a first corrected result;

[0018] The associated object suppression module is used to perform misdetection correction on the first corrected result based on the first mask to obtain the target detection result.

[0019] In some embodiments, performing shadow correction on the third feature based on the second mask to obtain a first corrected result includes:

[0020] Calculating the non-shadow probability based on the second mask;

[0021] Obtaining a fourth feature based on the product of the non-shadow probability and the third feature;

[0022] Obtaining the first corrected result based on the fourth feature.

[0023] In some embodiments, performing misdetection correction on the first corrected result based on the first mask to obtain the target detection result includes:

[0024] Extracting a third mask of the road disease from the first corrected result; calculating the contour similarity between the first mask and the third mask;

[0025] Retaining the third mask whose contour similarity is less than a preset threshold;

[0026] Based on the road diseases corresponding to the reserved third mask, obtain the target detection result.

[0027] In some embodiments, the training process of the road disease detection model includes:

[0028] Obtain a training data set; the training data set includes road image data and annotation data for shadows, shadow-related objects, and road diseases in the road image data;

[0029] Input the road image data into an initial road disease detection model to obtain prediction results for the shadows, the shadow-related objects, and the road diseases;

[0030] Based on the prediction results and the annotation data, establish a total loss function;

[0031] Based on the total loss function, perform backpropagation to train the initial road disease detection model to obtain a trained complete road disease detection model.

[0032] In some embodiments, establishing a total loss function based on the prediction results and the annotation data includes:

[0033] Based on the prediction results of the shadows and the shadow-related objects and the annotation data of the shadows and the shadow-related objects, establish a first loss function;

[0034] Based on the prediction results of the road diseases and the shadow-related objects and the annotation data of the road diseases and the shadow-related objects, establish a second loss function;

[0035] Based on the first loss and the second loss, establish a total loss function.

[0036] In a second aspect, in the present embodiment, a road disease detection device is provided, and the device includes:

[0037] A model detection module, configured to input the received image to be measured into the trained complete one to obtain a target detection result of the road disease;

[0038] Wherein, the road disease detection model is used to perform preliminary detection on each target object in the image to be measured to obtain a preliminary detection result; the target objects include road diseases, shadows, and shadow-related objects; and based on the preliminary detection results of the shadows and / or the shadow-related objects, for the road diseases.

[0039] In a third aspect, the present application also provides a computer device. The computer device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the road disease detection method described in the first aspect is implemented.

[0040] In a fourth aspect, the present application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, and when the computer program is executed by a processor, the road disease detection method described in the first aspect is implemented.

[0041] Compared with the related art, in the road disease detection method, device and computer device provided in this embodiment, by inputting the received image to be measured into a well-trained road disease detection model, a target detection result of road diseases is obtained; wherein, the road disease detection model is used to perform preliminary detection on each target object in the image to be measured to obtain a preliminary detection result; the target objects include road diseases, shadows and shadow-related objects; and based on the preliminary detection result of the shadow and / or the shadow-related object, the preliminary detection result of the road disease is corrected to obtain a target detection result, which solves the influence of shadows on the road surface on the accuracy of road disease detection and improves the road disease detection effect.

[0042] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more concise and understandable. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The drawings described herein are used to provide a further understanding of the present application and form a part of the present application. The illustrative embodiments and descriptions thereof of the present application are used to explain the present application and do not constitute an improper limitation of the present application. In the drawings:

[0044] Figure 1 is a hardware structure block diagram of a terminal for the road disease detection method in an embodiment of the present application;

[0045] Figure 2 is a flowchart of the road disease detection method in an embodiment of the present application;

[0046] Figure 3 is a schematic diagram of the road disease detection model in an embodiment of the present application;

[0047] Figure 4 is a schematic diagram of training an initial road disease detection model in an embodiment of the present application;

[0048] Figure 5 is a schematic diagram of the structure of a shadow correction module in an embodiment of the present application;

[0049] Figure 6Schematic diagram of the road disease detection method in the preferred embodiment of the present application;

[0050] Figure 7 Structural block diagram of the road disease detection device in the embodiment of the present application.

[0051] Reference numerals: 102, processor; 104, memory; 106, transmission device; 108, input / output device; 71, model detection module; 72, model training module. Detailed implementation manners

[0052] To more clearly understand the purpose, technical solution and advantages of the present application, the present application will be described and illustrated below with reference to the accompanying drawings and embodiments.

[0053] Unless otherwise defined, the technical terms or scientific terms involved in the present application shall have the general meaning understood by those with ordinary skills in the technical field to which the present application belongs. In the present application, words such as "a", "an", "one", "the", "these", etc. do not indicate a limitation in quantity, and they can be singular or plural. The terms "including", "comprising", "having" and any variants thereof involved in the present application are intended to cover non-exclusive inclusion; for example, a process, method, system, product or device including a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent in these processes, methods, products or devices. The terms "connected", "coupled" and the like involved in the present application are not limited to physical or mechanical connections, but may include electrical connections, whether directly or indirectly. The "plurality" involved in the present application means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" may mean: A exists alone, A and B exist simultaneously, and B exists alone. Usually, the character " / " indicates that the associated objects before and after are in an "or" relationship. The terms "first", "second", "third", etc. involved in the present application are only used to distinguish similar objects and do not represent a specific sorting of the objects.

[0054] The method embodiment provided in this embodiment can be executed on a terminal, a computer or a similar computing device. For example, running on a terminal, Figure 1 is the hardware structural block diagram of the terminal of the road disease detection method in this embodiment. As Figure 1 shown, the terminal may include one or more ( Figure 1Only one (shown only) processor 102 and a memory 104 for storing data are shown. Among them, the processor 102 may include, but is not limited to, a processing device such as a microprocessor MCU or a field programmable gate array FPGA. The above terminal may further include a transmission device 106 for communication functions and an input / output device 108. Those of ordinary skill in the art can understand that Figure 1 The structure shown is only illustrative and does not limit the structure of the above terminal. For example, the terminal may further include more or fewer components than Figure 1 shown therein, or have a different configuration from Figure 1 that shown.

[0055] The memory 104 can be used to store computer programs. For example, software programs and modules of application software, such as the computer program corresponding to the road disease detection method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, implements the above method. The memory 104 may include a high-speed random access memory and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memories. In some instances, the memory 104 may further include a memory remotely disposed relative to the processor 102, and these remote memories can be connected to the terminal through a network. Examples of the above network include, but are not limited to, the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0056] The transmission device 106 is used to receive or send data via a network. The above network includes a wireless network provided by the communication provider of the terminal. In one instance, the transmission device 106 includes a network adapter (abbreviated as NIC), which can be connected to other network devices through a base station and thus can communicate with the Internet. In one instance, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0057] In this embodiment, a road disease detection method is provided. Figure 2 is a flowchart of the road disease detection method in this embodiment, as Figure 2 shown, and the process includes the following steps:

[0058] Input the received image to be measured into a trained road disease detection model to obtain the target detection result of the road disease.

[0059] Among them, the road disease detection model is used to preliminarily detect each target object in the image to be detected, and obtain a preliminary detection result; the target objects include road diseases, shadows, and shadow-related objects; and the preliminary detection result of the road disease is corrected based on the preliminary detection result of the shadow and / or the shadow-related object to obtain a target detection result.

[0060] Specifically, the image to be detected is usually an image containing a road in a monitoring scenario. Road diseases include, but are not limited to, cracks, potholes, etc. When there is a shadow on the road, objects that form the shadow (such as lamp posts, trees, etc.) can usually be found around, that is, shadow-related objects. Therefore, in this embodiment, the detection effect of road diseases is improved by considering the shape of objects that are likely to form shadows around. For a shadow without a related object, the probability of misidentifying it as a road disease through the shadow in the preliminary detection result is directly reduced; for a shadow with a related object, the probability of being misidentified as a road disease can also be reduced through the shadow-related object.

[0061] Among them, the target detection result includes information such as the type, bounding box, mask graph, confidence level, etc. of the road disease. One or more of the above information can be configured as the target detection result for output according to needs. The output target detection result can continue to be used for display, warning, intelligent analysis, etc.

[0062] In this embodiment, by inputting the received image to be detected into a trained road disease detection model, a target detection result of the road disease is obtained; among them, the road disease detection model is used to preliminarily detect each target object in the image to be detected, and obtain a preliminary detection result; the target objects include road diseases, shadows, and shadow-related objects; and the preliminary detection result of the road disease is corrected based on the preliminary detection result of the shadow and / or the shadow-related object to obtain a target detection result, which solves the influence of shadows on the road surface on the accuracy of road disease detection and improves the road disease detection effect.

[0063] In some of these embodiments, correcting the preliminary detection result of the road disease based on the preliminary detection result of the shadow and / or the shadow-related object to obtain a target detection result includes:

[0064] Step S310, correcting the preliminary detection result of the road disease based on the preliminary detection result of the shadow to obtain a target detection result.

[0065] Specifically, if no shadow-related object is recognized in the image to be measured, step S310 can be triggered, and the road diseases are corrected only through the shadow. In one implementation, the process of correcting the preliminary detection result of road diseases based on the preliminary detection result of the shadow includes: extracting a shadow mask image from the preliminary detection result of the shadow, and extracting a road disease feature image from the preliminary detection result of the road diseases; calculating the non-shadow probability based on the shadow mask image; multiplying the non-shadow probability by the road disease feature image to obtain a road disease feature image corrected by the shadow; and generating a target detection result based on the road disease feature image.

[0066] Step S320: Correct the preliminary detection result of the road diseases based on the preliminary detection result of the shadow-related object to obtain a target detection result.

[0067] Specifically, if a shadow-related object is recognized in the image to be measured, step S320 can be triggered, and the road diseases are corrected only through the shadow-related object. In one implementation, the process of correcting the preliminary detection result of the road diseases based on the preliminary detection result of the shadow-related object includes: extracting a shadow-related object mask image from the preliminary detection result of the shadow-related object, and extracting a road disease feature image and the corresponding road disease mask image from the preliminary detection result of the road diseases; when the similarity between the shadow-related object mask image and the road disease mask image is higher than a threshold, screening out the corresponding road disease feature image; the remaining road disease feature images after the screening are the road disease feature images corrected by the shadow-related object; and generating a target detection result based on the road disease feature images.

[0068] Step S330: Double-correct the preliminary detection result of the road diseases based on the preliminary detection results of the shadow and the shadow-related object to obtain a target detection result.

[0069] Specifically, if both a shadow and a shadow-related object exist in the image to be measured, step S330 can be triggered. First, correct the road diseases once based on the shadow, and then perform a secondary correction on the road diseases corrected once based on the shadow-related object, or first correct the road diseases once based on the shadow-related object, and then perform a secondary correction on the road diseases corrected once based on the shadow, so as to achieve double correction.

[0070] In other implementations, if both a shadow and a shadow-related object exist in the image to be measured, according to the needs of factors such as computing performance, step S320 or step S310 can also be set to be triggered separately.

[0071] In this embodiment, through the combination of various correction methods for shadows and shadow-related objects, the needs of different application scenarios are fully met.

[0072] In some of these embodiments, refer to Figure 3, the road disease detection model includes a multi-task instance segmentation module and a segmentation result extraction module.

[0073] The multi-task instance segmentation module is used to identify and segment each target object in the image to be measured, and obtain the first feature corresponding to the shadow associated object, the second feature corresponding to the shadow, and the third feature corresponding to the road disease.

[0074] The segmentation result extraction module is used to obtain the first mask based on the first feature and the second mask based on the second feature.

[0075] The preliminary detection result includes the first feature, the second feature, the third feature, the first mask, and the second mask.

[0076] Specifically, the difference between the multi-task instance segmentation module and the conventional instance segmentation model is that there are three segmentation heads, which respectively correspond to the segmentation of three categories. Conventional instance segmentation models include but are not limited to YOLACT, Mask-R-CNN, etc.

[0077] In some of these embodiments, refer to Figure 3 , the road disease detection model further includes: a shadow correction module and an associated object suppression module.

[0078] The shadow correction module is used to perform shadow correction on the third feature based on the second mask to obtain a first corrected result.

[0079] The associated object suppression module is used to perform misdetection correction on the first corrected result based on the first mask to obtain a target detection result.

[0080] Specifically, in the second mask image, the pixel values of the shadow area are set to 0, and the pixel values of the remaining areas are set to 1. The third feature is multiplied element by element with the second mask. The feature values corresponding to the shadow area in the third feature will be set to 0, while the feature values of the remaining areas remain unchanged.

[0081] In this embodiment, first, the feature expression of the shadow area is suppressed, further improving the effect of road disease semantic segmentation, providing a more accurate input for the subsequent associated object suppression module, and thus reducing the probability of being misidentified as a road disease.

[0082] In some of these embodiments, performing shadow correction on the third feature based on the second mask to obtain a first corrected result includes:

[0083] Step S410, calculating the non-shadow probability based on the second mask.

[0084] Step S420, obtaining a fourth feature based on the product of the non-shadow probability and the third feature.

[0085] Step S430, obtain the first correction result based on the fourth feature.

[0086] Specifically, use 1 - sigmoid(x) to obtain the non - shadow probability, where x is the second mask.

[0087] In this embodiment, the shadow correction module performs shadow suppression at the feature level, without performing image - level shadow correction and removal, and will not affect the image to be measured itself.

[0088] In some of these embodiments, perform mis - detection correction on the first correction result based on the first mask to obtain the target detection result, including:

[0089] Step S510, extract the third mask of the road disease from the first correction result; calculate the contour similarity between the first mask and the third mask.

[0090] Step S520, retain the third mask whose contour similarity is less than the preset threshold.

[0091] Step S530, obtain the target detection result based on the road disease corresponding to the retained third mask.

[0092] Specifically, the calculation method of the contour similarity can adopt the Hausdorff distance, and its formula is as follows:

[0093] ;

[0094] where X and Y are the contours of the first mask and the third mask respectively. The meaning of the formula is to find the maximum value of the distances from each point x to the nearest points in the set. The smaller this distance is, the more similar and coincident the two point sets are.

[0095] In this embodiment, define objects such as street lights and trees on both sides of the road as shadow - related objects. By calculating the contour similarity between the road disease and the related objects, the effect of shadow detection is further improved, so as to remove the mis - detected road diseases.

[0096] In some of these embodiments, the training process of the road disease detection model includes:

[0097] Step S610, obtain the training data set; the training data set includes road image data and annotation data for shadows, shadow - related objects, and road diseases in the road image data.

[0098] Specifically, the training dataset is a road segmentation dataset, which includes two parts: image data and annotation data. The annotation data is made in the way of semantic segmentation. The categories to be annotated include: 1) associated objects such as lamp posts and trees, whose bounding box labels (bbox) and mask labels are denoted as LB1 and LM1 respectively; 2) shadows formed by associated objects such as lamp posts and trees, whose bounding box labels and mask labels are denoted as LB2 and LM2 respectively; 3) road diseases such as cracks and potholes, whose bounding box labels and mask labels are denoted as LB3 and LM3. After the road segmentation dataset is made, the dataset is divided into a training set, a validation set, and a test set according to a certain ratio.

[0099] Step S620: Input the road image data into the initial road disease detection model to obtain the prediction results of shadows, shadow-associated objects, and road diseases.

[0100] Specifically, refer to Figure 4 , design a road disease semantic segmentation network as the initial road disease detection model. The road disease semantic segmentation network includes an initial multi-task instance segmentation module, an initial shadow correction module, and an initial associated object suppression module.

[0101] Step S630: Establish a total loss function based on the prediction results and the annotation data.

[0102] Step S640: Perform backpropagation based on the total loss function to train the initial road disease detection model to obtain a well-trained road disease detection model.

[0103] In some of the embodiments, establishing a total loss function based on the prediction results and the annotation data includes:

[0104] Step S631: Establish a first loss function based on the prediction results of shadows and shadow-associated objects and the annotation data of shadows and shadow-associated objects.

[0105] Step S632: Establish a second loss function based on the prediction results of road diseases and shadow-associated objects and the annotation data of road diseases and shadow-associated objects.

[0106] Step S633: Establish a total loss function based on the first loss and the second loss.

[0107] Specifically, refer to Figure 4, the input of the initial multi-task instance segmentation module is the road image in the monitoring scenario. During forward propagation, first, the multi-task instance segmentation module is passed through to obtain the associated object segmentation feature F1, the shadow segmentation feature F2, and the road disease segmentation feature F3. Based on F1 and F2, the instance segmentation results R1 and R2 are further obtained. Among them, R1 contains the bounding box B1 and the mask M1, and R2 contains the bounding box B2 and the mask M2. During backpropagation, the mask calculates the binary cross-entropy loss BCELoss, and the bounding box calculates the smooth L1 loss Smooth_L1Loss to obtain loss1. The calculation formula is as follows:

[0108] .

[0109] The input of the shadow correction module includes two parts, namely the shadow segmentation result M2 and the road disease segmentation feature F3. See Figure 5 , first, M2 is passed through 1 - sigmoid(M2) to obtain the non-shadow probability, and then F3 is multiplied by the non-shadow probability to obtain the road disease segmentation feature F4. Based on F4, the instance segmentation result R3 is further obtained.

[0110] The input of the associated object suppression module includes two parts, namely the associated object segmentation results B1, M1 and the road disease segmentation result R3 (the bounding box and mask are denoted as B3 and M3 respectively). The method of suppressing associated objects is to calculate the contour similarity between the associated object and the road disease. When the similarity is greater than the given threshold, the detection result is removed. Through the associated object suppression module, the road disease segmentation result R4 (the bounding box and mask are denoted as B4 and M4 respectively) is obtained. R4 is a subset of R3. During backpropagation, loss2 is calculated:

[0111] .

[0112] Finally, the calculation formula of the overall loss is as follows: loss = α × loss1 + β × loss2, where α + β = 1.

[0113] Using the above loss function for backpropagation, after each training epoch is completed, the accuracy on the validation set is evaluated, and the model with the highest accuracy on the validation set is saved as the optimal model for subsequent accuracy evaluation.

[0114] The following describes and illustrates this embodiment through preferred embodiments.

[0115] Figure 6 is the flowchart of the road disease detection method of this preferred embodiment.

[0116] This preferred embodiment proposes a road disease detection method based on associated object suppression. This method includes the production of a road disease segmentation dataset, model training, model inference, and model application.

[0117] S1. Production of Road Disease Segmentation Dataset: The road disease segmentation dataset consists of two parts: image data and annotation data. The annotation data is produced in the way of semantic segmentation. The categories that need to be annotated include: 1) Associated objects such as lamp posts and trees, with the bounding box label (bbox) and mask label denoted as LB1 and LM1; 2) Shadows formed by associated objects such as lamp posts and trees, with the bounding box label and mask label denoted as LB2 and LM2; 3) Road diseases such as cracks and potholes, with the bounding box label and mask label denoted as LB3 and LM3. After the road segmentation dataset is produced, it is divided into a training set, a validation set, and a test set according to a certain ratio.

[0118] S2. Model Training: During training, a road disease semantic segmentation network is designed, which includes an initial multi-task instance segmentation module, an initial shadow correction module, and an initial associated object suppression module. The structure of the road disease semantic segmentation network is as Figure 4 shown.

[0119] 1) Multi-task Instance Segmentation: Road pictures in the monitoring scenario are used as input. During forward propagation, first, the multi-task instance segmentation module is passed through to obtain the associated object segmentation feature F1, the shadow segmentation feature F2, and the road disease segmentation feature F3. Based on F1 and F2, the instance segmentation results R1 and R2 are further obtained. Among them, R1 contains the bounding box B1 and the mask M1, and R2 contains the bounding box B2 and the mask M2. During backpropagation, the mask calculates the binary cross-entropy loss, and the bounding box calculates the smooth absolute value loss to obtain loss1. The calculation formula is as follows:

[0120] .

[0121] 2) Shadow Correction: The shadow segmentation result M2 and the road disease segmentation feature F3 are used as input. First, M2 is passed through 1 - sigmoid(M2) to obtain the non-shadow probability, and then F3 is multiplied by the non-shadow probability to obtain the road disease segmentation feature F4. Based on F4, the instance segmentation result R3 is further obtained. The initial shadow correction module is as Figure 5 shown.

[0122] 3) Associated Object Suppression: The input of the associated object suppression module consists of two parts, namely the associated object segmentation results B1 and M1 and the road disease segmentation result R3 (the bounding box and mask are denoted as B3 and M3 respectively). The method of suppressing associated objects is to calculate the contour similarity between the associated object and the road disease. When the similarity is greater than a given threshold, the detection result is removed. The calculation method of the contour similarity uses the Hausdorff distance, and its formula is as follows:

[0123] ;

[0124] Among them, X and Y are the contours of the first mask M1 and the third mask M3 respectively. The meaning of the formula is to find the maximum value of the distance from each point x to the nearest point on the set. The smaller this distance is, the more similar and coincident the two point sets are.

[0125] Through the associated object suppression module, the road disease segmentation result R4 (the bounding box and mask are denoted as B4 and M4 respectively) is obtained. R4 is a subset of R3. During backpropagation, the loss2 is calculated as follows:

[0126] 。

[0127] The calculation formula of the overall loss is as follows: loss = α × loss1 + β × loss2, where α + β = 1.

[0128] Using the above loss function for backpropagation, after each training epoch is completed, the accuracy on the validation set is evaluated, and the model with the highest accuracy on the validation set is saved as the optimal model for subsequent accuracy evaluation.

[0129] S3. Model inference: Use the above saved model with the highest accuracy on the validation set to predict the labels on the validation set. Finally, compare the labels with the model output, and use metrics such as accuracy to evaluate the model accuracy. Save the model whose accuracy evaluation meets the requirements as the trained road disease detection model.

[0130] S4. Model application: Input the received image to be tested into the trained road disease detection model to obtain the object detection result of the road disease.

[0131] In this preferred embodiment, the effect of road disease semantic segmentation is improved through the multi-task instance segmentation module, the associated object suppression module, and the shadow suppression module. Among them, for the shadow without associated objects, the probability of being mis-identified as a road disease is reduced by the shadow suppression module; for the shadow with associated objects, in addition to the shadow suppression module, the probability of being mis-identified as a road disease is also reduced by the associated object suppression module. In this preferred embodiment, image-level shadow correction and removal are not performed, which will not affect the image itself. In this preferred embodiment, objects such as street lights and trees on both sides of the road are defined as associated objects, and the contour similarity and distance between the shadow and the associated objects are calculated as the loss function to further improve the shadow detection effect.

[0132] It should be noted that the steps shown in the above process or the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order from here.

[0133] In this embodiment, a road disease detection device is also provided. This device is used to implement the above-mentioned embodiments and preferred implementation manners, and those that have been described will not be repeated here. The following terms such as "module", "unit", "sub-unit", etc. can be a combination of software and / or hardware that can achieve a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware is also possible and contemplated.

[0134] Figure 7 is the structural block diagram of the road disease detection device in this embodiment. As Figure 7 shown, the device includes: a model detection module 71.

[0135] The model detection module 71 is configured to input the received image to be measured into a well-trained one to obtain the target detection result of road diseases; wherein, the road disease detection model is used to perform a preliminary detection on each target object in the image to be measured to obtain a preliminary detection result; the target objects include road diseases, shadows, and shadow-related objects; and based on the preliminary detection result of the shadow and / or the shadow-related object, the preliminary detection result of the road disease is corrected to obtain the target detection result.

[0136] In some of these embodiments, the model detection module 71 is further configured to correct the preliminary detection result of the road disease based on the preliminary detection result of the shadow to obtain the target detection result; or, correct the preliminary detection result of the road disease based on the preliminary detection result of the shadow-related object to obtain the target detection result; or, perform a double correction on the preliminary detection result of the road disease based on the preliminary detection results of the shadow and the shadow-related object to obtain the target detection result.

[0137] In some of these embodiments, the road disease detection model includes a multi-task instance segmentation module and a segmentation result extraction module; the multi-task instance segmentation module is configured to identify and segment each target object in the image to be measured to obtain the first feature corresponding to the shadow-related object, the second feature corresponding to the shadow, and the third feature corresponding to the road disease; the segmentation result extraction module is configured to obtain a first mask based on the first feature and a second mask based on the second feature; the preliminary detection result includes the first feature, the second feature, the third feature, the first mask, and the second mask.

[0138] In some of these embodiments, the road disease detection model further includes: a shadow correction module and an associated object suppression module; the shadow correction module is configured to perform shadow correction on the third feature based on the second mask to obtain a first corrected result; the associated object suppression module is configured to perform false detection correction on the first corrected result based on the first mask to obtain a target detection result.

[0139] In some of these embodiments, performing shadow correction on the third feature based on the second mask to obtain a first corrected result includes: calculating a non-shadow probability based on the second mask; obtaining a fourth feature based on the product of the non-shadow probability and the third feature; and obtaining the first corrected result based on the fourth feature.

[0140] In some of these embodiments, performing false detection correction on the first corrected result based on the first mask to obtain a target detection result includes: extracting a third mask of the road disease from the first corrected result; calculating the contour similarity between the first mask and the third mask; retaining the third masks with a contour similarity less than a preset threshold; and obtaining the target detection result based on the road diseases corresponding to the retained third masks.

[0141] In some of these embodiments, the device further includes a model training module 72. The model training module 72 is configured to obtain a training data set; the training data set includes road image data and annotation data for shadows, shadow associated objects, and road diseases in the road image data; input the road image data into an initial road disease detection model to obtain prediction results for the shadows, the shadow associated objects, and the road diseases; establish a total loss function based on the prediction results and the annotation data; and perform backpropagation based on the total loss function to train the initial road disease detection model to obtain a trained complete road disease detection model.

[0142] In some of these embodiments, the model training module 72 is further configured to establish a first loss function based on the prediction results of the shadows and the shadow associated objects and the annotation data of the shadows and the shadow associated objects; establish a second loss function based on the prediction results of the road diseases and the shadow associated objects and the annotation data of the road diseases and the shadow associated objects; and establish a total loss function based on the first loss and the second loss.

[0143] It should be noted that the above-mentioned various modules can be functional modules or program modules, and can be implemented either by software or by hardware. For the modules implemented by hardware, the above-mentioned various modules can be located in the same processor; or the above-mentioned various modules can also be located in different processors in any combined form.

[0144] In this embodiment, a computer device is further provided, including a memory and a processor. A computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0145] Optionally, the above computer device may further include a transmission device and input / output devices. Among them, the transmission device is connected to the above processor, and the input / output devices are connected to the above processor.

[0146] It should be noted that the specific examples in this embodiment may refer to the examples described in the above embodiments and optional implementation manners, and will not be elaborated herein.

[0147] In addition, in combination with the road disease detection method provided in the above embodiments, a storage medium may also be provided in this embodiment to implement it. A computer program is stored on the storage medium; when the computer program is executed by a processor, any one of the road disease detection methods in the above embodiments is implemented.

[0148] It should be understood that the specific embodiments described here are only used to explain this application, rather than to limit it. According to the embodiments provided in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the protection scope of this application.

[0149] Obviously, the drawings are only some examples or embodiments of this application. For those of ordinary skill in the art, this application can also be applied to other similar situations based on these drawings without creative efforts. In addition, it can be understood that although the work done during the development process here may be complex and time-consuming, for those of ordinary skill in the art, certain design, manufacturing, or production changes based on the technical content disclosed in this application are only conventional technical means and should not be regarded as insufficient disclosure of this application.

[0150] The term "embodiment" in this application means that the specific features, structures, or characteristics described in combination with the embodiment may be included in at least one embodiment of this application. The phrase appears in various positions in the specification does not necessarily mean the same embodiment, nor does it mean independence or alternative to other embodiments. Those of ordinary skill in the art can clearly or implicitly understand that the embodiments described in this application can be combined with other embodiments without conflict.

[0151] The above-described embodiments merely represent several implementation manners of the present application. The description thereof is relatively specific and detailed, but it should not be construed as a limitation on the scope of patent protection. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all fall within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.

Claims

1. A road disease detection method, characterized in that: The method comprises: Input the received image to be tested into a well-trained road damage detection model to obtain a target detection result of road damage; The road damage detection model is used to perform preliminary detection on each target object in the image to be detected to obtain preliminary detection results; the target objects include road damage, shadows and shadow-associated objects; and based on the preliminary detection results of the shadows and the shadow-associated objects, the preliminary detection results of the road damage are corrected to obtain target detection results; Wherein, performing preliminary detection on each target object in the image to be tested to obtain preliminary detection results includes: identifying and segmenting each target object in the image to be tested to obtain a first feature corresponding to the shadow associated object, a second feature corresponding to the shadow, and a third feature corresponding to the road disease; obtaining a first mask based on the first feature, and obtaining a second mask based on the second feature; the preliminary detection results include the first feature, the second feature, the third feature, the first mask, and the second mask; Among them, based on the preliminary detection results of the shadow and the shadow associated objects, the preliminary detection results of the road damage are corrected to obtain the target detection results, including: performing shadow correction on the third feature based on the second mask to obtain a first correction result; extracting the third mask of the road damage from the first correction result; calculating the contour similarity between the first mask and the third mask; retaining the third mask whose contour similarity is less than a preset threshold; and obtaining the target detection result based on the road damage corresponding to the retained third mask.

2. The road damage detection method according to claim 1, characterized in that: Also includes: Correcting the preliminary detection result of the road damage based on the preliminary detection result of the shadow to obtain a target detection result; Alternatively, the preliminary detection result of the road damage is corrected based on the preliminary detection result of the shadow associated object to obtain a target detection result.

3. The road damage detection method according to claim 1, characterized in that: Performing shadow correction on the third feature based on the second mask to obtain a first correction result includes: Calculating a non-shadow probability based on the second mask; Obtaining a fourth feature based on a product of the non-shadow probability and the third feature; A first correction result is obtained based on the fourth feature.

4. The road damage detection method according to claim 1, characterized in that: The training process of the road damage detection model includes: Acquire a training data set; the training data set includes road image data and annotation data for shadows, shadow-related objects, and road damage in the road image data; Inputting the road image data into an initial road damage detection model to obtain prediction results of the shadow, the shadow associated object and the road damage; Establishing a total loss function based on the prediction result and the labeled data; Back propagation is performed based on the total loss function to train the initial road damage detection model to obtain a fully trained road damage detection model.

5. The road damage detection method according to claim 4, characterized in that: Establishing a total loss function based on the prediction result and the labeled data includes: Establishing a first loss function based on the prediction results of the shadow and the shadow associated object and the labeled data of the shadow and the shadow associated object; Establishing a second loss function based on the prediction results of the road damage and the shadow associated object and the labeled data of the road damage and the shadow associated object; A total loss function is established based on the first loss and the second loss.

6. A road disease detection device, characterized in that: The device comprises: The model detection module is used to input the received image to be tested into the fully trained model to obtain the target detection result of road diseases; The road damage detection model is used to perform preliminary detection on each target object in the image to be detected to obtain preliminary detection results; the target objects include road damage, shadows and shadow-associated objects; and based on the preliminary detection results of the shadows and the shadow-associated objects, the preliminary detection results of the road damage are corrected to obtain target detection results; Wherein, performing preliminary detection on each target object in the image to be tested to obtain preliminary detection results includes: identifying and segmenting each target object in the image to be tested to obtain a first feature corresponding to the shadow associated object, a second feature corresponding to the shadow, and a third feature corresponding to the road disease; obtaining a first mask based on the first feature, and obtaining a second mask based on the second feature; the preliminary detection results include the first feature, the second feature, the third feature, the first mask, and the second mask; Among them, based on the preliminary detection results of the shadow and the shadow associated objects, the preliminary detection results of the road damage are corrected to obtain the target detection results, including: performing shadow correction on the third feature based on the second mask to obtain a first correction result; extracting the third mask of the road damage from the first correction result; calculating the contour similarity between the first mask and the third mask; retaining the third mask whose contour similarity is less than a preset threshold; and obtaining the target detection result based on the road damage corresponding to the retained third mask.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

Images