Road disease detection method and device, readable storage medium and terminal equipment
By pre-training the backbone network and neck network and combining them with the edge detection training sample set, a road defect detection model is constructed, which solves the problem of low accuracy in existing road defect detection technologies and achieves higher detection accuracy and adaptability.
Patent Information
- Application Number
- CN202411095291.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-09
- Publication Date
- 2026-02-10
AI Technical Summary
Existing methods for detecting road defects have low accuracy and are difficult to accurately detect road defects in complex environments.
The artificial intelligence model trained is used for road defect detection. The model is constructed by unsupervised pre-training of the backbone network and neck network, combined with edge detection training sample set for contour segmentation.
It improves the accuracy and generalization ability of road defect detection, enabling it to better adapt to actual business scenarios.
Smart Images
Figure CN121504791A_ABST
Abstract
Description
Technical Field
[0001] This application belongs to the field of road inspection technology, and in particular relates to a method, apparatus, computer-readable storage medium and terminal equipment for road defect detection. Background Technology
[0002] With the continuous development and advancement of technology, automated inspection has been widely applied and promoted in various fields. In the road traffic sector, the detection and repair of road defects has always been a crucial issue, directly impacting driving and pedestrian safety. Traditional manual inspections suffer from inefficiency and are prone to missed detections and underreporting; therefore, intelligent inspection has become a current industry trend.
[0003] However, since road surfaces are mainly composed of bricks, lanes, etc., the forms of roads are diverse, and the road background may be quite complex, making it difficult for existing road defect detection methods to accurately detect road defects. Summary of the Invention
[0004] In view of this, embodiments of this application provide a method, apparatus, computer-readable storage medium, and terminal device for detecting road defects, in order to solve the problem of low accuracy in existing methods for detecting road defects.
[0005] A first aspect of this application provides a method for detecting road defects, which may include:
[0006] Acquire road images;
[0007] Based on a preset road defect detection model, road defects are detected in the road image to obtain the road defect detection results of the road image.
[0008] The road defect detection model is an artificial intelligence model trained for road defect detection, and the neck network of the road defect detection model is obtained by pre-training contour segmentation using an edge detection training sample set constructed based on edge detection.
[0009] In one specific implementation of the first aspect, the training process of the road defect detection model includes:
[0010] The initial backbone network is pre-trained to obtain the pre-trained backbone network.
[0011] Based on the pre-trained backbone network, the initial neck network is pre-trained to obtain the pre-trained neck network.
[0012] Based on the pre-trained backbone network and the pre-trained neck network, the initial artificial intelligence model is trained to obtain the road defect detection model.
[0013] In one specific implementation of the first aspect, the step of pre-training the initial backbone network to obtain the pre-trained backbone network includes:
[0014] Construct a reconstruction training sample set; wherein the reconstruction training sample set includes a first number of first mask-processed images and corresponding second mask-processed images;
[0015] Using each of the first mask-processed images as input and the corresponding second mask-processed images as the expected output, the initial first pre-trained model is trained to obtain the trained first pre-trained model; wherein, the initial first pre-trained model includes an initial backbone network and a decoding network;
[0016] Based on the first pre-trained model after training, the pre-trained backbone network is determined.
[0017] In one specific implementation of the first aspect, the step of pre-training the initial neck network based on the pre-trained backbone network to obtain the pre-trained neck network includes:
[0018] Construct the edge detection training sample set; wherein, the edge detection training sample set includes a second number of second sample images and corresponding contour images;
[0019] Using each of the second sample images as input and the corresponding contour image as the expected output, the initial second pre-trained model is trained to obtain the trained second pre-trained model; wherein, the initial second pre-trained model includes the pre-trained backbone network, the initial neck network, and the initial segmentation prediction head;
[0020] Based on the trained second pre-trained model, the pre-trained neck network is determined.
[0021] In one specific implementation of the first aspect, constructing the edge detection training sample set includes:
[0022] Obtain a second number of each of the second sample images;
[0023] Using a preset edge detection algorithm, edge detection is performed on each of the second sample images to obtain the corresponding contour image;
[0024] The edge detection training sample set is constructed based on each of the second sample images and the corresponding contour images.
[0025] In one specific implementation of the first aspect, the step of training an initial second pre-trained model using each of the second sample images as input and the corresponding contour image as the expected output to obtain a trained second pre-trained model includes:
[0026] Using the initial second pre-trained model, contour segmentation is performed on each of the second sample images to obtain the corresponding actual segmented images;
[0027] Based on the contour image, the actual segmented image, and the preset loss function, the training loss value of the second pre-trained model is calculated;
[0028] Based on the training loss value, the model parameters of the initial second pre-trained model are adjusted to obtain the trained second pre-trained model.
[0029] In one specific implementation of the first aspect, the step of training an initial artificial intelligence model based on the pre-trained backbone network and the pre-trained neck network to obtain the road defect detection model includes:
[0030] Construct a road defect training sample set; wherein, the road defect training sample set includes a third number of third sample images and corresponding expected detection labels;
[0031] Using each of the third sample images as input and the corresponding expected detection label as expected output, the initial artificial intelligence model is trained to obtain the road defect detection model;
[0032] The initial artificial intelligence model includes the pre-trained backbone network, the pre-trained neck network, and the initial detection head.
[0033] A second aspect of this application provides a road defect detection device, which may include:
[0034] The road image acquisition module is used to acquire road images;
[0035] The road defect detection module is used to perform road defect detection on the road image based on a preset road defect detection model, and obtain the road defect detection result of the road image; wherein, the road defect detection model is an artificial intelligence model trained for road defect detection, and the neck network of the road defect detection model is obtained by pre-training contour segmentation through an edge detection training sample set constructed based on edge detection.
[0036] In one specific implementation of the second aspect, the road defect detection device further includes:
[0037] The first training module is used to pre-train the initial backbone network to obtain the pre-trained backbone network.
[0038] The second training module is used to pre-train the initial neck network based on the pre-trained backbone network to obtain the pre-trained neck network.
[0039] The third training module is used to train the initial artificial intelligence model based on the pre-trained backbone network and the pre-trained neck network to obtain the road defect detection model.
[0040] In one specific implementation of the second aspect, the first training module includes:
[0041] The first sub-module for constructing the sample set is used to construct the reconstruction training sample set; wherein, the reconstruction training sample set includes a first number of first mask-processed images and corresponding second mask-processed images;
[0042] The model training first submodule is used to train the initial first pre-trained model with each of the first mask processing images as input and the corresponding second mask processing images as expected output, to obtain the trained first pre-trained model; wherein, the initial first pre-trained model includes an initial backbone network and a decoding network;
[0043] The backbone network determination submodule is used to determine the pre-trained backbone network based on the first pre-trained model after training.
[0044] In one specific implementation of the second aspect, the second training module includes:
[0045] The sample set construction second sub-module is used to construct the edge detection training sample set; wherein, the edge detection training sample set includes a second number of second sample images and corresponding contour images;
[0046] The second sub-module for model training is used to train the initial second pre-trained model with each of the second sample images as input and the corresponding contour image as the expected output, to obtain the trained second pre-trained model; wherein, the initial second pre-trained model includes the pre-trained backbone network, the initial neck network, and the initial segmentation prediction head;
[0047] The neck network determination submodule is used to determine the pre-trained neck network based on the trained second pre-trained model.
[0048] In one specific implementation of the second aspect, the sample set construction second sub-module includes:
[0049] The second sample image acquisition unit is used to acquire a second number of individual second sample images;
[0050] An edge detection unit is used to perform edge detection on each of the second sample images using a preset edge detection algorithm to obtain the corresponding contour image;
[0051] The sample set construction unit is used to construct the edge detection training sample set based on each of the second sample images and the corresponding contour images.
[0052] In one specific implementation of the second aspect, the model training second sub-module includes:
[0053] The actual output unit is used to perform contour segmentation on each of the second sample images using the initial second pre-trained model to obtain the corresponding actual segmentation image;
[0054] The loss calculation unit is used to calculate the training loss value of the second pre-trained model based on the contour image, the actual segmented image, and a preset loss function.
[0055] The parameter adjustment unit is used to adjust the model parameters of the initial second pre-trained model based on the training loss value, so as to obtain the trained second pre-trained model.
[0056] In one specific implementation of the second aspect, the third training module includes:
[0057] The third sub-module for constructing the sample set is used to construct a road defect training sample set; wherein, the road defect training sample set includes a third number of third sample images and corresponding expected detection labels;
[0058] The third sub-module for model training is used to train the initial artificial intelligence model with each of the third sample images as input and the corresponding expected detection label as expected output to obtain the road defect detection model; wherein, the initial artificial intelligence model includes the pre-trained backbone network, the pre-trained neck network, and the initial detection head.
[0059] A third aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of any of the road defect detection methods described above.
[0060] A fourth aspect of this application provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of any of the road defect detection methods described above.
[0061] The fifth aspect of this application provides a computer program product that, when run on a terminal device, causes the terminal device to execute the steps of any of the road defect detection methods described above.
[0062] The beneficial effects of this application embodiment compared with the prior art are as follows: This application embodiment acquires road images; based on a preset road defect detection model, road defects are detected in the road images to obtain road defect detection results; wherein, the road defect detection model is an artificial intelligence model trained for road defect detection, and the neck network of the road defect detection model is pre-trained by contour segmentation using an edge detection training sample set constructed based on edge detection. Since the road contour segmentation task and the road defect detection task have certain similarities, this application embodiment can construct an edge detection training sample set through edge detection, and can use the edge detection training sample set to pre-train the neck network for contour segmentation, so that the pre-trained neck network can perform feature fusion more specifically, thereby providing more powerful assistance in the road defect detection task and helping to improve the accuracy and generalization ability of the road defect detection model. Attached Figure Description
[0063] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0064] Figure 1 This is a schematic flowchart illustrating the training process of a road defect detection model in an embodiment of this application;
[0065] Figure 2 This is a schematic diagram of the mask image corresponding to the first sample image;
[0066] Figure 3 This is a schematic diagram illustrating the pre-training of the first pre-trained model;
[0067] Figure 4 This is a schematic diagram of the second sample image and its corresponding contour image;
[0068] Figure 5 This is a schematic diagram illustrating the pre-training of the second pre-trained model;
[0069] Figure 6 This is a schematic diagram illustrating the pre-training of an initial artificial intelligence model.
[0070] Figure 7 This is a flowchart of one embodiment of a road defect detection method according to the present application;
[0071] Figure 8 This is a structural diagram of one embodiment of a road defect detection device according to the present application.
[0072] Figure 9 This is a schematic block diagram of a terminal device in an embodiment of this application. Detailed Implementation
[0073] To make the inventive objectives, features, and advantages of this application more apparent and understandable, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described below are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0074] It should be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.
[0075] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.
[0076] It should also be further understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.
[0077] As used in this specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if [the described condition or event] is detected" may be interpreted, depending on the context, as "once determined," "in response to determination," "once [the described condition or event] is detected," or "in response to detection of [the described condition or event]."
[0078] Furthermore, in the description of this application, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.
[0079] With the continuous development and advancement of technology, automated inspection has been widely applied and promoted in various fields. In the road traffic sector, the detection and repair of road defects has always been a crucial issue, directly impacting driving and pedestrian safety. Traditional manual inspections suffer from inefficiency and are prone to missed detections and underreporting; therefore, intelligent inspection has become a current industry trend.
[0080] However, since road surfaces are mainly composed of bricks, lanes, etc., the forms of roads are diverse, and the road background may be quite complex, making it difficult for existing road defect detection methods to accurately detect road defects.
[0081] It should be noted that the implementing entity of the method in this application is a terminal device, which may include, but is not limited to, vehicle terminals (e.g., vehicle computers, vehicle mobile phones or vehicle tablets) and robots applied in road scenarios (e.g., inspection robots, cleaning robots or delivery robots).
[0082] In real life, road defects such as cracks, potholes, or subsidence can affect the safety of pedestrians and vehicles. Therefore, identifying road defects is crucial. This application embodiment uses a road defect detection model to detect road defects; wherein, the road defect detection model is a trained artificial intelligence model used for road defect detection.
[0083] In this embodiment of the application, the road defect detection model may specifically include a backbone network, a neck network, and a head. The backbone network is used to extract features from the input data, while the neck network is used to fuse the features extracted by the backbone network. Therefore, the neck network can also be called a feature fusion network. The head can predict road defects based on the feature fusion results of the neck network.
[0084] In this embodiment, the backbone network and neck network can be pre-trained unsupervised to improve the generalization ability of the artificial intelligence model. Then, based on the pre-trained backbone network and neck network, the artificial intelligence model is trained for downstream tasks so that the artificial intelligence model can better adapt to the actual business scenario, thus obtaining the road defect detection model in this embodiment.
[0085] Specifically, please refer to Figure 1 The training process of the road defect detection model in this embodiment may include the following steps S101 to S103:
[0086] Step S101: Pre-train the initial backbone network to obtain the pre-trained backbone network.
[0087] In this embodiment of the application, a preset camera acquisition device can be used to acquire images of a road area to obtain a preset first number of first sample images.
[0088] Next, the acquired first sample image can be masked according to a preset masking ratio to obtain a first masked image and a second masked image. The first masked image is the image after masking the first sample image, and the second masked image is the image of the masked area in the first masked image. Figure 2 As shown. The value of the above-mentioned occlusion ratio can be customized and contextualized according to actual needs, and this application does not limit it. For example, the occlusion ratio can be set to 75%, that is, occluding 75% of the pixels in the first sample image.
[0089] Based on each first mask processed image and the corresponding second mask processed image, a reconstructed training sample set can be constructed.
[0090] The value of the first number can be specified and contextualized according to actual needs, and this application does not limit it; since no manual annotation is required in the process of constructing the reconstructed training sample set, a large-scale reconstructed training sample set can be easily constructed. Therefore, the value of the first number can be set to a large value.
[0091] In this embodiment, the initial first pre-trained model can be trained based on a reconstructed training sample set. The first pre-trained model may include an initial backbone network and a decoding network. The initial backbone network can be specifically configured with reference to any existing network used for feature extraction, and this application does not limit its specific configuration. For example, the backbone network can be a Vision Transformer (ViT) based on a Transformer structure or a Hierarchical Vision Transformer (HiViT) based on a Transformer structure. In the first pre-trained model, the backbone network can be used to perform feature extraction and downsampling operations on the input image, transforming image information into an abstract feature representation. The decoding network can then upsample the data output by the backbone network to map the output back to the original image size, restoring the image details.
[0092] Specifically, the initial pre-trained model can be trained using each first mask-processed image as input and the corresponding second mask-processed image as the expected output, such as... Figure 3 As shown. In actual training, the specific pre-training method of the first pre-trained model can be any common generative self-supervised learning method in the existing technology, such as MAE (Masked Autoencoder), iTPN (Image Transformer Prediction Network) or MFF (Masked Feature Fusion).
[0093] During training, the actual reconstructed image obtained by the initial first pre-trained model from the first mask-processed image can be obtained. Then, the training loss value of the first pre-trained model can be calculated based on the actual reconstructed image, the second mask-processed image, and a preset loss function. Based on this training loss value, the model parameters of the first pre-trained model can be adjusted. When the number of training iterations exceeds a preset iteration threshold, or when the reconstruction accuracy of the first pre-trained model exceeds a preset accuracy threshold, the pre-training of the first pre-trained model is considered complete, and the pre-trained first pre-trained model is obtained. Subsequently, the backbone network in the pre-trained first pre-trained model can be determined as the pre-trained backbone network.
[0094] Therefore, the backbone network can be pre-trained in an unsupervised manner. Since there is no need to manually label the sample images, a large number of sample images can be constructed, enabling the backbone network to learn rich features, which helps to improve the performance of the road defect detection model.
[0095] Step S102: Based on the pre-trained backbone network, pre-train the initial neck network to obtain the pre-trained neck network.
[0096] Since road contour segmentation and road defect detection tasks have certain similarities, in this embodiment, the initial neck network can be pre-trained for contour segmentation, thereby enabling the neck network to provide more powerful assistance in the road defect detection task and helping to improve the accuracy and generalization ability of the road defect detection model.
[0097] In this embodiment, the parameters of the pre-trained backbone network can be frozen, and a second pre-trained model can be obtained based on the parameter-frozen backbone network. Specifically, the second pre-trained model may include a pre-trained backbone network, an initial neck network, and a segmentation prediction head; wherein, the segmentation prediction head can be used to segment the contours of the image, and the initial neck network can specifically be a decoder in a Transformer structure.
[0098] Roads are mainly composed of elements such as bricks and lanes, and their outline information is relatively clear. Therefore, before pre-training the initial neck network, an edge detection training sample set can be constructed based on edge detection. Specifically, a second number of second sample images can be acquired using a camera acquisition device. Here, the second sample image can be a sample image that is completely different from the first sample image, or the second sample image can be a sample image that is partially the same as or completely identical to the first sample image. That is, some or all of the first sample images can be determined as the second sample images.
[0099] Then, a preset edge detection algorithm can be used to perform edge detection on each second sample image to obtain the corresponding contour image, such as... Figure 4 As shown. The edge detection algorithm can be any common edge detection algorithm in the existing technology, including but not limited to the Canny edge detection algorithm, Roberts edge detection algorithm, Prewitt edge detection algorithm, Sobel edge detection algorithm, or Marr-Hidreth edge detection algorithm.
[0100] Based on the second number of second sample images and the corresponding contour images, an edge detection training sample set can be constructed. The value of the second number can be customized and contextualized according to actual needs; this application does not impose any limitations on it. Since no manual annotation is required during the construction of the edge detection training sample set, a large-scale edge detection training sample set can be easily constructed; therefore, the value of the second number can be set to a relatively large value.
[0101] Therefore, a large number of sample images can be efficiently constructed in an unsupervised manner, improving the pre-training effect of the neck network.
[0102] Then, using each second sample image as input and the corresponding contour image as the expected output, the initial second pre-trained model can be trained, such as... Figure 5 As shown.
[0103] Specifically, the initial second pre-trained model can be used to perform contour segmentation on each second sample image to obtain the corresponding actual segmented image. Then, based on the contour image, the actual segmented image, and the preset loss function, the training loss value of the second pre-trained model can be calculated. Based on the training loss value, the model parameters of the neck network and segmentation prediction head in the second pre-trained model can be adjusted using the gradient descent method. When the number of training iterations is greater than the preset iteration threshold, or when the segmentation accuracy of the second pre-trained model is greater than the preset accuracy threshold, the pre-training of the second pre-trained model can be determined to be complete, and the trained second pre-trained model can be obtained.
[0104] The loss function mentioned above can be any loss function used for training the computational model in the existing technology. For example, it can be the Dice coefficient loss function, which is widely used in image segmentation tasks.
[0105] Then, the neck network in the second pre-trained model can be identified as the pre-trained neck network.
[0106] Step S103: Based on the pre-trained backbone network and the pre-trained neck network, train the initial artificial intelligence model to obtain the road disease detection model.
[0107] In this embodiment of the application, an initial artificial intelligence model can be obtained based on the pre-trained backbone network, the pre-trained neck network, and the initial detection head.
[0108] Before training the initial artificial intelligence model, a training sample set of road defects can be pre-built.
[0109] Specifically, a camera acquisition device can be used to acquire images of the road area, obtaining a third number of third sample images; alternatively, a portion of the first or second sample images can be designated as third sample images, and these third sample images can be labeled to obtain corresponding expected detection labels. Here, the expected detection labels of the third sample images can include expected category labels and expected location labels; wherein, the expected category labels of the third sample images can be used to indicate whether road defects exist in the third sample images (i.e., the detection result of road defects), and the expected location labels can indicate the specific location of the road defects in the third sample images (marked in the form of detection boxes).
[0110] Based on each third-sample image and its corresponding expected detection label, a road defect training sample set can be constructed. Then, the initial artificial intelligence model can be trained using each third-sample image as input and the corresponding expected detection label as the expected output.
[0111] During training, the initial AI model can be used to detect road defects in third-sample images to obtain actual detection labels, such as... Figure 6 As shown, the actual detection label can include an actual detection category label and an actual detection location label. The actual detection category label can be used to indicate whether road defects are detected in the third sample image, while the actual detection location label can indicate the specific location of the road defects in the third sample image.
[0112] Subsequently, the training loss value of the initial artificial intelligence model can be calculated based on the actual detection label, the expected detection label, and the preset loss function. The model parameters of the initial artificial intelligence model can be adjusted based on the training loss value. When the number of training iterations is greater than the preset number of iterations threshold, or when the prediction accuracy of the artificial intelligence model is greater than the preset accuracy threshold, it can be determined that the pre-training of the artificial intelligence model is complete, and the road defect detection model of this application embodiment can be obtained.
[0113] The aforementioned loss function can be specifically set according to actual needs, and this application does not impose any limitations on it. For example, the initial classification loss value of the artificial intelligence model can be calculated based on the actual detected class label, the expected class label, and the cross-entropy loss function; the initial bounding box regression loss value of the artificial intelligence model can also be calculated based on the actual detected location label, the expected location label, and the L1 loss function; then, the weighted average of the classification loss value and the bounding box regression loss value can be calculated to obtain the initial training loss value of the artificial intelligence model.
[0114] After training the road defect detection model, it can be applied to actual road defect detection.
[0115] Please see Figure 7 One embodiment of a road defect detection method in this application may include:
[0116] Step S701: Obtain road image.
[0117] In this embodiment of the application, the terminal device may be equipped with a camera acquisition device for applications such as navigation, obstacle avoidance, and detection.
[0118] When road defects need to be detected, a camera acquisition device can be used to acquire images of the road area to be detected, thus obtaining road images.
[0119] Step S702: Based on the preset road defect detection model, perform road defect detection on the road image to obtain the road defect detection results of the road image.
[0120] In this embodiment of the application, a road image can be used as input to a road defect detection model to obtain the output result of the road defect detection model, which is the road defect detection result of the road image.
[0121] In one specific implementation of this application, in order to further improve the performance of the road defect detection model, samples can be collected during the actual detection process, and the road defect detection model can be trained periodically based on the collected samples, thereby improving the accuracy of the road defect detection model and better adapting it to actual application scenarios.
[0122] In summary, this application embodiment acquires road images; based on a preset road defect detection model, it performs road defect detection on the road images to obtain road defect detection results; wherein, the road defect detection model is a trained artificial intelligence model for road defect detection, and the neck network of the road defect detection model is pre-trained by contour segmentation using an edge detection training sample set constructed based on edge detection. Since road contour segmentation and road defect detection tasks have certain similarities, this application embodiment can construct an edge detection training sample set through edge detection and use this set to pre-train the neck network for contour segmentation. This allows the pre-trained neck network to perform more targeted feature fusion, thereby providing stronger assistance in the road defect detection task and helping to improve the accuracy and generalization ability of the road defect detection model.
[0123] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.
[0124] Corresponding to the road defect detection method described in the above embodiments, Figure 8 This illustration shows a structural diagram of one embodiment of a road defect detection device provided in this application.
[0125] In this embodiment of the application, a road defect detection device may include:
[0126] The road image acquisition module 801 is used to acquire road images;
[0127] The road defect detection module 802 is used to perform road defect detection on the road image based on a preset road defect detection model, and obtain the road defect detection result of the road image; wherein, the road defect detection model is an artificial intelligence model trained for road defect detection, and the neck network of the road defect detection model is obtained by pre-training contour segmentation through an edge detection training sample set constructed based on edge detection.
[0128] In one specific implementation of this application embodiment, the road defect detection device further includes:
[0129] The first training module is used to pre-train the initial backbone network to obtain the pre-trained backbone network.
[0130] The second training module is used to pre-train the initial neck network based on the pre-trained backbone network to obtain the pre-trained neck network.
[0131] The third training module is used to train the initial artificial intelligence model based on the pre-trained backbone network and the pre-trained neck network to obtain the road defect detection model.
[0132] In one specific implementation of this application embodiment, the first training module includes:
[0133] The first sub-module for constructing the sample set is used to construct the reconstruction training sample set; wherein, the reconstruction training sample set includes a first number of first mask-processed images and corresponding second mask-processed images;
[0134] The model training first submodule is used to train the initial first pre-trained model with each of the first mask processing images as input and the corresponding second mask processing images as expected output, to obtain the trained first pre-trained model; wherein, the initial first pre-trained model includes an initial backbone network and a decoding network;
[0135] The backbone network determination submodule is used to determine the pre-trained backbone network based on the first pre-trained model after training.
[0136] In one specific implementation of this application embodiment, the second training module includes:
[0137] The sample set construction second sub-module is used to construct the edge detection training sample set; wherein, the edge detection training sample set includes a second number of second sample images and corresponding contour images;
[0138] The second sub-module for model training is used to train the initial second pre-trained model with each of the second sample images as input and the corresponding contour image as the expected output, to obtain the trained second pre-trained model; wherein, the initial second pre-trained model includes the pre-trained backbone network, the initial neck network, and the initial segmentation prediction head;
[0139] The neck network determination submodule is used to determine the pre-trained neck network based on the trained second pre-trained model.
[0140] In one specific implementation of this application embodiment, the second sub-module for constructing the sample set includes:
[0141] The second sample image acquisition unit is used to acquire a second number of individual second sample images;
[0142] An edge detection unit is used to perform edge detection on each of the second sample images using a preset edge detection algorithm to obtain the corresponding contour image;
[0143] The sample set construction unit is used to construct the edge detection training sample set based on each of the second sample images and the corresponding contour images.
[0144] In one specific implementation of this application embodiment, the second sub-module for model training includes:
[0145] The actual output unit is used to perform contour segmentation on each of the second sample images using the initial second pre-trained model to obtain the corresponding actual segmentation image;
[0146] The loss calculation unit is used to calculate the training loss value of the second pre-trained model based on the contour image, the actual segmented image, and a preset loss function.
[0147] The parameter adjustment unit is used to adjust the model parameters of the initial second pre-trained model based on the training loss value, so as to obtain the trained second pre-trained model.
[0148] In one specific implementation of this application embodiment, the third training module includes:
[0149] The third sub-module for constructing the sample set is used to construct a road defect training sample set; wherein, the road defect training sample set includes a third number of third sample images and corresponding expected detection labels;
[0150] The third sub-module for model training is used to train the initial artificial intelligence model with each of the third sample images as input and the corresponding expected detection label as expected output to obtain the road defect detection model; wherein, the initial artificial intelligence model includes the pre-trained backbone network, the pre-trained neck network, and the initial detection head.
[0151] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.
[0152] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.
[0153] Figure 9 A schematic block diagram of a terminal device provided in an embodiment of this application is shown. For ease of explanation, only the parts related to the embodiment of this application are shown.
[0154] like Figure 9 As shown, the terminal device 9 in this embodiment includes: a processor 90, a memory 91, and a computer program 92 stored in the memory 91 and executable on the processor 90. When the processor 90 executes the computer program 92, it implements the steps in the various road defect detection method embodiments described above, for example... Figure 7 Steps S701 to S702 are shown. Alternatively, when the processor 90 executes the computer program 92, it implements the functions of each module / unit in the above-described device embodiments, for example... Figure 8 The functions of modules 801 to 802 are shown.
[0155] For example, the computer program 92 may be divided into one or more modules / units, which are stored in the memory 91 and executed by the processor 90 to complete this application. The one or more modules / units may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program 92 in the terminal device 9.
[0156] Those skilled in the art will understand that Figure 9 This is merely an example of terminal device 9 and does not constitute a limitation on terminal device 9. It may include more or fewer components than shown, or combine certain components, or different components. For example, terminal device 9 may also include input / output devices, network access devices, buses, etc.
[0157] The processor 90 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor. The processor 90 can be the nerve center and command center of the terminal device 9. The processor 90 can generate operation control signals based on instruction opcodes and timing signals to control instruction fetching and execution.
[0158] The memory 91 can be an internal storage unit of the terminal device 9, such as a hard disk or memory of the terminal device 9. The memory 91 can also be an external storage device of the terminal device 9, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the terminal device 9. Furthermore, the memory 91 can include both internal and external storage units of the terminal device 9. The memory 91 is used to store the computer program and other programs and data required by the terminal device 9. The memory 91 can also be used to temporarily store data that has been output or will be output.
[0159] The terminal device 9 may further include a communication module, which can provide communication solutions for network devices, including Wireless Local Area Networks (WLANs) (such as Wi-Fi networks), Bluetooth, Zigbee, mobile communication networks, Global Navigation Satellite System (GNSS), Frequency Modulation (FM), Near Field Communication (NFC), and Infrared (IR) technologies. The communication module may be one or more devices integrating at least one communication processing module. The communication module may include an antenna, which may have a single element or be an antenna array with multiple elements. The communication module can receive electromagnetic waves through the antenna, frequency-modulate and filter the electromagnetic wave signals, and send the processed signals to the processor. The communication module can also receive signals to be transmitted from the processor, frequency-modulate and amplify them, and then convert them into electromagnetic waves for radiation via the antenna.
[0160] The terminal device 9 may further include a power management module, which can receive input from an external power source, battery, and / or charger to power the processor, the memory, and the communication module, etc.
[0161] The terminal device 9 may further include a display module, which can be used to display information input by the user or information provided to the user. The display module may include a display panel, optionally configured as a Liquid Crystal Display (LCD), Organic Light-Emitting Diode (OLED), or similar display panel. Furthermore, a touch panel may cover the display panel. When the touch panel detects a touch operation on or near it, it transmits the information to the processor to determine the type of touch event. Subsequently, the processor provides corresponding visual output on the display panel based on the type of touch event.
[0162] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.
[0163] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.
[0164] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0165] In the embodiments provided in this application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling or direct coupling or communication connection may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.
[0166] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0167] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0168] This application provides a computer program product that, when run on a terminal device, enables the terminal device to implement the steps described in the various method embodiments above.
[0169] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable storage medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc. It should be noted that the content included in the computer-readable storage medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable storage medium does not include electrical carrier signals and telecommunication signals.
[0170] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.
Claims
1. A method for detecting road defects, characterized in that, include: Acquire road images; Based on a preset road defect detection model, road defects are detected in the road image to obtain the road defect detection results of the road image. The road defect detection model is an artificial intelligence model trained for road defect detection, and the neck network of the road defect detection model is obtained by pre-training contour segmentation using an edge detection training sample set constructed based on edge detection.
2. The road defect detection method according to claim 1, characterized in that, The training process of the road defect detection model includes: The initial backbone network is pre-trained to obtain the pre-trained backbone network. Based on the pre-trained backbone network, the initial neck network is pre-trained to obtain the pre-trained neck network. Based on the pre-trained backbone network and the pre-trained neck network, the initial artificial intelligence model is trained to obtain the road defect detection model.
3. The road defect detection method according to claim 2, characterized in that, The step of pre-training the initial backbone network to obtain the pre-trained backbone network includes: Construct a reconstruction training sample set; wherein the reconstruction training sample set includes a first number of first mask-processed images and corresponding second mask-processed images; Using each of the first mask-processed images as input and the corresponding second mask-processed images as the expected output, the initial first pre-trained model is trained to obtain the trained first pre-trained model; wherein, the initial first pre-trained model includes an initial backbone network and a decoding network; Based on the first pre-trained model after training, the pre-trained backbone network is determined.
4. The road defect detection method according to claim 2, characterized in that, The process of pre-training the initial neck network based on the pre-trained backbone network to obtain the pre-trained neck network includes: Construct the edge detection training sample set; wherein, the edge detection training sample set includes a second number of second sample images and corresponding contour images; Using each of the second sample images as input and the corresponding contour image as the expected output, the initial second pre-trained model is trained to obtain the trained second pre-trained model; wherein, the initial second pre-trained model includes the pre-trained backbone network, the initial neck network, and the initial segmentation prediction head; Based on the trained second pre-trained model, the pre-trained neck network is determined.
5. The road defect detection method according to claim 4, characterized in that, The construction of the edge detection training sample set includes: Obtain a second number of each of the second sample images; Using a preset edge detection algorithm, edge detection is performed on each of the second sample images to obtain the corresponding contour image; The edge detection training sample set is constructed based on each of the second sample images and the corresponding contour images.
6. The road defect detection method according to claim 4, characterized in that, The step of training the initial second pre-trained model using each of the second sample images as input and the corresponding contour image as the expected output to obtain the trained second pre-trained model includes: Using the initial second pre-trained model, contour segmentation is performed on each of the second sample images to obtain the corresponding actual segmented images; Based on the contour image, the actual segmented image, and the preset loss function, the training loss value of the second pre-trained model is calculated; Based on the training loss value, the model parameters of the initial second pre-trained model are adjusted to obtain the trained second pre-trained model.
7. The road defect detection method according to claim 2, characterized in that, The road defect detection model is obtained by training the initial artificial intelligence model based on the pre-trained backbone network and the pre-trained neck network, including: Construct a road defect training sample set; wherein, the road defect training sample set includes a third number of third sample images and corresponding expected detection labels; Using each of the third sample images as input and the corresponding expected detection label as expected output, the initial artificial intelligence model is trained to obtain the road defect detection model; The initial artificial intelligence model includes the pre-trained backbone network, the pre-trained neck network, and the initial detection head.
8. A road defect detection device, characterized in that, include: The road image acquisition module is used to acquire road images; The road defect detection module is used to perform road defect detection on the road image based on a preset road defect detection model, and obtain the road defect detection result of the road image; wherein, the road defect detection model is an artificial intelligence model trained for road defect detection, and the neck network of the road defect detection model is obtained by pre-training contour segmentation through an edge detection training sample set constructed based on edge detection.
9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the road defect detection method as described in any one of claims 1 to 7.
10. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the road defect detection method as described in any one of claims 1 to 7.