A method for identifying highway pavement diseases based on a capsule network

Abstract

The application discloses a highway pavement disease identification method based on a capsule network, relates to the technical field of pavement disease identification, and comprises three modules of database preprocessing, a capsule network model and an identification method, wherein the database preprocessing comprises the following contents: in the establishment of the model, the model database is first built, in the building, the types of highway pavement diseases are divided, then the original data of the pavement diseases are established according to the divided results, in the establishment, the picture information of various data is converted into primary picture data sets by shooting, and the information corresponding to the picture information is acquired by using a ground penetrating radar and is converted into primary internal picture sets, through the database preprocessing, secondary similarity determination is performed to pre-screen the database, so that the data is processed in detail and the detailed data of the data is increased, and therefore, fine internal picture sets and fine picture data are obtained.

Description

Technical Field

[0001] This invention relates to the field of pavement distress identification technology, and specifically to a method for identifying highway pavement distress based on capsule networks. Background Technology

[0002] Highway pavement defects refer to damage that occurs on highway surfaces after a period of use, typically including potholes, cracks, settlement, and water accumulation. These defects can seriously affect driving safety on highways and require timely repair.

[0003] To prevent highway pavement damage, a series of measures need to be taken, mainly to strengthen pavement maintenance and management: regular maintenance of highway pavement, including clearing snow from the road surface, replacing worn pavement materials, and repairing cracks.

[0004] Capsule networks offer a good approach to efficient image recognition and classification. They feature capsule units, which enhance the model's information representation capabilities, require less training data, and achieve higher accuracy. Furthermore, the dynamic routing algorithm used in the updates between two layers of vectors in capsule networks avoids the loss of precise image location information.

[0005] However, existing highway road surface identification methods mainly rely on road surface identification vehicles to identify the current road surface status after driving on-site, in order to identify road surface defects. However, there is a problem of untimely identification in the actual identification process. Therefore, this invention provides a highway road surface defect identification method based on capsule networks with a timely identification method. Summary of the Invention

[0006] In view of the aforementioned existing problems, the present invention is proposed.

[0007] Therefore, the purpose of this invention is to provide a method for identifying highway pavement defects based on capsule networks, which solves the above-mentioned problems.

[0008] To achieve the above objectives, the present invention provides the following technical solution:

[0009] A method for identifying highway pavement defects based on capsule networks includes three main modules: database preprocessing, capsule network model, and identification method. The database preprocessing includes the following:

[0010] S1. Database Preprocessing: In the process of building the model, the first step is to build the model database. During the construction, the types of road surface defects of highways are classified, and the original data of road surface defects are established based on the classification results. In the process of building, the image information of various data is taken and converted into a primary image dataset. At the same time, ground penetrating radar is used to obtain the corresponding information and convert it into a primary internal image set.

[0011] S2. After the primary image dataset and the primary internal image set are established, data preprocessing is required. During preprocessing, highly similar images of the same disease are removed. This removal process also involves judging the primary internal image set. Before removing images, the primary image dataset is rasterized. During rasterization, images are divided according to their size. The computer divides each image in the primary image dataset into 3x3 pixels. Rasterizing the image dataset to a ratio of 3, the high-similarity segmentation algorithm is as follows: Primary Image Dataset Similarity Determination Algorithm in This represents the amount of image information in the control group, while This indicates the amount of image information in the judgment group. This indicates the output result. The similarity level is determined based on the output result, and its value ranges from [value missing]. Between them, when the calculation is performed by calculating the amount of overlap in information, in the primary image dataset similarity determination algorithm, if the result is in This proves that the initial image dataset similarity assessment indicates dissimilar image information, and when the results are obtained... This indicates that the images are similar. When similar images are identified, a secondary similarity determination is performed on the initial internal image set. The algorithm is as follows: in This represents the amount of pixel information in the control group, while This represents the pixel information of primary internal images in similar primary image datasets, while This indicates the output value, in as well as This indicates dissimilarity, while when If the similarity is determined to be similar, then the data is deleted. If the data is not similar, then the data is optimized by refining the data and increasing the detail of the data, thereby obtaining a refined internal image set and a refined image dataset.

[0012] Furthermore, the capsule network model includes the following:

[0013] SS1 transforms each fine-grained internal image set and the fine-grained image dataset into an input size of 28. The specification is 28, and a 256-channel feature map is generated by using a convolution kernel of size 9. A stride of 1 is set, and Relativity is used as the activation function, resulting in an output size of 256. 20 The feature map of size 20 is used as the first convolutional layer and encapsulated by converting it into a convolutional kernel with a size of 9. 9. Simultaneously, after setting the step size to 2 and the number of channels to 256, the material is transformed into 32 capsule layers after passing through the main capsule layer, with each capsule layer limited to 6 cells. Composed of 6 capsules, resulting in a total output of 32. 6 There are 6 capsule units, and the capsule unit vector is limited to 8, where the output of the main capsule is; in This is the output squeeze vector value, and This is the vector value obtained during data processing;

[0014] SS2 transforms the main capsule layer into a digital capsule of 10 by using a dynamic routing algorithm, while controlling each layer to contain 10 capsule layers, thus converting it into 10. A matrix sequence of 10 is used to determine that when the angle and position of the image of the object being identified changes, the vector of the capsule unit will also change accordingly, thus completing the encoding.

[0015] The SS3 decoder processes the obtained matrix sequence through three fully connected layers. During this process, neurons are used for computation, with 512, 1024, and 784 neurons used for each layer. After processing through the 512 and 1024 neurons, the neurons pass through the Reye's activation function, while the last fully connected layer with 784 neurons uses the sigmoid function to complete the decoding.

[0016] Furthermore, the identification method includes the following:

[0017] Step 1: Include a dynamic routing algorithm in the capsule network model. The algorithm is as follows:

[0018] ,in express The There are output vectors, and This represents the transformation matrix, determined by setting the summary vector of the main capsule layer. , among them ,in For the predicted capsule weights, This is the prediction capsule connected to the output capsule. The verification probability is used to iteratively update the algorithm and adjust the weights accordingly. ;

[0019] Step 2, by passing through Continue through Obtain the output vector To update as well as , then To perform the next generation of iterations, the updated output vector is repeatedly calculated. This yields the final output vector, while utilizing the interval loss function. ,in This indicates a classification indicator, when If it exists, its value is set to 1; otherwise, it is set to 1. The value is 0, where This signifies punishment of the higher realms, and This indicates punishment for the lower realm, and This represents the scaling factor, obtained by updating the transformed matrix. And its convolution parameters, in practical use, through the middle The value is limited to 0.9, while Then it is limited to 0.1, and The value is then set to 0.5, thus yielding the probability of classification.

[0020] Furthermore, based on the probability obtained, the type of pavement distress is determined. According to the type of distress, the actual area of ​​distress inside the pavement is predicted. The actual distress is determined based on the type of pavement distress identified by the capsule network. Using the image information of the identified distress type, the shape of the identified distress is compared with the fine internal image set and fine image dataset stored in the database. During the comparison, the size ratio of the compared data shapes is transformed by zooming in and out to ensure that they are the same. When identifying and determining the distress depth, the predicted pavement distress information is used to predict the actual pavement distress depth.

[0021] Furthermore, the identification is aided by surveillance cameras on the highway. When there are no vehicles on the road, the data is uploaded to the cloud. After the capsule network identifies the road defects, maintenance personnel are arranged to conduct on-site identification. During the on-site identification, the construction personnel will identify the actual road information. After identification, the actual detection depth and shape are uploaded to the detailed internal image set in the capsule network for learning, which facilitates the identification of road defects and the prediction of the depth of road defects in the future.

[0022] Furthermore, the road surface defects include cracks, ruts, potholes, subsidence, shoving, and bumps.

[0023] The technical effects and advantages provided by the present invention in the above technical solution are as follows:

[0024] 1. This invention uses database preprocessing, which involves a second similarity assessment to pre-screen the database, thereby refining the data and increasing its detail, resulting in a refined internal image set and a refined image dataset.

[0025] 2. This invention utilizes surveillance cameras on highways for assisted identification. When there are no vehicles on the road, the road information is uploaded to the cloud. After identifying road defects through a capsule network, maintenance personnel are dispatched to conduct on-site identification. During on-site identification, the construction personnel identify the actual road information. After identification, the actual detection depth and shape are uploaded to a detailed internal image set in the capsule network for learning. This facilitates the subsequent identification of road defects and prediction of their depth. Through continuous use, new road defect depths are added. As actual use progresses, more actual road condition data sets are added, which are corroborated by actual measurement results in real-world scenarios to ensure the addition of new actual data sets, thereby improving the accuracy of subsequent road defect predictions. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0027] Figure 1 This is a schematic diagram of road surface cracks according to the present invention;

[0028] Figure 2 This is a schematic diagram of the road surface potholes of the present invention;

[0029] Figure 3 This is a schematic diagram of the identification process of the present invention. Detailed Implementation

[0030] To enable those skilled in the art to better understand the technical solution of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings.

[0031] This invention discloses a method for identifying highway pavement defects based on capsule networks.

[0032] This invention provides, for example Figure 1-3 The method for identifying highway pavement defects based on capsule networks, as shown, includes three main modules: database preprocessing, capsule network model, and identification method. The database preprocessing includes the following:

[0033] S1. Database Preprocessing: In model building, the first step is to construct the model database. This involves classifying the types of road surface defects on highways and then establishing the raw data of road surface defects based on the classification results. During this process, images of various data are captured and converted into a primary image dataset. Simultaneously, ground-penetrating radar is used to acquire corresponding information and convert it into a primary internal image set. The obtained road surface defect depth information is used to prepare for later predictions. The prepared internal defect depth information is used to prepare for later detections. Whenever actual road surface defects are detected later, matching is performed. After determining the type of defect, the defect type is classified to obtain the defect type at the actual defect location. The shape of the defect is then detected. After determining the defect shape, it is matched with the defect depth of similar defect shapes in the database to predict and estimate the defect depth.

[0034] S2. After the primary image dataset and the primary internal image set are established, data preprocessing is required. During preprocessing, highly similar images of the same disease are removed. This removal process also involves judging the primary internal image set. Before removing images, the primary image dataset is rasterized. During rasterization, images are divided according to their size. The computer divides each image in the primary image dataset into 3x3 pixels. Rasterizing the image dataset to a ratio of 3, the high-similarity segmentation algorithm is as follows: Primary Image Dataset Similarity Determination Algorithm in This represents the amount of image information in the control group, while This indicates the amount of image information in the judgment group. This indicates the output result. The similarity level is determined based on the output result, and its value ranges from [value missing]. Between them, when the calculation is performed by calculating the amount of overlap in information, in the primary image dataset similarity determination algorithm, if the result is in This proves that the initial image dataset similarity assessment indicates dissimilar image information, and when the results are obtained... This indicates that the images are similar. When similar images are identified, a secondary similarity determination is performed on the initial internal image set. The algorithm is as follows: in This represents the amount of pixel information in the control group, while This represents the pixel information of primary internal images in similar primary image datasets, while This indicates the output value, in as well as This indicates dissimilarity, while when If the similarity is determined by secondary similarity, the data is deleted. If the data is not similar, it is optimized by refining the data and increasing its detail. This results in a refined internal image set and a refined image dataset. By including images with internal depth in the image selection, the accuracy of image similarity selection can be increased, rather than simply filtering out images based on shape similarity. When selecting based on similar shapes, some diseases may have similar surfaces but different internal depths. By performing a further detection operation based on disease shape similarity, the prediction of disease depth can be further improved.

[0035] The capsule network model includes the following:

[0036] SS1 transforms each fine-grained internal image set and the fine-grained image dataset into an input size of 28. The specification is 28, and a 256-channel feature map is generated by using a convolution kernel of size 9. A stride of 1 is set, and Relativity is used as the activation function, resulting in an output size of 256. 20 The feature map of size 20 is used as the first convolutional layer and encapsulated by converting it into a convolutional kernel with a size of 9. 9. Simultaneously, after setting the step size to 2 and the number of channels to 256, the material is transformed into 32 capsule layers after passing through the main capsule layer, with each capsule layer limited to 6 cells. Composed of 6 capsules, resulting in a total output of 32. 6 There are 6 capsule units, and the capsule unit vector is limited to 8, where the output of the main capsule is; in This is the output squeeze vector value, and This is the vector value obtained during data processing;

[0037] SS2 transforms the main capsule layer into a digital capsule of 10 by using a dynamic routing algorithm, while controlling each layer to contain 10 capsule layers, thus converting it into 10. A matrix sequence of 10 is used to determine that when the angle and position of the image of the object being identified changes, the vector of the capsule unit will also change accordingly, thus completing the encoding.

[0038] The SS3 decoder processes the obtained matrix sequence through three fully connected layers. During this process, neurons are used for computation, with 512, 1024, and 784 neurons used for each layer. After processing through the 512 and 1024 neurons, the neurons pass through the Reye's activation function, while the last fully connected layer with 784 neurons uses the sigmoid function to complete the decoding.

[0039] The identification methods include the following:

[0040] Step 1: Include a dynamic routing algorithm in the capsule network model. The algorithm is as follows:

[0041] ,in express The There are output vectors, and This represents the transformation matrix, determined by setting the summary vector of the main capsule layer. , among them ,in For the predicted capsule weights, This is the prediction capsule connected to the output capsule. The verification probability is used to iteratively update the algorithm and adjust the weights accordingly. ;

[0042] Step 2, by passing through Continue through Obtain the output vector To update as well as , then To perform the next generation of iterations, the updated output vector is repeatedly calculated. This yields the final output vector, while utilizing the interval loss function. ,in This indicates a classification indicator, when If it exists, its value is set to 1; otherwise, it is set to 1. The value is 0, where This signifies punishment of the higher realms, and This indicates punishment for the lower realm, and This represents the scaling factor, obtained by updating the transformed matrix. And its convolution parameters, in practical use, through the middle The value is limited to 0.9, while Then it is limited to 0.1, and The value is then set to 0.5 to determine the probability of classification, which makes it easier to determine whether the road surface defects on site have been identified.

[0043] The process involves determining the type of road surface distress based on the probability obtained, predicting the actual area of ​​distress within the road surface, and then comparing the identified distress shape with a set of detailed internal images and a detailed image dataset stored in the database. During the comparison, the size of the compared data shapes is adjusted by zooming in and out to ensure consistency. Finally, the predicted road surface distress information is used to predict the actual depth of the distress after identification.

[0044] The system utilizes surveillance cameras on highways for assisted identification. When there are no vehicles on the road, the data is uploaded to the cloud. After identifying road defects through a capsule network, maintenance personnel are dispatched to conduct on-site identification. During this process, the construction workers assess the actual road surface conditions and upload the measured depth and shape to a detailed internal image set within the capsule network for learning. This learning facilitates the identification of road defects and the prediction of their depth. As the system is used, new road defect depth data is continuously added. Real-world measurement results from actual scenarios corroborate the findings, ensuring the accuracy of future road defect predictions is improved.

[0045] Among them, road surface defects include cracks (road surface cracking), rutting (the accumulation of permanent strip-shaped grooves on the road surface under repeated traffic loads), potholes (small pits on the road surface), subsidence (large pits on the road surface), and shoving (the shoving caused by asphalt displacement).

[0046] The foregoing has only described certain exemplary embodiments of the present invention by way of illustration. Undoubtedly, those skilled in the art can modify the described embodiments in various ways without departing from the spirit and scope of the present invention. Therefore, the foregoing drawings and descriptions are illustrative in nature and should not be construed as limiting the scope of protection of the claims of the present invention.

Claims

1. A method for identifying highway pavement defects based on capsule networks, characterized in that, It comprises three main modules: database preprocessing, capsule network model, and recognition method. The database preprocessing includes the following: S1. Database Preprocessing: In the process of building the model, the first step is to build the model database. During the construction, the types of road surface defects of highways are classified, and the original data of road surface defects are established based on the classification results. In the process of building, the image information of various data is taken and converted into a primary image dataset. At the same time, ground penetrating radar is used to obtain the corresponding information and convert it into a primary internal image set. S2. After the primary image dataset and the primary internal image set are established, data preprocessing is required. During preprocessing, highly similar images of the same disease are removed. This removal process includes initial internal image set evaluation. Before image data removal, the initial image dataset is rasterized. During rasterization, images are divided according to their size, and the computer divides each image in the initial image dataset into 3x3 pixels. Rasterize at a ratio of 3; The methods for dividing images into high-similarity categories are as follows: Methods for determining similarity in primary image datasets. in This represents the amount of image information in the control group, while This indicates the amount of image information in the judgment group. This indicates the output result. The similarity level is determined based on the output result, and its value ranges from [value missing]. Between them, when the calculation is performed by calculating the amount of overlap in information, in the primary image dataset similarity determination method, if the result is in This proves that the initial image dataset similarity determination indicates that the images are dissimilar. And when the result is obtained This indicates that the images are similar. When similar images are identified, a secondary similarity determination is performed on the initial internal image set. The method is as follows: in This represents the amount of pixel information in the control group, while This represents the pixel information of primary internal images in similar primary image datasets, while This indicates the output value, in as well as This indicates dissimilarity, while when Then it is considered similar; If the secondary similarity test determines that the data is still similar, then the data is deleted. If the data is not similar, then the data is optimized by refining its details and increasing the data's detail, thereby obtaining a refined internal image set and a refined image dataset.

2. The method for identifying highway pavement defects based on capsule networks according to claim 1, characterized in that, The capsule network model includes the following: SS1 transforms each fine-grained internal image set and the fine-grained image dataset into an input size of 28. The specification is 28, and a 256-channel feature map is generated by using a convolution kernel of size 9. A stride of 1 is set, and Relativity is used as the activation function, resulting in an output size of 256. 20 The feature map of size 20 is used as the first convolutional layer and encapsulated by converting it into a convolutional kernel with a size of 9.

9. Simultaneously, after setting the step size to 2 and the number of channels to 256, the material is transformed into 32 capsule layers after passing through the main capsule layer, with each capsule layer limited to 6 cells. Composed of 6 capsules, resulting in a total output of 32. 6 There are 6 capsule units, and the capsule unit vector is limited to 8, where the output of the main capsule is; in This is the output squeeze vector value, and This is the vector value obtained during data processing; SS2, by using a dynamic routing method to output the main capsule layer as a digital capsule of 10, and controlling each layer to contain 10 capsule layers, thus transforming it into a 10... A matrix sequence of 10 is used to determine that when the angle and position of the image of the object being identified changes, the vector of the capsule unit will also change accordingly, thus completing the encoding. The SS3 decoder processes the obtained matrix sequence through three fully connected layers. During this process, neurons are used for computation, with 512, 1024, and 784 neurons used for each layer. After processing through the 512 and 1024 neurons, the neurons pass through the Reye's activation function, while the last fully connected layer with 784 neurons uses the sigmoid function to complete the decoding.

3. The method for identifying highway pavement defects based on capsule networks according to claim 1, characterized in that, The identification method includes the following: Step 1: Include a dynamic routing method in the capsule network model. The method is as follows: ,in express The There are output vectors, and This represents the transformation matrix, determined by setting the summary vector of the main capsule layer. , among them ,in For the predicted capsule weights, This is the prediction capsule connected to the output capsule. The verification probability is used to iteratively update the method and adjust the weights accordingly. ; Step 2, by passing through Continue through Obtain the output vector To update as well as , then To perform the next generation of iterations, the updated output vector is repeatedly calculated. This yields the final output vector, while utilizing the interval loss function. ,in This indicates a classification indicator, when If it exists, its value is set to 1; otherwise, it is set to 1. The value is 0, where This signifies punishment of the higher realms, and This indicates punishment for the lower realm, and This represents the scaling factor, obtained by updating the transformed matrix. And its convolution parameters, in practical use, through the middle The value is limited to 0.9, while Then it is limited to 0.1, and The value is then set to 0.5, thus yielding the probability of classification.

4. The method for identifying highway pavement defects based on capsule networks according to claim 1, characterized in that, The probability of obtaining the probability is used to determine the type of road surface distress. Based on the type of distress, the actual area of ​​distress inside the road surface is predicted. The actual distress is determined by the actual capsule network. Using the image information of the identified distress type, the shape of the identified distress is compared with the fine internal image set and fine image dataset stored in the database. During the comparison, the size ratio of the compared data shapes is transformed by zooming in and out to ensure that they are the same. When identifying and determining the distress depth, the predicted road surface distress information is used to predict the actual depth of the road surface distress.

5. The method for identifying highway pavement defects based on capsule networks according to claim 4, characterized in that, With the assistance of surveillance cameras on the highway, when there are no vehicles on the road, the data is uploaded to the cloud. After the capsule network identifies the road defects, maintenance personnel are arranged to conduct on-site identification. During the on-site identification, the construction personnel will identify the actual road information. After identification, the actual detection depth and shape are uploaded to the detailed internal image set in the capsule network for learning, which facilitates the identification of road defects and the prediction of the depth of road defects in the future.

6. The method for identifying highway pavement defects based on capsule networks according to claim 1, characterized in that, The road surface defects include cracks, ruts, potholes, subsidence, shoving, and bumps.

Images