Method and device for automatically generating intelligent pavement maintenance scheme

Abstract

The invention provides an automatic generation method and device of an intelligent pavement maintenance scheme. According to the method, the point cloud data of the road surface are obtained through the three-dimensional laser scanning technology, the point cloud data of the road surface are automatically processed and analyzed through the neural network algorithm, and the point cloud data are classified, recognized, extracted and the like so as to automatically detect information such as the disease type, position, severity and the like of the road surface; furthermore, a maintenance scheme is automatically generated in response to the pavement disease, the scheme comprises detailed information such as maintenance content, a maintenance method and a disposal drawing, and dynamic adjustment and optimization can be performed according to actual conditions. According to the technical scheme, the automation degree of pavement maintenance is improved, manual intervention is reduced, and the maintenance efficiency and quality are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of artificial intelligence application, and specifically relates to a method and device for automatically generating an intelligent road maintenance plan. Background Art

[0002] With the development of transportation and the continuous construction of roads, the total mileage of roads in my country has continued to increase, and road maintenance issues have received more and more attention. In particular, road surface maintenance is closely related to road traffic safety and road service life, and occupies an important position in infrastructure maintenance.

[0003] Traditional pavement maintenance mainly relies on manual inspections and repairs, which is inefficient and costly. In addition, the pavement maintenance requirements and processes for different types of roads are different, and targeted maintenance plans need to be formulated based on the different characteristics of the roads. Existing pavement maintenance plans usually rely on manual inspections and empirical judgments, which are not only time-consuming and labor-intensive, but the accuracy of the inspection results is limited by the technical level of the personnel. At present, there is no systematic system that can automatically generate maintenance plans.

[0004] At present, various information and intelligent technical means have begun to be widely used in the field of pavement maintenance, including pavement condition detection by 3D laser scanning. However, although 3D laser scanning technology can obtain 3D point cloud data of the road surface and truly restore the geometric shape and subtle changes of the road surface, how to efficiently and accurately identify pavement diseases based on these data and automatically generate targeted maintenance plans has always been the main challenge in this field. In the existing technology, the disease identification and analysis of point cloud data and the generation of maintenance plans often require complex algorithms and professional knowledge, resulting in a low degree of automation and low maintenance efficiency. Summary of the invention

[0005] In response to the deficiencies of the above-mentioned prior art, the present application provides a method and device for automatically generating an intelligent pavement maintenance plan. The present application obtains the point cloud data of the pavement through three-dimensional laser scanning technology, and uses a neural network algorithm to automatically process and analyze the point cloud data of the pavement, and classifies, identifies, extracts and other operations on the point cloud data to automatically detect the type, location, severity and other information of the pavement disease; then, a maintenance plan is automatically generated in response to the pavement disease, and the plan includes detailed information such as maintenance content, maintenance methods, and disposal drawings, and can be dynamically adjusted and optimized according to actual conditions. This technical solution aims to improve the degree of automation of pavement maintenance, reduce manual intervention, and improve maintenance efficiency and quality.

[0006] In a first aspect, the present application proposes a method for automatically generating an intelligent pavement maintenance plan, comprising the following steps: Use 3D laser scanning technology to obtain 3D point cloud data of the road surface, measure and record the spatial coordinates of each point on the road surface, and thus construct a 3D model of the road surface; The collected 3D point cloud data is processed and analyzed using a neural network to automatically detect the road surface damage type, location, and severity level description information. For the 3D point cloud data, a neural network module including a CNN backbone network model, a Transformers codec structure, a road surface damage level classifier, and a damage area locator is used to perform a training phase, thereby automatically detecting the road surface damage type, location, and severity level description information. In response to the automatically detected disease description information, a pavement maintenance plan is automatically generated; the maintenance plan includes detailed information on maintenance content, maintenance methods, and disposal drawings, and can be dynamically adjusted and optimized according to actual conditions.

[0007] Preferably, in the training phase, a large number of three-dimensional point cloud data samples of the road surface are collected, including positive samples consisting of three-dimensional point cloud data of N types of diseased road surface targets of different types and severity, and the positive samples are annotated with bounding boxes corresponding to the diseased road surface targets. The samples also include negative samples consisting of three-dimensional point cloud data of normal road surfaces; the CNN backbone network model extracts its feature map from the three-dimensional point cloud data and straightens it into a feature vector of the three-dimensional point cloud data; then the feature vector is output to the Transformers encoder-decoder structure, wherein the encoder is used to further learn global feature information, and the decoder performs self-attention interactive decoding with the global feature vector output by the encoder through the target query amount of the given N types of diseased road surface targets and can output the prediction boxes of N disease positioning bounding boxes in parallel; the prediction loss between the prediction box and the bounding box corresponding to the diseased road surface target is calculated with the help of bipartite graph matching, and iterative training is achieved by optimizing the loss function.

[0008] Preferably, the data set is constructed with the three-dimensional point cloud data samples; the samples are represented as:

[0009] Among them, here is the sample size, The sample is 3D point cloud data, is the height of the 3D point cloud data, is the width of the 3D point cloud data; The sample The most 3D point cloud data The label is composed of the disease classification category and the positioning vector of the disease positioning bounding box; , The sample is Three-dimensional point cloud data The category labels of the diseased road surface targets, It is Three-dimensional point cloud data The 4D vector is composed of the standardized center point coordinates of the defect location bounding box of the defective pavement target and the height and width relative to the size of the 3D point cloud data.

[0010] Preferably, the CNN backbone network model, the Transformers codec structure, and the parameters of the pavement damage classification classifier and the damage area locator of the neural network module are initialized; wherein the CNN backbone network model adopts the convolution layer part after removing the classification layer in the pre-trained ResNet neural network, Represents the convolutional layer part of the CNN backbone network model, It is a parameter tensor consisting of all parameters of the pre-trained ResNet convolutional layer. A sample of the above three-dimensional point cloud data input into the CNN backbone network model; Represents the CNN backbone network model used to reduce the resolution of the feature map of the full convolutional network, initialize the parameter tensor of the model ,in is the low-resolution feature map generated by the fully convolutional network; Represents the Transformers encoder-decoder structure, which performs the operation of extracting global features and performing self-attention interaction with the target query volume of N types of diseased road targets, initializing its parameter tensor ; Represents a pavement disease classification classifier, generates disease classification categories for diseased pavement targets, and initializes the parameter tensor , It is for decoded features of each target; initialize the bounding box model for each target location, Represents the defect area locator, which generates a defect location bounding box for the defect road target and initializes the parameter tensor ; Initialize the parameter tensor for object queries ; Initialize the parameter tensor for positional encoding .

[0011] Preferably, the three-dimensional point cloud data of the sample is input into the CNN backbone network model to generate a low-resolution feature map ; Then the low-resolution feature map Through the Transformers encoder-decoder structure, we get A prediction set of diseased pavement targets.

[0012] Preferably, the three-dimensional point cloud data of the sample Input based on CNN backbone network model ,get

[0013] Said It is the convolutional layer part of the ResNet neural network. is the parameter tensor of the convolutional layer of the ResNet neural network, is the generated low-resolution feature map, where the number of channels of the feature map is , ;use indivual The convolution kernel Spliced , the feature map The dimension is reduced to the number of channels New high-level feature map ,Right now ;symbol Representation feature map Convolution operation between and convolution kernel; ; The feature map and positional encoding parameters Add them together, and then find the sum of the two tensors. The flat shape is straightened, thereby changing the shape , and transposed to The tensor of is used as the input of the Transformers encoder, denoted as ,Right now

[0014] here, is the input of the Transformers encoder; Through the Transformers encoder with multi-head self-attention mechanism, we get Feature encoding of the same shape , Right now: ;here , is the parameter tensor of the Transformer encoder module; and the parameter tensor of the target query volume Through the Transformers decoder with cross-attention mechanism, the predicted decoding features of N diseased road targets are obtained , Right now: , is the prediction decoding of N diseased road targets The tensor composed of is the parameter tensor of the Transformer decoder module, the parameters of the encoder and decoder Together they form the parameter tensor of the Transformer ;Will Pavement Disorder Classification and disease area locator The above model uses FFN feedforward neural network, and the output includes a set of predicted outputs of N disease classification types and disease location bounding boxes (output) ,Right now .

[0015] Preferably, according to the predicted output set ,calculate and the true label of the sample The set prediction loss between includes: first, using the Hungarian algorithm for bipartite graph matching, find a bipartite graph matching with the minimum loss as follows, that is, find a permutation of the sequence [1,2,...,N] , so that

[0016] here, represents the set of all possible permutations of the sequence [1,2,...,N] as elements, is the bipartite graph matching loss, for

[0017] in is the indicative function, that is, when hour, ;when hour, , Representing a collection The The prediction is Probability of class; is the intersection-over-union loss, expressed as:

[0018] in, express The corresponding ground-truth box and Generalized Intersection over Union (GIoU) ​​loss between corresponding prediction boxes; represents the vector 1-norm, where a vector The 1-norm of is defined as:

[0019] in is a hyperparameter; according to the optimal permutation obtained ,calculate and the true label The set prediction loss between is: .

[0020] Preferably, by optimizing and the true label The ensemble prediction loss between , obtain parameter estimates for the CNN backbone network model, Transformers encoder-decoder structure, pavement damage classification classifier, and damage area locator models; use the Adam algorithm in pytorch to optimize the loss , obtain the parameter estimates of each optimized model, expressed as: , , , , , , .

[0021] Preferably, the pre-processed three-dimensional point cloud data of the real road surface is input in the automatic detection stage, and the trained neural network module is used to detect the input three-dimensional point cloud data to be detected, and identify the damaged road surface targets therein, including the damage classification type and the damage location boundary box.

[0022] Preferably, the target 3D point cloud data x to be detected is input into the trained CNN backbone network model and Transformers encoder-decoder structure, and then the trained pavement disease classification classifier is used. and disease area locator , get a prediction set containing N disease classification types and disease location bounding boxes ,Right now , It is The vector of probability values ​​of the diseased road surface target belonging to all possible disease classification categories, The The positioning vector of the disease positioning bounding box; the predicted set of N disease classification types and disease positioning bounding boxes , generating descriptive information representing the road surface disease type, severity classification, and disease description information of the target location coordinates of the diseased road surface.

[0023] The present application also provides an automatic generation device for an intelligent road maintenance plan, which executes the above method, and includes: 3D laser sensing unit, which uses 3D laser scanning technology to obtain 3D point cloud data of the road surface; The defect analysis and detection unit processes and analyzes the collected 3D point cloud data using a neural network, and automatically detects defect description information indicating the road surface defect type, location, and severity level. The maintenance plan automatic generation unit automatically generates a pavement maintenance plan in response to the automatically detected disease description information.

[0024] Beneficial effects of the present invention: In the implementation of the technical solution of the present invention, the pavement maintenance technology that combines three-dimensional laser scanning with artificial intelligence algorithms can be widely used in the pavement maintenance of various infrastructures, and can be specifically applied in scenarios such as highway pavements, municipal road pavements, and airport pavements; by deploying three-dimensional laser scanning equipment on highways, municipal roads, or airport pavements, a comprehensive road surface scan is performed to collect three-dimensional point cloud data; the collected point cloud data is imported into the artificial intelligence algorithm to perform disease detection, classification, and evaluation, and generate a pavement condition report. Based on the pavement condition report and in combination with the maintenance knowledge base, a maintenance plan suitable for the infrastructure is automatically generated. The technology of the present application can adapt to the complexity of the urban environment, accurately and hierarchically detect tiny cracks and deformations on the pavement, formulate maintenance measures in a timely manner, realize efficient and accurate maintenance decision support, and improve the quality and efficiency of maintenance work. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 This is an overall flow chart of the method for automatically generating an intelligent pavement maintenance plan according to the present invention.

[0026] Figure 2 This is a framework diagram of the automatic generation device of the intelligent pavement maintenance solution of the present invention. DETAILED DESCRIPTION

[0027] The exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments described herein; on the contrary, these embodiments are provided to enable a more thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.

[0028] In the first aspect, the present application proposes a method for automatically generating an intelligent road maintenance plan, such as Figure 1 As shown, the following steps are included: S100: Use 3D laser scanning technology to obtain 3D point cloud data of the road surface.

[0029] Detection equipment that carries multiple sensors such as lidar can be used to collect three-dimensional point cloud data of the road surface through three-dimensional laser scanning.

[0030] The 3D point cloud data can be used to truly restore the pavement geometry and subtle changes. The 3D point cloud data captured by the 3D laser scanning technology can accurately measure and record the spatial coordinates of each point on the pavement, thereby constructing a 3D model of the pavement. In this way, the pavement geometry can be reproduced with high precision, including the accurate capture of subtle changes. Such data collection methods provide detailed and reliable basic information for subsequent pavement disease analysis and maintenance plan formulation.

[0031] S200: The collected three-dimensional point cloud data is processed and analyzed using a neural network to automatically detect the damage description information representing the type, location, and severity level of the road surface damage.

[0032] In implementing the technical solution of the present invention, one of the key steps is to perform in-depth processing and analysis of the collected 3D point cloud data. Specifically, this step involves importing the 3D point cloud data into a specially designed neural network artificial intelligence algorithm. The algorithm combines machine learning and deep learning techniques to process data efficiently and accurately.

[0033] The following is the detailed technical implementation of this step: S201: Data preprocessing: First, the collected 3D point cloud data is cleaned and filtered to remove noise and unnecessary points to improve data quality.

[0034] S202: For the three-dimensional point cloud data, a neural network module including a CNN backbone network model, a Transformers codec structure, a road surface disease classification classifier, and a disease area locator is used to perform a training phase, thereby automatically detecting disease description information representing the road surface disease type, location, and severity classification.

[0035] In step S202, the global feature extraction and analysis capabilities of the Transformers encoder-decoder structure are used to directly output a unique prediction box for road surface defects as well as the defect type and severity classification at one time, greatly simplifying the training and deployment of the model without requiring much manual intervention and prior knowledge.

[0036] The actual execution process of step S202 is divided into the training phase and the automatic detection phase.

[0037] During the training phase, a large number of 3D point cloud data samples of the road surface are collected, including positive samples consisting of 3D point cloud data of N types of diseased road surface targets of different types and severity. The positive samples are annotated with the bounding boxes corresponding to the diseased road surface targets. The samples also include negative samples consisting of 3D point cloud data of normal road surfaces. The CNN backbone network model extracts its feature map from the 3D point cloud data and straightens it into a feature vector of the 3D point cloud data. The feature vector is then output to the Transformers encoder-decoder structure, where the encoder is used to further learn global feature information. The decoder performs self-attention interactive decoding on the target query (object query) of N types of diseased road surface targets and the global feature vector output by the encoder, and can output the prediction boxes of N disease location bounding boxes in parallel. The prediction loss function between the prediction box and the bounding box corresponding to the diseased road surface target is calculated with the help of bipartite graph matching, and iterative training is achieved by optimizing the loss function.

[0038] The following is a more detailed description of the training process: Step S2021: construct a data set of the above samples; specifically, the samples of the three-dimensional point cloud data are represented as:

[0039] Among them, here is the sample size, The sample is 3D point cloud data, is the height of the 3D point cloud data, is the width of the 3D point cloud data; The sample The most 3D point cloud data The label is composed of the disease classification category and the positioning vector of the disease positioning bounding box; , The sample is Three-dimensional point cloud data The category label of the diseased road surface target (when the target is a 3D point cloud data background, the value is ), It is Three-dimensional point cloud data The 4D vector is composed of the standardized center point coordinates of the defect location bounding box of the defective pavement target and the height and width relative to the size of the 3D point cloud data.

[0040] Step S2022: Initialize the CNN backbone network model of the neural network module, the Transformers codec structure, and various parameters of the pavement disease grading classifier and the disease area locator.

[0041] Specifically, the CNN backbone network model uses the convolutional layer after removing the classification layer from the pre-trained ResNet neural network. Represents the convolutional layer part of the CNN backbone network model, It is a parameter tensor consisting of all parameters of the pre-trained ResNet convolutional layer. A sample of the above three-dimensional point cloud data input into the CNN backbone network model; Represents the CNN backbone network model used to reduce the resolution of the feature map of the full convolutional network, initialize the parameter tensor of the model ,in is the low-resolution feature map generated by the fully convolutional network; Represents the Transformers encoder-decoder structure, which performs the operation of extracting global features and performing self-attention interaction with the target query volume of N types of diseased road targets, initializing its parameter tensor .

[0042] Represents a pavement disease classification classifier, generates disease classification categories for diseased pavement targets, and initializes the parameter tensor , It is for decoded features of each target; initialize the bounding box model for each target location, Represents the defect area locator, which generates a defect location bounding box for the defect road target and initializes the parameter tensor ; Initialize the parameter tensor for object queries ; Initialize the parameter tensor for positional encoding .

[0043] Step S2023: Input the sample's 3D point cloud data into the CNN backbone network model to generate a low-resolution feature map ; Then the low-resolution feature map Through the Transformers encoder-decoder structure, we get A prediction set of diseased pavement targets.

[0044] The three-dimensional point cloud data of the sample Input based on CNN backbone network model ,get

[0045] As mentioned above, here, It is the convolutional layer part of the ResNet neural network. is the parameter tensor of the convolutional layer of the ResNet neural network. is the generated low-resolution feature map, where the number of channels of the feature map is , .

[0046] use indivual The convolution kernel concatenated , the feature map The dimension is reduced to the number of channels New high-level feature map ,Right now

[0047] symbol Representation feature map Convolution operation between and convolution kernel; .

[0048] Next, the feature map and positional encoding parameters Add them together, and then find the sum of the two tensors. The plane shape is straightened, thereby changing the shape (expressed as reshape) , and transposed to The tensor of is used as the input of the Transformers encoder, denoted as ,Right now

[0049] here, is the input of the Transformers encoder; Through the Transformers encoder with multi-head self-attention mechanism, we get Feature encoding of the same shape , Right now , here , is the parameter tensor of the Transformer encoder module; and the parameter tensor of the target query volume Through the Transformers decoder with cross-attention mechanism, the predicted decoding features of N diseased road targets are obtained , Right now: , is the prediction decoding of N diseased road targets The tensor composed of is the parameter tensor of the Transformer decoder module, the parameters of the encoder and decoder Together they form the parameter tensor of the Transformer ; Will Pavement Disorder Classification and disease area locator The above model uses FFN feedforward neural network, and the output includes a set of predicted outputs of N disease classification types and disease location bounding boxes (output) , Right now

[0050] Step S2024: Output set based on prediction ,calculate and the true label of the sample The ensemble prediction loss between .

[0051] First, we use the Hungarian algorithm to find a bipartite graph matching with the minimum loss, that is, to find a permutation of the sequence [1,2,...,N] , so that

[0052] here, represents the set of all possible permutations of the sequence [1,2,...,N] as elements, is the bipartite graph matching loss, for

[0053] in is the indicative function, that is, when hour, ;when hour, , Representing a collection The The prediction is Probability of class; is the intersection-over-union loss, expressed as:

[0054] in, express The corresponding ground-truth box and Generalized Intersection over Union (GIoU) ​​loss between corresponding prediction boxes; Represents the vector 1-norm, where a vector The 1-norm of is defined as:

[0055] is a hyperparameter.

[0056] Then, according to the optimal replacement ,calculate and the true label The set prediction loss between is:

[0057] Step S2025: By optimizing and the true label The ensemble prediction loss between , obtain the parameter estimates of the CNN backbone network model, Transformers encoder-decoder structure, pavement damage classification classifier, and damage area locator models. You can use the Adam algorithm in pytorch to optimize the loss , obtain the parameter estimates of each optimized model, expressed as: , , , , , , .

[0058] In the automatic detection stage of step S202, the three-dimensional point cloud data of the real road surface collected and preprocessed by S100 is input, and the trained neural network module is used to detect the input three-dimensional point cloud data to be detected, and identify the damaged road surface targets therein, including the damage classification type and the damage location boundary box.

[0059] Specifically, step S2026: input the target 3D point cloud data x to be detected into the trained CNN backbone network model and Transformers encoder-decoder structure, and then pass the trained pavement disease classification classifier and disease area locator , get a prediction set containing N disease classification types and disease location bounding boxes , Right now , It is The vector of probability values ​​of the diseased road surface target belonging to all possible disease classification categories, The The localization vector of the disease localization bounding box.

[0060] Then, step S2027: based on the predicted set of N disease classification types and disease location bounding boxes , generate descriptive information representing the road surface disease type, severity classification, and disease description information of the target location coordinates of the diseased road surface, that is, transform the prediction set represented by the above vectors into disease description information containing search terms and drawing coordinate information that can be used as knowledge base query statements.

[0061] Back to Figure 1 , step S300 of the method: responding to the automatically detected disease description information, automatically generating a road maintenance plan. The maintenance plan includes detailed information such as maintenance content, maintenance methods, disposal drawings, etc., and can be dynamically adjusted and optimized according to actual conditions.

[0062] Specifically, first, according to the description information of the disease, the maintenance measures that match the disease type and severity classification are searched in the knowledge base; the content of the maintenance measures includes but is not limited to the type of materials required, the selection of maintenance processes, the configuration of the construction team, etc.; based on the matching results, a preliminary maintenance plan is generated, including: the order of maintenance, the construction period, and the equipment used; then, considering the actual environmental factors and budget constraints, the feasibility of the preliminary maintenance plan is evaluated. If the feasibility does not meet the standard (for example, the budget exceeds the standard), the knowledge base is queried again for automatic optimization; finally, the disposal drawings are automatically generated based on the location of the disease and the maintenance content. The drawings will mark key information such as the maintenance area, material distribution, and construction steps in detail. In the actual maintenance process, dynamic adjustment is supported, and the maintenance plan can be quickly adjusted according to the real-time feedback from the construction site; for example, if a new disease is found on site or the original disease is more serious than expected, the system can immediately provide the knowledge base to update the maintenance content and methods.

[0063] This application proposes an automatic generation system for intelligent pavement maintenance plans, such as Figure 2As shown, including: 3D laser sensing unit, which uses 3D laser scanning technology to obtain 3D point cloud data of the road surface; The defect analysis and detection unit processes and analyzes the collected 3D point cloud data using a neural network, and automatically detects defect description information indicating the road surface defect type, location, and severity level. The maintenance plan automatic generation unit automatically generates a pavement maintenance plan in response to the automatically detected disease description information.

[0064] In the technical solution of the present invention, the pavement maintenance technology that combines three-dimensional laser scanning with artificial intelligence algorithms can be widely used in the pavement maintenance of various infrastructures, and can be specifically applied in scenarios such as highway pavements, municipal road pavements, and airport pavements; by deploying three-dimensional laser scanning equipment on highways, municipal roads, or airport pavements, a comprehensive road surface scan is performed to collect three-dimensional point cloud data; the collected point cloud data is imported into the artificial intelligence algorithm to perform disease detection, classification, and evaluation, and generate a pavement condition report. Based on the pavement condition report and in combination with the maintenance knowledge base, a maintenance plan suitable for the infrastructure is automatically generated. The technology of this application can adapt to the complexity of the urban environment, accurately and hierarchically detect tiny cracks and deformations on the pavement, formulate maintenance measures in a timely manner, achieve efficient and accurate maintenance decision support, and improve the quality and efficiency of maintenance work.

[0065] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0066] The above are only preferred implementations of the present invention. It should be pointed out that a number of modifications and improved technical solutions made by those skilled in the art without departing from the technical solution should also be deemed to fall within the scope of protection required by the claims.

Claims

1. A method for automatically generating an intelligent pavement maintenance plan, characterized in that: The following steps are involved: Use 3D laser scanning technology to obtain 3D point cloud data of the road surface, measure and record the spatial coordinates of each point on the road surface, and thus construct a 3D model of the road surface; The collected 3D point cloud data is processed and analyzed using a neural network to automatically detect the road surface damage type, location, and severity level description information. For the 3D point cloud data, a neural network module including a CNN backbone network model, a Transformers codec structure, a road surface damage level classifier, and a damage area locator is used to perform a training phase, thereby automatically detecting the road surface damage type, location, and severity level description information. In response to the automatically detected disease description information, a pavement maintenance plan is automatically generated; the maintenance plan includes detailed information on maintenance content, maintenance methods, and disposal drawings, and can be dynamically adjusted and optimized according to actual conditions.

2. The method for automatically generating an intelligent pavement maintenance plan according to claim 1, characterized in that: In the training phase, a large number of three-dimensional point cloud data samples of the road surface are collected, including positive samples consisting of three-dimensional point cloud data of N types of diseased road surface targets of different types and severity. The positive samples are marked with bounding boxes corresponding to the diseased road surface targets, and the samples also include negative samples consisting of three-dimensional point cloud data of normal road surfaces; the CNN backbone network model extracts its feature map from the three-dimensional point cloud data and straightens it into a feature vector of the three-dimensional point cloud data; then the feature vector is output to the Transformers encoder-decoder structure, wherein the encoder is used to further learn global feature information, and the decoder performs self-attention interactive decoding on the target query volume of the given N types of diseased road surface targets and the global feature vector output by the encoder, and can output the prediction boxes of N disease positioning bounding boxes in parallel; with the help of bipartite graph matching, the prediction loss between the prediction box and the bounding box corresponding to the diseased road surface target is calculated, and iterative training is achieved by optimizing the loss function.

3. The method for automatically generating an intelligent pavement maintenance plan according to claim 2, characterized in that: A data set is constructed using the three-dimensional point cloud data sample; the sample is represented as: Among them, here is the sample size, The sample is 3D point cloud data, is the height of the 3D point cloud data, is the width of the 3D point cloud data; The sample The most 3D point cloud data The label is composed of the disease classification category and the positioning vector of the disease positioning bounding box; , The sample is Three-dimensional point cloud data The category labels of the diseased road surface targets, It is Three-dimensional point cloud data The 4D vector is composed of the standardized center point coordinates of the defect location bounding box of the defective pavement target and the height and width relative to the size of the 3D point cloud data.

4. The method for automatically generating an intelligent pavement maintenance plan according to claim 3, characterized in that: Initialize the CNN backbone network model, Transformers codec structure, and various parameters of the pavement disease classification classifier and disease area locator of the neural network module; wherein the CNN backbone network model uses the convolutional layer part after removing the classification layer from the pre-trained ResNet neural network, Represents the convolutional layer part of the CNN backbone network model, It is a parameter tensor consisting of all parameters of the pre-trained ResNet convolutional layer. A sample of the above three-dimensional point cloud data input into the CNN backbone network model; Represents the CNN backbone network model used to reduce the resolution of the feature map of the full convolutional network, initialize the parameter tensor of the model ,in is the low-resolution feature map generated by the fully convolutional network; Represents the Transformers encoder-decoder structure, which performs the operation of extracting global features and performing self-attention interaction with the target query volume of N types of diseased road targets, initializing its parameter tensor ; Represents a pavement disease classification classifier, generates disease classification categories for diseased pavement targets, and initializes the parameter tensor , It is for decoded features of each target; initialize the bounding box model for each target location, Represents the defect area locator, which generates a defect location bounding box for the defect road target and initializes the parameter tensor ; Initialize the parameter tensor for object queries ; Initialize the parameter tensor for positional encoding .

5. The method for automatically generating an intelligent pavement maintenance plan according to claim 4, characterized in that: Input the sample's 3D point cloud data into the CNN backbone network model to generate a low-resolution feature map ; Then the low-resolution feature map Through the Transformers encoder-decoder structure, we get A prediction set of diseased pavement targets.

6. The method for automatically generating an intelligent pavement maintenance plan according to claim 5, characterized in that: The three-dimensional point cloud data of the sample Input based on CNN backbone network model ,get Said It is the convolutional layer part of the ResNet neural network. is the parameter tensor of the convolutional layer of the ResNet neural network, is the generated low-resolution feature map, where the number of channels of the feature map is , ;use indivual The convolution kernel Spliced , the feature map The dimension is reduced to the number of channels New high-level feature map ,Right now ;symbol Representation feature map Convolution operation between and convolution kernel; ; The feature map and positional encoding parameters Add them together, and then take the sum of the two into the tensor The flat shape is straightened, thereby changing the shape , and transposed to The tensor of is used as the input of the Transformers encoder, denoted as ,Right now here, is the input of the Transformers encoder; Through the Transformers encoder with multi-head self-attention mechanism, we get Feature encoding of the same shape ,Right now: ;here , is the parameter tensor of the Transformer encoder module; and the parameter tensor of the target query volume Through the Transformers decoder with cross-attention mechanism, the predicted decoding features of N diseased road targets are obtained , Right now: , is the prediction decoding of N diseased road targets The tensor composed of is the parameter tensor of the Transformer decoder module, the parameters of the encoder and decoder Together they form the parameter tensor of the Transformer ;Will Pavement Disorder Classification and disease area locator The above model uses FFN feedforward neural network, and the output includes a set of predicted outputs of N disease classification types and disease location bounding boxes (output) , Right now .

7. The method for automatically generating an intelligent pavement maintenance plan according to claim 6, characterized in that: Output set based on prediction ,calculate and the true label of the sample The set prediction loss between includes: first, using the Hungarian algorithm for bipartite graph matching, find a bipartite graph matching with the minimum loss as follows, that is, find a permutation of the sequence [1,2,...,N] , so that here, represents the set of all possible permutations of the sequence [1,2,...,N] as elements, is the bipartite graph matching loss, for in is the indicative function, that is, when hour, ;when hour, , Representing a collection The The prediction is Probability of class; is the intersection-over-union loss, expressed as: in, express The corresponding ground-truth box and Generalized Intersection over Union (GIoU) ​​loss between corresponding prediction boxes; Represents the vector 1-norm, where a vector The 1-norm of is defined as: in is a hyperparameter; according to the optimal permutation obtained ,calculate and the true label The set prediction loss between is: 。 8. The method for automatically generating an intelligent pavement maintenance plan according to claim 7, characterized in that: By optimizing and the true label The ensemble prediction loss between , obtain parameter estimates for the CNN backbone network model, Transformers encoder-decoder structure, pavement damage classification classifier, and damage area locator models; use the Adam algorithm in pytorch to optimize the loss , obtain the parameter estimates of each optimized model, expressed as: , , , , , , .

9. The method for automatically generating an intelligent pavement maintenance plan according to claim 8, characterized in that: In the automatic detection stage, the pre-processed 3D point cloud data of the real road surface is input, and the trained neural network module is used to detect the input 3D point cloud data to be detected, and the damaged road surface targets are identified, including the damage classification type and the damage location boundary box; the 3D point cloud data x of the target to be detected is input into the trained CNN backbone network model and Transformers encoder-decoder structure, and then the trained road surface damage classification classifier is used to classify the road surface damage. and disease area locator , get a prediction set containing N disease classification types and disease location bounding boxes ,Right now , It is The vector of probability values ​​of the diseased road surface target belonging to all possible disease classification categories, The The positioning vector of the disease positioning bounding box; the predicted set of N disease classification types and disease positioning bounding boxes , generating descriptive information representing the road surface disease type, severity classification, and disease description information of the target location coordinates of the diseased road surface.

10. An automatic generation device for intelligent pavement maintenance plan, used to execute the method according to any one of claims 1 to 9, characterized in that: include: 3D laser sensing unit, which uses 3D laser scanning technology to obtain 3D point cloud data of the road surface; The defect analysis and detection unit processes and analyzes the collected 3D point cloud data using a neural network, and automatically detects defect description information indicating the road surface defect type, location, and severity level. The maintenance plan automatic generation unit automatically generates a pavement maintenance plan in response to the automatically detected disease description information.

Images