Method and device for detecting pavement disease of sidewalk, and storage medium

Abstract

The application discloses a slow lane pavement disease detection method and device and a storage medium, relates to the technical field of artificial intelligence, and discloses a slow lane pavement disease detection method, which comprises the following steps: acquiring a slow lane pavement image and pavement height data; analyzing the pavement image and the pavement height data to determine whether the slow lane has pavement diseases; if the slow lane has pavement diseases, constructing a first prompt text according to the pavement image, the pavement height data and pavement disease detection requirements; and inputting the first prompt text into a pavement disease detection model to obtain the pavement disease type of the slow lane, wherein the pavement disease detection model is obtained by fine-tuning a preset pavement disease type sample, a preset pavement image sample and a preset pavement height data sample, so that the accuracy of the pavement disease detection result is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to a slow lane pavement disease detection method, device and storage medium. BACKGROUND

[0002] As an important part of urban transportation system, the safety and comfort of slow lane are increasingly valued. However, due to the dual influence of natural factors and human activities, various pavement diseases such as cracks, potholes, ruts and peeling may occur in slow lane during use. These diseases not only affect the appearance and service life of the road, but also pose a potential threat to the safety of pedestrians and non-motor vehicles. Therefore, timely and accurate pavement disease detection is of great significance to ensure the quality and safety of slow lane. At present, slow lane pavement disease detection mainly adopts manual inspection method. This traditional detection method relies on the on-site investigation of maintenance personnel, who records the disease conditions on the road surface by observing with the naked eye while driving slowly or walking on the road. However, manual inspection largely depends on the experience and subjective judgment of maintenance personnel, and there may be individual differences between different maintenance personnel, leading to different judgments of the same disease, thereby affecting the accuracy of pavement disease detection results. SUMMARY

[0003] The main purpose of the present application is to provide a slow lane pavement disease detection method, device and storage medium, aiming to solve the technical problem of low accuracy of slow lane pavement disease detection by manual detection.

[0004] To achieve the above purpose, the present application provides a slow lane pavement disease detection method, comprising:

[0005] obtaining pavement images and pavement height data of the slow lane;

[0006] analyzing the pavement images and pavement height data to determine whether the slow lane has pavement diseases;

[0007] if the slow lane has pavement diseases, constructing a first prompt text according to the pavement images, pavement height data and pavement disease detection requirements;

[0008] inputting the first prompt text into a pavement disease detection model to obtain the pavement disease type of the slow lane, wherein the pavement disease detection model is obtained by fine-tuning a preset pavement disease type sample, a preset pavement image sample and a preset pavement height data sample.

[0009] In an embodiment, analyzing the pavement images and pavement height data to determine whether the slow lane has pavement diseases comprises:

[0010] analyzing the road surface image and the road surface height data to obtain a road surface analysis result;

[0011] determining, according to the road surface analysis result, whether the sidewalk has a road surface disease;

[0012] The road surface analysis result includes whether the road surface has a crack and whether the road surface is flat.

[0013] In an embodiment, analyzing the road surface image and the road surface height data to obtain the road surface analysis result includes:

[0014] performing image analysis on the road surface image corresponding to the target area of the sidewalk to determine whether the target area has a crack, and performing flatness analysis on the road surface height data of the target area to determine whether the target area is flat;

[0015] determining, according to the road surface analysis result, whether the sidewalk has a road surface disease:

[0016] If the target area has a crack and / or the target area is not flat, it is determined that the sidewalk has a road surface disease.

[0017] Or, if the target area does not have a crack and the target area is flat, it is determined that the sidewalk does not have a road surface disease.

[0018] In an embodiment, performing flatness analysis on the road surface height data of the target area to determine whether the target area is flat includes:

[0019] performing feature extraction on the road surface height data to obtain height features of the target area;

[0020] fitting the road surface height data based on a collection time of the road surface height data to obtain a road surface height change trend of the target area;

[0021] If the road surface height change trend of the target area conforms to a preset change trend, it is determined that the target area is flat.

[0022] Or, if the road surface height change trend of the target area does not conform to the preset change trend, it is determined that the target area is not flat.

[0023] In an embodiment, inputting the first prompt text into the road surface disease detection model to obtain the road surface disease type of the sidewalk includes:

[0024] inputting the first prompt text into the road surface disease detection model to perform vectorization representation on the road surface image and the road surface height data in the first prompt text to obtain a feature vector corresponding to the road surface image and a feature vector corresponding to the road surface height data;

[0025] The feature vectors corresponding to the road surface images and the feature vectors corresponding to the road surface height data are comprehensively represented to obtain a comprehensive vector representation;

[0026] The similarity between the comprehensive vector representation and each preset vector representation in a database is determined, wherein the database stores preset comprehensive vector representations corresponding to different road surface disease types, and the preset comprehensive vector representations are determined according to preset feature vectors corresponding to preset road surface images and preset feature vectors corresponding to preset road surface height data;

[0027] According to the similarity, the road surface disease type of the slow lane is determined.

[0028] In an embodiment, after inputting the first prompt text into the road surface disease detection model to obtain the road surface disease type of the slow lane, the method further comprises:

[0029] Obtaining the road surface disease distribution of the slow lane;

[0030] Determining a small sample example set corresponding to the road surface disease type and the road surface disease distribution, wherein the small sample example set includes a plurality of preset sample examples, and the preset sample examples include preset road surface disease type samples, preset road surface disease distributions, and corresponding preset treatment suggestion samples;

[0031] According to the road surface disease type, the road surface disease distribution, the small sample example set, and the treatment requirement of the road surface disease type, a second prompt text is constructed;

[0032] Inputting the second prompt text into a road surface disease treatment model to obtain a treatment suggestion for the road surface disease type and the road surface disease distribution, wherein the road surface disease treatment model is fine-tuned according to the preset road surface disease type samples, the preset road surface disease distributions, and the corresponding preset treatment suggestion samples.

[0033] In an embodiment, determining the small sample example set corresponding to the road surface disease type and the road surface disease distribution comprises:

[0034] Vector representing the road surface disease type to obtain a first feature vector of the road surface disease type, and vector representing the road surface disease distribution to obtain a second feature vector of the road surface disease distribution;

[0035] Retrieving a first preset sample example matching the first feature vector and a second preset sample example matching the second feature vector from an example library;

[0036] Merging the first preset sample example and the second preset sample example to obtain a merged preset sample example;

[0037] Performing similarity sorting and filtering on the merged preset sample example to obtain the small sample example set.

[0038] In an embodiment, the acquiring the pavement disease distribution of the slow lane comprises:

[0039] acquiring positioning data of the slow lane where the pavement disease exists;

[0040] determining the pavement disease distribution according to the positioning data where the pavement disease exists.

[0041] In addition, to achieve the above-mentioned purpose, the present application also proposes a pavement disease detection device for a slow lane, which comprises a memory, a processor and a computer program stored on the memory and executable on the processor, and the computer program is configured to implement the steps of the pavement disease detection method for a slow lane as described above.

[0042] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer readable storage medium, and the storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the pavement disease detection method for a slow lane as described above.

[0043] Compared with the related art, the present application first acquires the pavement image and the pavement height data of the slow lane, which describes the pavement from two dimensions of visual form and physical structure respectively. The image data can capture the visual features such as color and texture of the pavement, while the height data reflects the physical state such as undulation and deformation of the pavement. Through these two kinds of data, the detection process can more comprehensively and meticulously cover various disease characteristics of the pavement, reducing the detection blind area caused by a single data type. Then, the pavement image and the pavement height data are analyzed to determine whether the slow lane has pavement disease, realizing the preliminary identification of the existence of pavement disease. Compared with manual inspection, intelligent analysis can more accurately and quickly identify disease characteristics, reducing misjudgment and omissions caused by human factors. When detecting the pavement disease, a prompt text containing detailed information is automatically generated according to the pavement image, the height data and the detection requirement. This customized text not only facilitates the maintenance personnel to quickly understand the disease situation, but also provides targeted guidance for subsequent detection and repair work. The pavement disease detection model is fine-tuned based on the preset pavement disease type sample, the pavement image sample and the pavement height data sample. This fine-tuning process enables the model to more accurately adapt to the pavement disease characteristics of a specific area, and improves the accuracy of the pavement disease detection result. BRIEF DESCRIPTION OF DRAWINGS

[0044] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the application.

[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the accompanying drawings needed to be used in the embodiments or prior art description will be briefly introduced as follows. Obviously, the drawings are only for the purpose of illustrating the embodiments of the present application, and for those of ordinary skill in the art, other drawings can also be obtained based on these drawings without any creative effort.

[0046] Figure 1 A flowchart for a process in some embodiments of the present application;

[0047] Figure 2 A flowchart for a detailed process of step S20 in some embodiments of the present application;

[0048] Figure 3 A flowchart for a detailed process of step S211 in some embodiments of the present application;

[0049] Figure 4 A flowchart for a detailed process of step S40 in some embodiments of the present application;

[0050] Figure 5 Another flowchart for a process in some embodiments of the present application;

[0051] Figure 6 A flowchart for a detailed process of step S120 in some embodiments of the present application;

[0052] Figure 7 A structural diagram of a road surface disease detection device for a slow lane in some embodiments of the present application.

[0053] The purposes, functional features and advantages of the present application will be further illustrated with reference to the accompanying drawings in conjunction with the embodiments. DETAILED DESCRIPTION

[0054] It should be understood that the specific embodiments described herein are merely intended to explain the technical solutions of the present application, and are not intended to limit the present application.

[0055] In order to better understand the technical solutions of the present application, the following will be described in detail in conjunction with the accompanying drawings and specific embodiments.

[0056] Currently, slow-moving roads, as a crucial component of urban transportation systems, are receiving increasing attention for their safety and comfort. However, due to both natural factors and human activity, slow-moving roads can develop various pavement defects, such as cracks, potholes, rutting, and peeling. These defects not only impact the road's aesthetics and service life but also pose a potential threat to the safety of pedestrians and non-motorized vehicles. Therefore, timely and accurate pavement defect detection is crucial for ensuring the quality and safety of slow-moving roads. Pavement defect detection on slow-moving roads primarily relies on manual inspections. This traditional method relies on on-site inspections by maintenance personnel, who visually record the condition of the road surface from a slow-moving vehicle or on foot. However, manual inspections rely heavily on the maintenance personnel's experience and subjective judgment, which can vary between individual maintenance personnel, leading to varying judgments for the same defect and impacting the accuracy of pavement defect detection results.

[0057] In response to the above problems, the present application proposes a method for detecting pavement defects on slow lanes, and the main technical solutions include: obtaining a pavement image and pavement height data of the slow lane; analyzing the pavement image and the pavement height data to determine whether there are pavement defects on the slow lane; if there are pavement defects on the slow lane, constructing a first prompt text based on the pavement image, the pavement height data and pavement defect detection requirements; inputting the first prompt text into a pavement defect detection model to obtain the pavement defect type of the slow lane, wherein the pavement disease detection model is obtained by fine-tuning the preset pavement disease type sample, the preset pavement image sample and the preset pavement height data sample.

[0058] Compared with the related art, the present application first acquires the road surface image and the road surface height data of the slow lane, which respectively describe the road surface from two dimensions of visual form and physical structure. The image data can capture the visual features such as color and texture of the road surface, and the height data reflects the physical state such as undulation and deformation of the road surface. Through the two kinds of data, the detection process can more comprehensively and carefully cover various disease characteristics of the road surface, reducing the detection blind area caused by a single data type. Then, the road surface image and the road surface height data are analyzed to determine whether the slow lane has road surface diseases, realizing preliminary identification of the existence of road surface diseases. Compared with manual inspection, intelligent analysis can more accurately and quickly identify disease characteristics, reducing misjudgment and omissions caused by human factors. When detecting road surface diseases, prompt text containing detailed information is automatically generated according to the road surface image, height data and detection requirements. This customized text not only facilitates maintenance personnel to quickly understand the disease situation, but also provides targeted guidance for subsequent detection and repair work. The road surface disease detection model is fine-tuned based on preset road surface disease type samples, road surface image samples and road surface height data samples. This fine-tuning process enables the model to more accurately adapt to the road surface disease characteristics of a specific area, improving the accuracy of road surface disease detection results.

[0059] It should be noted that the execution subject of the present embodiment is a slow lane road surface disease detection device capable of realizing the above functions. The execution subject of the present application can also be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc.

[0060] In an application scenario, the slow lane road surface disease detection device is movable, which can be a bicycle or an electric bicycle. The detection module is mounted on the bicycle or electric bicycle to collect road surface images and road surface height data of the slow lane, and the processing module is mounted to perform road surface disease detection on the collected road surface images and road surface height data.

[0061] In another application scenario, it can be connected with the slow lane road surface disease detection device described above, receive the road surface image and the road surface height data collected by the slow lane road surface disease detection device, and perform road surface disease detection on the collected road surface image and road surface height data.

[0062] In another application scenario, the road surface image and the road surface height data of the slow lane can be collected by a slow lane road surface disease detection device, and the road surface image and the road surface height data of the slow lane are preliminarily analyzed to detect whether the slow lane has a road surface disease. If there is a road surface disease, the specific and accurate analysis of the type of the road surface disease and the treatment suggestion of the type of the road surface disease are performed by a computing service device. In this way, the problem that the determined detection result of the road surface disease of the slow lane is inaccurate due to insufficient computing power of the slow lane road surface disease detection device is avoided, the computing power of the slow lane road surface disease detection device is reduced, the slow lane road surface disease detection device can have more computing power for detection work, and the accuracy of the detection result of the road surface disease of the slow lane is improved.

[0063] Based on this, the embodiment of the present application provides a slow lane road surface disease detection method, referring to Figure 1 , Figure 1 is a flowchart of some embodiments of the slow lane road surface disease detection method of the present application.

[0064] In this embodiment, the slow lane road surface disease detection method comprises steps S10-S40:

[0065] Step S10, obtaining the road surface image and the road surface height data of the slow lane.

[0066] The slow lane is also called a non-motor vehicle lane or a bicycle lane.

[0067] The road surface height data refers to the vertical distance or elevation information of each point on the surface of the slow lane relative to a certain reference level.

[0068] The road surface image and the road surface height data of the slow lane can be the road surface image and the road surface height data of the entire road of the slow lane, or the road surface image and the road surface height data of a certain target area of the slow lane.

[0069] In a feasible embodiment, an image collection sensor and a gyroscope are mounted on a slow lane road surface disease detection device, the slow lane road surface disease detection device moves along the slow lane, and the road surface image of the slow lane is collected by the image collection sensor mounted thereon, and at the same time, the road surface height data of the slow lane is collected by the gyroscope mounted thereon.

[0070] In another possible implementation, the image collection sensor and the gyroscope are calibrated, and the calibrated image collection sensor is used to collect the pavement image of the slow lane, and the calibrated gyroscope is used to collect the pavement height data of the slow lane, so as to improve the accuracy of the collected pavement image and pavement height data of the slow lane. The working parameters of the image collection sensor and the gyroscope can also be adjusted according to different road conditions to meet the data collection requirements in different road scenes.

[0071] In another possible implementation, the pavement image and the pavement height data of the slow lane can be collected in real time, or the pavement image and the pavement height data of the slow lane can be collected when a collection instruction is received.

[0072] In step S20, the pavement image and the pavement height data are analyzed to determine whether the slow lane has a pavement disease.

[0073] The pavement disease refers to various damages, deformations and other defects that occur on the cement or asphalt pavement after a period of use. The pavement disease includes but is not limited to cracks, potholes, ruts, loose, subsidence, bump at bridge head and culvert top, surface damage, etc.

[0074] In a possible implementation, the pavement image and the pavement height data can be fused to form more complete and accurate multi-source pavement information. The pavement height data can be used to identify pavement diseases with depth information such as potholes and subsidence. The pavement image can be used to identify the pavement surface conditions such as cracks and surface damage. By comprehensively analyzing the multi-source pavement information, it can be more accurately determined whether there is a pavement disease.

[0075] In step S30, if the slow lane has a pavement disease, a first prompt text is constructed according to the pavement image, the pavement height data and the pavement disease detection requirement.

[0076] The pavement disease detection requirement is proposed by the maintenance personnel, and is used to guide the pavement disease detection model to identify the pavement disease type.

[0077] The first prompt information aims to provide sufficient context for the pavement disease detection model, so that the pavement disease detection model can accurately understand the task to be performed and identify the pavement disease type accordingly. The first prompt information of the present application at least includes the pavement image, the pavement height data and the pavement disease detection requirement, and can also include other contents.

[0078] In an implementable embodiment, the road surface image, the road surface height data and the road disease detection requirement can be combined to form the first prompt text. Specifically, annotations can be added at the beginning or end of the road surface image and the road surface height data to explain the road disease detection requirement, and a specific format or template can be used to organize the road surface image, the road surface height data and the road disease detection requirement so that the road disease detection model can easily parse. In this process, the first prompt text can also be appropriately formatted according to the requirements of the road disease detection model used, which can include adjusting the indentation of the road surface image and the road surface height data, adding necessary annotations or markers, etc. In addition, the first prompt text can be verified and tested before being input into the road disease detection model, including checking the clarity of the road surface image and whether there is interference data in the road surface height data to ensure its accuracy and integrity.

[0079] In step S40, the first prompt text is input into the road disease detection model to obtain the road disease type of the slow lane, wherein the road disease detection model is fine-tuned according to the preset road disease type sample, the preset road surface image sample and the preset road surface height data sample.

[0080] The road disease detection model is used to realize the road disease type of the slow lane.

[0081] The road disease detection model can be a large language model, which has natural language understanding ability and can extract information from the input irregular natural language, i.e., the first prompt text of the present application, and actively ask questions to the user when information is missing. The core technology of the large language model includes pre-training and fine-tuning:

[0082] Pre-training: In the pre-training stage, the large language model will use large-scale text data sets for training to learn the patterns, structures and semantic information of the language.

[0083] Fine-tuning: In the fine-tuning stage, the pre-trained large language model is applied to a specific task or data set, and the model parameters are adjusted through supervised learning to adapt to the requirements of the specific task.

[0084] It should be noted that the road disease detection model used in the present application can be a general large language model or a pre-trained special large language model, and the pre-trained special large language model is preferred, which has strong natural language processing ability and certain professional knowledge reserve to accurately understand the first prompt text and accurately identify the road disease type of the slow lane.

[0085] The embodiment of the application first acquires the road surface image and the road surface height data of the slow lane. The two types of data describe the road surface from two dimensions of visual form and physical structure. The image data can capture the visual features of the road surface such as color and texture, while the height data reflects the physical state of the road surface such as undulation and deformation. Through the two types of data, the detection process can more comprehensively and meticulously cover various disease characteristics of the road surface, reducing the detection blind area caused by a single data type. Then, the road surface image and the road surface height data are analyzed to determine whether the slow lane has road surface diseases, realizing preliminary identification of the existence of road surface diseases. Compared with manual inspection, intelligent analysis can more accurately and quickly identify disease characteristics, reducing misjudgment and omissions caused by human factors. When detecting road surface diseases, a prompt text containing detailed information is automatically generated according to the road surface image, height data and detection requirements. This customized text not only facilitates maintenance personnel to quickly understand the disease situation, but also provides targeted guidance for subsequent detection and repair work. The road surface disease detection model is fine-tuned based on preset road surface disease type samples, road surface image samples and road surface height data samples. This fine-tuning process enables the model to more accurately adapt to the road surface disease characteristics of a specific area, improving the accuracy of road surface disease detection results.

[0086] In some embodiments of the present application, with reference to Figure 2 analyzing the road surface image and the road surface height data to determine whether the slow lane has road surface diseases includes:

[0087] In step S21, the road surface image and the road surface height data are analyzed to obtain a road surface analysis result.

[0088] The road surface analysis result includes whether the road surface has cracks and whether the road surface is flat, or others.

[0089] In a feasible implementation, a high-definition camera or a drone equipped with a high-definition camera can be used to perform all-around and high-resolution image acquisition of the slow lane, ensuring that the image is clear and free of blur, and can accurately reflect the real condition of the road surface. The collected image is preprocessed, including denoising, contrast enhancement, color correction, etc., to improve the image quality; the image is converted into a format suitable for subsequent processing, such as JPEG, PNG, etc. Image processing algorithms such as edge detection and morphological processing are used to automatically identify cracks in the image, and by setting appropriate threshold values and parameters, the cracks are separated from the image and their position, shape and size are labeled. The road surface texture and color change in the image are analyzed, combined with the characteristics of the road surface material, to evaluate the flatness of the road surface. Texture analysis, gray level co-occurrence matrix, etc. can be used to extract the texture features of the road surface, and then judge the flatness of the road surface.

[0090] In another possible implementation, high-precision measurement equipment such as a laser scanner or radar can be used to collect road surface height data of the slow lane, ensuring the accuracy and completeness of data collection and covering the entire slow lane area. The collected data is preprocessed, including denoising, filtering, interpolation, and the like, to improve the accuracy and reliability of the data, and the data is converted into a format suitable for subsequent analysis, such as CSV, Excel, and the like. The change in road surface height data is analyzed, the elevation difference between adjacent points is calculated, and whether the road surface has diseases such as settlement and deformation is determined according to the size and distribution of the elevation difference. Using the road surface height data, the flatness index of the road surface, such as the international flatness index and the standard deviation, is calculated, and whether the flatness of the road surface meets the requirements is determined by comparing the flatness index with a preset threshold.

[0091] In another possible implementation, image analysis is performed on the road surface image corresponding to a target area of the slow lane to determine whether the target area has cracks, and flatness analysis is performed on the road surface height data of the target area to determine whether the target area is flat, so as to achieve the effect of identifying a target area of the slow lane.

[0092] In step S22, whether the slow lane has road surface diseases is determined according to the road surface analysis result.

[0093] In a possible implementation, if the target area has cracks and / or the target area is not flat, it is determined that the slow lane has road surface diseases; or if the target area has no cracks and the target area is flat, it is determined that the slow lane has no road surface diseases.

[0094] In another possible implementation, the crack detection result in the road surface image and the flatness calculation result in the road surface height data can be combined for correlation analysis. If the number of cracks is large, the width is large, and the flatness index is poor, it is determined that the slow lane has road surface diseases. According to the shape, distribution of the cracks and the change of the flatness index, the type and degree of the diseases such as transverse cracks, longitudinal cracks, and block cracks can be determined.

[0095] The embodiments of the present application can preliminarily determine whether the slow lane has road surface diseases by analyzing the road surface image and the road surface height data in detail, combining crack detection and flatness evaluation, and provide a scientific basis for subsequent maintenance and repair work.

[0096] In some embodiments of the present application, with reference to Figure 3 The flatness analysis based on the road surface height data of the target area to determine whether the target area is flat includes:

[0097] In step S2111, feature extraction is performed on the road surface height data to obtain height features of the target area.

[0098] In an embodiment, the road surface height data of the target area is collected to ensure the completeness and accuracy of the data, and the outliers and missing values in the data are cleaned to avoid interference with subsequent analysis. Key height features such as maximum height, minimum height, average height, and height standard deviation are extracted from the road surface height data. These features can reflect the overall situation and dispersion of the road surface height, and provide a basis for subsequent fitting and judgment.

[0099] In step S2112, the road surface height data is fitted based on the collection time of the road surface height data to obtain the road surface height change trend of the target area.

[0100] In an embodiment, the data is sorted in time sequence according to the collection time of the road surface height data, and the change trend of the time series data is analyzed, including linear trend, nonlinear trend, etc. A suitable fitting method such as linear regression, polynomial regression, exponential regression, etc. is selected to fit the road surface height data. The purpose of fitting is to find a curve that can describe the change of road surface height with time, i.e. the road surface height change trend line.

[0101] In another embodiment, after the road surface height change trend line is fitted, the fitting effect of the fitted curve can be evaluated, including goodness of fit, residual analysis, etc. to ensure that the fitted curve can accurately reflect the change trend of the road surface height data.

[0102] In step S2113, if the road surface height change trend of the target area meets the preset change trend, it is determined that the target area is flat.

[0103] The preset change trend can be set according to actual conditions, for example, according to the requirements of road flatness, the preset change trend is set, and the preset change trend can be flat, i.e. the height change is small, or it can have a certain slope but the change is smooth, such as a gentle slope.

[0104] Alternatively, in step S2114, if the road surface height change trend of the target area does not meet the preset change trend, it is determined that the target area is not flat.

[0105] In an embodiment, the fitted road surface height change trend line is compared with the preset change trend. It is judged whether the trend line meets the requirements of the preset change trend, i.e. whether the slope and curvature of the trend line are consistent with the preset value. If the road surface height change trend of the target area meets the preset change trend, it is determined that the target area is flat. If the road surface height change trend of the target area does not meet the preset change trend, such as sudden change, abnormal fluctuation, etc., it is determined that the target area is not flat.

[0106] In another possible implementation, the judgment result can be output in the form of a report, including the road surface height characteristics of the target area, the fitting curve, the judgment result, etc., and the report should include detailed analysis process and judgment basis for subsequent use and reference. The judgment result can also be fed back to the relevant department or maintenance unit for timely maintenance and repair work. According to the feedback result and the actual situation, the scheme is improved and optimized to improve the accuracy and reliability of the judgment.

[0107] According to the feature extraction, data fitting, and trend comparison methods, the embodiments of the present application can accurately judge whether the road surface of the target area is flat, and provide a scientific basis for whether the slow lane has road diseases.

[0108] In some embodiments of the present application, the first prompt text is input into the road disease detection model, and the road disease type of the slow lane is obtained. Figure 4

[0109] Step S41, input the first prompt text into the road disease detection model, and perform vectorization representation on the road surface image and the road surface height data in the first prompt text to obtain the feature vector corresponding to the road surface image and the feature vector corresponding to the road surface height data.

[0110] In a possible implementation, the feature extraction module in the road disease detection model is used to extract features from the preprocessed road surface image to obtain the feature vector corresponding to the road surface image. Similarly, the feature extraction is performed on the preprocessed road surface height data to obtain the feature vector corresponding to the road surface height data. The extracted features are converted into vector form for subsequent comprehensive representation and similarity calculation.

[0111] In another possible implementation, the image description generation model such as Show and Tell, CaptionBot, etc. can be used to convert the road surface image into a descriptive text; a simple script can be written to convert the road surface height data into a text form; and the generated text description is input into the road disease detection model for vectorization representation, so that the road disease detection model can recognize and understand the road surface image and the road surface height data. After the generated text description is input into the road disease detection model, the road disease detection model can use the word embedding technology such as Word2Vec, BERT, etc. to convert the text into a point in a high-dimensional vector space, in which similar texts or text descriptions have similar vector representations.

[0112] Step S42, the feature vector corresponding to the road surface image and the feature vector corresponding to the road surface height data are comprehensively represented to obtain a comprehensive vector representation.

[0113] ​In a feasible implementation, the feature vectors corresponding to the road surface images and the feature vectors corresponding to the road surface height data are spliced or weighted summed, and the like, to obtain a comprehensive vector representation. The comprehensive vector representation can reflect the information of the road surface images and the road surface height data at the same time, and provides a basis for subsequent similarity calculation.

[0114] In step S43, the similarity between the comprehensive vector representation and each preset vector representation in the database is determined, wherein the database stores preset comprehensive vector representations corresponding to different road surface disease types, and the preset comprehensive vector representations are determined according to preset feature vectors corresponding to preset road surface images and preset feature vectors corresponding to preset road surface height data.

[0115] In a feasible implementation, a database is established, and the database stores preset comprehensive vector representations corresponding to different road surface disease types. The preset comprehensive vector representations are obtained according to known preset road surface images and preset road surface height data through the same feature extraction and vectorization process. The similarity between the comprehensive vector representation and each preset comprehensive vector representation in the database is calculated using a suitable similarity calculation method, such as cosine similarity, Euclidean distance, and the like. The result of the similarity calculation can reflect the closeness between the comprehensive vector representation and the preset comprehensive vector representation.

[0116] In step S44, the road surface disease type of the slow lane is determined according to the similarity.

[0117] According to the result of the similarity calculation, the preset comprehensive vector representation that is most similar to the comprehensive vector representation is found. The road surface disease type corresponding to the preset comprehensive vector representation is determined as the road surface disease type of the slow lane.

[0118] In the embodiments of the present application, the first prompt text is input into the large language model for analysis, and the accurate road surface disease type of the slow lane is obtained.

[0119] In some embodiments of the present application, with reference to Figure 5 After inputting the first prompt text into the road surface disease detection model to obtain the road surface disease type of the slow lane, the method further includes:

[0120] In step S110, the road surface disease distribution of the slow lane is obtained.

[0121] The road surface disease distribution refers to the position of the road surface disease on the slow lane. The road surface disease distribution can be a road surface disease distribution map, and the map should clearly show the position, size and severity of each disease on the road surface.

[0122] In an implementable embodiment, the road surface image and the road surface height data of the slow lane can be acquired to determine the road surface disease, and the positioning data can be acquired based on the position detection sensor mounted on the road surface disease detection device of the slow lane. When it is determined that there is a road surface disease based on the acquired road surface image and road surface height data of the slow lane, the positioning data of the slow lane where the road surface disease exists is acquired, and then the distribution of the road surface disease is determined according to the positioning data where the road surface disease exists.

[0123] In step S120, a small sample example set corresponding to the road surface disease type and the road surface disease distribution is determined, wherein the small sample example set includes a plurality of preset sample examples, and the preset sample examples include preset road surface disease type samples, preset road surface disease distributions, and corresponding preset processing suggestion samples.

[0124] The small sample example set is also called a few-shot example set, which means that the example is not fixed, but similar examples are queried from the example library as examples according to the current input road surface disease type and road surface disease distribution. Therefore, when the road surface disease type and the road surface disease distribution change, the corresponding small sample example set also changes accordingly. Such examples are more referential and can further improve the accuracy of the processing suggestion of the road surface disease. The small sample example set can include one or more sample examples.

[0125] In an implementable embodiment, a small number of samples with representative samples are selected as the example set from existing road surface disease processing cases. These samples should cover different road surface disease types such as cracks and potholes and distribution modes such as concentrated distribution and scattered distribution.

[0126] The preset sample example refers to a sample example prepared in advance, which is used to assist in determining the processing suggestion process. The preset sample example includes a preset road surface disease type sample and a preset road surface disease distribution, and a corresponding preset processing suggestion sample. Through the corresponding relationship between the preset sample example and the corresponding preset processing suggestion sample, the subsequent road surface disease processing model can be guided to determine the processing suggestion corresponding to the road surface disease type and the road surface disease distribution. For example, the preset sample example includes a road surface disease type sample such as an image or description of a crack, a preset road surface disease distribution such as a position map of the disease on the road surface, and a corresponding preset processing suggestion sample such as a repair material and a repair method.

[0127] In step S130, a second prompt text is constructed according to the road surface disease type, the road surface disease distribution, the small sample example set, and the processing requirement of the road surface disease type.

[0128] Integrate the current slow road surface disease types, distribution, and treatment needs such as urgency, budget constraints, etc. into a descriptive text, i.e. the second prompt text. Ensure that the second prompt text is structured clearly and contains all necessary information to facilitate the subsequent road surface disease treatment model to understand and process.

[0129] Step S140, input the second prompt text into the road surface disease treatment model to obtain treatment suggestions for the road surface disease type and road surface disease distribution, wherein the road surface disease treatment model is fine-tuned according to the preset road surface disease type sample, the preset road surface disease distribution, and the corresponding preset treatment suggestion sample.

[0130] The road surface disease treatment model is used to recommend the treatment suggestions for the slow road surface disease.

[0131] The road surface disease treatment model used in this application can be a general large language model or a pre-trained special large language model, preferably a pre-trained special large language model. The pre-trained special large language model has powerful natural language processing ability and certain professional knowledge reserve, so as to accurately understand the second prompt text and accurately recommend the treatment suggestions for the slow road surface disease. This road surface disease treatment model can also be a prediction model based on deep learning or machine learning, which can learn the association between disease type, distribution, and treatment suggestion from the input data.

[0132] In a feasible implementation, the road surface disease treatment model outputs treatment suggestions for the current slow road surface disease type and distribution according to the input second prompt text. These treatment suggestions may include specific repair methods, required materials, budget estimates, etc.

[0133] In another feasible implementation, a detailed disease treatment plan can also be developed according to the treatment suggestions output by the road surface disease treatment model. According to the plan, the disease treatment is carried out, including material preparation, construction operation, etc. After the treatment is completed, the treatment effect is evaluated to ensure that the disease is effectively solved.

[0134] Through the above steps, the road surface disease treatment model can intelligently provide treatment suggestions for the slow road surface disease, improving the efficiency and accuracy of disease treatment.

[0135] In some embodiments of the present application, with reference to Figure 6 , the small sample set corresponding to the road surface disease type and the road surface disease distribution includes:

[0136] Step S121, vectorizing the road disease type to obtain a first feature vector of the road disease type, and vectorizing the road disease distribution to obtain a second feature vector of the road disease distribution.

[0137] In a feasible implementation, for the road disease type, a set of features capable of describing the road disease type, such as the shape, size, color, texture, etc. of the disease, needs to be defined. These features can be extracted from the disease image through image processing techniques such as edge detection, texture analysis, color space conversion, etc. The extracted feature values are combined into a vector, i.e. the first feature vector of the road disease type, which can uniquely or approximately uniquely represent the disease type.

[0138] In another feasible implementation, for the road disease distribution, spatial information such as the position, range, density, etc. of the disease on the road surface needs to be extracted. These information can be obtained through geographic information systems or image processing techniques such as region segmentation, shape analysis, etc. The extracted spatial information values are combined into a vector, i.e. the second feature vector of the road disease distribution, which should be able to accurately describe the distribution of the disease on the road surface.

[0139] Step S122, retrieving a first preset sample example matching the first feature vector from the example library, and retrieving a second preset sample example matching the second feature vector.

[0140] In a feasible implementation, the first feature vector of the road disease type is matched with the preset sample examples in the example library. The matching process can use vector space model, cosine similarity, Euclidean distance, etc. algorithm to evaluate the similarity between the feature vectors. From the matching results, the preset sample example most similar to the first feature vector is selected as the first preset sample example, or the preset sample example with a similarity greater than a preset similarity to the first feature vector is selected as the first preset sample example.

[0141] In another feasible implementation, the second feature vector of the road disease distribution is matched with the preset sample examples in the example library. From the matching results, the preset sample example most similar to the second feature vector is selected as the second preset sample example, or the preset sample example with a similarity greater than a preset similarity to the second feature vector is selected as the second preset sample example. Wherein, the matching process can use vector space model, cosine similarity, Euclidean distance, etc. algorithm to evaluate the similarity between the feature vectors.

[0142] It should be noted that the first preset sample example can be one or more, and the second preset sample example can be one or more.

[0143] Step S123, merging the first preset sample example and the second preset sample example to obtain a merged preset sample example.

[0144] In a possible implementation, the retrieved first preset sample example and the second preset sample example are merged. The merging process can involve integrating key information such as disease type description, distribution map, treatment suggestion, etc. in the two examples into a new example to obtain a merged preset sample example containing complete information.

[0145] In another possible implementation, since the retrieval process can be similarity score calculation by vector representation, the vector representation of each retrieved first preset sample example has a corresponding similarity score with the first feature vector. The vector representation of each retrieved second preset sample example also has a corresponding similarity score with the second feature vector. After obtaining the first preset sample example and the second preset sample example, each first preset sample example and each second preset sample example are sorted based on the corresponding similarity score, and the top N most similar preset sample examples are selected. Here, N can be 1 or an integer greater than 1.

[0146] Step S124, performing similarity sorting and screening on the merged preset sample example to obtain the small sample example set.

[0147] The merged preset sample example is sorted according to the similarity evaluation result, and the top several examples with the highest similarity are selected as the small sample example set. The screening process can ensure that the samples in the example set are highly relevant and representative to the original input.

[0148] In a possible implementation, the first preset sample example and the second preset sample example retrieved from the example library are merged; if the first preset sample example and the second preset sample example retrieved from the example library have duplicate sample examples, the one with the highest similarity score is retained; then the merged sample examples are sorted again according to the similarity scores; then the most relevant N sample examples are selected from the merged sample examples, and the most relevant N sample examples form the small sample example set. Here, N is 1 or an integer greater than 1.

[0149] The embodiments of the present application can efficiently construct a small sample example set for a specific road surface disease type and distribution, and provide strong support for subsequent processing and analysis.

[0150] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the road surface disease detection method of the present application. More forms of simple transformation based on this technical concept are within the protection scope of the present application.

[0151] Based on the same inventive concept, the application provides a slow lane pavement disease detection device, which comprises at least one processor, and a memory in communication connection with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the slow lane pavement disease detection method in the above embodiment I.

[0152] Reference will now be made to the following description Figure 7 which shows a structural schematic diagram of a slow lane pavement disease detection device suitable for implementing the embodiments of the application. The slow lane pavement disease detection device in the embodiments of the application can include a processing device 1001 (such as a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 1002 or programs loaded from a storage device 1003 into a random access memory (RAM) 1004. In the RAM 1004, various programs and data required for the operation of the slow lane pavement disease detection device are also stored. The processing device 1001, the ROM 1002, and the RAM 1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; the storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the slow lane pavement disease detection device to communicate with other devices wirelessly or by wire to exchange data. Although the slow lane pavement disease detection device with various systems is shown in the figure, it should be understood that it is not required to implement or have all the shown systems. More or fewer systems can be alternatively implemented or provided.

[0153] In particular, according to the embodiments disclosed in the application, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, the embodiments disclosed in the application include a computer program product comprising a computer program carried on a computer readable medium, which computer program contains program codes for executing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network through the communication device, or installed from the storage device 1003, or installed from the ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the methods of the embodiments disclosed in the application are executed.

[0154] The slow lane pavement disease detection device provided by the application adopts the slow lane pavement disease detection method in the above embodiment, and can solve the technical problem that manual slow lane pavement disease detection results in low accuracy of pavement disease detection results. Compared with the prior art, the slow lane pavement disease detection device provided by the application has the same beneficial effects as the slow lane pavement disease detection method provided by the above embodiment, and other technical features in the slow lane pavement disease detection device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0155] It should be understood that various parts of the present application can be realized by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0156] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of changes or replacements within the technical scope disclosed by the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

[0157] Based on the same inventive concept, the present application provides a computer readable storage medium having computer readable program instructions (i.e. computer programs) stored thereon, the computer readable program instructions being used to execute the slow lane pavement disease detection method in the above embodiment.

[0158] The computer readable storage medium provided in the present application may, for example, be a U disk, but is not limited to an electric, magnetic, optical, electromagnetic, infrared, or semiconductor system, system, or device, or any combination of the above. More specific examples of the computer readable storage medium can include, but are not limited to, an electric connection with one or more conductive wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present embodiment, the computer readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer readable storage medium can be transmitted by any suitable medium, including but not limited to an electric wire, an optical cable, an RF (Radio Frequency), and the like, or any suitable combination of the above.

[0159] The above computer readable storage medium can be included in the road surface disease detection device for slow lanes, or can exist separately without being assembled into the road surface disease detection device for slow lanes.

[0160] The above computer readable storage medium carries one or more programs, which, when executed by the road surface disease detection device for slow lanes, enable the road surface disease detection device for slow lanes to improve the accuracy of the road surface disease detection result.

[0161] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0162] The flow diagrams and the block diagrams in the drawings are illustrations of architectures, functionalities, and operations of possible implementations of systems, methods, and computer program products according to various embodiments of the present application. In this regard, each block in the flow diagrams or block diagrams can represent a module, a segment, or a portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks may

[0163] The modules involved in the embodiments of the present application can be implemented in the form of software or in the form of hardware. In some cases, the name of the module does not constitute a limitation on the module itself.

[0164] The readable storage medium provided by the present application is a computer readable storage medium, which stores computer readable program instructions (i.e., computer programs) for executing the slow lane pavement disease detection method described above, and can solve the technical problem of low accuracy of slow lane pavement disease detection by manual detection. Compared with the prior art, the computer readable storage medium provided by the present application has the same beneficial effects as the slow lane pavement disease detection method provided by the above-mentioned embodiments, and will not be described here.

[0165] The above merely provides part of embodiments of the present application, and does not limit the patent scope of the present application. Any equivalent structure transformation, direct / indirect application in other related technical fields, or the like, which is made based on the technical concept of the present application, and based on the content of the present application specification and drawings, is included in the patent protection scope of the present application.

Claims

1. A method for detecting road surface defects on a slow lane, characterized in that: The method for detecting road surface defects on a slow lane comprises: Obtaining road surface images and road height data of the slow lane; performing image analysis on a road surface image corresponding to a target area of ​​the slow lane to determine whether cracks exist in the target area, and performing flatness analysis based on road surface height data of the target area to determine whether the target area is flat; If the target area has cracks and / or the target area is uneven, it is determined that the slow lane has pavement damage; or if the target area has no cracks and the target area is flat, it is determined that the slow lane has no pavement damage; If there is a road surface defect on the slow lane, construct a first prompt text according to the road surface image, the road surface height data and the road surface defect detection requirement; Inputting the first prompt text into a pavement damage detection model to obtain the pavement damage type of the slow lane, wherein the pavement damage detection model is fine-tuned based on a preset pavement damage type sample, a preset pavement image sample, and a preset pavement height data sample; Obtaining the distribution of pavement damage on the slow lane; Performing a vector representation on the pavement damage type to obtain a first eigenvector of the pavement damage type, and performing a vector representation on the pavement damage distribution to obtain a second eigenvector of the pavement damage distribution; Retrieving a first preset sample example matching the first feature vector from a sample library, and retrieving a second preset sample example matching the second feature vector; Merging the first preset sample example and the second preset sample example to obtain a merged preset sample example; Sorting and screening the merged preset sample examples by similarity to obtain a small number of sample example sets, wherein the small number of sample example sets includes a plurality of preset sample examples, and the preset sample examples include preset pavement disease type samples, preset pavement disease distributions, and corresponding preset treatment suggestion samples; constructing a second prompt text according to the pavement damage type, the pavement damage distribution, the small number of sample example sets, and the processing requirements of the pavement damage type; The second prompt text is input into a pavement damage treatment model to obtain treatment suggestions for the pavement damage type and the pavement damage distribution, wherein the pavement disease treatment model is fine-tuned based on preset pavement disease type samples, preset pavement disease distribution and corresponding preset treatment suggestion samples.

2. The method for detecting road surface defects in a slow lane according to claim 1, wherein: The performing a flatness analysis based on the road surface height data of the target area to determine whether the target area is flat includes: Performing feature extraction on the road height data to obtain a height feature of the target area; Fitting the road height data based on the acquisition time of the road height data to obtain a road height change trend of the target area; If the road surface height variation trend of the target area meets the preset variation trend, determining that the target area is flat; Alternatively, if the road surface height variation trend of the target area does not conform to the preset variation trend, it is determined that the target area is uneven.

3. The method for detecting road surface defects in a slow lane according to claim 1, wherein: The inputting of the first prompt text into the pavement damage detection model to obtain the pavement damage type of the slow lane includes: Inputting the first prompt text into a pavement defect detection model, performing vectorization on the pavement image and pavement height data in the first prompt text to obtain a feature vector corresponding to the pavement image and a feature vector corresponding to the pavement height data; Performing a comprehensive representation on the feature vector corresponding to the road surface image and the feature vector corresponding to the road surface height data to obtain a comprehensive vector representation; Determining a similarity between the integrated vector representation and various preset integrated vector representations in a database, wherein the database stores preset integrated vector representations corresponding to different pavement defect types, the preset integrated vector representations being determined based on preset feature vectors corresponding to preset pavement images and preset feature vectors corresponding to preset pavement height data; The pavement damage type of the slow lane is determined according to the similarity.

4. The method for detecting road surface defects in a slow lane according to claim 1, wherein: The obtaining of the pavement damage distribution of the slow lane includes: Obtaining location data of road surface defects on the slow lane; The distribution of the pavement defects is determined based on the positioning data of the pavement defects.

5. A road surface disease detection device for a slow lane, characterized in that: The road surface defect detection device for the slow lane includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the road surface defect detection method for the slow lane as described in any one of claims 1 to 4.

6. A storage medium, characterized in that The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the method for detecting pavement defects on a slow lane as described in any one of claims 1 to 4 are implemented.

Images