A road disease detection method and device based on unmanned aerial vehicle images

Abstract

The application relates to a road disease detection method and device based on unmanned aerial vehicle images, and belongs to the technical field of road detection. The road disease detection method based on unmanned aerial vehicle images comprises the following steps: dividing a road surface image data region, extracting visual feature sequences from the road surface image data after region division, processing the visual feature sequences in multiple directions to obtain feature response results, normalizing the feature response results to obtain linear confidence sequences, weighting the visual feature sequences based on the linear confidence sequences to obtain visual feature enhanced sequences, extracting sparse feature sequences from the visual feature enhanced sequences based on iterative feature aggregation, determining a screening weight vector based on historical road disease description information, screening road disease feature sequences based on the screening weight vector to determine a road disease detection result. The application improves the accuracy of road disease detection by enhancing road disease features.

Description

Technical Field

[0001] This invention relates to the field of road inspection technology, and in particular to a method and apparatus for detecting road defects based on UAV imagery. Background Technology

[0002] Current road defect detection mainly relies on manual inspections or vehicle-mounted detection equipment, which suffers from low efficiency, high cost, and limited coverage. With the development of drone technology, using drones to acquire road surface images for defect detection has gradually become a research hotspot. However, existing technologies mostly focus on target detection or pixel-level segmentation of defects, and their output results are usually bounding boxes or segmentation masks, which are difficult to meet the actual needs of road maintenance departments for describing the type, spatial location, geometric shape, and extent of defects.

[0003] In addition, road defects in UAV imagery typically have three main characteristics: the defect area occupies a very small proportion of the entire image, exhibiting severe sparsity; crack-type defects have obvious directionality and connectivity, and their geometric features are difficult to accurately represent by traditional visual models; road scenes often contain highly salient interfering targets such as vehicles, vegetation, and shadows, which can easily lead to false detections or unstable descriptions.

[0004] Therefore, how to accurately detect road defects using drone imagery has become a pressing technical problem that needs to be solved. Summary of the Invention

[0005] In view of this, it is necessary to provide a road defect detection method and device based on UAV imagery to solve the problem of poor detection accuracy when using UAV imagery for road defect detection.

[0006] To address the aforementioned problems, in a first aspect, the present invention provides a road defect detection method based on UAV imagery, comprising: The system acquires road surface image data collected by drones, divides the road surface image data into regions based on multiple preset spatial regions, and extracts visual features from the divided road surface image data to obtain a visual feature sequence. Multi-directional response processing is performed on the visual feature sequence to obtain the feature response results of the visual feature sequence in multiple directions. The feature response results of the visual feature sequence in multiple directions are then normalized to obtain a linear confidence sequence. The linear confidence sequence is used to represent the probability that each spatial region belongs to a linear or strip-shaped disease structure. The visual feature sequence is weighted based on the linear confidence sequence to obtain the visual feature enhancement sequence, and sparse feature sequence is extracted from the visual feature enhancement sequence based on iterative feature aggregation. The filtering weight vector is determined based on the correlation between historical road defect description information and visual feature sequence. Based on the filtering weight vector, road defect feature sequence is selected from sparse feature sequence, and the road defect detection result is determined based on the road defect feature sequence.

[0007] In one possible implementation, the multi-directional response processing of the visual feature sequence to obtain the feature response results of the visual feature sequence in multiple directions includes: By applying a multi-directional response operator to the visual feature sequence, the feature response results of the visual feature sequence in multiple directions are obtained.

[0008] In one possible implementation, the normalization of the feature response results of the visual feature sequence in multiple directions to obtain a linear confidence sequence includes: The linear confidence sequence is determined based on the maximum value of the normalized feature response results in each direction.

[0009] In one possible implementation, the weighting of the visual feature sequence based on the linear confidence sequence to obtain the visual feature enhancement sequence includes: The visual feature sequence is obtained by multiplying the linear confidence sequence element by element.

[0010] In one possible implementation, the extraction of sparse feature sequences from visual feature enhancement sequences based on iterative feature aggregation includes: In the first iteration, features are extracted from the visual feature enhancement sequence using the initial query vector. In each iteration, the query vector is updated based on the updated feature sequence obtained from the previous iteration. Features are then extracted from the visual feature enhancement sequence based on the updated query vector. After multiple iterations, a sparse feature sequence is obtained.

[0011] In one possible implementation, determining the screening weight vector based on the correlation between historical road defect description information and visual feature sequences includes: A memory prior representation is constructed based on historical road defect description information, and a selection weight vector is determined based on the correlation between the memory prior representation and the visual feature sequence.

[0012] In one possible implementation, determining the road defect detection result based on the road defect feature sequence includes: The road defect feature sequence is processed based on fully connected layers and activation functions to determine the probability of various road defects, and the road defect with the highest probability is identified as the road defect type information. Regression prediction is performed on the characteristic sequence of road defects to determine the geometric attribute information of road defects; The road defect type information and road defect geometric attribute information are used as the road defect detection results.

[0013] On the other hand, the present invention also provides a road defect detection device based on UAV imagery, comprising: The acquisition module is used to acquire road surface image data collected by UAV, divide the road surface image data into regions based on multiple preset spatial regions, and extract visual features from the divided road surface image data to obtain a visual feature sequence. The processing module is used to perform multi-directional response processing on the visual feature sequence to obtain the feature response results of the visual feature sequence in multiple directions, and to normalize the feature response results of the visual feature sequence in multiple directions to obtain a linear confidence sequence. The linear confidence sequence is used to represent the probability that each spatial region belongs to a linear or strip-shaped disease structure. The extraction module is used to perform weighted processing on the visual feature sequence based on the linear confidence sequence to obtain the visual feature enhancement sequence, and to extract the sparse feature sequence from the visual feature enhancement sequence based on iterative feature aggregation. The detection module is used to determine the screening weight vector based on the correlation between historical road defect description information and visual feature sequence, to filter out road defect feature sequence from sparse feature sequence based on the screening weight vector, and to determine the road defect detection result based on the road defect feature sequence.

[0014] Secondly, the present invention also provides a detection device, including a memory and a processor, wherein, The memory is used to store programs; The processor, coupled to the memory, is used to execute the program stored in the memory to implement the steps in the road defect detection method based on UAV imagery as described in any of the above implementations.

[0015] Thirdly, the present invention also provides a computer-readable storage medium for storing a computer-readable program or instructions, which, when executed by a processor, can implement the steps in the road defect detection method based on UAV imagery described in any of the above implementations.

[0016] The beneficial effects of this invention are as follows: The road defect detection method and apparatus based on UAV imagery provided by this invention firstly extracts regional features to model the geometric attributes and spatial relationships of road defects. Then, it obtains a linear confidence sequence through multi-directional response processing and normalization. The visual feature sequence is then weighted using the linear confidence sequence to obtain a visual feature enhancement sequence. This explicitly introduces the directionality and connectivity information of road defects at the visual feature level, enhancing the road defect features. Next, iterative feature aggregation extracts a sparse feature sequence from the visual feature enhancement sequence, enhancing the expressive power of defect-related features and further improving the accuracy of subsequent road defect detection. Finally, a screening weight vector is determined based on the correlation between historical road defect description information and the visual feature sequence. This screening weight vector is used to further focus on road defect features, further enhancing the expression of road defect features, resulting in a road defect feature sequence, and thus determining the road defect detection result. This invention improves the accuracy of road defect detection by enhancing road defect features. Attached Figure Description

[0017] Figure 1 This is a schematic flowchart of an embodiment of the road defect detection method based on UAV imagery provided by the present invention; Figure 2 This is a schematic flowchart of an embodiment of the road defect detection process provided by the present invention; Figure 3 This is a schematic diagram of an embodiment of the road defect detection system architecture provided by the present invention; Figure 4 This is a schematic diagram of an embodiment of the road defect detection device based on UAV imagery provided by the present invention; Figure 5 This is a schematic diagram of an embodiment of the detection device provided by the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0019] In the description of the embodiments of the present invention, unless otherwise stated, "multiple" means two or more. "And / or" describes the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0020] The terms "first," "second," etc., used in the embodiments of this invention are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a technical feature defined with "first" or "second" may explicitly or implicitly include at least one of that feature.

[0021] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0022] This invention provides a method and apparatus for detecting road defects based on UAV imagery, which will be described in detail below.

[0023] Figure 1 This is a schematic flowchart of an embodiment of the road defect detection method based on UAV imagery provided by the present invention, as shown below. Figure 1 As shown, the road defect detection method based on UAV imagery includes: S101. Acquire road surface image data collected by UAV, divide the road surface image data into regions based on multiple preset spatial regions, and extract visual features from the divided road surface image data to obtain a visual feature sequence.

[0024] It should be noted that the road defect detection method based on UAV imagery provided by this invention can be applied to road defect detection scenarios, especially road defect detection scenarios based on UAV imagery.

[0025] When conducting road defect detection, the detection equipment (such as a desktop or portable computer) can first acquire road surface image data collected by a drone. Then, the road surface image data can be divided into multiple spatial regions (for example, the road can be divided into multiple spatial regions at 500m intervals). Next, visual features can be extracted from the divided road surface image data (for example, feature extraction can be performed through ResNet or YOLO networks) to obtain a visual feature sequence, thereby converting the two-dimensional road image into a serialized feature representation, so as to model the geometric attributes and spatial relationships of road defects in the future.

[0026] S102. Perform multi-directional response processing on the visual feature sequence to obtain the feature response results of the visual feature sequence in multiple directions, and normalize the feature response results of the visual feature sequence in multiple directions to obtain a linear confidence sequence. The linear confidence sequence is used to represent the probability that each spatial region belongs to a linear or strip-shaped disease structure.

[0027] It should be noted that after obtaining the visual feature sequence, since crack-type road defects have obvious directionality and spatial connectivity, the visual feature sequence can be processed in multiple directions to obtain the feature response results of the visual feature sequence in multiple directions. Then, the feature response results of the visual feature sequence in multiple directions are normalized to obtain a linear confidence sequence, which reflects the possibility that each spatial area belongs to a linear or strip-shaped defect structure, thereby improving the accuracy of subsequent road defect detection.

[0028] S103. The visual feature sequence is weighted based on the linear confidence sequence to obtain the visual feature enhancement sequence, and sparse feature sequence is extracted from the visual feature enhancement sequence based on iterative feature aggregation.

[0029] It should be noted that after obtaining the linear confidence sequence, the visual feature sequence can be weighted using the linear confidence sequence to obtain the visual feature enhancement sequence. This explicitly introduces the directionality and connectivity information of road defects at the visual feature level. Then, sparse feature sequences are extracted from the visual feature enhancement sequence through iterative feature aggregation to enhance the expressive power of defect-related features and further improve the accuracy of subsequent road defect detection.

[0030] S104. Determine the screening weight vector based on the correlation between historical road defect description information and visual feature sequence. Based on the screening weight vector, select the road defect feature sequence from the sparse feature sequence and determine the road defect detection result based on the road defect feature sequence.

[0031] It should be noted that: Finally, a screening weight vector can be determined based on the correlation between historical road defect description information and visual feature sequence. Then, the road defect feature sequence can be selected from the sparse feature sequence using the screening weight vector. The road defect features can be further enhanced based on historical road defect description information. Finally, the road defect detection result can be determined based on the road defect feature sequence, thus realizing road defect detection.

[0032] In summary, the road defect detection method based on UAV imagery provided in this invention first extracts regional features to model the geometric attributes and spatial relationships of road defects. Then, it obtains a linear confidence sequence through multi-directional response processing and normalization. The visual feature sequence is then weighted using this linear confidence sequence to obtain a visual feature enhancement sequence. This explicitly introduces the directionality and connectivity information of road defects at the visual feature level, enhancing the road defect features. Next, iterative feature aggregation extracts a sparse feature sequence from the visual feature enhancement sequence, enhancing the expressive power of defect-related features and further improving the accuracy of subsequent road defect detection. Finally, a filtering weight vector is determined based on the correlation between historical road defect description information and the visual feature sequence. This filtering weight vector is used to further focus on road defect features, further enhancing their expression and obtaining a road defect feature sequence, thereby determining the road defect detection result. This invention improves the accuracy of road defect detection by enhancing road defect features.

[0033] In some embodiments of the present invention, the step of performing multi-directional response processing on the visual feature sequence to obtain feature response results of the visual feature sequence in multiple directions includes: By applying a multi-directional response operator to the visual feature sequence, the feature response results of the visual feature sequence in multiple directions are obtained.

[0034] It should be noted that when performing multi-directional response processing on visual feature sequences, multi-directional response operators can be applied to the visual feature sequences to obtain the feature response results of the visual feature sequences in multiple directions.

[0035] In some embodiments of the present invention, the normalization processing of the feature response results of the visual feature sequence in multiple directions to obtain a linear confidence sequence includes: The linear confidence sequence is determined based on the maximum value of the normalized feature response results in each direction.

[0036] It should be noted that when normalizing the feature response results of the visual feature sequence in multiple directions, the linear confidence sequence can be determined based on the maximum value of the normalized feature response results in each direction.

[0037] In some embodiments of the present invention, the weighting of the visual feature sequence based on the linear confidence sequence to obtain the visual feature enhancement sequence includes: The visual feature sequence is obtained by multiplying the linear confidence sequence element by element.

[0038] It should be noted that when weighting the visual feature sequence based on the linear confidence sequence, the visual feature sequence and the linear confidence sequence can be multiplied element by element to obtain the visual feature enhancement sequence.

[0039] In some embodiments of the present invention, the extraction of sparse feature sequences from visual feature enhancement sequences based on iterative feature aggregation includes: In the first iteration, features are extracted from the visual feature enhancement sequence using the initial query vector. In each iteration, the query vector is updated based on the updated feature sequence obtained from the previous iteration. Features are then extracted from the visual feature enhancement sequence based on the updated query vector. After multiple iterations, a sparse feature sequence is obtained.

[0040] It should be noted that when extracting sparse feature sequences from visual feature enhancement sequences based on iterative feature aggregation, features can be extracted from the visual feature enhancement sequences using the initial query vector in the first iteration. Then, in each iteration, the query vector is updated based on the updated feature sequence obtained from the previous iteration. Features are then extracted from the visual feature enhancement sequences using the updated query vector. After multiple iterations, a sparse feature sequence is obtained.

[0041] In some embodiments of the present invention, determining the screening weight vector based on the correlation between historical road defect description information and visual feature sequences includes: A memory prior representation is constructed based on historical road defect description information, and a selection weight vector is determined based on the correlation between the memory prior representation and the visual feature sequence.

[0042] It should be noted that when determining the screening weight vector based on the correlation between historical road damage description information and visual feature sequence, a memory prior representation can be constructed first based on the historical road damage description information, and then the screening weight vector can be determined based on the correlation between the memory prior representation and the visual feature sequence.

[0043] In some embodiments of the present invention, determining the road defect detection result based on the road defect feature sequence includes: The road defect feature sequence is processed based on fully connected layers and activation functions to determine the probability of various road defects, and the road defect with the highest probability is identified as the road defect type information. Regression prediction is performed on the characteristic sequence of road defects to determine the geometric attribute information of road defects; The road defect type information and road defect geometric attribute information are used as the road defect detection results.

[0044] It should be noted that when determining road defect detection results based on road defect feature sequences, the feature sequences can be processed using fully connected layers and activation functions to determine the probabilities of various road defects. The road defect with the highest probability is then identified as the road defect type. Next, regression prediction can be performed on the road defect feature sequences to determine the geometric attributes of the road defects. Finally, the road defect type and geometric attribute information are used as the road defect detection results.

[0045] Combination Figure 2 The road defect detection method provided by this invention specifically includes the following steps: 1. Use drones to acquire road surface image data.

[0046] 2. Extract features from road surface images to obtain a sequence of visual features representing the image content.

[0047] Feature extraction processing is performed on road surface images captured by drones, representing the images as a visual feature sequence X composed of multiple feature vectors:

[0048] in, Indicates the first in the image Visual feature vectors corresponding to each spatial region For feature dimension, This represents the number of spatial regions.

[0049] The above method converts two-dimensional road images into sequential feature representations, enabling subsequent modeling of the geometric attributes and spatial relationships of road defects.

[0050] 3. Extract the geometric features of road defects based on visual feature sequences. The geometric features include at least the directional features and spatial connectivity features of the defects.

[0051] In view of the obvious directionality and spatial connectivity of crack-type road defects, multi-directional response processing is applied to the visual feature sequence to extract the geometric features of road defects.

[0052] Apply multi-directional response operators to visual feature sequences (N is the number of multi-directional response operators), to obtain the characteristic response results in different directions. :

[0053] Based on this, the directional response results are normalized, and the linear confidence level of road defects is calculated. :

[0054] Linear confidence is used to characterize the probability that each spatial region belongs to a linear or strip-shaped disease structure.

[0055] Geometric enhancement features are obtained by weighting visual features using linear confidence scores. :

[0056] in, This indicates element-weighted operations. Through this method, directional and connectivity information about road defects is explicitly introduced at the visual feature level.

[0057] 4. By using an iterative feature aggregation method, sparse feature information related to road defects in the visual feature sequence is gradually focused.

[0058] Because road defects typically exhibit a highly sparse distribution in UAV imagery, an iterative feature aggregation approach is employed to enhance the expressive power of defect-related features. Specifically, a query vector set is set. :

[0059] The query vector is iteratively updated through multiple rounds of feature interaction:

[0060] in, This represents the update amount calculated based on the current geometric enhancement features. As the number of iterations increases, the query vector gradually focuses on sparse feature regions related to road defects.

[0061] 5. In the process of generating road defect reports, a memory prior is constructed based on the historical descriptive information that has been generated, and visual features are dynamically filtered accordingly.

[0062] In the process of generating road defect reports, a memory prior representation is constructed based on the generated defect description information. The selection weights of visual features are calculated based on the correlation between prior memory and visual features. :

[0063] in, This is a function for calculating feature relevance. Based on the selection weights, a preset number of features are selected from visual features for generating road defect reports for the current stage, in order to reduce the interference of background information on defect descriptions.

[0064] 6. Based on the filtered features, generate a structured road defect report that includes road defect type, spatial location, and geometric attributes.

[0065] Based on the filtered visual feature set A multi-task prediction network is used to generate structured reports. Mapping relationships. Specifically, this is achieved through the following three parallel processing sub-steps: 1) Disease category determination.

[0066] Processing feature sets using classification prediction branches The probability of a feature belonging to a different disease category is calculated using a fully connected layer and an activation function. The label with the highest probability is selected as the disease type information (e.g., transverse cracks, longitudinal cracks, pits, etc.).

[0067] 2) Geometric and positional regression.

[0068] Processing feature sets using regression prediction branches Combining the directional features extracted in step 3 It directly regresses to predict the bounding box coordinates or center point coordinates of the disease to obtain spatial location information; at the same time, it predicts the pixel-level length, width and damaged area of ​​the disease, and combines the image scale to convert it into physical units of measurement to obtain geometric attribute information.

[0069] 3) Structured information slot filling.

[0070] Based on a predefined report template, the obtained disease type labels are semantically aligned and assembled with spatial coordinates and geometric attribute values. Discrete prediction data are then populated into the corresponding attribute fields to finally generate a standardized, structured road disease report.

[0071] The above-mentioned road defect detection process can be achieved through methods such as Figure 3 The road defect detection system shown is implemented.

[0072] To better implement the road defect detection method based on UAV imagery in this embodiment of the invention, based on the road defect detection method based on UAV imagery, correspondingly, as follows: Figure 4 As shown, this embodiment of the invention also provides a road defect detection device based on UAV imagery. The road defect detection device 400 based on UAV imagery includes: The acquisition module 401 is used to acquire road surface image data collected by UAV, divide the road surface image data into regions based on multiple preset spatial regions, and extract visual features from the divided road surface image data to obtain a visual feature sequence. The processing module 402 is used to perform multi-directional response processing on the visual feature sequence to obtain the feature response results of the visual feature sequence in multiple directions, and to normalize the feature response results of the visual feature sequence in multiple directions to obtain a linear confidence sequence. The linear confidence sequence is used to represent the probability that each spatial region belongs to a linear or strip-shaped disease structure. The extraction module 403 is used to perform weighted processing on the visual feature sequence based on the linear confidence sequence to obtain the visual feature enhancement sequence, and to extract the sparse feature sequence from the visual feature enhancement sequence based on iterative feature aggregation. The detection module 404 is used to determine the screening weight vector based on the correlation between historical road defect description information and visual feature sequence, to screen out road defect feature sequence from sparse feature sequence based on the screening weight vector, and to determine the road defect detection result based on the road defect feature sequence.

[0073] The road defect detection device 400 based on UAV imagery provided in the above embodiments can realize the technical solutions described in the above embodiments of the road defect detection method based on UAV imagery. The specific implementation principles of each module or unit can be found in the corresponding content in the above embodiments of the road defect detection method based on UAV imagery, and will not be repeated here.

[0074] like Figure 5 As shown, the present invention also provides a detection device 500. The detection device 500 includes a processor 501, a memory 502, and a display 503. Figure 5 Only some components of the testing device 500 are shown; however, it should be understood that implementation of all shown components is not required, and more or fewer components may be implemented instead.

[0075] In some embodiments, processor 501 may be a central processing unit (CPU), microprocessor, or other data processing chip, used to run program code stored in memory 502 or process data, such as the road defect detection method based on UAV imagery in this invention.

[0076] In some embodiments, processor 501 may be a single server or a group of servers. The server group may be centralized or distributed. In some embodiments, processor 501 may be local or remote. In some embodiments, processor 501 may be implemented on a cloud platform. In one embodiment, the cloud platform may include a private cloud, public cloud, hybrid cloud, community cloud, distributed cloud, internal cloud, multi-cloud, etc., or any combination thereof.

[0077] In some embodiments, memory 502 may be an internal storage unit of the detection device 500, such as a hard disk or memory of the detection device 500. In other embodiments, memory 502 may also be an external storage device of the detection device 500, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the detection device 500.

[0078] Furthermore, the memory 502 may include both internal storage units of the detection device 500 and external storage devices. The memory 502 is used to store the application software and various types of data installed on the detection device 500.

[0079] In some embodiments, display 503 may be an LED display, a liquid crystal display, a touch-screen liquid crystal display, or an organic light-emitting diode (OLED) touchscreen, etc. Display 503 is used to display information from the detection device 500 and to display a visual user interface. Components 501-503 of the detection device 500 communicate with each other via a system bus.

[0080] In one embodiment, when processor 501 executes a road defect detection program based on UAV imagery stored in memory 502, the following steps can be performed: The system acquires road surface image data collected by drones, divides the road surface image data into regions based on multiple preset spatial regions, and extracts visual features from the divided road surface image data to obtain a visual feature sequence. Multi-directional response processing is performed on the visual feature sequence to obtain the feature response results of the visual feature sequence in multiple directions. The feature response results of the visual feature sequence in multiple directions are then normalized to obtain a linear confidence sequence. The linear confidence sequence is used to represent the probability that each spatial region belongs to a linear or strip-shaped disease structure. The visual feature sequence is weighted based on the linear confidence sequence to obtain the visual feature enhancement sequence, and sparse feature sequence is extracted from the visual feature enhancement sequence based on iterative feature aggregation. The filtering weight vector is determined based on the correlation between historical road defect description information and visual feature sequence. Based on the filtering weight vector, road defect feature sequence is selected from sparse feature sequence, and the road defect detection result is determined based on the road defect feature sequence.

[0081] It should be understood that when the processor 501 executes the road defect detection program based on UAV imagery in the memory 502, in addition to the functions mentioned above, it can also perform other functions, as detailed in the description of the corresponding method embodiments above.

[0082] Furthermore, this embodiment of the invention does not specifically limit the type of the detection device 500 mentioned. The detection device 500 can be a portable electronic device such as a mobile phone, tablet computer, personal digital assistant (PDA), wearable device, or laptop computer. Exemplary embodiments of portable electronic devices include, but are not limited to, portable electronic devices running iOS, Android, Microsoft, or other operating systems. The aforementioned portable electronic devices can also be other portable electronic devices, such as laptop computers with touch-sensitive surfaces (e.g., touch panels). It should also be understood that in some other embodiments of the invention, the detection device 500 may not be a portable electronic device, but rather a desktop computer with a touch-sensitive surface (e.g., a touch panel).

[0083] Accordingly, this application also provides a computer-readable storage medium for storing computer-readable programs or instructions. When the programs or instructions are executed by a processor, they can implement the steps or functions of the road defect detection method based on UAV imagery provided in the above-described method embodiments.

[0084] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware (such as a processor, controller, etc.), and the computer program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0085] The road defect detection method and device based on UAV imagery provided by the present invention have been described in detail above. Specific examples have been used to illustrate the principle and implementation of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation and application scope based on the idea of ​​the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for detecting road defects based on UAV imagery, characterized in that, include: The system acquires road surface image data collected by drones, divides the road surface image data into regions based on multiple preset spatial regions, and extracts visual features from the divided road surface image data to obtain a visual feature sequence. Multi-directional response processing is performed on the visual feature sequence to obtain the feature response results of the visual feature sequence in multiple directions. The feature response results of the visual feature sequence in multiple directions are then normalized to obtain a linear confidence sequence. The linear confidence sequence is used to represent the probability that each spatial region belongs to a linear or strip-shaped disease structure. The visual feature sequence is weighted based on the linear confidence sequence to obtain the visual feature enhancement sequence, and sparse feature sequence is extracted from the visual feature enhancement sequence based on iterative feature aggregation. The filtering weight vector is determined based on the correlation between historical road defect description information and visual feature sequence. Based on the filtering weight vector, road defect feature sequence is selected from sparse feature sequence, and the road defect detection result is determined based on the road defect feature sequence.

2. The road defect detection method based on UAV imagery according to claim 1, characterized in that, The multi-directional response processing of the visual feature sequence to obtain the feature response results of the visual feature sequence in multiple directions includes: By applying a multi-directional response operator to the visual feature sequence, the feature response results of the visual feature sequence in multiple directions are obtained.

3. The road defect detection method based on UAV imagery according to claim 1, characterized in that, The normalization process of the feature response results of the visual feature sequence in multiple directions to obtain a linear confidence sequence includes: The linear confidence sequence is determined based on the maximum value of the normalized feature response results in each direction.

4. The road defect detection method based on UAV imagery according to claim 1, characterized in that, The weighted processing of the visual feature sequence based on the linear confidence sequence to obtain the visual feature enhancement sequence includes: The visual feature sequence is obtained by multiplying the linear confidence sequence element by element.

5. The road defect detection method based on UAV imagery according to claim 1, characterized in that, The extraction of sparse feature sequences from visual feature enhancement sequences based on iterative feature aggregation includes: In the first iteration, features are extracted from the visual feature enhancement sequence using the initial query vector. In each iteration, the query vector is updated based on the updated feature sequence obtained from the previous iteration. Features are then extracted from the visual feature enhancement sequence based on the updated query vector. After multiple iterations, a sparse feature sequence is obtained.

6. The road defect detection method based on UAV imagery according to claim 1, characterized in that, The process of determining the screening weight vector based on the correlation between historical road defect description information and visual feature sequences includes: A memory prior representation is constructed based on historical road defect description information, and a selection weight vector is determined based on the correlation between the memory prior representation and the visual feature sequence.

7. The road defect detection method based on UAV imagery according to claim 1, characterized in that, The method of determining road defect detection results based on road defect feature sequences includes: The road defect feature sequence is processed based on fully connected layers and activation functions to determine the probability of various road defects, and the road defect with the highest probability is identified as the road defect type information. Regression prediction is performed on the characteristic sequence of road defects to determine the geometric attribute information of road defects; The road defect type information and road defect geometric attribute information are used as the road defect detection results.

8. A road defect detection device based on UAV imagery, characterized in that, include: The acquisition module is used to acquire road surface image data collected by UAV, divide the road surface image data into regions based on multiple preset spatial regions, and extract visual features from the divided road surface image data to obtain a visual feature sequence. The processing module is used to perform multi-directional response processing on the visual feature sequence to obtain the feature response results of the visual feature sequence in multiple directions, and to normalize the feature response results of the visual feature sequence in multiple directions to obtain a linear confidence sequence. The linear confidence sequence is used to represent the probability that each spatial region belongs to a linear or strip-shaped disease structure. The extraction module is used to perform weighted processing on the visual feature sequence based on the linear confidence sequence to obtain the visual feature enhancement sequence, and to extract the sparse feature sequence from the visual feature enhancement sequence based on iterative feature aggregation. The detection module is used to determine the screening weight vector based on the correlation between historical road defect description information and visual feature sequence, to filter out road defect feature sequence from sparse feature sequence based on the screening weight vector, and to determine the road defect detection result based on the road defect feature sequence.

9. A testing device, characterized in that, Including memory and processor, among which, The memory is used to store programs; The processor, coupled to the memory, is used to execute the program stored in the memory to implement the steps in the road defect detection method based on UAV imagery as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, Used to store computer-readable programs or instructions, which, when executed by a processor, can implement the steps in the road defect detection method based on UAV imagery as described in any one of claims 1 to 7.

Images