A pavement disease image processing method

Abstract

The present invention discloses a pavement defect image processing method, which relates to the technical field of pavement defect detection. The method comprises the following steps: after obtaining an oblique photographic image of the road surface in front of a vehicle captured by a vehicle-mounted binocular camera, first using a target detection algorithm to perform pavement defect recognition processing on the oblique photographic image; then, for each identified pavement defect identification frame, intercepting the corresponding oblique photographic image of the pavement defect; combining the three-dimensional coordinates of the corresponding frame center and / or frame corner in the camera coordinate system, applying an AIGC-based AI drawing tool to obtain the corresponding pavement defect overhead photographic image, and also converting the corresponding pavement defect location; finally, extracting pavement defect parameters based on the overhead photographic image of the pavement defect corresponding to the pavement defect location and loading and displaying them; thereby effectively improving the accuracy of the final pavement defect detection result while still using oblique photography.

Description

Technical Field

[0001] The present invention belongs to the technical field of pavement disease detection, and in particular relates to a pavement disease image processing method. Background Art

[0002] With the rapid development of new infrastructure, smart highways based on "human-vehicle-road-environment collaboration" have become an inevitable trend. Infrastructure such as bridges, slopes, tunnels and pavements, as important components of smart highways, face their huge stock and stable increase. The intelligence level of their maintenance and management urgently needs to keep up with the development trend and be adapted.

[0003] Currently, existing road inspection methods primarily rely on patrol vehicles, which are typically modified or refitted from existing cars. For example, a camera is installed on the front of the vehicle to capture images of the road ahead. These images are then processed for road defect identification and detection. However, due to the limitations of the camera's mounting position, the images captured are primarily oblique, rather than bird's-eye, images, limiting the accuracy of the resulting road defect detection results. Summary of the Invention

[0004] The purpose of the present invention is to provide a pavement defect image processing method, device, computer equipment, computer-readable storage medium and computer program product to solve the problem of limited accuracy of the final pavement defect detection results obtained by existing pavement inspection methods due to oblique photography.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] In a first aspect, a method for processing pavement damage images is provided, comprising:

[0007] Obtaining an oblique photographic image of the road surface in front of the vehicle captured in real time by a vehicle-mounted binocular camera, wherein the vehicle-mounted binocular camera is mounted on the front of the vehicle body with the camera lens tilted toward the front and downward;

[0008] Using a target detection algorithm in real time to perform road surface defect recognition processing on the oblique photographic image of the road surface in front of the vehicle to obtain a road surface defect recognition result;

[0009] If at least one pavement defect identification frame is found according to the pavement defect identification result, for each pavement defect identification frame in the at least one pavement defect identification frame, a corresponding pavement defect oblique photographic image is captured from the oblique photographic image of the road surface in front of the vehicle;

[0010] For each pavement defect identification frame, the corresponding oblique photographic image of the pavement defect and the three-dimensional coordinates of the frame center and / or frame corners in the camera coordinate system of the vehicle-mounted binocular camera are imported into an AI drawing tool based on AIGC, and the corresponding overhead photographic image of the pavement defect is output;

[0011] For each road surface defect identification frame, the location of the corresponding road surface defect is determined based on the satellite positioning result of the front of the vehicle at the time of image acquisition and the three-dimensional coordinates of the center of the corresponding frame in the camera coordinate system of the vehicle-mounted binocular camera, wherein the image acquisition time refers to the time when the oblique photographic image of the road surface in front of the vehicle is acquired;

[0012] Extracting pavement disease parameters at the location of the pavement disease based on the overhead photographic image of the pavement disease corresponding to the location of the pavement disease;

[0013] The location of the pavement damage and the corresponding pavement damage parameters are loaded and displayed on the road map display page.

[0014] Based on the above invention content, a new solution for pavement defect detection based on a target detection algorithm and an AIGC drawing tool is provided. That is, after obtaining an oblique photographic image of the road surface in front of the vehicle captured by an on-board binocular camera, a target detection algorithm is first used to perform pavement defect recognition processing on the oblique photographic image. Then, for each identified pavement defect identification frame, the corresponding oblique photographic image of the pavement defect is intercepted. In combination with the three-dimensional coordinates of the corresponding frame center and / or frame corner in the camera coordinate system, an AI drawing tool based on AIGC is applied to obtain the corresponding pavement defect overhead photographic image, and also convert the corresponding pavement defect location. Finally, based on the pavement disease overhead photographic image corresponding to the pavement disease location, pavement disease parameters are extracted and loaded for display. In this way, the accuracy of the final pavement disease detection result can be effectively improved while still using oblique photography, which is convenient for practical application and promotion.

[0015] In one possible design, a target detection algorithm is used to perform road surface defect recognition processing on the oblique photographic image of the road surface in front of the vehicle to obtain a road surface defect recognition result, including:

[0016] The oblique photographic image of the road surface in front of the vehicle is imported into a pavement defect recognition model pre-trained based on a target detection algorithm, and a pavement defect recognition result is outputted, which can show whether there is a pavement disease identification box on the oblique photographic image of the road surface in front of the vehicle. The target detection algorithm adopts a three-stage network structure including a first-stage network, a second-stage network and a third-stage network. The first-stage network, the second-stage network and the third-stage network respectively use Resnet50 networks as backbone networks, and respectively use different void convolution methods with increasing void rates, so that the first-stage network, the second-stage network and the third-stage network are reverse cascaded in sequence from coarse to fine according to the features of the deep convolution layer.

[0017] In one possible design, in the last layer of the first-stage network, M1 sliding windows of different sizes are used to perform window scanning on the oblique photographic image of the road surface in front of the vehicle to obtain multiple first marked frames. Based on the normalized results of the probabilities of the multiple first marked frames containing the target road surface defect, a first non-maximum suppression algorithm is used to screen out N1 optimal first marked frames from the multiple first marked frames, where M1 represents a positive integer greater than or equal to 30, and N1 represents a positive integer greater than or equal to 3500.

[0018] In the last layer of the second-stage network, M2 sliding windows of different sizes are used to perform window scanning on the oblique photographic image of the road surface in front of the vehicle within the N1 best first marked frames to obtain a plurality of second marked frames, and based on normalized results of the probabilities of the plurality of second marked frames containing target road surface defects, a second non-maximum suppression algorithm is used to screen out N2 best second marked frames from the plurality of second marked frames, and a third non-maximum suppression algorithm is further used to aggregate the N2 best second marked frames and screen out N3 best third marked frames, where M2 represents a positive integer less than M1, N2 represents a positive integer less than N1, and N3 represents a positive integer less than N2;

[0019] A structured random forest is trained in the last layer of the third-stage network using the convolutional features of the last layer to estimate the edge contour of the target pavement defect. Based on the edge contour, a greedy iterative algorithm is used to search for a pavement defect identification frame with the maximum probability of containing the target pavement defect at different positions and aspect ratios within the N3 best third marking frames.

[0020] In one possible design, for each pavement defect identification frame, the corresponding oblique photographic image of the pavement defect and the three-dimensional coordinates of the frame center and / or frame corners in the camera coordinate system of the vehicle-mounted binocular camera are imported into an AI drawing tool based on AIGC, and the corresponding overhead photographic image of the pavement defect is output, including the following steps S41 to S43:

[0021] S41. For a pavement defect identification frame, the corresponding oblique photographic image of the pavement defect and the three-dimensional coordinates of the frame center and / or frame corners in the camera coordinate system of the vehicle-mounted binocular camera are imported into an AI drawing tool based on AIGC, and the corresponding overhead photographic image of the pavement defect is output, and then step S42 is executed;

[0022] S42. The image clarity evaluation algorithm is used to calculate the image clarity of the overhead photographic image of the road surface disease of the road surface disease identification frame, and then step S43 is executed;

[0023] S43. Determine whether the image clarity of the overhead photographic image of the pavement disease of the certain pavement disease identification frame is lower than a first preset threshold value. If so, return to step S41. Otherwise, use the most recently output overhead photographic image of the pavement disease as the final overhead photographic image of the pavement disease of the certain pavement disease identification frame.

[0024] In one possible design, when the oblique photographic image of the road surface in front of the vehicle is any frame of video data collected in real time by the on-board binocular camera, pavement defect parameters at the location of the pavement defect are extracted based on the overhead photographic image of the pavement defect corresponding to the location of the pavement defect, including:

[0025] Aggregating all the pavement disease overhead photographic images corresponding to the same pavement disease location to obtain a pavement disease overhead photographic image set;

[0026] For each of the pavement defect overhead photographic images in the pavement defect overhead photographic image set, if any frame edge of the corresponding pavement defect identification frame is found to be located at an edge area of ​​the inclined photographic image of the road surface in front of the vehicle, the corresponding image is discarded;

[0027] Arrange all remaining images in the pavement defect overhead photographic image set in order from latest to earliest according to the corresponding image acquisition moments to obtain an image sequence;

[0028] Selecting the first K pavement defect overhead photographic images from the image sequence, and calculating the corresponding image clarity of each of the first K pavement defect overhead photographic images using an image clarity assessment algorithm, where K represents a positive integer;

[0029] The pavement disease parameters at the location of the same pavement disease are extracted based on a pavement disease overhead photographic image having the highest image clarity among the first K pavement disease overhead photographic images.

[0030] In one possible design, pavement defect parameters at the location of the same pavement defect are extracted based on a pavement defect overhead photographic image with the highest image clarity among the first K pavement defect overhead photographic images, including:

[0031] Selecting a pavement disease overhead photographic image with the highest image clarity from the first K pavement disease overhead photographic images;

[0032] If it is determined that the highest image clarity reaches a second preset threshold, pavement disease parameters at the location of the same pavement disease are extracted based on the overhead photographic image of the pavement disease;

[0033] If it is determined that the maximum image clarity is lower than the second preset threshold, an image acquisition task is generated to instruct the drone to go to the sky above the same pavement defect location to collect a bird's-eye view photographic image of the road surface, and then the image acquisition task is dispatched to the vehicle-mounted drone. Finally, based on the bird's-eye view photographic image of the road surface collected by the vehicle-mounted drone above the same pavement defect location, the pavement disease parameters at the same pavement defect location are extracted.

[0034] In a second aspect, a pavement defect image processing device is provided, comprising a photographic image acquisition unit, a pavement defect recognition unit, an oblique image interception unit, a bird's-eye view image generation unit, a defect location determination unit, a defect parameter extraction unit, and a detection result loading unit;

[0035] The photographic image acquisition unit is used to acquire an oblique photographic image of the road surface in front of the vehicle collected in real time by a vehicle-mounted binocular camera, wherein the vehicle-mounted binocular camera is installed at the front of the vehicle body with the camera lens tilted toward the front and downward;

[0036] The road surface defect recognition unit is communicatively connected to the photographic image acquisition unit and is used to perform road surface defect recognition processing on the oblique photographic image of the road surface in front of the vehicle using a target detection algorithm in real time to obtain a road surface defect recognition result;

[0037] The oblique image capture unit is communicatively connected to the pavement defect identification unit and is configured to capture, for each pavement defect identification frame within the at least one pavement defect identification frame, a corresponding oblique pavement defect image from the oblique photographic image of the road surface in front of the vehicle;

[0038] The overhead image generation unit is communicatively connected to the oblique image capture unit and is configured to import, for each pavement defect identification frame, the corresponding oblique photographic image of the pavement defect and the three-dimensional coordinates of the frame center and / or frame corners in the camera coordinate system of the on-board binocular camera into an AI drawing tool based on AIGC, thereby outputting the corresponding overhead photographic image of the pavement defect;

[0039] The defect location determination unit is communicatively connected to the oblique image interception unit and is configured to determine the location of the corresponding road surface defect for each road surface defect identification frame based on the satellite positioning result of the front of the vehicle at the time of image acquisition and the three-dimensional coordinates of the center of the corresponding frame in the camera coordinate system of the vehicle-mounted binocular camera, wherein the image acquisition time refers to the time when the oblique photographic image of the road surface in front of the vehicle is acquired;

[0040] The disease parameter extraction unit is communicatively connected to the overhead image generation unit and the disease location determination unit, and is used to extract the pavement disease parameter at the location of the pavement disease based on the overhead photographic image of the pavement disease corresponding to the location of the pavement disease;

[0041] The detection result loading unit is communicatively connected to the disease parameter extraction unit, and is used to load and display the location of the road disease and the corresponding road disease parameters on the road map display page.

[0042] In a third aspect, the present invention provides a computer device comprising a memory, a processor, and a transceiver communicatively connected in sequence, wherein the memory is used to store a computer program, the transceiver is used to send and receive messages, and the processor is used to read the computer program and execute the pavement disease image processing method as described in the first aspect or any possible design of the first aspect.

[0043] In a fourth aspect, the present invention provides a computer-readable storage medium having instructions stored thereon. When the instructions are run on a computer, the pavement disease image processing method as described in the first aspect or any possible design of the first aspect is executed.

[0044] In a fifth aspect, the present invention provides a computer program product, comprising a computer program or instructions, which, when executed by a computer, implements the pavement disease image processing method as described in the first aspect or any possible design of the first aspect.

[0045] Beneficial effects of the above scheme:

[0046] (1) The present invention creatively provides a new solution for pavement disease detection based on target detection algorithm and AIGC drawing tool, that is, after obtaining the oblique photographic image of the road surface in front of the vehicle captured by the vehicle-mounted binocular camera, the target detection algorithm is first used to perform pavement disease recognition processing on the oblique photographic image, and then for each identified pavement disease identification frame, the corresponding pavement disease oblique photographic image is intercepted, and combined with the three-dimensional coordinates of the corresponding frame center and / or frame corner in the camera coordinate system, the AI ​​drawing tool based on AIGC is used to obtain the corresponding pavement disease overhead photographic image, and also convert the corresponding pavement disease location, and finally, according to the pavement disease overhead photographic image corresponding to the pavement disease location, the pavement disease parameters are extracted and loaded for display. In this way, the accuracy of the final pavement disease detection result can be effectively improved while still using oblique photography, which is convenient for practical application and promotion. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0048] Figure 1 A schematic flow chart of the pavement damage image processing method provided in an embodiment of the present application.

[0049] Figure 2 Schematic diagram of the installation position of the vehicle-mounted binocular camera provided in an embodiment of the present application on the front of the vehicle body.

[0050] Figure 3 A schematic diagram of a three-stage network structure provided in an embodiment of the present application.

[0051] Figure 4 This is a schematic diagram of the structure of the road surface disease image processing device provided in an embodiment of the present application.

[0052] Figure 5 A schematic diagram of the structure of a computer device provided in an embodiment of the present application.

[0053] In the above drawings: 100 - front part of the vehicle body; 101 - first binocular camera; 102 - second binocular camera. DETAILED DESCRIPTION

[0054] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the present invention will be briefly introduced below in conjunction with the drawings and the description of the embodiments or the prior art. Obviously, the following description of the structure of the drawings is only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work. It should be noted that the description of these embodiments is used to help understand the present invention, but does not constitute a limitation of the present invention.

[0055] It should be understood that although the terms first, second, etc. may be used herein to describe various objects, these objects should not be limited by these terms. These terms are merely used to distinguish one object from another. For example, a first object can be referred to as a second object, and similarly, a second object can be referred to as a first object without departing from the scope of the exemplary embodiments of the present invention.

[0056] It should be understood that the term "and / or" that may appear in this document is merely a description of the association relationship between associated objects, indicating that there may be three relationships. For example, A and / or B can indicate three situations: A exists alone, B exists alone, or A and B exist at the same time. For another example, A, B and / or C can indicate the existence of any one of A, B and C or any combination of them. The term " / and" that may appear in this document describes another type of association object relationship, indicating that there may be two relationships. For example, A / and B can indicate two situations: A exists alone or A and B exist at the same time. In addition, the character " / " that may appear in this document generally indicates that the previous and next associated objects are in an "or" relationship.

[0057] Example:

[0058] like Figure 1 As shown, the pavement disease image processing method provided in the first aspect of this embodiment can be, but is not limited to, executed by a computer device with certain computing resources and in communication with a vehicle-mounted binocular camera, such as a vehicle-mounted computer (also known as a computer control module, in English, Electronic Control Unit, abbreviated as ECU), a platform server, a personal computer (Personal Computer, PC, refers to a multi-purpose computer with a size, price and performance suitable for personal use; desktops, laptops, small laptops and tablets and ultrabooks are all personal computers), a smart phone, a personal digital assistant (Personal Digital Assistant, PDA) or a wearable device. Figure 1 As shown, the pavement damage image processing method may include, but is not limited to, the following steps S1 to S7.

[0059] S1. Obtain an oblique photographic image of the road surface in front of the vehicle captured in real time by a vehicle-mounted binocular camera, wherein the vehicle-mounted binocular camera is installed at the front of the vehicle body with the camera lens tilted toward the front and downward.

[0060] In step S1, the binocular camera is a camera based on the binocular vision principle (i.e., the principle is: to achieve three-dimensional measurement through parallax measurement of two cameras; specifically, the farther the object is from the camera, the more obvious the difference in images captured by the left and right cameras is. By measuring this difference, the binocular camera can calculate information such as the distance and shape of the object). It consists of two cameras and an image processing unit, and can obtain three-dimensional information of the object, including the distance, shape, and size of the object, through parallax measurement of the two cameras. The lens field of view of the on-board binocular camera will cover the road area directly in front of the front of the vehicle body, and is used to collect oblique photographic images of the road area directly in front in real time, and obtain video data containing several continuous video frame images (all of which are oblique photographic images of the road surface); the specific installation position of the on-board binocular camera can be, but is not limited to, Figure 2 As shown: the vehicle-mounted binocular camera can be the first binocular camera 101 or the second binocular camera 102, and the angle between the lens of the vehicle-mounted binocular camera and the front is less than 30 degrees ( Figure 2 In addition, the vehicle-mounted binocular camera can transmit the collected data to the local device in a conventional manner.

[0061] S2. Using a target detection algorithm in real time to perform road surface defect recognition processing on the oblique photographic image of the road surface in front of the vehicle to obtain a road surface defect recognition result.

[0062] In step S2, the target detection algorithm is an existing artificial intelligence recognition algorithm for identifying objects in an image and marking the location of the objects. Specifically, it can be, but is not limited to, Faster R-CNN (Faster Regions with Convolutional Neural Networks features, a target detection algorithm proposed by He Kaiming et al. in 2015, which won multiple first places in the 2015 ILSVRV and COCO competitions) target detection algorithm, SSD (Single Shot MultiBox Detector, a target detection algorithm proposed by Wei Liu at ECCV 2016, which is one of the currently popular detection frameworks) target detection algorithm or YOLO (You only look Once, the latest version has been developed to V4 and is widely used in the industry. Its basic principle is: first, the input image is divided into a 7x7 grid, two bounding boxes are predicted for each grid, and then the target windows with relatively low probability are removed based on a threshold. Finally, the redundant windows are removed by using bounding box merging to obtain the detection result. (Target detection algorithm, etc.). Therefore, road surface defect identification processing can be performed based on this target detection algorithm to obtain the road surface defect identification result: whether there is a road surface defect identification box on the oblique photographic image of the road surface in front of the vehicle. In addition, the road surface defects that need to be identified include, but are not limited to, manhole covers, well holes, potholes to be repaired, and cracked road surfaces.

[0063] In the step S2, preferably, a target detection algorithm is used to perform pavement disease recognition processing on the oblique photographic image of the road in front of the vehicle to obtain a pavement disease recognition result, including but not limited to the following steps: importing the oblique photographic image of the road in front of the vehicle into a pavement disease recognition model pre-trained based on the target detection algorithm, and outputting a pavement disease recognition result that can show whether there is a pavement disease identification box on the oblique photographic image of the road in front of the vehicle, wherein the target detection algorithm adopts a three-stage network structure including a first-stage network, a second-stage network and a third-stage network, wherein the first-stage network, the second-stage network and the third-stage network respectively use the Resnet50 network as the backbone network, and respectively use different void convolution methods with increasing void rates, so that the first-stage network, the second-stage network and the third-stage network are reverse cascaded in sequence from coarse to fine according to the features of the deep convolution layer. The specific structure of the three-stage network structure is as follows: Figure 3As shown, the specific structure of the Resnet50 network is the existing network structure, which not only proposes a residual structure, but also proposes a skeleton network design paradigm, namely stem+n stage+cls head. For ResNet, its actual forward process is stem->4 stage->classification head, where the output stride of the stem (whose Input is a three-channel image, which first undergoes convolution operation, normalization operation, RELU operation, and then undergoes maximum pooling operation to obtain Output) (which represents the downsampling rate of the model, assuming that the input image size is 320×320, stride=10, then the output feature map size is 32×32, assuming that there are 9 anchors at each position, then the output feature map has a total of 32×32×9 position anchors) is 4, and the output strides of the four stages are 4, 8, 16 and 32 respectively. The dilated convolution method refers to filling the middle of the 3×3 convolution kernel with 0. The calculation method is the same as the standard convolution. The dilated convolution can expand the receptive field without losing the resolution problem caused by the previous downsampling step, and can capture multi-scale contextual information. Similar to the YOLO algorithm, a dilated convolution with a small dilation rate is used to detect large targets, and a dilated convolution with a large dilation rate is used to detect small targets. Therefore, through the three-stage network structure design of the aforementioned target detection algorithm, the following road surface disease detection process can be implemented: using the coarse-to-fine inverse cascade method of the features of the deep convolution layer, possible object suggestions (that is, the detected targets) are obtained in the image; dense suggestion sampling is performed in the last convolution layer of the first stage network, and then irrelevant boxes are gradually filtered until the last layer of the third stage network; in the third stage network, the contours extracted from the last layer of the third stage network are used to refine the suggestions; the finally generated road surface disease identification box can be used in the object detection process; finally, the problem of imbalance between background noise and foreground object information is solved, and the performance of multi-scale target detection is improved.

[0064] In the step S2, specifically, in the last layer of the first-stage network, M1 sliding windows with different sizes are used to perform window scanning on the oblique photographic image of the road surface in front of the vehicle, so as to obtain a plurality of first marking frames, and according to the normalized results of the probabilities of the plurality of first marking frames containing the target road surface disease, N1 best first marking frames are screened out from the plurality of first marking frames using a first non-maximum suppression algorithm, wherein M1 represents a positive integer greater than or equal to 30 (for example, 50), and N1 represents a positive integer greater than or equal to 3500 (for example, 4000); in the last layer of the second-stage network, M2 sliding windows with different sizes are used to perform window scanning on the oblique photographic image of the road surface in front of the vehicle within the N1 best first marking frames, so as to obtain a plurality of second marking frames, and according to the normalized results of the probabilities of the plurality of first marking frames containing the target road surface disease, N1 best first marking frames are screened out from the plurality of first marking frames. and contains the normalized result of the probability of the target pavement disease, using a second non-maximum suppression algorithm to screen out N2 best second labeling frames from the multiple second labeling frames, and also using a third non-maximum suppression algorithm to aggregate the N2 best second labeling frames and screen out N3 best third labeling frames, wherein M2 represents a positive integer less than M1 (for example, 10), N2 represents a positive integer less than N1 (for example, 3000), and N3 represents a positive integer less than N2 (for example, 1000); in the last layer of the third stage network, a structured random forest is trained using the convolutional features of the last layer to estimate the edge contour of the target pavement disease, and according to the edge contour, a greedy iterative algorithm is used to search and try to find a pavement disease identification frame with the maximum probability of containing the target pavement disease at different positions and aspect ratios within the N3 best third labeling frames.

[0065] In the step S2, specifically, the first-stage network, the second-stage network, and the third-stage network also respectively use the Focal Loss loss function as the loss function. The aforementioned Focal Loss loss function is introduced based on the cross entropy (CE) loss function of the binary classification problem. It is a dynamically scaled cross entropy loss. Through a dynamic scaling factor, the weight of easily distinguishable samples can be dynamically reduced during training, thereby quickly focusing on those difficult-to-distinguish samples. The specific form is as follows:

[0066] FL(p t )=-α×(1-p t ) γ ×log(p t )

[0067] Among them, α is used to balance positive and negative samples, for example, the value is 0.25. p represents the probability when the predicted sample is equal to 1, y represents the predicted value of the sample, and γ represents a preset constant (for example, 2). Figure 3 As shown, specifically, the first-stage network adopts a dilated convolution method with a dilation rate of 2, the second-stage network adopts a dilated convolution method with a dilation rate of 8, and the third-stage network adopts a dilated convolution method with a dilation rate of 16.

[0068] S3. If at least one pavement defect identification frame is found according to the pavement defect identification result, then for each pavement defect identification frame in the at least one pavement defect identification frame, a corresponding pavement defect oblique photography image is captured from the oblique photography image of the road surface in front of the vehicle.

[0069] In step S3, the at least one road surface damage identification frame specifically includes but is not limited to a manhole cover identification frame, a manhole identification frame, a pothole to be repaired identification frame and / or a cracked road surface identification frame.

[0070] S4. For each of the pavement defect identification frames, the corresponding oblique photographic image of the pavement defect and the three-dimensional coordinates of the frame center and / or frame corners in the camera coordinate system of the on-board binocular camera are imported into an AI drawing tool based on AIGC (Artificial Intelligence Generated Content, generative artificial intelligence, which is an important symbol of the transition from the artificial intelligence 1.0 era to the 2.0 era), and the corresponding overhead photographic image of the pavement defect is output.

[0071] In step S4, since the oblique photographic image of the road ahead of the vehicle is acquired in real time by the onboard binocular camera, each pixel in the oblique photographic image of the road ahead of the vehicle has three-dimensional information, and thus the three-dimensional coordinates of the frame center and / or any frame corner of the road surface defect identification frame in the camera coordinate system of the onboard binocular camera can be easily obtained. The AI ​​drawing tool is an existing drawing software, which may include but is not limited to AI drawing software such as Midjourrney, Dell-E, Stable Diffusion, NovelAI, and Disco Diffusion. Since these software are large models with many parameters, long training time, good generalization, strong versatility, and high practicality, and are applicable to drawing in various scenarios, they are well suited for this embodiment, that is, based on the oblique photographic image of the road surface defect and the frame center and / or frame corners of the road surface defect identification frame, the road surface defect overhead photographic image corresponding to the road surface defect identification frame is generated. In order to ensure that the generated top-view photographic image of the pavement defect has ideal image clarity, preferably, for each pavement defect identification frame, the corresponding oblique photographic image of the pavement defect and the three-dimensional coordinates of the frame center and / or frame corners in the camera coordinate system of the vehicle-mounted binocular camera are imported into an AI drawing tool based on AIGC, and the corresponding top-view photographic image of the pavement defect is output, including but not limited to the following steps S41 to S43.

[0072] S41. For a certain pavement defect identification frame, the corresponding oblique photographic image of the pavement defect and the three-dimensional coordinates of the frame center and / or frame corners in the camera coordinate system of the on-board binocular camera are imported into the AI ​​drawing tool based on AIGC, and the corresponding overhead photographic image of the pavement defect is output, and then step S42 is executed.

[0073] S42. Use an image clarity evaluation algorithm to calculate the image clarity of the pavement disease overhead photographic image of the pavement disease identification frame, and then execute step S43.

[0074] In step S42, the image clarity evaluation algorithm may be implemented by, but is not limited to, the existing Brenner gradient method, Tenegrad gradient method, Laplace gradient method, variance method, or energy gradient method.

[0075] S43. Determine whether the image clarity of the overhead photographic image of the pavement disease of the certain pavement disease identification frame is lower than a first preset threshold value. If so, return to step S41. Otherwise, use the most recently output overhead photographic image of the pavement disease as the final overhead photographic image of the pavement disease of the certain pavement disease identification frame.

[0076] In step S43, the first preset threshold, which can be preset by the user, serves as a criterion for determining whether the generated overhead photographic image of the pavement defect meets the minimum image clarity requirement. Furthermore, if the image clarity of the multiple overhead photographic images of the pavement defect is lower than the first preset threshold, step S41 may be terminated, and the single overhead photographic image of the pavement defect with the highest image clarity among the multiple overhead photographic images of the pavement defect is used as the final overhead photographic image of the pavement defect for the pavement defect identification frame.

[0077] S5. For each of the pavement defect identification frames, determine the location of the corresponding pavement defect based on the satellite positioning result of the front of the vehicle at the time of image acquisition and the three-dimensional coordinates of the corresponding frame center in the camera coordinate system of the on-board binocular camera, wherein the image acquisition time refers to the time when the oblique photographic image of the road surface in front of the vehicle is acquired.

[0078] In step S5, the satellite positioning result can be conventionally acquired by a satellite positioning module (e.g., a Beidou satellite positioning module) fixedly mounted on the front of the vehicle body, and is specifically a three-dimensional coordinate in a geodetic coordinate system. Since the on-board binocular camera is fixedly mounted on the front of the vehicle body, the positional relationship between the front of the vehicle body and the on-board binocular camera is fixed, and thus the three-dimensional coordinates of the center of the pavement defect identification frame in the geodetic coordinate system, i.e., the location of the pavement defect, can be converted based on a conventional coordinate system transformation method. In addition, since different oblique photographic images of the road surface in front of the vehicle are acquired at different acquisition times for the same pavement defect object by the on-board binocular camera, when the oblique photographic image of the road surface in front of the vehicle is any frame of video data acquired in real time by the on-board binocular camera, there may be a situation where multiple pavement defect identification frames correspond to the same pavement defect object (i.e., multiple pavement defect identification frames with different acquisition times may correspond to the same pavement defect location).

[0079] S6. Extracting pavement disease parameters at the location of the pavement disease based on the overhead photographic image of the pavement disease corresponding to the location of the pavement disease.

[0080] In step S6, different pavement defects have different pavement defect parameters. For example, the pavement defect parameters may include, but are not limited to, the radius and / or size of a manhole cover, the radius and / or opening area of ​​a manhole, the width, length, and / or opening area of ​​a pothole to be repaired, or the width, length, and / or area of ​​cracks in a cracked pavement. These parameters can be obtained by conventionally measuring the overhead photographic image of the pavement defect. Because the location of the pavement defect corresponds to the pavement defect identification frame (either a one-to-one or a one-to-many relationship), and the pavement defect identification frame corresponds to the overhead photographic image of the pavement defect, the location of the pavement defect corresponds to at least one overhead photographic image of the pavement defect. In order to achieve the purpose of obtaining the most accurate pavement disease parameters based on multiple overhead photographic images of the pavement disease, preferably, when the oblique photographic image of the road surface in front of the vehicle is any frame of video image in the video data collected in real time by the on-board binocular camera, the pavement disease parameters at the location of the pavement disease are extracted based on the overhead photographic image of the pavement disease corresponding to the location of the pavement disease, including but not limited to the following steps S61 to S65.

[0081] S61. Summarize all the pavement defect overhead photographic images corresponding to the same pavement defect location to obtain a pavement defect overhead photographic image set.

[0082] S62. For each pavement defect overhead photographic image in the pavement defect overhead photographic image set, if any frame edge of the corresponding pavement defect identification frame is found to be located at the edge area of ​​the inclined photographic image of the road surface in front of the vehicle, the corresponding image is discarded.

[0083] In step S62, it is considered that during the inspection process, the position of the pavement defect image in the oblique photographic image of the road ahead of the vehicle may shift from the central area to the edge area until it disappears. This may result in the detected pavement defect image being incomplete when the inspection vehicle passes by the pavement defect. Therefore, based on step S62, these suspected incomplete pavement defect overhead photographic images can be eliminated. Furthermore, the edge area is preferably the edge area of ​​a predetermined region of interest.

[0084] S63. Arrange all remaining images in the pavement damage overhead photographic image set in order from latest to earliest according to the corresponding image acquisition time to obtain an image sequence.

[0085] In step S63, since the overhead photographic image of the pavement defect is generated based on the oblique photographic image of the pavement defect, and the oblique photographic image of the pavement defect is captured from the oblique photographic image of the pavement in front of the vehicle, each overhead photographic image of the pavement defect will correspond to a corresponding image acquisition moment.

[0086] S64. Select the first K overhead photographic images of pavement defects from the image sequence, and use an image clarity assessment algorithm to calculate the corresponding image clarity for each of the first K overhead photographic images of pavement defects, where K represents a positive integer.

[0087] In step S64, K can be, for example, set to 10 and can be positively correlated with the acquisition frequency of the oblique photographic image of the road ahead of the vehicle (i.e., the higher the acquisition frequency, the larger the value of K). Furthermore, the image clarity assessment algorithm can also be implemented using, but is not limited to, existing Brenner gradient method, Tenegrad gradient method, Laplace gradient method, variance method, or energy gradient method.

[0088] S65. Extract the pavement disease parameters at the location of the same pavement disease based on a pavement disease overhead photographic image with the highest image clarity among the first K pavement disease overhead photographic images.

[0089] In step S65, considering that the highest image clarity may also be undesirable, in order to further ensure the accuracy of the extracted pavement disease parameters, preferably, based on a pavement disease overhead photographic image with the highest image clarity among the first K pavement disease overhead photographic images, the pavement disease parameters at the location of the same pavement disease are extracted, including but not limited to the following steps S651 to S653.

[0090] S651. Select a pavement damage overhead photographic image with the highest image clarity from the first K pavement damage overhead photographic images.

[0091] S652. If it is determined that the highest image clarity reaches a second preset threshold, pavement disease parameters at the location of the same pavement disease are extracted based on the overhead photographic image of the pavement disease.

[0092] In step S652, the second preset threshold is used as a criterion for determining whether a road surface defect overhead photographic image with the highest image clarity meets the minimum image clarity requirement. The second preset threshold can be preset by the user. Furthermore, the second preset threshold is generally greater than or equal to the first preset threshold.

[0093] S653. If it is determined that the maximum image clarity is lower than the second preset threshold, an image acquisition task is generated to instruct the drone to go to the sky above the same pavement defect location to collect a bird's-eye view photographic image of the road surface, and then the image acquisition task is dispatched to the vehicle-mounted drone. Finally, based on the bird's-eye view photographic image of the road surface collected by the vehicle-mounted drone above the same pavement defect location, the pavement disease parameters at the same pavement defect location are extracted.

[0094] In step S653, the vehicle-mounted drone can be in flight, following the inspection vehicle, or parked on the vehicle. Upon receiving the image acquisition task, it will immediately fly above the location of the same pavement defect and capture a bird's-eye view of the road surface using a camera located on its abdomen with its lens facing vertically downward (preferably a binocular camera). It will then return or continue to follow the inspection vehicle. Considering that tunnel sections are not conducive to drone flight, the image acquisition task may be generated and assigned to the vehicle-mounted drone only when the location of the same pavement defect is in a non-tunnel section. In addition, the target detection algorithm can be used to first perform pavement disease recognition processing on the overhead photographic image of the pavement, and then a overhead photographic image of the pavement disease at the location of the same pavement disease can be intercepted based on the recognition result. Finally, the pavement disease parameters at the location of the same pavement disease can be extracted based on the overhead photographic image of the pavement disease. The overhead photographic image of the pavement disease can also be used as a model output item, and the oblique photographic image of the pavement disease corresponding to the location of the same pavement disease and the frame center and / or frame corners of the pavement disease identification frame can be used as model input items to perform online training on the AI ​​drawing tool to obtain a better AI drawing tool.

[0095] S7. Load and display the location of the pavement damage and the corresponding pavement damage parameters on the road map display page.

[0096] Therefore, based on the pavement defect image processing method described in the aforementioned steps S1 to S7, a new solution for pavement defect detection based on a target detection algorithm and an AIGC drawing tool is provided. That is, after obtaining an oblique photographic image of the road surface in front of the vehicle captured by a vehicle-mounted binocular camera, a target detection algorithm is first used to perform pavement defect recognition processing on the oblique photographic image. Then, for each identified pavement defect identification frame, the corresponding pavement disease oblique photographic image is intercepted, and combined with the three-dimensional coordinates of the corresponding frame center and / or frame corner in the camera coordinate system, the AI ​​drawing tool based on AIGC is applied to obtain the corresponding pavement disease overhead photographic image, and also convert the corresponding pavement disease location. Finally, based on the pavement disease overhead photographic image corresponding to the pavement disease location, the pavement disease parameters are extracted and loaded for display. In this way, the accuracy of the final pavement disease detection result can be effectively improved while still using oblique photography, which is convenient for practical application and promotion.

[0097] like Figure 4 As shown, the second aspect of this embodiment provides a virtual device for implementing the pavement defect image processing method described in the first aspect, comprising a photographic image acquisition unit, a pavement defect recognition unit, an oblique image capture unit, a bird's-eye view image generation unit, a defect location determination unit, a defect parameter extraction unit, and a detection result loading unit;

[0098] The photographic image acquisition unit is used to acquire an oblique photographic image of the road surface in front of the vehicle collected in real time by a vehicle-mounted binocular camera, wherein the vehicle-mounted binocular camera is installed at the front of the vehicle body with the camera lens tilted toward the front and downward;

[0099] The road surface defect recognition unit is communicatively connected to the photographic image acquisition unit and is used to perform road surface defect recognition processing on the oblique photographic image of the road surface in front of the vehicle using a target detection algorithm in real time to obtain a road surface defect recognition result;

[0100] The oblique image capture unit is communicatively connected to the pavement defect identification unit and is configured to capture, for each pavement defect identification frame within the at least one pavement defect identification frame, a corresponding oblique pavement defect image from the oblique photographic image of the road surface in front of the vehicle;

[0101] The overhead image generation unit is communicatively connected to the oblique image capture unit and is configured to import, for each pavement defect identification frame, the corresponding oblique photographic image of the pavement defect and the three-dimensional coordinates of the frame center and / or frame corners in the camera coordinate system of the on-board binocular camera into an AI drawing tool based on AIGC, thereby outputting the corresponding overhead photographic image of the pavement defect;

[0102] The defect location determination unit is communicatively connected to the oblique image interception unit and is configured to determine the location of the corresponding road surface defect for each road surface defect identification frame based on the satellite positioning result of the front of the vehicle at the time of image acquisition and the three-dimensional coordinates of the center of the corresponding frame in the camera coordinate system of the vehicle-mounted binocular camera, wherein the image acquisition time refers to the time when the oblique photographic image of the road surface in front of the vehicle is acquired;

[0103] The disease parameter extraction unit is communicatively connected to the overhead image generation unit and the disease location determination unit, and is used to extract the pavement disease parameter at the location of the pavement disease based on the overhead photographic image of the pavement disease corresponding to the location of the pavement disease;

[0104] The detection result loading unit is communicatively connected to the disease parameter extraction unit, and is used to load and display the location of the road disease and the corresponding road disease parameters on the road map display page.

[0105] The working process, working details and technical effects of the aforementioned device provided in the second aspect of this embodiment can be found in the pavement disease image processing method described in the first aspect, and will not be described in detail here.

[0106] like Figure 5 As shown, the third aspect of this embodiment provides a computer device for executing the pavement disease image processing method as described in the first aspect, comprising a memory, a processor, and a transceiver that are sequentially connected in communication, wherein the memory is used to store a computer program, the transceiver is used to send and receive messages, and the processor is used to read the computer program and execute the pavement disease image processing method as described in the first aspect. For example, the memory may include, but is not limited to, a random access memory (RAM), a read-only memory (ROM), a flash memory, a first-in first-out memory (FIFO), and / or a first-in last-out memory (FILO), etc.; the processor may include, but is not limited to, a microprocessor of the STM32F105 series. In addition, the computer device may also include, but is not limited to, a power module, a display screen, and other necessary components.

[0107] The working process, working details and technical effects of the aforementioned computer device provided in the third aspect of this embodiment can be found in the pavement disease image processing method described in the first aspect, and will not be described in detail here.

[0108] A fourth aspect of this embodiment provides a computer-readable storage medium storing instructions including the pavement damage image processing method described in the first aspect. Specifically, the computer-readable storage medium stores instructions that, when executed on a computer, execute the pavement damage image processing method described in the first aspect. The computer-readable storage medium refers to a data storage medium and may include, but is not limited to, a floppy disk, an optical disk, a hard disk, a flash memory, a USB flash drive, and / or a memory stick. The computer may be a general-purpose computer, a dedicated computer, a computer network, or other programmable device.

[0109] The working process, working details and technical effects of the aforementioned computer-readable storage medium provided in the fourth aspect of this embodiment can be referred to the pavement disease image processing method described in the first aspect, and will not be repeated here.

[0110] A fifth aspect of this embodiment provides a computer program product, including a computer program or instructions, which, when executed by a computer, implements the pavement defect image processing method described in the first aspect. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device.

[0111] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention shall be included in the scope of protection of the present invention.

Claims

1. A method for processing pavement disease images, characterized in that: include: Obtaining an oblique photographic image of the road surface in front of the vehicle captured in real time by a vehicle-mounted binocular camera, wherein the vehicle-mounted binocular camera is mounted on the front of the vehicle body with the camera lens tilted toward the front and downward; Using a target detection algorithm in real time to perform road surface defect recognition processing on the oblique photographic image of the road surface in front of the vehicle to obtain a road surface defect recognition result; If at least one pavement defect identification frame is found according to the pavement defect identification result, for each pavement defect identification frame in the at least one pavement defect identification frame, a corresponding pavement defect oblique photographic image is captured from the oblique photographic image of the road surface in front of the vehicle; For each pavement defect identification frame, the corresponding oblique photographic image of the pavement defect and the three-dimensional coordinates of the frame center and / or frame corners in the camera coordinate system of the vehicle-mounted binocular camera are imported into an AI drawing tool based on AIGC, and the corresponding overhead photographic image of the pavement defect is output; For each road surface defect identification frame, the location of the corresponding road surface defect is determined based on the satellite positioning result of the front of the vehicle at the time of image acquisition and the three-dimensional coordinates of the center of the corresponding frame in the camera coordinate system of the vehicle-mounted binocular camera, wherein the image acquisition time refers to the time when the oblique photographic image of the road surface in front of the vehicle is acquired; Extracting pavement disease parameters at the location of the pavement disease based on the overhead photographic image of the pavement disease corresponding to the location of the pavement disease, specifically including: aggregating all the overhead photographic images of the pavement disease corresponding to the same location of the pavement disease to obtain a pavement disease overhead photographic image set; for each pavement disease overhead photographic image in the pavement disease overhead photographic image set, if any frame edge of the corresponding pavement disease identification frame is found to be located at the edge area of ​​the inclined photographic image of the road surface in front of the vehicle, the corresponding image will be eliminated; arranging all the remaining images in the pavement disease overhead photographic image set in order from the latest to the earliest time of the corresponding image acquisition time to obtain an image sequence; selecting the front image from the image sequence. A bird's-eye view of the road surface disease photographs, and for the For each of the pavement disease overhead photographic images, the corresponding image clarity is calculated using the image clarity evaluation algorithm, where: Represents a positive integer; according to the A pavement disease overhead photographic image with the highest image clarity among the pavement disease overhead photographic images is extracted to obtain pavement disease parameters at the location of the same pavement disease; The location of the pavement damage and the corresponding pavement damage parameters are loaded and displayed on the road map display page.

2. The pavement damage image processing method according to claim 1, characterized in that: The target detection algorithm is used to perform road surface defect recognition processing on the oblique photographic image of the road surface in front of the vehicle to obtain a road surface defect recognition result, including: The oblique photographic image of the road surface in front of the vehicle is imported into a pavement defect recognition model pre-trained based on a target detection algorithm, and a pavement defect recognition result is outputted, which can show whether there is a pavement disease identification box on the oblique photographic image of the road surface in front of the vehicle. The target detection algorithm adopts a three-stage network structure including a first-stage network, a second-stage network and a third-stage network. The first-stage network, the second-stage network and the third-stage network respectively use Resnet50 networks as backbone networks, and respectively use different void convolution methods with increasing void rates, so that the first-stage network, the second-stage network and the third-stage network are reverse cascaded in sequence from coarse to fine according to the features of the deep convolution layer.

3. The pavement damage image processing method according to claim 2, characterized in that: In the last layer of the first stage network, different sizes of A sliding window is used to perform a window scan on the oblique photographic image of the road surface in front of the vehicle to obtain a plurality of first marking frames, and a first non-maximum suppression algorithm is used to filter out the target road surface defects from the plurality of first marking frames according to the normalized results of the probabilities of the plurality of first marking frames containing the target road surface defects. The best first marked box, where represents a positive integer greater than or equal to 30, Represents a positive integer greater than or equal to 3500; In the last layer of the second stage network, different sizes of A sliding window is used to perform the tilted photographic image of the road surface in front of the vehicle on the The window within the best first mark frame is scanned to obtain multiple second mark frames, and the second non-maximum suppression algorithm is used to filter out the target road disease from the multiple second mark frames according to the normalized results of the probabilities of the multiple second mark frames containing the target road disease. The best second marker box, and the third non-maximum suppression algorithm is used to The best second marker boxes are aggregated and filtered out The best third marked box, where Indicates less than A positive integer, Indicates less than A positive integer, Indicates less than A positive integer; In the last layer of the third stage network, a structured random forest is trained using the convolutional features of the last layer to estimate the edge contour of the target pavement disease, and a greedy iterative algorithm is used to search for the edge contour of the target pavement disease. The pavement disease identification frame with the maximum probability of containing the target pavement disease at different positions and aspect ratios within the best third marking frame is selected.

4. The pavement damage image processing method according to claim 1, characterized in that: For each pavement defect identification frame, the corresponding oblique photographic image of the pavement defect and the three-dimensional coordinates of the frame center and / or frame corners in the camera coordinate system of the vehicle-mounted binocular camera are imported into an AI drawing tool based on AIGC, and the corresponding overhead photographic image of the pavement defect is output, including the following steps S41 to S43: S41. For a pavement defect identification frame, the corresponding oblique photographic image of the pavement defect and the three-dimensional coordinates of the frame center and / or frame corners in the camera coordinate system of the vehicle-mounted binocular camera are imported into an AI drawing tool based on AIGC, and the corresponding overhead photographic image of the pavement defect is output, and then step S42 is executed; S42. The image clarity evaluation algorithm is used to calculate the image clarity of the overhead photographic image of the road surface disease of the road surface disease identification frame, and then step S43 is executed; S43. Determine whether the image clarity of the overhead photographic image of the pavement disease of the certain pavement disease identification frame is lower than a first preset threshold value. If so, return to step S41. Otherwise, use the most recently output overhead photographic image of the pavement disease as the final overhead photographic image of the pavement disease of the certain pavement disease identification frame.

5. The pavement damage image processing method according to claim 1, characterized in that: According to the aforementioned A pavement disease overhead photographic image with the highest image clarity among the pavement disease overhead photographic images is used to extract pavement disease parameters at the location of the same pavement disease, including: Before the Select a pavement disease overhead photographic image with the highest image clarity from the pavement disease overhead photographic images; If it is determined that the highest image clarity reaches a second preset threshold, pavement disease parameters at the location of the same pavement disease are extracted based on the overhead photographic image of the pavement disease; If it is determined that the maximum image clarity is lower than the second preset threshold, an image acquisition task is generated to instruct the drone to go to the sky above the same pavement defect location to collect a bird's-eye view photographic image of the road surface, and then the image acquisition task is dispatched to the vehicle-mounted drone. Finally, based on the bird's-eye view photographic image of the road surface collected by the vehicle-mounted drone above the same pavement defect location, the pavement disease parameters at the same pavement defect location are extracted.

6. A road surface disease image processing device, characterized in that: It includes a photographic image acquisition unit, a road surface disease identification unit, an inclined image interception unit, a bird's-eye view image generation unit, a disease location determination unit, a disease parameter extraction unit and a detection result loading unit; The photographic image acquisition unit is used to acquire an oblique photographic image of the road surface in front of the vehicle collected in real time by a vehicle-mounted binocular camera, wherein the vehicle-mounted binocular camera is installed at the front of the vehicle body with the camera lens tilted toward the front and downward; The road surface defect recognition unit is communicatively connected to the photographic image acquisition unit and is used to perform road surface defect recognition processing on the oblique photographic image of the road surface in front of the vehicle using a target detection algorithm in real time to obtain a road surface defect recognition result; The oblique image capture unit is communicatively connected to the pavement defect identification unit and is configured to capture, for each pavement defect identification frame within the at least one pavement defect identification frame, a corresponding oblique pavement defect image from the oblique photographic image of the road surface in front of the vehicle; The overhead image generation unit is communicatively connected to the oblique image capture unit and is configured to import, for each pavement defect identification frame, the corresponding oblique photographic image of the pavement defect and the three-dimensional coordinates of the frame center and / or frame corners in the camera coordinate system of the on-board binocular camera into an AI drawing tool based on AIGC, thereby outputting the corresponding overhead photographic image of the pavement defect; The defect location determination unit is communicatively connected to the oblique image interception unit and is configured to determine the location of the corresponding road surface defect for each road surface defect identification frame based on the satellite positioning result of the front of the vehicle at the time of image acquisition and the three-dimensional coordinates of the center of the corresponding frame in the camera coordinate system of the vehicle-mounted binocular camera, wherein the image acquisition time refers to the time when the oblique photographic image of the road surface in front of the vehicle is acquired; The disease parameter extraction unit is communicatively connected to the overhead image generation unit and the disease position determination unit, and is used to extract the pavement disease parameters at the location of the pavement disease based on the overhead photographic image of the pavement disease corresponding to the location of the pavement disease, specifically including: aggregating all the overhead photographic images of the pavement disease corresponding to the same location of the pavement disease to obtain a pavement disease overhead photographic image set; for each pavement disease overhead photographic image in the pavement disease overhead photographic image set, if any frame edge of the corresponding pavement disease identification frame is found to be located at the edge area of ​​the inclined photographic image of the road surface in front of the vehicle, the corresponding image is eliminated; all remaining images in the pavement disease overhead photographic image set are arranged in order from the latest to the earliest according to the corresponding image acquisition time, to obtain an image sequence; and the front image is selected from the image sequence. A bird's-eye view of the road surface disease photographs, and for the For each of the pavement disease overhead photographic images, the corresponding image clarity is calculated using the image clarity evaluation algorithm, where: Represents a positive integer; according to the A pavement disease overhead photographic image with the highest image clarity among the pavement disease overhead photographic images is extracted to obtain pavement disease parameters at the location of the same pavement disease; The detection result loading unit is communicatively connected to the disease parameter extraction unit, and is used to load and display the location of the road disease and the corresponding road disease parameters on the road map display page.

7. A computer device, characterized in that: The method comprises a memory, a processor and a transceiver which are communicatively connected in sequence, wherein the memory is used to store a computer program, the transceiver is used to send and receive messages, and the processor is used to read the computer program and execute the pavement disease image processing method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores instructions, and when the instructions are executed on the computer, the pavement disease image processing method according to any one of claims 1 to 5 is executed.

9. A computer program product comprising a computer program or instructions, characterized in that When the computer program or the instruction is executed by a computer, the pavement damage image processing method according to any one of claims 1 to 5 is implemented.

Images