Pavement disease image processing method

Abstract

The invention discloses a pavement disease image processing method, and relates to the technical field of pavement disease detection. The method comprises the following steps: after acquiring an oblique photography image of a road surface in front of a vehicle collected by a vehicle-mounted binocular camera, carrying out road surface disease identification processing on the oblique photography image by adopting a target detection algorithm, and then intercepting a corresponding oblique photography image of the road surface disease for each identified road surface disease identification frame; the three-dimensional coordinates of the corresponding frame body center and / or frame body angular point in a camera coordinate system are combined, a corresponding pavement disease overlook photographic image is obtained by applying an AI drawing tool based on AIGC, the position of the corresponding pavement disease is obtained through conversion, and finally the pavement disease is determined according to the pavement disease overlook photographic image corresponding to the position of the pavement disease. And the pavement disease parameters are extracted, loaded and displayed, so that the accuracy of the finally obtained pavement disease detection result can be effectively improved under the condition of oblique photography.

Description

Technical Field

[0001] The invention belongs to the technical field of pavement damage detection, and in particular relates to a pavement damage image processing method. Background Art

[0002] With the rapid development of new infrastructure, smart highways based on "human-vehicle-road-environment coordination" have become an inevitable trend. Infrastructure such as bridges, slopes, tunnels and pavements, as important components of smart highways, face their huge stock and stable increase. The intelligence of their maintenance and management urgently needs to keep up with the development trend and be adapted.

[0003] At present, the existing road inspection method is mainly based on the inspection of inspection vehicles, and the inspection vehicles are generally obtained by modifying or refitting existing cars. For example, a camera is installed on the front of the car body to collect the road surface image in front of the car, and then the road surface image in front of the car is processed for road surface disease recognition to obtain the road surface disease detection result. However, due to the limitation of the installation position of the camera, the road surface image collected in front of the car is mainly an oblique photography image, rather than a bird's-eye view photography image, which limits the accuracy of the final road surface disease detection result. Summary of the invention

[0004] The purpose of the present invention is to provide a pavement disease image processing method, device, computer equipment, computer readable storage medium and computer program product to solve the problem of limited accuracy of the final pavement disease detection results obtained in the existing pavement inspection method due to oblique photography.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] In a first aspect, a method for processing a pavement disease image is provided, comprising:

[0007] Obtaining an oblique photographic image of the road surface in front of the vehicle collected in real time by a vehicle-mounted binocular camera, wherein the vehicle-mounted binocular camera is installed at the front of the vehicle body and the camera lens is tilted toward the front and downward;

[0008] Using a target detection algorithm in real time to perform road surface disease recognition processing on the oblique photographic image of the road surface in front of the vehicle to obtain a road surface disease recognition result;

[0009] If at least one pavement defect identification frame is found according to the pavement defect identification result, for each pavement defect identification frame in the at least one pavement defect identification frame, a corresponding pavement defect oblique photography image is captured from the oblique photography image of the pavement in front of the vehicle;

[0010] For each pavement defect identification frame, the corresponding pavement defect oblique photographic image and the three-dimensional coordinates of the frame center and / or frame corners in the camera coordinate system of the vehicle-mounted binocular camera are imported into an AI drawing tool based on AIGC, and the corresponding pavement defect top-view photographic image is output;

[0011] For each road surface defect identification frame, the location of the corresponding road surface defect is determined according to the satellite positioning result of the front part of the vehicle body at the time of image acquisition and the three-dimensional coordinates of the center of the corresponding frame in the camera coordinate system of the vehicle-mounted binocular camera, wherein the image acquisition time refers to the time of acquisition of the oblique photography image of the road surface in front of the vehicle;

[0012] Extracting pavement disease parameters at the location of the pavement disease based on the pavement disease overhead photographic image corresponding to the location of the pavement disease;

[0013] The location of the pavement damage and the corresponding pavement damage parameters are loaded and displayed on the road map display page.

[0014] Based on the above invention content, a new solution for road surface disease detection based on target detection algorithm and AIGC drawing tool is provided, that is, after obtaining the oblique photographic image of the road surface in front of the vehicle collected by the vehicle-mounted binocular camera, the target detection algorithm is first used to perform road surface disease recognition processing on the oblique photographic image, and then for each identified road surface disease identification frame, the corresponding road surface disease oblique photographic image is intercepted, and combined with the three-dimensional coordinates of the corresponding frame center and / or frame corner in the camera coordinate system, the AI ​​drawing tool based on AIGC is used to obtain the corresponding road surface disease top-view photographic image, and also convert the corresponding road surface disease location, and finally, according to the road surface disease top-view photographic image corresponding to the road surface disease location, the road surface disease parameters are extracted and loaded for display, so that the accuracy of the final road surface disease detection result can be effectively improved under the condition of oblique photography, which is convenient for practical application and promotion.

[0015] In a possible design, a target detection algorithm is used to perform road surface disease recognition processing on the oblique photographic image of the road surface in front of the vehicle to obtain a road surface disease recognition result, including:

[0016] The oblique photographic image of the road surface in front of the vehicle is imported into a pavement defect recognition model pre-trained based on a target detection algorithm, and a pavement defect recognition result that can show whether there is a pavement disease identification box on the oblique photographic image of the road surface in front of the vehicle is output, wherein the target detection algorithm adopts a three-stage network structure including a first-stage network, a second-stage network and a third-stage network, and the first-stage network, the second-stage network and the third-stage network respectively use Resnet50 network as the backbone network, and respectively use different void convolution methods with increasing void rates, so that the first-stage network, the second-stage network and the third-stage network are reverse cascaded in sequence from coarse to fine according to the features of the deep convolution layer.

[0017] In a possible design, in the last layer of the first-stage network, M1 sliding windows with different sizes are used to perform window scanning on the oblique photographic image of the road surface in front of the vehicle, so as to obtain a plurality of first marking frames, and according to the normalized results of the probabilities of the plurality of first marking frames containing the target road surface disease, N1 best first marking frames are screened out from the plurality of first marking frames using a first non-maximum suppression algorithm, wherein M1 represents a positive integer greater than or equal to 30, and N1 represents a positive integer greater than or equal to 3500;

[0018] In the last layer of the second-stage network, M2 sliding windows with different sizes are used to perform window scanning on the oblique photographic image of the road surface in front of the vehicle within the N1 best first marking frames, so as to obtain a plurality of second marking frames, and according to the normalized results of the probabilities of the plurality of second marking frames containing target road surface diseases, N2 best second marking frames are screened out from the plurality of second marking frames using a second non-maximum suppression algorithm, and N3 best third marking frames are aggregated and screened out using a third non-maximum suppression algorithm, wherein M2 represents a positive integer less than M1, N2 represents a positive integer less than N1, and N3 represents a positive integer less than N2;

[0019] In the last layer of the third-stage network, a structured random forest is trained using the convolutional features of the last layer to estimate the edge contour of the target pavement disease, and based on the edge contour, a greedy iterative algorithm is used to search for a pavement disease identification frame with the maximum probability of containing the target pavement disease at different positions and aspect ratios within the N3 best third marking frames.

[0020] In a possible design, for each pavement defect identification frame, the corresponding pavement defect oblique photographic image and the three-dimensional coordinates of the frame center and / or frame corner in the camera coordinate system of the vehicle-mounted binocular camera are imported into an AI drawing tool based on AIGC, and the corresponding pavement defect top-view photographic image is output, including the following steps S41 to S43:

[0021] S41. For a road surface disease identification frame, the corresponding road surface disease oblique photographic image and the three-dimensional coordinates of the frame center and / or frame corners in the camera coordinate system of the vehicle-mounted binocular camera are imported into the AI ​​drawing tool based on AIGC, and the corresponding road surface disease top-view photographic image is output, and then step S42 is executed;

[0022] S42. Calculate the image clarity of the road surface disease overhead photographic image of the road surface disease identification frame using an image clarity evaluation algorithm, and then execute step S43;

[0023] S43. Determine whether the image clarity of the pavement disease overhead photographic image of the pavement disease identification frame is lower than a first preset threshold value. If so, return to step S41. Otherwise, use the pavement disease overhead photographic image most recently output as the final pavement disease overhead photographic image of the pavement disease identification frame.

[0024] In a possible design, when the oblique photographic image of the road surface in front of the vehicle is any frame of video image in the video data collected in real time by the on-board binocular camera, the pavement disease parameter at the location of the pavement disease is extracted based on the top-view photographic image of the pavement disease corresponding to the location of the pavement disease, including:

[0025] Summarizing all the pavement disease overhead photographic images corresponding to the same pavement disease location to obtain a pavement disease overhead photographic image set;

[0026] For each of the pavement defect overhead photographic images in the pavement defect overhead photographic image set, if any frame edge of the corresponding pavement defect identification frame is found to be located at an edge area of ​​the inclined photographic image of the road surface in front of the vehicle, the corresponding image is discarded;

[0027] Arrange all the remaining images in the pavement disease overhead photography image set in order from the latest to the earliest corresponding image acquisition time to obtain an image sequence;

[0028] Selecting the first K pavement disease overhead photographic images from the image sequence, and calculating the corresponding image clarity of each pavement disease overhead photographic image in the first K pavement disease overhead photographic images by using an image clarity evaluation algorithm, wherein K represents a positive integer;

[0029] According to a certain pavement disease overhead photographic image with the highest image clarity among the first K pavement disease overhead photographic images, pavement disease parameters at the location of the same pavement disease are extracted.

[0030] In a possible design, a pavement disease parameter at the location of the same pavement disease is extracted based on a pavement disease overhead photographic image with the highest image clarity among the first K pavement disease overhead photographic images, including:

[0031] Selecting a pavement disease overhead photographic image with the highest image clarity from the first K pavement disease overhead photographic images;

[0032] If it is determined that the highest image clarity reaches a second preset threshold, then, based on the overhead photographic image of a certain road surface disease, pavement disease parameters at the location of the same road surface disease are extracted;

[0033] If it is determined that the maximum image clarity is lower than the second preset threshold, an image acquisition task is generated to instruct the UAV to go to the sky above the same pavement defect location to collect a bird's-eye view photographic image of the road surface, and then the image acquisition task is dispatched to a vehicle-mounted UAV. Finally, based on the bird's-eye view photographic image of the road surface collected by the vehicle-mounted UAV above the same pavement defect location, the pavement disease parameters at the same pavement defect location are extracted.

[0034] In a second aspect, a pavement disease image processing device is provided, comprising a photographic image acquisition unit, a pavement disease recognition unit, an inclined image interception unit, a bird's-eye view image generation unit, a disease location determination unit, a disease parameter extraction unit, and a detection result loading unit;

[0035] The photographic image acquisition unit is used to acquire an oblique photographic image of the road surface in front of the vehicle collected in real time by a vehicle-mounted binocular camera, wherein the vehicle-mounted binocular camera is installed at the front of the vehicle body and the camera lens is tilted toward the front and downward;

[0036] The road surface disease recognition unit is communicatively connected to the photographic image acquisition unit and is used to perform road surface disease recognition processing on the inclined photographic image of the road surface in front of the vehicle using a target detection algorithm in real time to obtain a road surface disease recognition result;

[0037] The oblique image interception unit is communicatively connected to the pavement defect recognition unit, and is used to intercept the corresponding pavement defect oblique photography image from the oblique photography image of the road surface in front of the vehicle for each pavement defect identification frame in the at least one pavement defect identification frame if at least one pavement defect identification frame is found according to the pavement defect recognition result;

[0038] The overhead image generation unit is communicatively connected to the oblique image interception unit, and is used to import the corresponding oblique photographic image of the road surface disease and the three-dimensional coordinates of the frame center and / or frame corners in the camera coordinate system of the vehicle-mounted binocular camera into the AI ​​drawing tool based on AIGC for each road surface disease identification frame, and output the corresponding overhead photographic image of the road surface disease;

[0039] The defect location determination unit is communicatively connected to the oblique image interception unit, and is used to determine the location of the corresponding road surface defect for each road surface defect identification frame according to the satellite positioning result of the front part of the vehicle body at the time of image acquisition and the three-dimensional coordinates of the center of the corresponding frame in the camera coordinate system of the vehicle-mounted binocular camera, wherein the image acquisition time refers to the acquisition time of the oblique photography image of the road surface in front of the vehicle;

[0040] The disease parameter extraction unit is respectively connected to the overhead image generation unit and the disease position determination unit for communication, and is used to extract the pavement disease parameter at the location of the pavement disease based on the pavement disease overhead photographic image corresponding to the location of the pavement disease;

[0041] The detection result loading unit is communicatively connected to the disease parameter extraction unit, and is used to load and display the location of the road disease and the corresponding road disease parameters on the road map display page.

[0042] In a third aspect, the present invention provides a computer device comprising a memory, a processor and a transceiver which are communicatively connected in sequence, wherein the memory is used to store a computer program, the transceiver is used to send and receive messages, and the processor is used to read the computer program to execute the pavement disease image processing method as described in the first aspect or any possible design of the first aspect.

[0043] In a fourth aspect, the present invention provides a computer-readable storage medium having instructions stored thereon, which, when executed on a computer, executes the pavement disease image processing method as described in the first aspect or any possible design of the first aspect.

[0044] In a fifth aspect, the present invention provides a computer program product, comprising a computer program or instructions, which, when executed by a computer, implements the pavement damage image processing method as described in the first aspect or any possible design of the first aspect.

[0045] Beneficial effects of the above scheme:

[0046] (1) The present invention creatively provides a new solution for pavement disease detection based on a target detection algorithm and an AIGC drawing tool, that is, after obtaining an oblique photographic image of the road surface in front of the vehicle collected by a vehicle-mounted binocular camera, the target detection algorithm is first used to perform pavement disease recognition processing on the oblique photographic image, and then for each identified pavement disease identification frame, the corresponding pavement disease oblique photographic image is intercepted, and combined with the three-dimensional coordinates of the corresponding frame center and / or frame corner in the camera coordinate system, the AIGC-based AI drawing tool is used to obtain the corresponding pavement disease overhead photographic image, and also convert the corresponding pavement disease location, and finally, according to the pavement disease overhead photographic image corresponding to the pavement disease location, the pavement disease parameters are extracted and loaded for display. In this way, the accuracy of the final pavement disease detection result can be effectively improved while still using oblique photography, which is convenient for practical application and promotion. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0048] Figure 1 A schematic flow chart of a pavement damage image processing method provided in an embodiment of the present application.

[0049] Figure 2 Schematic diagram of the installation position of the vehicle-mounted binocular camera provided in an embodiment of the present application at the front of the vehicle body.

[0050] Figure 3 A schematic diagram of a three-stage network structure provided in an embodiment of the present application.

[0051] Figure 4 A schematic diagram of the structure of a road surface disease image processing device provided in an embodiment of the present application.

[0052] Figure 5 A schematic diagram of the structure of a computer device provided in an embodiment of the present application.

[0053] In the above drawings: 100 - front part of the vehicle body; 101 - first binocular camera; 102 - second binocular camera. DETAILED DESCRIPTION

[0054] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the present invention will be briefly introduced below in combination with the drawings and the description of the embodiments or the prior art. Obviously, the following description of the structure of the drawings is only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. It should be noted that the description of these embodiments is used to help understand the present invention, but does not constitute a limitation of the present invention.

[0055] It should be understood that although the terms first and second, etc. may be used herein to describe various objects, these objects should not be limited by these terms. These terms are only used to distinguish one object from another object. For example, a first object can be referred to as a second object, and similarly, a second object can be referred to as a first object without departing from the scope of the exemplary embodiments of the present invention.

[0056] It should be understood that the term "and / or" that may appear in this document is merely a description of the association relationship between associated objects, indicating that there may be three relationships. For example, A and / or B can indicate three situations: A exists alone, B exists alone, or A and B exist at the same time. For another example, A, B and / or C can indicate the existence of any one of A, B and C or any combination of them. The term " / and" that may appear in this document describes another type of association object relationship, indicating that there may be two relationships. For example, A / and B can indicate two situations: A exists alone or A and B exist at the same time. In addition, the character " / " that may appear in this document generally indicates that the previous and next associated objects are in an "or" relationship.

[0057] Example:

[0058] like Figure 1 As shown, the pavement disease image processing method provided in the first aspect of this embodiment can be, but is not limited to, executed by a computer device having certain computing resources and communicating with a vehicle-mounted binocular camera, such as a vehicle computer (also known as a computer control module, Electronic Control Unit in English, abbreviated as ECU), a platform server, a personal computer (Personal Computer, PC, refers to a multi-purpose computer with a size, price and performance suitable for personal use; desktops, laptops, small laptops, tablets and ultrabooks are all personal computers), smart phones, personal digital assistants (Personal Digital Assistant, PDA) or wearable devices and other electronic devices. Figure 1 As shown, the pavement damage image processing method may include but is not limited to the following steps S1 to S7.

[0059] S1. Obtain an oblique photographic image of the road surface in front of the vehicle collected in real time by a vehicle-mounted binocular camera, wherein the vehicle-mounted binocular camera is installed at the front of the vehicle body and the camera lens is tilted downward and directly in front.

[0060] In step S1, the binocular camera is a camera based on the binocular vision principle (i.e., the principle is: to achieve three-dimensional measurement through parallax measurement of two cameras; specifically, the farther the object is from the camera, the more obvious the difference between the images taken by the left and right cameras is. By measuring this difference, the binocular camera can calculate the distance and shape of the object and other information). It is composed of two cameras and an image processing unit, and can obtain the three-dimensional information of the object, including the distance, shape and size of the object, through the parallax measurement of the two cameras. The lens field of view of the vehicle-mounted binocular camera will cover the road area directly in front of the front of the vehicle body, and is used to collect oblique photographic images of the road area directly in front in real time to obtain video data containing a number of continuous video frame images (all of which are oblique photographic images of the road surface); the specific installation position of the vehicle-mounted binocular camera can be, but is not limited to, Figure 2 As shown: the vehicle-mounted binocular camera can be the first binocular camera 101 or the second binocular camera 102, and the angle between the lens of the vehicle-mounted binocular camera and the front is less than 30 degrees ( Figure 2 In addition, the vehicle-mounted binocular camera can transmit the collected data to the local device in a conventional manner.

[0061] S2. Using a target detection algorithm in real time to perform road surface disease recognition processing on the oblique photographic image of the road surface in front of the vehicle to obtain a road surface disease recognition result.

[0062] In step S2, the target detection algorithm is an existing artificial intelligence recognition algorithm for identifying objects in a picture and marking the location of the objects, and specifically can be but not limited to Faster R-CNN (Faster Regions with Convolutional Neural Networks features, a target detection algorithm proposed by He Kaiming et al. in 2015, which won multiple first places in the 2015 ILSVRV and COCO competitions) target detection algorithm, SSD (Single Shot MultiBox Detector, a target detection algorithm proposed by Wei Liu at ECCV 2016, which is one of the currently popular main detection frameworks) target detection algorithm or YOLO (Youonly look Once, the latest version has been developed to V4, and is widely used in the industry. Its basic principle is: first divide the input image into 7x7 grids, predict 2 frames for each grid, then remove the target window with a relatively low probability according to the threshold, and finally use the frame merging method to remove the redundant window to obtain the detection result) target detection algorithm, so the road surface disease recognition processing can be performed based on the target detection algorithm to obtain the road surface disease recognition result: whether there is a road surface disease identification frame on the oblique photography image of the road surface in front of the vehicle. In addition, the road surface diseases that need to be identified include but are not limited to manhole covers, well holes, potholes to be repaired, and cracked road surfaces.

[0063] In the step S2, preferably, a target detection algorithm is used to perform pavement disease recognition processing on the oblique photographic image of the road surface in front of the vehicle to obtain a pavement disease recognition result, including but not limited to the following steps: importing the oblique photographic image of the road surface in front of the vehicle into a pavement disease recognition model pre-trained based on the target detection algorithm, and outputting a pavement disease recognition result that can show whether there is a pavement disease identification box on the oblique photographic image of the road surface in front of the vehicle, wherein the target detection algorithm adopts a three-stage network structure including a first-stage network, a second-stage network and a third-stage network, wherein the first-stage network, the second-stage network and the third-stage network respectively use Resnet50 network as the backbone network, and respectively use different void convolution methods with increasing void rates, so that the first-stage network, the second-stage network and the third-stage network are reverse cascaded in sequence from coarse to fine according to the features of the deep convolution layer. The specific structure of the three-stage network structure is as follows: Figure 3As shown in the figure, the specific structure of the Resnet50 network is the existing network structure, which not only proposes a residual structure, but also proposes a skeleton network design paradigm, namely stem+n stage+cls head. For ResNet, its actual forward process is stem->4 stages->classification head, where the output stride (which represents the downsampling rate of the model, assuming that the input image size is 320×320, stride=10, then the output feature map size is 32×32, assuming that there are 9 anchors at each position, then the output feature map has a total of 32×32×9 anchors) of the stem (whose Input is a three-channel image, which first undergoes convolution operation, normalization operation, RELU operation, and then undergoes maximum pooling operation to obtain Output) is 4, and the output strides of the four stages are 4, 8, 16 and 32 respectively. The hole convolution method refers to filling 0 in the middle of the 3×3 convolution kernel. The calculation method is the same as the standard convolution. The hole convolution can expand the receptive field without losing the resolution problem caused by the previous step of downsampling, and can capture multi-scale context information; similar to the YOLO algorithm, a hole convolution with a small hole rate rate is used to detect large targets, and a hole convolution with a large hole rate rate is used to detect small targets. Therefore, through the three-stage network structure design of the aforementioned target detection algorithm, the following road surface disease detection process can be implemented: using the coarse to fine inverse cascade method of the features of the deep convolution layer, possible object suggestions (that is, the detected target) are obtained in the image; dense suggestion sampling is performed in the last convolution layer of the first stage network, and then irrelevant frames are gradually filtered until the last layer of the third stage network; in the third stage network, the contour extracted from the last layer of the third stage network is used to refine the suggestion; the finally generated road surface disease identification frame can be used in the object detection process; finally, the problem of imbalance between background noise and foreground object information is solved, and the performance of multi-scale target detection is improved.

[0064] In the step S2, specifically, in the last layer of the first-stage network, M1 sliding windows with different sizes are used to perform window scanning on the oblique photographic image of the road surface in front of the vehicle, so as to obtain a plurality of first marking frames, and according to the normalized results of the probability of the plurality of first marking frames containing the target road surface disease, N1 best first marking frames are screened out from the plurality of first marking frames using a first non-maximum suppression algorithm, wherein M1 represents a positive integer greater than or equal to 30 (for example, 50), and N1 represents a positive integer greater than or equal to 3500 (for example, 4000); in the last layer of the second-stage network, M2 sliding windows with different sizes are used to perform window scanning on the oblique photographic image of the road surface in front of the vehicle within the N1 best first marking frames, so as to obtain a plurality of second marking frames, and according to the plurality of second marking frames, and containing the target pavement disease, using a second non-maximum suppression algorithm to filter out N2 best second marking frames from the multiple second marking frames, and also using a third non-maximum suppression algorithm to aggregate the N2 best second marking frames and filter out N3 best third marking frames, wherein M2 represents a positive integer less than M1 (for example, 10), N2 represents a positive integer less than N1 (for example, 3000), and N3 represents a positive integer less than N2 (for example, 1000); in the last layer of the third stage network, a structured random forest is trained using the convolutional features of the last layer to estimate the edge contour of the target pavement disease, and according to the edge contour, a greedy iterative algorithm is used to search and try to find a pavement disease identification frame with the maximum probability of containing the target pavement disease at different positions and aspect ratios within the N3 best third marking frames.

[0065] In the step S2, specifically, the first stage network, the second stage network and the third stage network also respectively use the Focal Loss loss function as the loss function. The aforementioned Focal Loss loss function is introduced on the basis of the cross entropy (CE) loss function of the binary classification problem. It is a dynamically scaled cross entropy loss. Through a dynamic scaling factor, the weight of easily distinguishable samples can be dynamically reduced during the training process, thereby quickly focusing on those difficult-to-distinguish samples. The specific form is as follows:

[0066] FL(p t )=-α×(1-p t ) γ ×log(p t )

[0067] Among them, α is used to balance positive and negative samples, for example, the value is 0.25. p represents the probability when the predicted sample is equal to 1, y represents the predicted value of the sample, and γ represents a preset constant (for example, 2). Figure 3 As shown, specifically, the first-stage network adopts a dilated convolution method with a dilation rate of 2, the second-stage network adopts a dilated convolution method with a dilation rate of 8, and the third-stage network adopts a dilated convolution method with a dilation rate of 16.

[0068] S3. If at least one pavement defect identification frame is found according to the pavement defect identification result, for each pavement defect identification frame in the at least one pavement defect identification frame, a corresponding pavement defect oblique photography image is captured from the oblique photography image of the road surface in front of the vehicle.

[0069] In step S3, the at least one road surface damage identification frame specifically includes but is not limited to a manhole cover identification frame, a manhole identification frame, a pothole identification frame to be repaired, and / or a cracked road surface identification frame.

[0070] S4. For each pavement defect identification frame, the corresponding pavement defect oblique photographic image and the three-dimensional coordinates of the frame center and / or frame corners in the camera coordinate system of the vehicle-mounted binocular camera are imported into an AI drawing tool based on AIGC (Artificial Intelligence Generated Content, generative artificial intelligence, which is an important symbol of the transition from artificial intelligence 1.0 era to 2.0 era), and the corresponding pavement defect overhead photographic image is output.

[0071] In step S4, since the oblique photographic image of the road surface in front of the vehicle is acquired in real time by the on-board binocular camera, each pixel in the oblique photographic image of the road surface in front of the vehicle will have three-dimensional information, and thus the three-dimensional coordinates of the frame center and / or any frame corner of the road surface disease identification frame in the camera coordinate system of the on-board binocular camera can be easily obtained. The AI ​​drawing tool is an existing drawing software, which may include but is not limited to AI drawing software such as Midjourrney, Dell-E, Stable Diffusion, NovelAI and Disco Diffusion. Since these software belong to large models, they have the characteristics of many parameters, long training time, good generalization, strong versatility and high practicality, and can be applied to drawing of various scenes. Therefore, they can be well applied to this embodiment, that is, based on the oblique photographic image of the road surface disease and the frame center and / or frame corner of the road surface disease identification frame, the road surface disease overhead photographic image corresponding to the road surface disease identification frame is generated. In order to ensure that the generated pavement disease overhead photographic image has ideal image clarity, preferably, for each pavement disease identification frame, the corresponding pavement disease oblique photographic image and the three-dimensional coordinates of the frame center and / or frame corners in the camera coordinate system of the vehicle-mounted binocular camera are imported into an AI drawing tool based on AIGC, and the corresponding pavement disease overhead photographic image is output, including but not limited to the following steps S41 to S43.

[0072] S41. For a certain road surface disease identification frame, the corresponding road surface disease oblique photographic image and the three-dimensional coordinates of the frame center and / or frame corners in the camera coordinate system of the vehicle-mounted binocular camera are imported into the AI ​​drawing tool based on AIGC, and the corresponding road surface disease top-view photographic image is output, and then step S42 is executed.

[0073] S42. Use an image clarity evaluation algorithm to calculate the image clarity of the road surface disease overhead photographic image of the road surface disease identification frame, and then execute step S43.

[0074] In step S42, the image clarity evaluation algorithm may be implemented by, but is not limited to, the existing Brenner gradient method, Tenegrad gradient method, Laplace gradient method, variance method or energy gradient method.

[0075] S43. Determine whether the image clarity of the pavement disease overhead photographic image of the pavement disease identification frame is lower than a first preset threshold value. If so, return to step S41. Otherwise, use the pavement disease overhead photographic image most recently output as the final pavement disease overhead photographic image of the pavement disease identification frame.

[0076] In the step S43, the first preset threshold is used as a criterion for confirming whether the generated pavement damage overhead photographic image meets the minimum image clarity requirement, and can be specifically preset by the user. In addition, if the image clarity of the pavement damage overhead photographic images generated multiple times is lower than the first preset threshold, the step S41 can be stopped, and any pavement damage overhead photographic image with the highest image clarity among the pavement damage overhead photographic images generated multiple times is used as the final pavement damage overhead photographic image of the pavement damage identification frame.

[0077] S5. For each road surface defect identification frame, determine the location of the corresponding road surface defect based on the satellite positioning result of the front of the vehicle body at the time of image acquisition and the three-dimensional coordinates of the corresponding frame center in the camera coordinate system of the vehicle-mounted binocular camera, wherein the image acquisition time refers to the time when the oblique photographic image of the road surface in front of the vehicle is acquired.

[0078] In step S5, the satellite positioning result can be conventionally acquired by a satellite positioning module (such as a Beidou satellite positioning module) fixedly installed on the front of the vehicle body, which is specifically a three-dimensional coordinate in the geodetic coordinate system. Since the on-board binocular camera is fixedly installed on the front of the vehicle body, the positional relationship between the front of the vehicle body and the on-board binocular camera is fixed, and then the three-dimensional coordinates of the frame center of the pavement disease identification frame in the geodetic coordinate system can be converted based on the conventional coordinate system transformation method, that is, the location of the pavement disease. In addition, since different oblique photographic images of the road surface in front of the vehicle will be acquired by the on-board binocular camera at different acquisition times for the same pavement disease object, when the oblique photographic image of the road surface in front of the vehicle is any frame of video image in the video data acquired in real time by the on-board binocular camera, there will be a situation where multiple pavement disease identification frames correspond to the same pavement disease object (that is, multiple pavement disease identification frames with different acquisition times may correspond to the same pavement disease location).

[0079] S6. Extracting pavement disease parameters at the location of the pavement disease based on the overhead photographic image of the pavement disease corresponding to the location of the pavement disease.

[0080] In step S6, different pavement diseases will have different pavement disease parameters, for example, the pavement disease parameters may specifically include but are not limited to the radius and / or size of the manhole cover, the radius and / or opening area of ​​the well hole, the groove width, groove length and / or groove opening area of ​​the pothole to be repaired, or the crack width, crack length and / or crack area of ​​the cracked pavement, etc. These parameters can be obtained by conventionally measuring the overhead photographic image of the pavement disease. Since the location of the pavement disease has a corresponding relationship with the pavement disease identification frame (which can be a one-to-one correspondence or a one-to-many relationship), and the pavement disease identification frame has a one-to-one correspondence with the pavement disease overhead photographic image, the location of the pavement disease will correspond to at least one of the pavement disease overhead photographic images. In order to achieve the purpose of obtaining the most accurate pavement disease parameters based on multiple overhead photographic images of pavement diseases, preferably, when the oblique photographic image of the road surface in front of the vehicle is any frame of video image in the video data collected in real time by the on-board binocular camera, the pavement disease parameters at the location of the pavement disease are extracted based on the overhead photographic image of the pavement disease corresponding to the location of the pavement disease, including but not limited to the following steps S61 to S65.

[0081] S61. Summarize all the pavement disease overhead photographic images corresponding to the same pavement disease location to obtain a pavement disease overhead photographic image set.

[0082] S62. For each pavement defect overhead photographic image in the pavement defect overhead photographic image set, if any frame edge of the corresponding pavement defect identification frame is found to be located at the edge area of ​​the inclined photographic image of the road surface in front of the vehicle, the corresponding image is discarded.

[0083] In step S62, it is considered that during the inspection process, the position of the road surface defect image in the oblique photographic image of the road surface in front of the vehicle will move from the middle area to the edge area until it moves out and disappears, so that at the moment when the inspection vehicle passes by the road surface defect, the detected road surface defect image appears to be incomplete. Therefore, based on the aforementioned step S62, these suspected incomplete road surface defect overhead photographic images can be eliminated. In addition, the edge area is preferably the edge area of ​​the preset area of ​​interest.

[0084] S63. Arrange all the remaining images in the pavement damage overhead photography image set in order from late to early according to the corresponding image acquisition time to obtain an image sequence.

[0085] In step S63, since the pavement disease overhead photographic image is generated based on the pavement disease oblique photographic image, and the pavement disease oblique photographic image is captured from the oblique photographic image of the pavement in front of the vehicle, each of the pavement disease overhead photographic images will correspond to a corresponding image acquisition moment.

[0086] S64. Select the first K pavement disease overhead photographic images from the image sequence, and use an image clarity evaluation algorithm to calculate the corresponding image clarity for each of the first K pavement disease overhead photographic images, where K represents a positive integer.

[0087] In the step S64, K can be 10 for example, and can be positively correlated with the acquisition frequency of the oblique photography image of the road surface in front of the vehicle (i.e., the higher the acquisition frequency, the larger the value of K). In addition, the image clarity evaluation algorithm can also be implemented by, but not limited to, the existing Brenner gradient method, Tenegrad gradient method, Laplace gradient method, variance method or energy gradient method.

[0088] S65. Extract the pavement disease parameters at the location of the same pavement disease based on a pavement disease overhead photographic image with the highest image clarity among the first K pavement disease overhead photographic images.

[0089] In the step S65, considering that the highest image clarity is not ideal, in order to further ensure the accuracy of the extracted pavement damage parameters, preferably, based on a pavement damage overhead photographic image with the highest image clarity among the first K pavement damage overhead photographic images, the pavement damage parameters at the location of the same pavement damage are extracted, including but not limited to the following steps S651 to S653.

[0090] S651. Select a pavement damage overhead photographic image with the highest image clarity from the first K pavement damage overhead photographic images.

[0091] S652. If it is determined that the highest image clarity reaches a second preset threshold, pavement disease parameters at the location of the same pavement disease are extracted based on the overhead photographic image of the pavement disease.

[0092] In step S652, the second preset threshold is used as a criterion for determining whether a road surface disease overhead photographic image with the highest image clarity meets the minimum image clarity requirement, which can be specifically preset by the user. In addition, the second preset threshold is generally greater than or equal to the first preset threshold.

[0093] S653. If it is determined that the maximum image clarity is lower than the second preset threshold, an image acquisition task is generated to instruct the UAV to go to the sky above the same pavement defect location to collect a bird's-eye view photographic image of the road surface, and then the image acquisition task is dispatched to a vehicle-mounted UAV. Finally, based on the bird's-eye view photographic image of the road surface collected by the vehicle-mounted UAV above the same pavement defect location, the pavement disease parameters at the same pavement defect location are extracted.

[0094] In step S653, the vehicle-mounted drone can be in a flying state following the inspection vehicle, or in a stopped state parked on the inspection vehicle. Once the image acquisition task is received, it will immediately go to the air above the location of the same road surface disease, and use a camera located on the abdomen with the lens facing vertically downward (preferably a binocular camera here) to collect the overhead photographic image of the road surface, and finally return or continue to fly behind the inspection vehicle. Considering that tunnel sections are not conducive to drone flight, the image acquisition task can be generated and assigned to the vehicle-mounted drone only when the location of the same road surface disease is in a non-tunnel section. In addition, the target detection algorithm can be used to first perform pavement disease recognition processing on the pavement overhead photographic image, and then a pavement disease overhead photographic image at the location of the same pavement disease can be captured based on the recognition result. Finally, the pavement disease parameters at the location of the same pavement disease can be extracted based on the pavement disease overhead photographic image. The pavement disease overhead photographic image can also be used as a model output item, and the pavement disease oblique photographic image corresponding to the same pavement disease location and the frame center and / or frame corners of the pavement disease identification frame can be used as model input items to perform online training on the AI ​​drawing tool to obtain a better AI drawing tool.

[0095] S7. Loading and displaying the location of the road surface disease and the corresponding road surface disease parameters on the road map display page.

[0096] Therefore, based on the pavement disease image processing method described in the aforementioned steps S1 to S7, a new solution for pavement disease detection based on a target detection algorithm and an AIGC drawing tool is provided, that is, after obtaining the oblique photographic image of the road surface in front of the vehicle collected by the on-board binocular camera, the target detection algorithm is first used to perform pavement disease recognition processing on the oblique photographic image, and then for each identified pavement disease identification frame, the corresponding pavement disease oblique photographic image is intercepted, and combined with the three-dimensional coordinates of the corresponding frame center and / or frame corner in the camera coordinate system, the AI ​​drawing tool based on AIGC is used to obtain the corresponding pavement disease overhead photographic image, and also convert the corresponding pavement disease location, and finally, according to the pavement disease overhead photographic image corresponding to the pavement disease location, the pavement disease parameters are extracted and loaded for display, so that the accuracy of the final pavement disease detection result can be effectively improved under the condition of oblique photography, which is convenient for practical application and promotion.

[0097] like Figure 4 As shown, the second aspect of this embodiment provides a virtual device for implementing the pavement disease image processing method described in the first aspect, comprising a photographic image acquisition unit, a pavement disease recognition unit, an inclined image interception unit, a bird's-eye view image generation unit, a disease location determination unit, a disease parameter extraction unit, and a detection result loading unit;

[0098] The photographic image acquisition unit is used to acquire an oblique photographic image of the road surface in front of the vehicle collected in real time by a vehicle-mounted binocular camera, wherein the vehicle-mounted binocular camera is installed at the front of the vehicle body and the camera lens is tilted toward the front and downward;

[0099] The road surface disease recognition unit is communicatively connected to the photographic image acquisition unit and is used to perform road surface disease recognition processing on the inclined photographic image of the road surface in front of the vehicle using a target detection algorithm in real time to obtain a road surface disease recognition result;

[0100] The oblique image interception unit is communicatively connected to the pavement defect recognition unit, and is used to intercept the corresponding pavement defect oblique photography image from the oblique photography image of the road surface in front of the vehicle for each pavement defect identification frame in the at least one pavement defect identification frame if at least one pavement defect identification frame is found according to the pavement defect recognition result;

[0101] The overhead image generation unit is communicatively connected to the oblique image interception unit, and is used to import the corresponding oblique photographic image of the road surface disease and the three-dimensional coordinates of the frame center and / or frame corners in the camera coordinate system of the vehicle-mounted binocular camera into the AI ​​drawing tool based on AIGC for each road surface disease identification frame, and output the corresponding overhead photographic image of the road surface disease;

[0102] The defect location determination unit is communicatively connected to the oblique image interception unit, and is used to determine the location of the corresponding road surface defect for each road surface defect identification frame according to the satellite positioning result of the front part of the vehicle body at the time of image acquisition and the three-dimensional coordinates of the center of the corresponding frame in the camera coordinate system of the vehicle-mounted binocular camera, wherein the image acquisition time refers to the acquisition time of the oblique photography image of the road surface in front of the vehicle;

[0103] The disease parameter extraction unit is respectively connected to the overhead image generation unit and the disease position determination unit for communication, and is used to extract the pavement disease parameter at the location of the pavement disease based on the pavement disease overhead photographic image corresponding to the location of the pavement disease;

[0104] The detection result loading unit is communicatively connected to the disease parameter extraction unit, and is used to load and display the location of the road disease and the corresponding road disease parameters on the road map display page.

[0105] The working process, working details and technical effects of the aforementioned device provided in the second aspect of this embodiment can be referred to the pavement disease image processing method described in the first aspect, and will not be described in detail here.

[0106] like Figure 5 As shown, the third aspect of this embodiment provides a computer device for executing the pavement disease image processing method as described in the first aspect, including a memory, a processor and a transceiver that are sequentially connected in communication, wherein the memory is used to store a computer program, the transceiver is used to send and receive messages, and the processor is used to read the computer program to execute the pavement disease image processing method as described in the first aspect. For example, the memory may include, but is not limited to, a random access memory (RAM), a read-only memory (ROM), a flash memory, a first-in-first-out memory (FIFO) and / or a first-in-last-out memory (FILO), etc.; the processor may include, but is not limited to, a microprocessor of the STM32F105 series. In addition, the computer device may also include, but is not limited to, a power module, a display screen and other necessary components.

[0107] The working process, working details and technical effects of the aforementioned computer device provided in the third aspect of this embodiment can be referred to the pavement disease image processing method described in the first aspect, and will not be described in detail here.

[0108] In a fourth aspect of this embodiment, there is provided a computer-readable storage medium storing instructions including the pavement disease image processing method as described in the first aspect, that is, the computer-readable storage medium stores instructions, and when the instructions are executed on a computer, the pavement disease image processing method as described in the first aspect is executed. The computer-readable storage medium refers to a carrier for storing data, which may include but is not limited to computer-readable storage media such as floppy disks, optical disks, hard disks, flash memories, USB flash drives, and / or memory sticks, and the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.

[0109] The working process, working details and technical effects of the aforementioned computer-readable storage medium provided in the fourth aspect of this embodiment can be referred to the pavement disease image processing method described in the first aspect, and will not be repeated here.

[0110] In a fifth aspect, the present embodiment provides a computer program product, including a computer program or an instruction, wherein when the computer program or the instruction is executed by a computer, the pavement damage image processing method as described in the first aspect is implemented. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device.

[0111] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for processing pavement damage images, characterized in that: include: Obtaining an oblique photographic image of the road surface in front of the vehicle collected in real time by a vehicle-mounted binocular camera, wherein the vehicle-mounted binocular camera is installed at the front of the vehicle body and the camera lens is tilted toward the front and downward; Using a target detection algorithm in real time to perform road surface disease recognition processing on the oblique photographic image of the road surface in front of the vehicle to obtain a road surface disease recognition result; If at least one pavement defect identification frame is found according to the pavement defect identification result, for each pavement defect identification frame in the at least one pavement defect identification frame, a corresponding pavement defect oblique photography image is captured from the oblique photography image of the pavement in front of the vehicle; For each pavement defect identification frame, the corresponding pavement defect oblique photographic image and the three-dimensional coordinates of the frame center and / or frame corners in the camera coordinate system of the vehicle-mounted binocular camera are imported into an AI drawing tool based on AIGC, and the corresponding pavement defect top-view photographic image is output; For each road surface defect identification frame, the location of the corresponding road surface defect is determined according to the satellite positioning result of the front part of the vehicle body at the time of image acquisition and the three-dimensional coordinates of the center of the corresponding frame in the camera coordinate system of the vehicle-mounted binocular camera, wherein the image acquisition time refers to the time of acquisition of the oblique photography image of the road surface in front of the vehicle; Extracting pavement disease parameters at the location of the pavement disease based on the overhead photographic image of the pavement disease corresponding to the location of the pavement disease; The location of the pavement damage and the corresponding pavement damage parameters are loaded and displayed on the road map display page.

2. The pavement damage image processing method according to claim 1, characterized in that: The target detection algorithm is used to perform road surface disease recognition processing on the oblique photographic image of the road surface in front of the vehicle to obtain a road surface disease recognition result, including: The oblique photographic image of the road surface in front of the vehicle is imported into a pavement defect recognition model pre-trained based on a target detection algorithm, and a pavement defect recognition result that can show whether there is a pavement disease identification box on the oblique photographic image of the road surface in front of the vehicle is output, wherein the target detection algorithm adopts a three-stage network structure including a first-stage network, a second-stage network and a third-stage network, and the first-stage network, the second-stage network and the third-stage network respectively use Resnet50 network as the backbone network, and respectively use different void convolution methods with increasing void rates, so that the first-stage network, the second-stage network and the third-stage network are reverse cascaded in sequence from coarse to fine according to the features of the deep convolution layer.

3. The pavement damage image processing method according to claim 2, characterized in that: In the last layer of the first-stage network, M1 sliding windows with different sizes are used to perform window scanning on the oblique photographic image of the road surface in front of the vehicle, so as to obtain a plurality of first marking frames, and according to the normalized results of the probabilities of the plurality of first marking frames containing the target road surface disease, N1 best first marking frames are screened out from the plurality of first marking frames using a first non-maximum suppression algorithm, wherein M1 represents a positive integer greater than or equal to 30, and N1 represents a positive integer greater than or equal to 3500; In the last layer of the second-stage network, M2 sliding windows with different sizes are used to perform window scanning on the oblique photographic image of the road surface in front of the vehicle within the N1 best first marking frames, so as to obtain a plurality of second marking frames, and according to the normalized results of the probabilities of the plurality of second marking frames containing target road surface diseases, N2 best second marking frames are screened out from the plurality of second marking frames using a second non-maximum suppression algorithm, and N3 best third marking frames are aggregated and screened out using a third non-maximum suppression algorithm, wherein M2 represents a positive integer less than M1, N2 represents a positive integer less than N1, and N3 represents a positive integer less than N2; In the last layer of the third-stage network, a structured random forest is trained using the convolutional features of the last layer to estimate the edge contour of the target pavement disease, and based on the edge contour, a greedy iterative algorithm is used to search for a pavement disease identification frame with the maximum probability of containing the target pavement disease at different positions and aspect ratios within the N3 best third marking frames.

4. The pavement damage image processing method according to claim 1, characterized in that: For each pavement defect identification frame, the corresponding pavement defect oblique photographic image and the three-dimensional coordinates of the frame center and / or frame corners in the camera coordinate system of the vehicle-mounted binocular camera are imported into an AI drawing tool based on AIGC, and the corresponding pavement defect top-view photographic image is output, including the following steps S41 to S43: S41. For a road surface disease identification frame, the corresponding road surface disease oblique photographic image and the three-dimensional coordinates of the frame center and / or frame corners in the camera coordinate system of the vehicle-mounted binocular camera are imported into the AI ​​drawing tool based on AIGC, and the corresponding road surface disease top-view photographic image is output, and then step S42 is executed; S42. Calculate the image clarity of the road surface disease overhead photographic image of the road surface disease identification frame using an image clarity evaluation algorithm, and then execute step S43; S43. Determine whether the image clarity of the pavement disease overhead photographic image of the pavement disease identification frame is lower than a first preset threshold value. If so, return to step S41. Otherwise, use the pavement disease overhead photographic image most recently output as the final pavement disease overhead photographic image of the pavement disease identification frame.

5. The pavement damage image processing method according to claim 1, characterized in that: When the oblique photographic image of the road surface in front of the vehicle is any frame of video image in the video data collected in real time by the on-board binocular camera, the pavement disease parameter at the location of the pavement disease is extracted according to the top-view photographic image of the pavement disease corresponding to the location of the pavement disease, including: Summarizing all the pavement disease overhead photographic images corresponding to the same pavement disease location to obtain a pavement disease overhead photographic image set; For each of the pavement defect overhead photographic images in the pavement defect overhead photographic image set, if any frame edge of the corresponding pavement defect identification frame is found to be located at an edge area of ​​the inclined photographic image of the road surface in front of the vehicle, the corresponding image is discarded; Arrange all the remaining images in the pavement disease overhead photography image set in order from the latest to the earliest corresponding image acquisition time to obtain an image sequence; Selecting the first K pavement disease overhead photographic images from the image sequence, and calculating the corresponding image clarity of each pavement disease overhead photographic image in the first K pavement disease overhead photographic images by using an image clarity evaluation algorithm, wherein K represents a positive integer; According to a certain pavement disease overhead photographic image with the highest image clarity among the first K pavement disease overhead photographic images, pavement disease parameters at the location of the same pavement disease are extracted.

6. The pavement damage image processing method according to claim 5, characterized in that: According to a certain pavement disease overhead photographic image with the highest image clarity among the first K pavement disease overhead photographic images, pavement disease parameters at the location of the same pavement disease are extracted, including: Selecting a pavement disease overhead photographic image with the highest image clarity from the first K pavement disease overhead photographic images; If it is determined that the highest image clarity reaches a second preset threshold, then, based on the overhead photographic image of a certain road surface disease, pavement disease parameters at the location of the same road surface disease are extracted; If it is determined that the maximum image clarity is lower than the second preset threshold, an image acquisition task is generated to instruct the UAV to go to the sky above the same pavement defect location to collect a bird's-eye view photographic image of the road surface, and then the image acquisition task is dispatched to a vehicle-mounted UAV. Finally, based on the bird's-eye view photographic image of the road surface collected by the vehicle-mounted UAV above the same pavement defect location, the pavement disease parameters at the same pavement defect location are extracted.

7. A road surface disease image processing device, characterized in that: It includes a photographic image acquisition unit, a road surface disease recognition unit, an inclined image interception unit, a bird's-eye view image generation unit, a disease location determination unit, a disease parameter extraction unit and a detection result loading unit; The photographic image acquisition unit is used to acquire an oblique photographic image of the road surface in front of the vehicle collected in real time by a vehicle-mounted binocular camera, wherein the vehicle-mounted binocular camera is installed at the front of the vehicle body and the camera lens is tilted toward the front and downward; The road surface disease recognition unit is communicatively connected to the photographic image acquisition unit and is used to perform road surface disease recognition processing on the inclined photographic image of the road surface in front of the vehicle using a target detection algorithm in real time to obtain a road surface disease recognition result; The oblique image interception unit is communicatively connected to the pavement defect recognition unit, and is used to intercept the corresponding pavement defect oblique photography image from the oblique photography image of the road surface in front of the vehicle for each pavement defect identification frame in the at least one pavement defect identification frame if at least one pavement defect identification frame is found according to the pavement defect recognition result; The overhead image generation unit is communicatively connected to the oblique image interception unit, and is used to import the corresponding oblique photographic image of the road surface disease and the three-dimensional coordinates of the frame center and / or frame corners in the camera coordinate system of the vehicle-mounted binocular camera into the AI ​​drawing tool based on AIGC for each road surface disease identification frame, and output the corresponding overhead photographic image of the road surface disease; The defect location determination unit is communicatively connected to the oblique image interception unit, and is used to determine the location of the corresponding road surface defect for each road surface defect identification frame according to the satellite positioning result of the front part of the vehicle body at the time of image acquisition and the three-dimensional coordinates of the center of the corresponding frame in the camera coordinate system of the vehicle-mounted binocular camera, wherein the image acquisition time refers to the acquisition time of the oblique photography image of the road surface in front of the vehicle; The disease parameter extraction unit is respectively connected to the overhead image generation unit and the disease position determination unit for communication, and is used to extract the pavement disease parameter at the location of the pavement disease based on the pavement disease overhead photographic image corresponding to the location of the pavement disease; The detection result loading unit is communicatively connected to the disease parameter extraction unit, and is used to load and display the location of the road disease and the corresponding road disease parameters on the road map display page.

8. A computer device, characterized in that: It includes a memory, a processor and a transceiver which are communicatively connected in sequence, wherein the memory is used to store a computer program, the transceiver is used to send and receive messages, and the processor is used to read the computer program to execute the pavement disease image processing method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores instructions, and when the instructions are executed on the computer, the pavement damage image processing method as described in any one of claims 1 to 6 is executed.

10. A computer program product comprising a computer program or instructions, characterized in that When the computer program or the instruction is executed by a computer, the pavement damage image processing method according to any one of claims 1 to 6 is implemented.

Images