A real-time detection and identification method for road surface diseases in a road inspection image

Abstract

The application discloses a kind of road inspection image in real-time detection and identification method of road surface disease, it is related to road intelligent inspection and image target detection technical field, the specific steps of this method are as follows: 8 frames sliding image pool is constructed by collecting road surface image, control frame overlapping pixel is more than 65%;After constructing pixel time sequence feature matrix by correcting image, output disease candidate frame by double-branch time sequence convolution network;Real disease is filtered by fusing score and 5 frames continuous determination, and contour labeling is completed and identification data package is generated;The application does not need to position hardware, relies on sliding image pool and alignment image of mark correction, extracts pixel time sequence feature to distinguish damage and transient interference, multiple frame determination reduces false detection, adapts to lightweight vehicle-mounted equipment real-time detection;Double-branch convolution network distinguishes disease and interference, and disease contour is labeled by fusing time sequence score grading, without positioning data, disease can also be recorded completely, storage capacity is small, and recognition stability is stronger in complex working condition.

Description

Technical Field

[0001] This invention relates to the field of intelligent road inspection and image target detection technology, specifically a method for real-time detection and identification of road surface defects in road inspection images. Background Technology

[0002] Road surfaces endure long-term vehicle loads, natural weathering, and rainwater erosion, continuously developing various structural defects such as cracks, potholes, subsidence, and alligator cracks. If these defects are not promptly identified and addressed, the damage will continue to expand, reducing road comfort and potentially leading to vehicle accidents. Therefore, routine road inspections are a core component of the road traffic maintenance system. Currently, intelligent inspection technology is gradually replacing manual foot patrols. It relies on vehicle-mounted image acquisition equipment to capture images of the entire road surface, and then uses image recognition algorithms to automatically locate damaged areas. Most mainstream recognition solutions in the industry rely on single... The system extracts and determines the features of road defects from frame images. Some solutions combine this with positioning sensors to collect geographic coordinate information and archive defects based on spatial location. Road inspection operations are usually accompanied by continuous movement of the vehicle, and the collected images have problems such as inter-frame perspective shift and image overlap. Outdoor scenes also have various temporary interference elements such as fallen leaves, water reflections, momentary shadows, and scattered gravel. Various dynamic interferences and image distortions will directly affect the output results of the recognition algorithm. The industry continues to explore detection solutions based on collaborative analysis of multi-frame temporal images to improve the overall performance of road defect recognition in complex outdoor scenes.

[0003] Existing road surface defect image detection technologies suffer from several unavoidable shortcomings. Most traditional identification schemes heavily rely on external positioning sensors such as GPS and inertial measurement units, resulting in complex hardware configurations for the entire inspection equipment. The costs of equipment procurement, debugging, and subsequent maintenance are high. Positioning sensors are susceptible to signal loss due to tunnels and elevated roads, interrupting the identification process. Traditional algorithms only perform feature analysis on single-frame images, lacking multi-frame temporal pixel correlation analysis logic. They cannot distinguish between long-term real defects and short-term temporary interference, easily generating a large number of false alarm detection boxes, significantly increasing the workload of subsequent manual review. Conventional image correction methods do not uniformly handle perspective shifts in consecutive multi-frame images. Frame-to-frame pixel coordinate misalignment can cause cross-frame feature fragmentation of the same defect area, making it difficult to fully extract the temporal change features of defects. Traditional network models do not distinguish the identification link between defects and interference targets. Mixed operations of various pixel regions cause feature confusion. There is a lack of a unified quantitative scoring system and hierarchical labeling mechanism. The identification output data has a single dimension, making it impossible to intuitively distinguish the confidence level of defects. Under high-speed inspection conditions, it is difficult to simultaneously achieve both recognition accuracy and real-time performance. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of existing technologies and provide a real-time detection and identification method for road surface defects in road inspection images. The entire process relies solely on the pixel information of the road surface images for computation, without the need for various positioning sensors. The scheme first constructs a sliding temporal image pool to acquire continuous road surface images, then uses the road marking features to solve the homography matrix to correct inter-frame image offset, establishes a two-dimensional temporal pixel association feature matrix, extracts pixel temporal features through a gray-scale edge coupling fluctuation algorithm, and uses a dual-branch temporal convolutional network to identify road surface defects and various interference areas in parallel. Combined with a multi-index index steady-state fusion algorithm, it generates regional steady-state scores, filters short-term false alarms based on continuous multi-frame retention conditions, and generates hierarchical contour annotations through a confidence continuous mapping annotation algorithm. Finally, it outputs an image recognition data package containing multi-dimensional quantitative information. This invention relies on temporal pixel association logic to suppress outdoor environmental interference, reduces hardware configuration costs, and improves the accuracy and real-time performance of defect identification in mobile inspection scenarios. It is applicable to various road vehicle-mounted intelligent inspection scenarios.

[0005] To solve the above-mentioned technical problems, the present invention provides the following technical solution: a method for real-time detection and identification of road surface defects in road inspection images, the specific steps of which are as follows: S1. A fixed frame rate image acquisition device is used to continuously acquire road images and build a sliding time sequence image pool that stores 8 consecutive frames of images. The speed control unit configured on the carrier is used to correct the driving speed in real time so as to control the overlapping pixel ratio of adjacent frames of the road surface to be no less than 65%. The acquired data only contains image pixel information and does not access external data such as positioning sensors. In S1, the vehicle travels at a speed of 20 km / h. The vehicle is equipped with a speed control unit, which continuously corrects the vehicle's travel speed. After the sliding temporal image pool identifies the image content corresponding to the intersection and road widening, it extends the image index retention time. When the sliding temporal image pool performs pixel compression, it encodes the image texture data, maintains the original precision of the pixel grayscale values, and does not perform numerical rounding. The image group output by the sliding temporal image pool is completely transmitted to S2. There is no loss of pixel data or disorder of image temporal order within the image group. S2. Extract the feature points of fixed road markings in the image pool, remove mismatched feature points, solve the global homography matrix, and correct the pixel coordinates of all images based on the homography matrix to generate a time-aligned pixel dataset. S3. Based on the time-aligned pixel dataset, construct a two-dimensional time-series pixel association feature matrix, fill the association feature matrix with pixel gradient, edge continuity, and gray-level variance feature parameters, and use the gray-level edge coupling fluctuation algorithm to calculate the time-series feature quantization value of each pixel. S4. Input the temporal pixel association feature matrix into a dual-branch temporal convolutional network. The dual-branch temporal convolutional network is used to identify the disease area and the road interference area in parallel and generate a disease candidate detection box. Extract three types of temporal indicators for the identified disease connected pixel areas and run a multi-indicator index steady-state fusion algorithm to calculate the temporal steady-state fusion score. S5. The condition that the disease appears continuously for no less than 5 frames in the time-series image pool is used as the threshold condition for judging the real disease. Short-term interference areas are filtered out. For the disease areas that meet the judgment threshold, the time-series steady-state fusion score is retrieved. The confidence continuous mapping annotation algorithm is run. The matching contour lines are calculated according to the time-series steady-state fusion score and the image annotation is completed. The annotation information is integrated to generate an image recognition data package. In step S5, the image recognition data packets are sorted and stored from high to low according to the temporal steady-state fusion score values. Image recognition data packets with temporal steady-state fusion scores below 0.5 are divided into independent storage partitions. The image recognition data packets store pixel-level recognition parameters and do not contain road segment location values. The image recognition data packets retain the original pixel calculation values ​​and the original temporal steady-state fusion score values ​​of the composite feature dataset.

[0006] Further, in S1, the image acquisition device is a visible light industrial camera vertically oriented towards the road surface. The frame rate of the image acquisition device ranges from 25fps to 30fps, and the resolution of the output image is 1920×1080 or 2560×1440. When the image acquisition device is working, the carrier moves at a constant speed, and the adjustment range of the carrier's speed corresponds to the overlapping pixel ratio of adjacent frames of the road surface being between 65% and 78%. The sliding time-series image pool adopts a ring memory cache architecture and executes a first-in-first-out update logic. For each new road surface image output by the image acquisition device, the sliding time-series image pool removes the image with the earliest internal storage time. The sliding time-series image pool is divided into a pixel storage partition and a time index partition. The pixel storage partition completely retains the original pixel values ​​of the image, and the time index partition records the image acquisition timestamp. The sliding time-series image pool is equipped with a lossless pixel compression module, and the pixel data is restored losslessly before being transmitted to S2.

[0007] Furthermore, in step S2, the inflection points of solid lane lines, right-angled contour points of zebra crossings, and boundary inflection points of road directional arrows are selected as fixed road marking feature points. In road sections where road markings are identifiable, the number of effective feature points extracted from a single frame image is no less than 30 sets. Bidirectional matching is performed on feature points of adjacent frames, and mismatched feature points with pixel coordinate offsets are removed. The global homography matrix is ​​obtained by solving the remaining effective matching feature points. The global homography matrix is ​​used to perform perspective transformation correction on all 8 frames of images in the temporal image pool. The correction process eliminates the horizontal and vertical pixel coordinate offsets of the images. The temporal aligned pixel dataset obtained after correction is sorted according to the road space coordinates. Non-road pixels at the image edges are removed, and the gray values ​​in the dataset are uniformly normalized to the range of 0 to 255. A pixel space index table recording the relative coordinates of pixels inside the image is generated.

[0008] Furthermore, in S2, the temporally aligned pixel dataset removes irrelevant pixels from the image edges, retaining only the valid pixels corresponding to the road surface of the driving lane in the center of the image; the temporally aligned pixel dataset is configured with a pixel spatial index table, which records the horizontal and vertical coordinates of pixels inside the image, and the pixel spatial index table does not set road segment mileage parameters and geographical latitude and longitude parameters; the time-aligned pixel dataset that has completed the standardization process is packaged in a fixed file format, and the packaged dataset file serves as the sole input data source for the S3 operation step.

[0009] Furthermore, in S3, the rows of the two-dimensional temporal pixel association feature matrix correspond to the coordinates of independent road surface pixels, and the columns of the two-dimensional temporal pixel association feature matrix are sorted according to the image acquisition time to correspond to 8 frames of images in the sliding temporal image pool; the pixel gradient parameter is obtained by calculating the mean value of the horizontal gray-level gradient and the vertical gray-level gradient of the pixel respectively using the Sobel operator; the edge continuity parameter is the ratio of the number of edge pixels in the eight-neighborhood of the pixel to the total number of pixels in the eight-neighborhood; the gray-level variance parameter is the discrete variance of the temporal gray-level values ​​of a single pixel corresponding to 8 frames; the two-dimensional temporal pixel association feature matrix filters out solid-color road surface pixels with gradient values ​​lower than a fixed gradient threshold, and all feature values ​​in the two-dimensional temporal pixel association feature matrix are stored in single-precision floating-point format.

[0010] Furthermore, in S3, the mathematical expression for the grayscale edge coupling fluctuation algorithm is: in: This represents the temporal fluctuation enhancement feature of the p-th road surface pixel, where T is the total number of frames in the temporal window and is fixed at 8. This represents the grayscale value of pixel p in the image at frame t. This represents the average of the temporal grayscale values ​​of pixel p across 8 frames. The edge continuity ratio of pixel p in the t-th frame image is represented by a value ranging from 0 to 1. All pixel rows remaining in the two-dimensional temporal pixel association feature matrix are substituted into the grayscale edge coupling fluctuation algorithm to complete the calculation. All pixel operations generate... The numerical data are aggregated to form a composite feature dataset, which is then sorted in ascending order of road surface pixel coordinates.

[0011] Furthermore, in S4, the data input channel of the dual-branch temporal convolutional network cuts off the data transmission path of GPS and the inertial measurement unit. The dual-branch temporal convolutional network sets up two independent parallel operation links. The first operation link is the real disease identification link, and the second operation link is the false alarm interference identification link. The real disease identification link divides the disease identification into four categories: cracks, pits, subsidence, and crazing. The real disease identification link counts the total number of frames in which abnormal features continuously appear in the connected pixel areas of the disease. The false alarm interference identification link divides the interference identification into four categories: fallen leaves, water reflection, scattered sand and gravel, and momentary shadows. After identifying the corresponding pixel area, the false alarm interference identification link blocks the detection box of the corresponding pixel area.

[0012] Furthermore, in S4, the mathematical expression for the multi-index exponential steady-state fusion algorithm is: Where: S represents the temporal steady-state fusion score of the disease-connected region. All pixels within the connected area of ​​the disease The average value, This is the temporal fluctuation enhancement feature for the p-th road surface pixel; For all pixels of all 8 frames in the sliding time-series image pool The global maximum value, and when season The value is set to 0 to avoid division by zero exceptions; The total number of frames within an 8-frame time window where the region edge continuity meets the standard; The total number of frames within an 8-frame time window whose regional grayscale fluctuations meet the stability standard, wherein the stability standard is that the grayscale fluctuation amplitude is lower than a fixed stability threshold. , , The exponential weighting coefficients are satisfied. The index weighting coefficient has a fixed value. , , T is fixed at a value of 8.

[0013] Furthermore, in S4, the samples used in the offline training phase of the dual-branch temporal convolutional network include road surface images labeled with disease categories and temporal duration frames. The samples do not contain geolocation labels. The real disease identification link only receives pixel matrices with temporal fluctuation amplitudes lower than a fixed stable threshold and edge continuity parameters higher than a fixed complete threshold. The false alarm interference identification link only receives pixel matrices with temporal fluctuation amplitudes higher than a fixed stable threshold and edge continuity parameters lower than a fixed complete threshold.

[0014] Furthermore, in S5, the mathematical expression for the confidence-based continuous mapping annotation algorithm is: in: Here, S represents the visual confidence mapping value of the disease boundary, and S is the time-series steady-state fusion score, ranging from 0 to 1. , To visualize the calibration constant, the values ​​of the calibration constant are visualized. , ; for the calculated Numerical truncation is performed, limiting the lower limit to 0 and the upper limit to 1. After truncation, the values ​​are remapped to three numerical intervals. Corresponding to the solid line outline, Corresponding to the dashed outline, For the corresponding dot-line contour, the pixel areas corresponding to the dot-line contour are marked with text labels for verification; for pixel areas that are continuously identified as diseases for 1, 2, 3, or 4 frames, the corresponding detection box data is directly deleted; only pixel areas that are continuously identified for 5 or more frames are input into the confidence continuous mapping annotation algorithm for calculation; the image recognition data package contains six types of data: hierarchical contour vector, disease pixel coordinates, disease classification type, total area of ​​disease pixels, temporal steady-state fusion score, and number of consecutive disease frames.

[0015] Compared with existing technologies, this method for real-time detection and identification of road surface defects in road inspection images has the following advantages: I. This invention constructs a fixed-capacity sliding temporal image pool and completes multi-frame image temporal alignment processing based on road marking features. It does not access external data such as positioning sensors throughout the process, eliminating the dependence of inspection equipment on positioning hardware such as GPS and inertial measurement units. This reduces the hardware deployment cost and equipment failure probability of the entire inspection and recognition system. It uses homography matrix to complete perspective correction of multi-frame images to eliminate pixel offset, constructs a unified two-dimensional temporal pixel association feature matrix to extract multi-dimensional pixel temporal features, and uses a grayscale edge coupling fluctuation algorithm to quantify the long-term grayscale change pattern of pixels. This allows it to distinguish between static road surface damage and instantaneous image anomalies. It uses continuous multi-frame judgment rules in the temporal dimension to filter short-term interference sources, suppressing false alarm targets such as fallen leaves and instantaneous shadows from the perspective of temporal change patterns, significantly reducing the false detection rate of road defects. Furthermore, the entire image acquisition, correction, and feature extraction process is based on pure pixel information computation, with a simple data link. It can be adapted to lightweight vehicle-mounted hardware to achieve real-time online detection of road defects, ensuring the real-time nature of the recognition output during inspection operations.

[0016] II. This invention separates the defect identification and interference identification into two independent computational links by setting up a dual-branch parallel temporal convolutional network. It processes pixel regions with different feature attributes separately, specifically distinguishing between various real road defects and multiple road interferences, thereby improving the model's ability to extract and adapt to different target features. It adopts a multi-index index steady-state fusion algorithm to integrate regional temporal multi-dimensional features to obtain a unified steady-state score. Combined with a confidence continuous mapping labeling algorithm, it automatically generates differentiated defect contour labels based on the score, realizing visual hierarchical labeling of defect boundaries, distinguishing between high-confidence defects and suspected areas that require manual review. The image recognition data package output by the invention integrates multi-dimensional defect quantitative information and stores and manages it hierarchically. It can complete the complete quantitative record of defects without binding geographic location data, which reduces the data storage volume and simplifies the subsequent defect ledger organization process. The entire recognition logic relies on temporal pixel features to complete the full-domain judgment, improving the stability and reliability of defect recognition results in complex lighting and high-speed driving inspection scenarios.

[0017] Other advantages, objectives and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination or study, or may be learned from the practice of the invention. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0019] Figure 1 A flowchart of the method for real-time detection and recognition of road surface defects in road inspection images; Figure 2 Sub-flowchart for constructing the S3 temporal pixel feature matrix and generating the composite feature dataset; Figure 3 The flowchart shows the sub-process of S4 dual-branch temporal convolutional network for recognition and steady-state fusion scoring calculation. Detailed Implementation

[0020] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description of the specific implementation methods, structures, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided below.

[0021] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] Reference Figure 1 One embodiment of the present invention proposes a real-time detection and identification method for road surface defects in road inspection images. This method employs a data processing mechanism that utilizes an 8-frame circular sliding temporal image pool, temporal alignment of road marking feature homography matrix, gray-level edge coupling fluctuation pixel feature extraction, dual-branch temporal convolution parallel classification, multi-index steady-state fusion scoring, temporal frame number threshold filtering, and hierarchical contour annotation. This mechanism enables accurate differentiation between real road surface defects and interfering areas such as fallen leaves, reflections, and shadows without the need for external sensor data such as GPS or inertial positioning. Furthermore, it achieves high-accuracy, low-false-alarm-rate online real-time identification of four types of road surface defects: cracks, potholes, subsidence, and alligator cracks. The specific methods of this embodiment include: A visible light industrial camera oriented vertically toward the road surface is used to continuously acquire road surface images at a fixed frame rate. A circular sliding time-series image pool that can store 8 consecutive images is built. By adjusting the speed of the inspection vehicle in real time, the percentage of overlapping pixels of adjacent road surfaces is stably controlled in the range of 65%-78%. During the acquisition phase, only the original pixel information of the image is retained, and no external data such as positioning sensors is connected. Inflection point features of solid lane lines, zebra crossings, and guide arrows in each frame of the temporal image pool are extracted. Feature points of adjacent frames are matched bidirectionally and mismatched points with coordinate offset are removed. The global homography matrix is ​​solved using the retained effective feature points. Perspective correction is performed on all 8 frames of images based on the homography matrix to generate a spatially aligned temporal aligned pixel dataset. Based on a temporally aligned pixel dataset, a two-dimensional temporal pixel association feature matrix is ​​constructed, with rows corresponding to pixel coordinates and columns corresponding to temporal frames. The matrix is ​​then filled with three types of feature parameters: pixel gradient, edge continuity, and gray-level variance. All valid pixels in the matrix are substituted into a gray-level edge coupling fluctuation algorithm to calculate temporal fluctuation enhancement features pixel by pixel. The composite feature dataset is then generated. The two-dimensional temporal pixel association feature matrix is ​​input into a dual-branch temporal convolutional network with the input channel of the positioning sensor cut off. The network is operated synchronously through two independent parallel links for real diseases and false alarm interference, and outputs candidate detection boxes for diseases. For each disease-connected pixel region, three types of temporal indicators are extracted: the number of edge-compliant frames, the number of gray-level stable frames, and the average fluctuation feature of the region. These are then substituted into a multi-indicator index steady-state fusion algorithm to calculate the temporal steady-state fusion score S. The condition of a disease appearing continuously for no less than 5 frames within an 8-frame time window is used as the filtering condition for a real disease. Short-term interference detection boxes that last for 1-4 frames are directly deleted. For disease areas that meet the frame number threshold, the temporal steady-state fusion score is retrieved. The confidence continuous mapping annotation algorithm is run to calculate the visual confidence mapping value of the disease boundary. Based on the mapping value, the three levels of contours, namely solid line, dashed line and dotted line, are divided and the image is annotated. Six types of information, including pixel coordinates, disease type, and fusion score, are integrated to generate an image recognition data package.

[0023] Specifically, the execution logic of this invention lies in building an image temporal processing system that does not rely on positioning sensors. This system first completes lossless storage and temporal management of continuous road surface images through a circular buffer temporal image pool. It then uses inherent road marking features to achieve unified alignment of spatial coordinates across multiple frames, eliminating pixel shifts caused by vehicle movement and camera perspective. Next, it extracts three-dimensional low-level features—gradient, edge, and grayscale changes—from the aligned temporal pixels. A grayscale edge coupling fluctuation algorithm is used to fuse the temporal change patterns of a single pixel across eight frames, quantifying the degree of pixel anomalies. Subsequently, a dual-branch temporal convolutional network is used to distinguish between four types of real defects and four types of road surface interference in parallel, avoiding false detections caused by instantaneous environmental clutter. Finally, a dual screening mechanism combining a hard threshold for the number of temporally stored frames and a multi-index fusion soft scoring system filters out short-term false alarm targets and draws defect contours based on score differences, outputting a complete structured recognition data package.

[0024] Optionally, the step of using a visible light industrial camera vertically oriented towards the road surface to continuously acquire road surface images at a fixed frame rate, constructing a circular sliding temporal image pool capable of storing 8 consecutive frames of images, and stabilizing the percentage of overlapping pixels in adjacent road surface frames within the range of 65%-78% by adjusting the travel speed of the inspection vehicle in real time includes: A vertically mounted visible light industrial camera was selected as the image acquisition device, with a frame rate set to 25fps-30fps. The output image resolution was either 1920×1080 or 2560×1440. The inspection vehicle traveled at a constant speed, with a speed control unit adjusting the speed in real time to ensure that the percentage of overlapping pixels on the road surface in adjacent frames remained stable at 65% to 78%. The sliding time-series image pool adopted a ring-shaped memory cache architecture, following a first-in-first-out update logic. Each time the camera output a new road surface image, the oldest image acquired in the cache was automatically removed. The sliding time-series image pool was internally divided into a pixel storage partition and a time index partition. The pixel storage partition completely preserved the original pixel grayscale values ​​of the image, while the time index partition separately recorded the acquisition timestamp corresponding to each frame. The image pool had a built-in lossless pixel compression module, which performed lossless restoration of image data before transmission to subsequent steps, without causing any loss of grayscale values.

[0025] Specifically, the vertical deployment of industrial cameras avoids road perspective distortion caused by side-view shooting. A fixed frame rate of 25-30fps ensures that enough continuous road images can be captured in a short time. Two resolutions are available to adapt to different computing power scenarios of lightweight vehicle-mounted embedded devices and high-precision inspection equipment. Compared with ordinary linear cache, the ring memory cache architecture has lower read and write latency, making it more suitable for the low latency requirements of real-time vehicle inspection. Pixel partitions and time index partitions are physically isolated for storage, and pixel grayscale is not rounded or truncated throughout the process. Combined with a lossless compression module, it ensures that the image data entering the feature extraction stage is completely consistent with the camera's original output, and will not cause subsequent loss of defect features due to compression distortion. The speed control unit reads the vehicle's driving speed in real time. When changes in road scene such as intersections or road widening are detected in the image, the index retention time of the corresponding image is automatically extended to prevent vehicles at intersections and debris in widened areas from being misjudged as defects. The standard cruising speed of the vehicle is set to 20km / h. At this speed, the overlapping pixels of adjacent frames are stably maintained at about 72%, balancing inspection efficiency and timing matching accuracy.

[0026] For example, during the image acquisition and temporal image pool construction phase, the inspection vehicle is equipped with a 25fps, 2560×1440 resolution vertical visible light industrial camera. The standard cruising speed is locked at 20km / h. The speed control unit collects vehicle speed data in real time and dynamically fine-tunes it. According to pixel overlap calculation, the percentage of overlapping pixels between adjacent road surfaces is stable at 72%. The sliding temporal image pool enables a circular cache, continuously retaining the latest 8 original road surface images in memory. When the 9th new image is acquired, the earliest cached image (the 1st frame) is automatically removed. The pixel storage partition in the cache fully retains the original grayscale values ​​in the 0-255 range. Lossless encoding compression is performed on the image texture for transmission, and lossless decompression is completed during the feature extraction step, with no loss of grayscale value precision. When the camera captures an intersection area, the image pool identifies intersection features such as multiple lanes and stop lines in the image, extending the retention time of the time index of the corresponding image to twice that of ordinary road sections, preventing instantaneous pedestrian and vehicle shadows at intersections from being judged as road surface defects.

[0027] Optionally, the step of extracting the inflection point feature points of solid lane lines, zebra crossings, and guide arrows in each frame of the temporal image pool, bidirectionally matching feature points of adjacent frames and removing mismatched points with coordinate offsets, solving the global homography matrix using the retained valid feature points, and performing perspective correction on all 8 frames of images based on the homography matrix to generate a spatially aligned temporally aligned pixel dataset includes: The inflection points of solid lane lines, right-angled contours of zebra crossings, and inflection points of directional arrows on the road surface are selected as unique fixed road marking feature points. For unmarked road sections, globally uniformly distributed pixels are retained as auxiliary matching points. For complete road marking sections, at least 30 sets of effective feature points are extracted from a single frame image. Bidirectional matching logic is executed on the feature points extracted from adjacent frames, calculating the horizontal and vertical offsets of the matching point pixel coordinates. Matches with offsets exceeding a preset pixel threshold are judged as mismatches and directly discarded. The high-precision matching feature point sets retained after filtering are used to solve the global homography matrix. All 8 frames in the temporal image pool are then substituted into the homography matrix to perform perspective transformation. The calibration process eliminates lateral and longitudinal pixel coordinate offsets caused by vehicle vibration and camera pitch. After calibration, the pixels of the eight frames are reordered according to the real spatial coordinates of the road surface. Pixels on the non-driving lane road surface at the top, bottom, left, and right edges of the image are cropped and removed, retaining only the effective driving area in the center of the image. All pixel grayscale values ​​are standardized to the standard range of 0-255, and a pixel spatial index table is generated simultaneously, recording only the lateral and longitudinal relative coordinates of pixels. The index table does not include location parameters such as road mileage and geographic latitude and longitude. The standardized temporally aligned pixel dataset is packaged in a fixed binary file format and serves as the sole input data source for the next stage of feature matrix construction.

[0028] Specifically, road marking inflection points are selected as feature points. Since road markings are permanent and fixed road markers, they are unaffected by temporary disturbances such as fallen leaves, puddles, and shadows, resulting in significantly higher matching stability than random road surface pixels. The bidirectional matching mechanism, compared to unidirectional matching, can significantly reduce the probability of feature point mismatches and decrease the deviation in homography matrix calculation. A global homography matrix is ​​uniformly corrected across 8 frames, ensuring complete correspondence of pixel coordinates for the same road surface location within the 8-frame time sequence, providing a foundation for subsequent time-series pixel feature association calculations. Cropping non-road surface pixels at the edges significantly reduces the amount of invalid computational data, lowering the computational power consumption of embedded devices. Uniform grayscale regularization to the 0-255 range eliminates grayscale scale deviations caused by camera exposure differences under varying lighting conditions. The pixel spatial index table only records the relative positions of pixels, without introducing any positioning sensor data throughout the process, satisfying the core constraint of this invention: no external positioning dependency. The encapsulated fixed-format dataset can be directly read and parsed, eliminating the overhead of real-time data format conversion.

[0029] For example, during the image temporal alignment correction stage, 46 sets of valid road marking inflection point feature points are extracted from the complete single-frame image of the road markings. Bidirectional matching is performed between the feature points of the previous and current frames, generating 52 initial matching pairs. Among these, 11 matching points have horizontal and vertical pixel offsets exceeding a preset threshold of 10 pixels, and are therefore identified as mismatched points and removed. Finally, 41 sets of high-precision matching feature points are retained. Based on these 41 matching points, the global homography matrix is ​​calculated. Perspective transformation is then performed on all 8 frames of images in the cache. After correction, the pixel coordinate deviation of the same road surface crack within these 8 frames is controlled to within 1 pixel. After correction, the non-road areas of the image are cropped by 150 pixels at the top and bottom edges and 200 pixels at the left and right edges, retaining only the effective pixels of the middle lane. The grayscale of overexposed and underexposed pixels in the image is uniformly stretched and normalized to the range of 0-255, and a corresponding pixel spatial index table is generated. The table only records the relative coordinates (x, y) of each effective pixel, without GPS mileage or latitude and longitude fields. All 8 corrected images are packaged into a unified binary time-series dataset file and directly input into the two-dimensional time-series pixel association feature matrix construction module.

[0030] Optionally, the step involves constructing a two-dimensional temporal pixel association feature matrix based on the temporally aligned pixel dataset, where rows correspond to pixel coordinates and columns correspond to temporal frames. This matrix is ​​then filled with three types of feature parameters: pixel gradient, edge continuity, and grayscale variance. All valid pixels within the matrix are substituted into a grayscale edge coupling fluctuation algorithm to calculate the temporal fluctuation enhancement feature pixel by pixel. The aggregated composite feature dataset includes: A two-dimensional temporal pixel association feature matrix is ​​constructed. Each row of the matrix corresponds to the spatial coordinates of an independent road surface effective pixel, and each column of the matrix corresponds to the 1st to 8th frames of the sliding image pool in temporal order. The pixel gradient parameter is calculated using the Sobel operator to calculate the horizontal and vertical gray-level gradients of a single pixel, and the arithmetic mean of the two is taken as the gradient feature value; the edge continuity parameter is defined as the ratio of the number of edge pixels in the eight-neighborhood of a pixel to the total number of all eight pixels in the eight-neighborhood, and the value range is 0-1; the gray-level variance parameter is the discrete variance of all gray-level values ​​of the same pixel in the eight-frame temporal image. Set a fixed gradient threshold, traverse all pixel rows of the matrix, filter out solid color smooth road surface pixels with gradient values ​​lower than the threshold, and only retain pixel rows with grayscale changes and potential defects to participate in subsequent calculations; all gradient, edge continuity, and grayscale variance feature values ​​in the matrix are stored in a single-precision floating-point format. The mathematical expression for the grayscale edge coupling fluctuation algorithm is: in: This represents the temporal fluctuation enhancement feature of the p-th road surface pixel, where T is the total number of frames in the temporal window and is fixed at 8. This represents the grayscale value of pixel p in the image at frame t. This represents the average of the temporal grayscale values ​​of pixel p across 8 frames. The mathematical expression for calculation is: , This represents the percentage of edge continuity of pixel p within the t-th frame of the image. The mathematical expression for calculation is: ,in Let p be the number of edge pixels in the eight-neighborhood of pixel p in frame t. The above formula is given by... This refers to the single-pixel temporal fluctuation enhancement feature; T represents the fixed temporal frame number (8); p represents the current road surface pixel number involved in the calculation; and t represents the temporal frame number. For the corresponding pixel and the corresponding frame grayscale value, The average grayscale value of this pixel across 8 frames. Count the pixels at the edge of the eight-neighborhood. This represents the edge continuity ratio of a single frame. The calculation is performed by substituting each retained pixel row in the two-dimensional temporal pixel association feature matrix into the formula, and then summing up the results from all pixels. The numerical values ​​are rearranged in ascending order according to the spatial coordinates of the road surface pixels to generate an ordered composite feature dataset.

[0031] Specifically, the two-dimensional matrix is ​​arranged in a pixel-temporal dual-dimensional manner, which can directly store the temporal changes of pixels at the same spatial location for 8 frames, facilitating parallel batch calculation of temporal features. Gradient, edge, and gray-level variance are three types of low-level features that respectively characterize pixel brightness abrupt changes, contour integrity, and temporal brightness fluctuations. The combination of these three features can completely distinguish between defects and smooth road surfaces. Filtering low-gradient solid-color pixels can significantly compress the matrix data size, reducing the computational load of subsequent convolutional networks. Single-precision floating-point storage ensures computational accuracy while reducing memory usage. The gray-level edge coupling fluctuation algorithm multiplies and fuses the pixel temporal gray-level fluctuation amplitude with the temporal edge integrity. Smooth road surfaces have stable pixel temporal gray levels and low edge proportions, resulting in... The values ​​approach 0; the temporal grayscale of pixels with defects such as cracks and pits fluctuates continuously, while the edge features remain stable. The values ​​are significantly higher than expected; interference pixels such as fallen leaves and momentary shadows only show edge fluctuations in a few frames. The value is low, and this feature can be used to distinguish between diseases and short-term disturbances in advance.

[0032] For example, in the generation and execution stage of the two-dimensional temporal pixel association feature matrix and composite feature dataset, a total of 126,420 effective road surface pixels are extracted from the temporal alignment dataset, constructing a two-dimensional feature matrix of 126,420 rows × 8 columns. Three basic features are calculated pixel-by-pixel: the horizontal gradient value of a crack pixel is 18.6, the vertical gradient value is 16.2, and the average gradient parameter is 17.4; the edge continuity parameter of the 6 edge pixels within the eight-neighborhood is 6 / 8 = 0.75; the grayscale values ​​of the 8 frames are 62, 58, 60, 57, 61, 59, 56, and 60, respectively, and the calculated grayscale variance is 3.12. Since the gradient value of this pixel is higher than the preset gradient threshold of 2, it is retained for subsequent calculations. This pixel parameter is substituted into the grayscale edge coupling fluctuation formula, with T fixed at 8, and the calculation is performed for each frame. The summation and averaging are then performed to obtain the temporal fluctuation enhancement feature of the pixel. =12.86. After traversing the matrix and completing the calculation of all retained pixels, all pixels... Sort the pixels by (x, y) coordinates from smallest to largest to generate an ordered composite feature dataset, which is then passed in its entirety to the dual-branch temporal convolutional network recognition module, such as... Figure 2 As shown.

[0033] Optionally, the two-dimensional temporal pixel association feature matrix is ​​input into a dual-branch temporal convolutional network that cuts off the positioning sensor input channel. Simultaneous computation is performed through two independent parallel links for real lesions and false alarm interference, outputting lesion candidate detection boxes. For each lesion-connected pixel region, three types of temporal indicators are extracted: the number of edge-compliant frames, the number of gray-level stable frames, and the region's average fluctuation characteristics. These are then substituted into a multi-indicator exponential steady-state fusion algorithm to calculate the temporal steady-state fusion score S, including: The hardware data input channel of the dual-branch temporal convolutional network physically cuts off the data transmission paths of positioning sensors such as GPS and inertial measurement units, and does not receive any geographic positioning information; the network internally sets up two completely independent and parallel feature processing links, the first of which is the real disease identification link and the second of which is the false alarm interference identification link. The real-world defect identification link is preset with four types of identification targets: cracks, potholes, subsidence, and crazing. The link counts the total number of time-series frames in which abnormal features continuously appear within the connected pixel area. The false alarm interference identification link is preset with four types of road surface interference targets: fallen leaves, water reflection, scattered sand and gravel, and momentary shadows. After the link identifies the corresponding interference area, it directly blocks the candidate detection boxes generated in that area. During the offline training phase of the network, road surface image samples with only the disease category and the number of time-series frames labeled are used, and the samples do not have any geolocation tags; the two links are divided into differentiated input pixel matrices: the real disease link only receives pixel matrices with time-series fluctuation amplitude below a fixed stable threshold and edge continuity parameters above a fixed integrity threshold; the false alarm interference link only receives pixel matrices with time-series fluctuation amplitude above a fixed stable threshold and edge continuity parameters below a fixed integrity threshold. After the network parallel inference is completed, it outputs all unmasked disease-affected connected pixel candidate detection boxes, and extracts three types of time-series indicators for each connected region: The total number of frames within an 8-frame temporal window where the region edge continuity meets the standard. This refers to the total number of frames within an 8-frame time window where the grayscale fluctuation amplitude is lower than the stability threshold. All pixels within the region The arithmetic mean, The mathematical expression for calculation is: ,in This represents the total number of pixels within the currently connected region of the disease. This is to enhance the temporal fluctuation feature of the i-th pixel within the region; the mathematical expression of the multi-index exponential steady-state fusion algorithm is: Where: S is the temporal steady-state fusion score of the disease-connected region, with a value range of 0~1; For all pixels of the sliding time-series image pool for 8 frames The global maximum value, if Then let The overall value is 0 to avoid division by zero errors; T is fixed at 8; the exponential weighting coefficient satisfies... Fixed configuration , , , , The mathematical expression for the allocation calculation is as follows: , , ,in For pixel fluctuation feature weight base, As the weight base for edge-stabilized frame count, The weighting base for grayscale stable frame count is a fixed value used in engineering. , , In the above complete set of public data, S represents the disease steady-state fusion score, The average temporal fluctuation characteristics of pixels representing the diseased area Represents the maximum value of pixel fluctuation features across the entire image. Represents the index weights of volatility characteristic terms, The index weights representing the number of frames that meet the edge criteria Represents the index weight of the grayscale stable frame rate term. Represents the total number of frames that meet the time-series edge target. T represents the total number of frames with stable grayscale within the time sequence, and T represents the fixed number of frames in the time sequence. Represents the total number of pixels in the connected region of the disease. Single-pixel fluctuation features within the representative region , , These are the base weights for the three types of indicators; Specifically, the positioning sensor input channel is cut off, ensuring that the entire identification process relies entirely on the temporal features of image pixels, without depending on external positioning equipment, thus reducing the hardware modification costs of inspection vehicles. Dual-branch parallel links enable simultaneous identification of defects and interference, eliminating the need for two separate serial inference steps, significantly shortening the processing latency of a single frame image and meeting real-time detection requirements. Differentiated input filtering rules for the two links pre-select pixel regions with significantly different feature attributes, reducing the probability of network recognition confusion. A multi-index fusion algorithm simultaneously integrates three dimensions: average pixel fluctuation features of the region, the number of stable edge frames, and the number of unstable grayscale frames. Defects and interference that cannot be completely distinguished by a single index can be differentiated through multi-index weighted fusion to obtain differentiated scores: For genuine defect areas, the edges are continuously stable, and the grayscale fluctuates over a long period, with an S value approaching 1; for short-term interference areas such as fallen leaves and momentary shadows, the number of qualifying edge frames is low, and the number of stable grayscale frames is high, with an S value approaching 0. The reliability of the defect is quantified by the score.

[0034] For example, during the inference and steady-state fusion scoring calculation phase of the dual-branch temporal convolutional network, a two-dimensional temporal pixel association feature matrix is ​​input into the network, while the positioning sensor channel remains closed, with no GPS or inertial navigation data flowing in. A cracked connected region is identified in the actual defect link, while the false alarm interference link fails to match the four types of interference: fallen leaves, reflections, sand, and shadows. The candidate detection box for this region is retained. Three types of temporal indicators are extracted for this region: average pixel fluctuation characteristics within the region. Global maximum of the entire map The number of frames that meet the edge continuity standard within 8 frames of the time sequence. The number of frames whose grayscale fluctuations meet the standard of stability T=8, weighting coefficient , , Substitute each item into the fusion formula to calculate: First item ; Second item ; Third item ; Multiplying the three values ​​yields the temporal steady-state fusion score. This value is close to 1, indicating a very high degree of confidence that the area represents a genuine disease. Simultaneously, another transiently shadowed connected region was identified, and the calculation yielded... The reliability is low, and it will be downgraded by the contour annotation module. Figure 3 As shown.

[0035] Optionally, the step of using the condition that the lesion appears continuously for no less than 5 frames within an 8-frame time window as the filtering condition for real lesions directly deletes short-term interference detection boxes that last for 1-4 frames; for lesion areas that meet the frame number threshold, the time-series steady-state fusion score is retrieved, the confidence continuous mapping annotation algorithm is run to calculate the visual confidence mapping value of the lesion boundary, and the three-level contours of solid line, dashed line, and dotted line are divided according to the mapping value and the image annotation is completed; the image recognition data package is generated by integrating six types of information, including pixel coordinates, lesion type, and fusion score, including: Set a hard filtering threshold to retain detection boxes only when the connected disease area has been continuously existing for ≥5 frames in the time-series image pool. Pixel areas with consecutive identification frames of 1, 2, 3, and 4 frames will have their corresponding detection box data directly deleted to eliminate short-term interference such as instantaneous shadows and falling sand. The mathematical expression for the confidence-based continuous mapping labeling algorithm is: in: The visual confidence mapping value for the disease boundary is given by S, which is the time-series steady-state fusion score, ranging from 0 to 1; the visualization calibration constant is fixed. , , , The mathematical expression for calibration calculation is as follows: , ,in The upper limit of confidence mapping is 1. A perfect score for fusion: 1. The lowest valid score is 0, within the entire mapping formula. Represents the boundary confidence mapping value, Represents high-resolution visual calibration coefficients, S represents the low-segment visualization calibration coefficient, and S represents the steady-state fusion score. Represents the upper limit of the mapped value, Represents the maximum score, Represents the minimum score; for the calculated Perform numerical truncation, forcibly limiting the lower limit to 0 and the upper limit to 1; after truncation, draw the outline according to the three-segment interval division rules: Use solid line outlines for dimensioning. Use dashed outlines for labeling. Use dotted line outlines for annotation, and add text labels to the disease areas corresponding to all dotted line outlines for review; Image recognition data packets are uniformly encapsulated into six types of structured data: hierarchical contour vector, disease pixel coordinates, disease classification type, total disease pixel area, temporal steady-state fusion score, and number of consecutive disease frames. Data packets are stored in descending order of temporal steady-state fusion score, with data packets scoring below 0.5 assigned to a separate storage partition. Data packets only store pixel-level identification parameters and do not include location values ​​such as road segment mileage and latitude / longitude. The original pixel calculation values ​​of the composite feature dataset and the original calculation results of the temporal steady-state fusion score are fully retained without any value discarding or simplification.

[0036] Specifically, a continuous 5-frame hard threshold filters out transient interference from a temporal persistence dimension, coupled with multi-index fusion soft scoring for dual verification, significantly reducing the false alarm rate of road defect identification. The confidence mapping formula converts the fusion score in the 0-1 interval into a visual boundary confidence level. The three-segment differentiated contours intuitively distinguish between high-confidence defects, moderately suspicious defects, and low-confidence areas requiring review, facilitating rapid defect priority identification by inspection personnel. The image recognition data package standardizes and encapsulates six types of core recognition information, eliminating redundant positioning fields, resulting in smaller data volume and compatibility with in-vehicle embedded local storage. Low-score data packages are stored in partitions for easy batch review later, with original calculated values ​​fully preserved, supporting subsequent defect identification model iteration and accuracy backtesting.

[0037] For example, during the disease filtering, contour annotation, and identification data packet generation and execution stages, the network outputs two candidate detection boxes: the first crack region has been continuously existing for 8 frames, meeting the 25-frame threshold, so the detection box is retained, and the fusion score S=0.933 is retrieved and substituted into the mapping formula for calculation: Perform numerical truncation, with an upper limit of 1, and the final mapped value. ,satisfy The disease boundary was drawn using solid lines without any verification markers; the second suspected area only existed for 3 consecutive frames, which did not reach the 5-frame threshold, so all data of the detection box was directly deleted and it was not included in the subsequent annotation calculation.

[0038] The crack area marked with solid lines integrates six types of data to generate an independent identification data package: including the set of solid line contour vectors, the (x, y) coordinates of all pixels of the disease, the disease type crack, the total area of ​​disease pixels 1864, the fusion score 0.933, and the number of consecutive frames 8; Data packets are stored in storage units in descending order of their scores. Data packets with scores higher than 0.5 are stored in a high-confidence partition, preserving all pixels in that region. The original calculated values ​​and S original calculation results contain no road mileage or geographic coordinate positioning parameters in the data package.

[0039] Optionally, the method further includes: After the image recognition data packet is stored, the data packets to be reviewed with a time-series steady-state fusion score of less than 0.5 in the independent storage partition are read periodically, and the original pavement images and composite feature original data of the corresponding time series of 8 frames are extracted in batches to generate a pavement defect review list. For the low-confidence regions marked with dot and line outlines in the review list, the temporal fluctuation enhancement features of the corresponding pixels are re-extracted. The multi-index steady-state fusion algorithm is run a second time to update the score. If the score improves to 0.5 or above after the update, the data packet is automatically migrated to a high-reliability storage partition, and the dotted line outline is simultaneously corrected to a dashed line outline.

[0040] Specifically, this post-review mechanism is used to compensate for the problem of low single-identification scores in extreme lighting and large-area water accumulation reflective scenarios. By recalculating the temporal features a second time, the credibility of the disease is re-determined, avoiding minor diseases from being mistakenly classified into the low-credibility review queue due to environmental interference. The batch review list automatically summarizes low-credibility data, eliminating the need for manual image retrieval frame by frame, thus reducing the workload of subsequent inspection and review.

[0041] For example, during the post-batch review execution phase, the device automatically scans the low-confidence independent storage partition every 30 minutes, reads 27 recognition data packets with scores below 0.5, and extracts 8 frames of original time-series images and composite feature datasets corresponding to each data packet. One data packet corresponds to a minor crack in the road surface. The initial recognition score was 0.42 due to reflection from localized water accumulation. The outline was marked as a dotted line and accompanied by text to be reviewed. All pixels in the cracked area were then re-extracted. The original values ​​are used to remove pixels that interfere with water reflection. The three types of time-series indicators are then calculated again and substituted into the fusion algorithm to update the score to 0.58, which meets the ≥0.5 standard. The system automatically migrates the data packet to a high-confidence storage partition, and simultaneously modifies the image annotation contour to a dashed line, and updates the original value of the time-series steady-state fusion score in the data packet.

[0042] Optionally, the method further includes the following steps after generating the temporally aligned pixel dataset and before constructing the two-dimensional temporally associated pixel feature matrix: Perform global illumination equalization preprocessing on each frame of the temporally aligned pixel dataset; An adaptive histogram equalization algorithm was used to unify the brightness distribution of 8 frames of images, eliminating the differences in grayscale brightness of a single frame caused by cloud cover, tree shade, and backlighting in the morning and evening. After the illumination equalization process is completed, all pixel grayscale values ​​are realigned to the 0-255 range, the corresponding pixel grayscale records in the pixel space index table are updated, and then the two-dimensional temporal pixel correlation feature matrix construction process is input.

[0043] Specifically, during daytime inspections, sunlight, shade, and clouds can cause excessive differences in brightness between adjacent frames, directly affecting the accuracy of pixel gradient and grayscale variance calculations. Adaptive histogram equalization can smooth out global illumination differences, ensuring a uniform grayscale baseline across the eight temporal images and enhancing temporal fluctuation features. Calculation accuracy.

[0044] For example, in the illumination equalization preprocessing stage, a frame in the temporally aligned pixel dataset has a large area of ​​tree shadow occlusion, resulting in a huge difference between the brightness and darkness of the image. Adaptive histogram equalization is performed simultaneously on this frame and the other 7 frames to stretch the grayscale of the dark disease pixels and compress the grayscale of the overexposed road surface. After equalization, the brightness and darkness distribution of all images tends to be consistent. Then, the grayscale of all pixels is locked in the range of 0-255 again, and the grayscale storage value of each pixel in the pixel space index table is updated synchronously. The processed dataset is then sent to the two-dimensional temporal pixel association feature matrix construction module.

[0045] Optionally, the method of continuously acquiring road surface images using a fixed frame rate image acquisition device further includes: Before the inspection vehicle starts the inspection mission, the industrial camera is calibrated to correct the radial and tangential distortion of the camera lens; After calibration, fix the camera installation height and vertical tilt angle, and do not change the camera hardware installation position throughout the inspection process; At fixed inspection intervals, a secondary camera distortion correction is automatically triggered to avoid image distortion errors caused by camera shift due to vehicle bumps.

[0046] Specifically, lens distortion can cause the pixel coordinates of road surface edges to become distorted, which can impair the accuracy of marking feature point matching. Pre-calibration and periodic secondary correction can maintain the geometric accuracy of the image in the long term and ensure the stability of homography matrix correction and temporal pixel alignment.

[0047] For example, during the camera calibration preprocessing stage, before the inspection vehicle goes on the road for inspection, the visible light industrial camera distortion calibration is completed using a standard checkerboard calibration board. The radial and tangential distortion coefficients are solved and stored in the device's non-volatile memory. The camera is fixedly installed at a height of 1.8m and a vertical downward 0° tilt angle, with the bracket screws locked throughout. Every 5km the vehicle travels, a secondary distortion correction program is automatically triggered, which calls the stored distortion coefficients to recalibrate the real-time acquired images, eliminating the slight lens shift distortion caused by long-term bumps and ensuring that the geometric shape of each frame of input image is not distorted.

[0048] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A method for real-time detection and identification of road surface defects in road inspection images, characterized in that the specific steps of the method are as follows: S1. A fixed frame rate image acquisition device is used to continuously acquire road images and build a sliding time sequence image pool that stores 8 consecutive frames of images. The speed control unit configured on the carrier is used to correct the driving speed in real time so as to control the overlapping pixel ratio of adjacent frames of the road surface to be no less than 65%. The acquired data only contains image pixel information and does not access external data such as positioning sensors. S2. Extract the feature points of fixed road markings in the image pool, remove mismatched feature points, solve the global homography matrix, and correct the pixel coordinates of all images based on the homography matrix to generate a time-aligned pixel dataset. S3. Based on the time-aligned pixel dataset, construct a two-dimensional time-series pixel association feature matrix, fill the association feature matrix with pixel gradient, edge continuity, and gray-level variance feature parameters, and use the gray-level edge coupling fluctuation algorithm to calculate the time-series feature quantization value of each pixel. S4. Input the temporal pixel association feature matrix into a dual-branch temporal convolutional network. The dual-branch temporal convolutional network is used to identify the disease area and the road interference area in parallel and generate a disease candidate detection box. Extract three types of temporal indicators for the identified disease connected pixel areas and run a multi-indicator index steady-state fusion algorithm to calculate the temporal steady-state fusion score. S5. The condition that the disease appears continuously for no less than 5 frames in the time-series image pool is used as the threshold condition for judging the real disease. Short-term interference areas are filtered out. For the disease areas that meet the judgment threshold, the time-series steady-state fusion score is retrieved. The confidence continuous mapping annotation algorithm is run. The matching contour lines are calculated according to the time-series steady-state fusion score and the image annotation is completed. The annotation information is integrated to generate an image recognition data package.

2. The method for real-time detection and identification of road surface defects in road inspection images according to claim 1, characterized in that, in step S1, the image acquisition device is a visible light industrial camera arranged vertically towards the road surface, the frame rate of the image acquisition device is between 25fps and 30fps, the resolution of the image output by the image acquisition device is 1920×1080 or 2560×1440, the carrier moves at a constant speed when the image acquisition device is working, and the adjustment range of the carrier's speed corresponds to the overlapping pixel ratio of adjacent frames of the road surface being between 65% and 78%; the sliding time-series image pool adopts a ring memory cache architecture, the sliding time-series image pool executes a first-in-first-out update logic, for each new road surface image output by the image acquisition device, the sliding time-series image pool removes the image with the earliest internal storage time, the sliding time-series image pool is divided into a pixel storage partition and a time index partition, the pixel storage partition completely retains the original pixel values ​​of the image, the time index partition records the image acquisition timestamp, the sliding time-series image pool is configured with a lossless pixel compression module, and the pixel data is restored losslessly before being transmitted to step S2.

3. The method for real-time detection and identification of road surface defects in road inspection images according to claim 1, characterized in that, in step S2, the inflection points of solid lane lines, right-angle contour points of zebra crossings, and inflection points of road directional arrow boundaries are selected as fixed road marking feature points. In road sections where road markings can be identified, the number of effective feature points extracted from a single frame image is not less than 30 sets. Bidirectional matching is performed on feature points of adjacent frames, and mismatched feature points with pixel coordinate offsets are removed. The global homography matrix is ​​obtained by solving the remaining effective matching feature points. The global homography matrix is ​​used to perform perspective transformation correction on all 8 frames of images in the time-series image pool. The correction process eliminates the horizontal and vertical pixel coordinate offsets of the image. The time-series aligned pixel dataset obtained after correction is sorted according to the road surface spatial coordinates. Non-road surface pixels at the image edges are removed. The gray values ​​in the dataset are uniformly normalized to the range of 0 to 255, and a pixel space index table recording the relative coordinates of pixels inside the image is generated.

4. The method for real-time detection and identification of road surface defects in road inspection images according to claim 3, characterized in that, in step S2, the temporally aligned pixel dataset removes irrelevant pixels from the image edges and retains only the valid pixels corresponding to the road surface of the driving lane in the middle of the image; the temporally aligned pixel dataset is configured with a pixel spatial index table, which records the horizontal and vertical coordinates of pixels inside the image, and the pixel spatial index table does not set road segment mileage parameters and geographical latitude and longitude parameters; the time-aligned pixel dataset that has completed the standardization process is packaged in a fixed file format, and the packaged dataset file serves as the only input data source for the S3 operation step.

5. A method for real-time detection and identification of road surface defects in road inspection images according to claim 1, characterized in that, in step S3, the rows of the two-dimensional temporal pixel association feature matrix correspond to independent road surface pixel coordinates, and the columns of the two-dimensional temporal pixel association feature matrix are sorted according to the image acquisition time to correspond to 8 frames of images in the sliding temporal image pool; the pixel gradient parameter is obtained by calculating the mean value of the pixel's horizontal gray-level gradient and vertical gray-level gradient respectively using the Sobel operator; the edge continuity parameter is the ratio of the number of edge pixels in the eight-neighborhood of a pixel to the total number of pixels in the eight-neighborhood; the gray-level variance parameter is the discrete variance of the gray-level values ​​of a single pixel corresponding to 8 frames of temporal gray-level values; the two-dimensional temporal pixel association feature matrix filters out solid-color road surface pixels with gradient values ​​lower than a fixed gradient threshold, and all feature values ​​in the two-dimensional temporal pixel association feature matrix are stored in single-precision floating-point format.

6. The method for real-time detection and identification of road surface defects in road inspection images according to claim 1, characterized in that, In S3, the mathematical expression for the grayscale edge coupling fluctuation algorithm is: in: This represents the temporal fluctuation enhancement feature of the p-th road surface pixel, where T is the total number of frames in the temporal window and is fixed at 8. This represents the grayscale value of pixel p in the image at frame t. This represents the average of the temporal grayscale values ​​of pixel p across 8 frames. The edge continuity percentage of pixel p in the t-th frame of the image is represented by a value ranging from 0 to 1.

7. The method for real-time detection and identification of road surface defects in road inspection images according to claim 1, characterized in that, In step S4, the data input channel of the dual-branch temporal convolutional network cuts off the data transmission paths of GPS and the inertial measurement unit. The dual-branch temporal convolutional network sets up two independent parallel computing links. The first computing link is the real disease identification link, and the second computing link is the false alarm interference identification link. The real disease identification link divides the disease identification into four categories: cracks, pits, subsidence, and crazing. The real disease identification link counts the total number of frames in which abnormal features continuously appear in the connected pixel areas of the disease. The false alarm interference identification link divides the interference identification into four categories: fallen leaves, water accumulation and reflection, scattered sand and gravel, and momentary shadows. After identifying the corresponding pixel area, the false alarm interference identification link blocks the detection box of the corresponding pixel area.

8. The method for real-time detection and identification of road surface defects in road inspection images according to claim 1, characterized in that, In S4, the mathematical expression of the multi-index exponential steady-state fusion algorithm is: Where: S represents the temporal steady-state fusion score of the disease-connected region. All pixels within the connected area of ​​the disease The average value, This is the temporal fluctuation enhancement feature for the p-th road surface pixel; For all pixels of all 8 frames in the sliding time-series image pool The global maximum value, and when season The value is set to 0 to avoid division by zero exceptions; The total number of frames within an 8-frame time window where the region edge continuity meets the standard; The total number of frames within an 8-frame time window whose regional grayscale fluctuations meet the stability standard, wherein the stability standard is that the grayscale fluctuation amplitude is lower than a fixed stability threshold. , , The exponential weighting coefficients are satisfied. .

9. The method for real-time detection and identification of road surface defects in road inspection images according to claim 1, characterized in that, In S4, the samples used in the offline training phase of the dual-branch temporal convolutional network include road surface images labeled with disease categories and temporal duration frames. The samples do not contain geolocation labels. The real disease identification link only receives pixel matrices with temporal fluctuation amplitudes lower than a fixed stable threshold and edge continuity parameters higher than a fixed complete threshold. The false alarm interference identification link only receives pixel matrices with temporal fluctuation amplitudes higher than a fixed stable threshold and edge continuity parameters lower than a fixed complete threshold.

10. A method for real-time detection and identification of road surface defects in road inspection images according to claim 1, characterized in that, In S5, the mathematical expression for the confidence-based continuous mapping annotation algorithm is: in: Here, S represents the visual confidence mapping value of the disease boundary, and S is the time-series steady-state fusion score, ranging from 0 to 1. , Fixed constants are used for visualization calibration.

Images