Road disease feature extraction and analysis method based on image recognition

Abstract

The present application relates to the field of road disease intelligent detection and image recognition, in particular to a road disease feature extraction and analysis method based on image recognition; containing image recognition, feature extraction, down-sampling compression and adaptive transmission review steps; the system obtains initial recognition results through a vehicle-mounted model, filters and extracts disease edge information based on a preliminary screening confidence threshold; the core is to calculate the image compression ratio to generate a low-resolution thumbnail according to the confidence score and the area ratio of the region of interest, and to determine a dynamic transmission strategy according to the reference signal received power evaluation network; after the cloud completes data review, the system will compare time stamps and output accurate disease levels containing geographic coordinates; the present application realizes front-end filtering and on-demand compression transmission on the vehicle side, and solves the technical problems of communication bandwidth overload and data loss in weak signal network environment.

Description

Technical Field

[0001] This invention relates to the field of intelligent detection and image recognition technology for road defects, specifically a method for extracting and analyzing road defect features based on image recognition. Background Technology

[0002] In the daily operation of road inspection, inspection vehicles continuously collect a large number of road surface images and combine vehicle-mounted recognition and remote analysis methods to detect and determine road defects such as cracks, potholes, and settlement. The efficiency of defect identification and reporting is positively correlated with the vehicle-mounted terminal's ability to filter image data, the stability of the transmission process, and the accuracy of subsequent verification results. In traditional methods, the entire road surface image collected during the inspection process is usually directly uploaded or processed uniformly. When the majority of the images are of normal road surface, the wireless network signal is below the preset communication threshold, or the complex background interference exceeds the preset feature variance, problems such as large data transmission volume, untimely feedback, unclear edge features of road defects, and low stability of recognition results are likely to occur. Therefore, the real-time performance and accuracy of road defect inspection are still relatively low. Summary of the Invention

[0003] The purpose of this invention is to provide a method for road defect feature extraction and analysis based on image recognition, addressing the following technical problems: Existing road defect inspection technologies suffer from drawbacks such as large data transmission volumes due to direct uploading of entire image frames, data feedback delays caused by network fluctuations, and lower-than-preset accuracy in defect feature extraction and recognition stability under complex background interference. There is an urgent need for an image recognition-based method for road defect feature extraction and analysis that can more accurately perform initial image screening and adaptive feature compression on vehicle-mounted devices, dynamically adjust transmission strategies based on network conditions, and combine cloud-based joint verification to improve defect detection efficiency and accuracy. This invention can achieve its objective through the following technical solutions: Methods for extracting and analyzing road defect features based on image recognition include: The road surface image and the geographical coordinates of the vehicle are obtained. The road surface image is input into the preset vehicle-mounted defect recognition model through the vehicle edge computing unit to extract image texture information and category information, and obtain an initial recognition result containing the initial recognition timestamp, suspected defect category, bounding box coordinates and confidence score. The region corresponding to the bounding box coordinates where the confidence score is not lower than the preset initial screening confidence threshold is taken as the target bounding box region. The image texture information and category information within the target bounding box region are channel-weighted to generate lesion edge information. Using the original pixel region of the target bounding box region as the region of interest, the image compression ratio is calculated based on the corresponding confidence score and the area ratio of the region of interest to the road surface image. Based on the image compression ratio, the region of interest is downsampled to generate a low-resolution thumbnail. The reference signal receiving power of the access wireless network is obtained, and a transmission strategy is determined based on the comparison result between the reference signal receiving power and the preset communication threshold. The lesion edge information and the low-resolution thumbnail are then sent to the cloud. Receive the accurate disease category and severity level returned from the cloud, match and compare the accurate disease category and severity level with the initial identification timestamp, and output result information including the geographical location coordinates and the corresponding accurate disease category and severity level.

[0004] Furthermore, the generation process of the preset vehicle defect recognition model includes: Obtain the original deep learning model and calculate the scaling factor of the batch normalization layer of each convolutional layer in the original deep learning model. Channels whose absolute value of the scaling factor is lower than the preset pruning threshold are identified as redundant channels and deleted, while channels whose absolute value of the scaling factor is greater than or equal to the preset pruning threshold are retained, thus obtaining the pruned initial model. The model parameters of the pruned initial model are quantized to generate the preset vehicle-mounted defect identification model.

[0005] Furthermore, determining the preset pruning threshold and quantizing the model parameters of the pruned initial model includes: On a preset validation set, the preset initial parameter threshold is gradually increased, and the decrease in the recall rate of the original deep learning model for road defects is calculated. When the recall rate decreases to a preset critical point, the corresponding initial parameter threshold is determined as the preset pruning threshold. The input feature values ​​and floating-point weight values ​​of the pruned initial model are converted from floating-point to fixed-point using a linear quantization mapping formula to obtain quantized model weights. The preset vehicle defect recognition model is then generated based on these quantized model weights. The linear quantization mapping formula is as follows: In the formula, These are the quantized fixed-point values. For input feature values ​​or floating-point values ​​of model weights, This is the quantization scaling factor. This is the zero-point offset; This is the floor function.

[0006] Further, channel weighting is applied to the image texture information and category information within the target bounding box region to generate lesion edge information, including: The activation values ​​of each feature channel in the image texture information are extracted within the target bounding box region, and the spatial variance of the activation values ​​is calculated. Channels with spatial variance greater than a preset variance threshold are identified as the disease edge feature channels, and channels with spatial variance less than or equal to the preset variance threshold are identified as the road surface background feature channels. A first weight value is assigned to the disease edge feature channels, and a second weight value is assigned to the road surface background feature channels, wherein the first weight value is greater than the second weight value. The image texture information and category information within the target bounding box region are weighted and calculated based on the first weight value and the second weight value to generate the lesion edge information.

[0007] Further, the image compression ratio is calculated based on the corresponding confidence score and the area ratio of the region of interest to the road surface image, including: Obtain the first pixel area of ​​the region of interest and the second pixel area of ​​the road surface image; Calculate the ratio of the area of ​​the first pixel to the area of ​​the second pixel, and use it as the area ratio; Based on a preset first proportion coefficient and a second proportion coefficient, the confidence score and the area ratio are weighted and fused to obtain the image compression ratio.

[0008] Furthermore, the process of acquiring road surface images from vehicle-mounted cameras also includes: Real-time monitoring of the frequency of defects detected in current road surface images; When the detection frequency of the disease is lower than the preset detection frequency threshold, the operating frequency of the vehicle edge computing unit is reduced. When the detection frequency of the disease is higher than or equal to the preset detection frequency threshold, the peak operating frequency of the vehicle edge computing unit is restored.

[0009] Further, a transmission strategy is determined based on a comparison between the received power of the reference signal and a preset communication threshold. The disease edge information and the low-resolution thumbnail are then sent to the cloud according to the transmission strategy, including: When the power of the reference signal reception is higher than or equal to the preset communication threshold, a first transmission strategy is determined, and the disease edge information and the low-resolution thumbnail are directly uploaded to the cloud. When the power of the reference signal reception is lower than the preset communication threshold, a second transmission strategy is determined, the lesion edge information and the low-resolution thumbnail are temporarily stored, and the corresponding temporary storage timestamp is recorded. When the received power of the reference signal is detected to be higher than or equal to the preset communication threshold again, the breakpoint resume mechanism is triggered according to the order of the temporary timestamps, and the temporary disease edge information and the low-resolution thumbnail are transmitted to the cloud.

[0010] Furthermore, the process for generating the accurate disease category and severity level includes: After receiving the disease edge information and the low-resolution thumbnail, the cloud inputs the disease edge information and the low-resolution thumbnail into a preset cloud disease verification model; The preset cloud-based disease verification model is used to analyze the disease edge information, and to perform feature extraction and background separation. Based on the analysis results after filtering out background interference and the low-resolution thumbnail, the accurate disease category and severity level are calculated.

[0011] Further, the accurate disease category and severity level are compared with the timestamp of the initial identification result locally, and the result information containing the geographical coordinates and the corresponding accurate disease category and severity level is output, including: Receive the judgment result sent from the cloud, which includes the accurate disease category, severity level, and unique timestamp from the cloud. Extract the initial identification timestamp recorded in the initial identification result locally; The unique timestamp in the cloud is matched and compared with the initial identification timestamp, and the judgment results of successful timestamp matching are selected as the target verification data of the corresponding local initial identification record. Based on the matched accurate disease category and severity level, update the local road disease status and output the result information including the geographical coordinates and the corresponding accurate disease category and severity level.

[0012] Furthermore, after outputting the result information including the aforementioned geographical coordinates and the corresponding accurate disease category and severity level, the following is also included: Receive the model parameter update file generated by the cloud after retraining based on periodically collected review and correction samples; The model parameter update file is used to replace and iteratively optimize the parameters of the local preset vehicle defect recognition model, thereby generating an updated preset vehicle defect recognition model.

[0013] Compared with the prior art, the present invention has the following beneficial effects: 1. This method filters normal road surfaces at the vehicle end by using an initial confidence threshold, and calculates the image compression ratio based on the confidence score and area ratio to generate a low-resolution thumbnail. At the same time, it dynamically switches between direct upload and temporary storage and retransmission strategies based on the reference signal reception power of the access network. This method avoids communication bandwidth overload caused by uploading full high-definition images and solves the data loss problem in weak signal network environments. 2. This method assigns different weights to the feature channels of the disease edge and the road background within the target bounding box area for weighted calculation, which enhances the disease outline and suppresses background information; the extracted disease edge information is used in the cloud for feature extraction and background separation, and then combined with low-resolution thumbnails for comprehensive calculation; this method avoids the unclear outline caused by indiscriminate equal weighting and improves the accuracy of disease verification in complex backgrounds. 3. This method compares the initial local identification timestamp with the unique timestamp sent from the cloud, and uses the successfully matched timestamp to eliminate data differences between records, ensuring the accurate correspondence between disease status and geographical location. In addition, it uses the model parameter update files sent from the cloud periodically to replace and iteratively optimize the parameters of the vehicle model, which solves the problem of decreased scene adaptability caused by the long-term fixation of the vehicle model. 4. This method utilizes the normalization layer scaling factor to eliminate redundant channels, combines the recall rate reduction magnitude to lock the pruning threshold and quantize parameters to generate a lightweight model adapted to the vehicle's computing power; simultaneously, it monitors the defect detection frequency in real time, reduces the operating frequency of the vehicle's edge computing unit when the frequency is low, and restores the peak frequency when the frequency is high; this approach achieves dynamic and reasonable allocation of hardware computing resources and effectively controls computing power consumption. Attached Figure Description

[0014] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0015] Figure 1 This is a flowchart illustrating the method for extracting and analyzing road defect features based on image recognition, as provided in an embodiment of this application. Detailed Implementation

[0016] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments.

[0017] like Figure 1 As shown, the method for extracting and analyzing road defect features based on image recognition includes: The road surface image and the geographical coordinates of the vehicle are obtained. The road surface image is input into the preset vehicle-mounted defect recognition model through the vehicle edge computing unit to extract image texture information and category information, and obtain the initial recognition result containing the initial recognition timestamp, suspected defect category, bounding box coordinates and confidence score. The region corresponding to the bounding box coordinates with a confidence score not lower than the preset initial screening confidence threshold is taken as the target bounding box region. The image texture information and category information within the target bounding box region are channel-weighted to generate disease edge information. The original pixel region of the target bounding box is taken as the region of interest. The image compression ratio is calculated based on the corresponding confidence score and the area ratio between the region of interest and the road surface image. The region of interest is downsampled based on the image compression ratio to generate a low-resolution thumbnail. The system obtains the reference signal received power of the access wireless network, determines the transmission strategy based on the comparison result between the reference signal received power and the preset communication threshold, and sends the disease edge information and low-resolution thumbnail to the cloud. Receive the accurate disease category and severity level returned from the cloud, match and compare the accurate disease category and severity level with the initial identification timestamp, and output the result information including geographical coordinates and the corresponding accurate disease category and severity level.

[0018] The generation process of the preset vehicle defect recognition model includes: Obtain the original deep learning model and calculate the scaling factor of the batch normalization layer of each convolutional layer in the original deep learning model. Channels with an absolute value of scaling factor lower than a preset pruning threshold are identified as redundant channels and deleted, while channels with an absolute value of scaling factor greater than or equal to the preset pruning threshold are retained, resulting in the pruned initial model. The initial model after pruning is quantified to generate a preset vehicle-mounted defect identification model.

[0019] The determination of the preset pruning threshold and the quantization of model parameters in the initial model after pruning include: On a preset validation set, the preset initial parameter threshold is gradually increased, and the decrease in the recall rate of the original deep learning model for road defects is calculated. When the recall rate drops to a preset critical point, the corresponding initial parameter threshold is determined as the preset pruning threshold. The input feature values ​​and floating-point weight values ​​of the pruned initial model are converted from floating-point to fixed-point using a linear quantization mapping formula to obtain quantized model weights. A pre-defined vehicle defect recognition model is then generated based on these quantized model weights. The linear quantization mapping formula is as follows: In the formula, These are the quantized fixed-point values. For input feature values ​​or floating-point values ​​of model weights, This is the quantization scaling factor; This is the zero-point offset; This is the floor function.

[0020] Channel weighting is applied to the image texture and category information to generate disease edge information, including: Extract the feature map activation values ​​of each feature channel in the image texture information within the target bounding box area, and calculate the spatial variance of the feature map activation values. Channels with spatial variance greater than a preset variance threshold are identified as disease edge feature channels, and channels with spatial variance less than or equal to the preset variance threshold are identified as road surface background feature channels. A first weight value is assigned to the defect edge feature channel; a second weight value is assigned to the road surface background feature channel, wherein the first weight value is greater than the second weight value; the image texture information and category information within the target bounding box area are weighted based on the first weight value and the second weight value to generate defect edge information; the weighting calculation process is as follows: the first weight value and the second weight value are multiplied element-wise with the activation values ​​of the feature maps in the corresponding defect edge feature channel and road surface background feature channel, respectively, to obtain the channel-weighted target feature map, which is used as defect edge information.

[0021] The image compression ratio is calculated based on the corresponding confidence score and the area ratio of the region of interest to the road surface image, including: Obtain the area of ​​the first pixel of the region of interest and the area of ​​the second pixel of the road surface image; Calculate the ratio of the area of ​​the first pixel to the area of ​​the second pixel, and use it as the area ratio; Based on the preset first and second proportion coefficients, the confidence score and area ratio are weighted and fused to obtain the image compression ratio.

[0022] In this embodiment, the inspection vehicle continuously collects road images through the vehicle-mounted camera during the road inspection process, and the vehicle-mounted terminal simultaneously records the current geographical coordinates of the vehicle. The vehicle-mounted terminal does not adopt the processing method of uploading the entire frame of high-definition image indiscriminately in the existing technology, but first completes the preliminary identification locally using a lightweight preset vehicle-mounted defect identification model, and only retains the information corresponding to the suspected defect area. For areas that pass the initial screening, the edge information of the disease is further extracted, and a low-resolution thumbnail is generated based on the confidence score and the area ratio of the region of interest to the road surface image. Then, the edge information of the disease and the low-resolution thumbnail are sent to the cloud according to the current wireless network conditions. After the cloud completes further verification, it returns the accurate disease category and severity level. The vehicle terminal then combines the original timestamp to complete the correspondence and output. Through this processing method, the output results include not only the disease category and severity level, but also their corresponding geographical coordinates, which facilitates subsequent road maintenance positioning. Furthermore, based on the above implementation method, the generation process of the preset vehicle disease identification model is explained. The original deep learning model is a convolutional neural network model containing convolutional layers and batch normalization layers. The original deep learning model is obtained, and the channel of the original deep learning model is filtered according to the scaling factor of each convolutional layer and batch normalization layer to generate the pruned initial model. The scaling factor refers to the response strength of each channel in the batch normalization layer. Its actual role in this scheme is to reflect the feature response weight of a certain channel to road disease features. When the absolute value of the scaling factor corresponding to a certain channel is lower than the preset pruning threshold, it indicates that the channel responds to the smooth road background, the area with gradually changing brightness, or the weakly distinguishable features, rather than effective disease features such as crack edges and pothole outlines. This invention uses the scaling factor of the batch normalization layer to selectively filter channels, thereby suppressing redundant background channels and retaining channels that are more sensitive to the morphology of defects. This reduces the number of parameters and floating-point operations in the preset vehicle defect recognition model, thus meeting the computing power constraints of the vehicle edge computing unit.

[0023] Furthermore, a validation set is obtained, and the initial parameter threshold is incrementally evaluated based on the decrease in recall rate to determine the preset pruning threshold. The parameters of the pruned initial model are then quantized to generate a preset vehicle-mounted defect recognition model. Here, the decrease in recall rate refers to the degree to which the original deep learning model's detection capability on road defect samples is reduced. In this scheme, its practical significance is to constrain the lightweighting of the model without sacrificing significant missed detections. The preset critical point can preferably be 2%, that is, when the threshold continues to increase and the defect recall rate decreases by 2%, the corresponding initial parameter threshold is locked as the preset pruning threshold. The pruning threshold determined in this way is an engineering parameter directly linked to the risk of missed detection of road defects. In practical implementation, when converting the initial model after pruning from floating-point to fixed-point, the above-mentioned linear quantization mapping formula is used to calculate the quantized fixed-point value. In the above formula, The actual input feature values ​​or floating-point values ​​of model weights originate from the feature response of road images or network parameters. It is used to map the current floating-point range to a fixed-point numerical range to control the correspondence between the strength of road image features and the step size of the fixed-point representation; it is calculated based on the extreme values ​​of the input feature values ​​or model weight floating-point values ​​that reflect the range of data distribution, and the upper and lower limits of the corresponding fixed-point representation. Zero offset Calculated based on the upper limit of the corresponding fixed-point representation, the maximum value of the input feature value or the floating-point value of the model weights, and the quantization scaling factor; used to ensure that features near zero values ​​can still be accurately represented, which is beneficial for preserving low-response disease textures in this scenario, its calculation formula is: in, This is the upper limit of the corresponding fixed-point representation. This represents the maximum value of the input feature value or the floating-point value of the model weights. Furthermore, the quantization scaling factor The calculation formula is: in, This refers to the maximum value of the input feature value or model weight floating-point value that reflects the current data distribution range within the training batch; This is the minimum value of the input feature value or model weight floating-point value that reflects the current data distribution range within the training batch. This is the upper limit of the corresponding fixed-point representation. As the lower limit of the corresponding fixed-point representation; the present invention coordinates the pruning threshold and the recall rate reduction magnitude constraint, and superimposes quantization processing, so that the lightweight process is constrained by both the disease detection requirements and the edge deployment conditions.

[0024] Furthermore, in the vehicle-mounted defect identification stage, road surface images captured by the vehicle-mounted camera are acquired, and the current geographical coordinates of the vehicle are simultaneously obtained. Based on the preset vehicle-mounted defect identification model, the road surface images are identified, generating an initial identification result that includes an initial identification timestamp, suspected defect category, bounding box coordinates, and confidence score. Here, the road surface images refer to road surface images collected in real time during the inspection vehicle's operation, such as images containing transverse cracks, longitudinal cracks, potholes, settlement areas, or images containing only normal asphalt texture. The initial recognition timestamp refers to the time stamp recorded when the image frame is processed by the local model, which is used to establish a correspondence with the cloud judgment results; the bounding box coordinates refer to the suspected disease location defined by the model in the image; the confidence score is the degree of confidence of the model that the bounding box belongs to the disease target; this invention first completes the local recognition on the vehicle terminal, so that normal road surface frames are filtered on the vehicle terminal, avoiding the network bandwidth being occupied by invalid images for a long time.

[0025] The initial identification results are obtained and filtered according to the preset initial screening confidence threshold to generate target bounding box regions and remove low confidence regions. The initial screening confidence threshold is used to distinguish between suspected defects and normal road surface interference, and to set the trigger conditions for data upload. As an example, the threshold can be set to 0.6; preferably, when the confidence score... When the value is greater than or equal to 0.6, the corresponding bounding box is retained as the target bounding box region; when... When the confidence level is less than 0.6, the region is directly removed; low confidence regions correspond to spurious responses caused by tree shadows, water stains, patch edges, or changes in lighting; this invention uses a preset threshold for pre-screening, so that suspected diseased regions are first quantitatively evaluated, and then it is decided whether to enter subsequent processing, thereby avoiding excessive consumption of subsequent computing and transmission resources by noisy regions.

[0026] Furthermore, the generation process of the defect edge information is explained; the target bounding box region is obtained, and the image texture information and category information within the target bounding box region are weighted according to the defect edge feature channel and the road surface background feature channel to generate defect edge information; among them, the defect edge feature channel refers to the feature channel that is more sensitive to crack edge sharpness, pothole shadow changes, and local texture abrupt changes; the road surface background feature channel refers to the feature channel that responds strongly to smooth road surface textures and areas of uniform brightness; In the specific implementation of the distinction, the spatial variance of the activation values ​​of the feature map within the target bounding box area of ​​each feature channel can be calculated; channels with spatial variance greater than a preset variance threshold are identified as disease edge feature channels with rich high-frequency information. The preset variance threshold is adaptively calculated by the maximum inter-class variance method or set based on the variance distribution statistical threshold of historical road background feature maps. Channels with spatial variance less than or equal to the preset variance threshold are identified as road surface background feature channels. The first weight value is used to enhance the contribution of the disease edge feature channel in the result, and the second weight value is used to suppress the influence of the pavement background feature channel. The first weight value is greater than the second weight value. The difference between the first weight value and the second weight value is used to enhance the disease edge features and suppress the pavement background features. This invention extracts high-response disease contour characterization information through a channel weighting mechanism, while simultaneously suppressing feature activation in smooth background regions, thereby improving the signal-to-noise ratio of disease regions in complex backgrounds.

[0027] Furthermore, the calculation method of image compression ratio is explained: the area of ​​the first pixel of the region of interest and the area of ​​the second pixel of the road image are obtained, and the compression ratio of the region of interest is calculated based on the confidence score and the area ratio to generate a low-resolution thumbnail; The region of interest (ROI) refers to the original pixel region corresponding to the target bounding box, such as the local rectangular region where a crack or pit is located; the area of ​​the first pixel is the area of ​​this local region, denoted as . The area of ​​the second pixel is the area of ​​the entire image frame, denoted as . Divide the area of ​​the first pixel by the area of ​​the second pixel to obtain the area ratio. The area ratio is used to characterize the proportion of the diseased area in the entire frame of the image. This area ratio is positively correlated with the proportion of the suspected diseased area in the current image. To determine the compression strength, the image compression ratio is calculated. The formula is: in, The confidence score indicates the degree to which the area is considered a real disease. The higher the confidence score, the greater the probability that the area is a real disease. This leads to a higher image compression ratio in the weighted fusion calculation, thus preserving more original image details for the target bounding box area during downsampling to ensure the accuracy of cloud verification. Conversely, the lower the confidence score, the lower the image compression ratio in the weighted fusion calculation, thus increasing the compression of the area and minimizing transmission bandwidth consumption in weak signal network environments. The first pre-set proportion coefficient is used to adjust its influence in compression decisions; The preset second proportion coefficient is used to adjust the degree of influence of the area proportion; and All are greater than and less than A floating-point number that satisfies , and The specific values ​​are determined by a grid search algorithm based on a preset compression preference index. The compression preference index is an objective function that jointly maximizes the recall rate of the diseased area and minimizes the number of bytes transmitted in the image. The objective function is specifically configured to assign a positive weight to the recall rate of the defective region and a negative penalty weight to the number of bytes transmitted in the image. The optimal first and second proportion coefficients that converge the objective function are obtained through multi-objective optimization. In a specific implementation scenario, the first proportion coefficient... The preferred value is 0.6 to 0.8, the second proportion coefficient. Preferably, it is 0.2 to 0.4; The image compression ratio CR is configured as a control parameter for local image downsampling processing. It is used to characterize the pixel retention coefficient or target sampling rate during the downsampling process, and its value is positively correlated with the proportion of details retained in the original image. In practical implementation, the calculated image compression ratio is obtained through a preset linear mapping function. Map the image to the target scaling range and compress the mapped image. As a control parameter for downsampling the region of interest, the resolution of the generated low-resolution thumbnail is controlled so that the thumbnail resolution is adaptively reduced to 10% to 20% of the corresponding region of the original image; This invention constructs a joint evaluation mechanism based on the confidence level of diseased areas and the area proportion, so that the compression process matches the salience of disease characteristics.

[0028] Under the conventional full-upload scheme, even if the road surface is mostly normal, the entire frame of high-definition image still needs to be continuously transmitted. When the 4G / 5G link fluctuates or multiple vehicles are connected concurrently, there is a technical defect that the normal road surface image occupies the communication bandwidth and causes the cloud data feedback to be delayed. In this implementation, local filtering is first performed by using a confidence threshold of 0.6, and then the edge information of the disease is extracted by channel weighting. The region of interest is then compressed to generate a low-resolution thumbnail, so that the uploaded content is focused on the set of verification criteria of edge information of disease + thumbnail. The above method optimizes the transmission of the entire frame image into the local transmission of the region of interest, and optimizes the indiscriminate full upload mechanism into a controlled transmission strategy based on confidence and region area ratio, so as to adapt to weak signal network environments such as tunnel entrances and remote road sections. The system obtains the reference signal received power of the current wireless network. Based on the comparison between the reference signal received power and the preset communication threshold, it uploads the edge information of the lesion and the low-resolution thumbnail to generate data for cloud verification. The reference signal received power is a signal strength indicator measured when the vehicle communication module accesses a 4G or 5G network, which is used to reflect whether the current network upload conditions are suitable for real-time transmission. The preset communication threshold is used to distinguish whether the current link is suitable for real-time upload. The preset communication threshold is a reference signal power parameter mapped according to the minimum basic transmission rate required for cloud verification. This invention incorporates network status assessment into the upload decision, ensuring that the data to be uploaded matches the link's carrying capacity, thereby reducing the risk of upload failure or accumulated delays in weak signal network environments.

[0029] In this embodiment, the process of acquiring road surface images captured by the vehicle-mounted camera also includes: Real-time monitoring of the frequency of defects detected in current road surface images; When the detection frequency of defects is lower than the preset detection frequency threshold, the operating frequency of the on-board edge computing unit is reduced. When the detection frequency of defects is higher than or equal to the preset detection frequency threshold, the peak operating frequency of the vehicle edge computing unit is restored.

[0030] Based on the comparison between the received power of the reference signal and the preset communication threshold, a transmission strategy is determined. Following this strategy, the lesion edge information and low-resolution thumbnails are sent to the cloud, including: When the power of the reference signal reception is higher than or equal to the preset communication threshold, the first transmission strategy is determined, and the disease edge information and low-resolution thumbnail are directly uploaded to the cloud. When the power of the reference signal reception is lower than the preset communication threshold, a second transmission strategy is determined, the edge information of the lesion and the low-resolution thumbnail are temporarily stored, and the corresponding temporary storage timestamp is recorded. When the received power of the reference signal is detected to be higher than or equal to the preset communication threshold again, the breakpoint resume mechanism is triggered in the order of the temporary timestamps to transmit the temporary disease edge information and low-resolution thumbnail to the cloud.

[0031] The process of generating accurate disease categories and severity levels includes: After receiving the disease edge information and low-resolution thumbnail in the cloud, the cloud inputs the disease edge information and low-resolution thumbnail into the preset cloud disease verification model; The pre-set cloud-based disease verification model is used to analyze the edge information of diseases, and to perform feature extraction and background separation. Based on the analysis results after filtering out background interference and the low-resolution thumbnail, the accurate disease category and severity level are calculated comprehensively.

[0032] Furthermore, based on the above implementation method, the power consumption control of the vehicle-mounted terminal, the network transmission strategy, and the cloud verification process are explained; the defect detection frequency of the current road image is obtained, and the working frequency of the vehicle-mounted edge computing unit is adjusted according to the preset detection frequency threshold to determine the current power consumption control state; wherein, the defect detection frequency is used to characterize the current computing load state of the vehicle-mounted edge computing unit; in specific implementation, when the vehicle is running on a newly built smooth road surface and the defect detection frequency is low, maintaining the peak working frequency will generate redundant hardware computing power overhead; The preset detection frequency threshold can be set according to historical statistical data of the actual road section, for example, it can be set to detect 3 defects per minute. When the defect detection frequency is lower than the preset threshold, the working frequency of the vehicle edge computing unit that performs the recognition is reduced. Conversely, when the vehicle enters a road section with dense defects and the defect detection frequency increases and reaches or exceeds the threshold, the peak working frequency of the vehicle edge computing unit is restored to ensure that the current image processing does not experience significant backlog. This invention adaptively adjusts the detection frequency according to the defect, so as to reduce power consumption under low load and ensure recognition capability under high load.

[0033] Obtain the reference signal received power of the current wireless network, and determine the first transmission strategy or the second transmission strategy based on the comparison result between the reference signal received power and the preset communication threshold, and generate the transmission path of the lesion edge information and the low-resolution thumbnail. The first transmission strategy refers to directly uploading the disease edge information and low-resolution thumbnail to the cloud when the network signal meets the upload conditions; the second transmission strategy refers to temporarily storing the disease edge information and low-resolution thumbnail when the network signal is lower than the upload conditions, and recording the corresponding temporary storage timestamp; the actual function of the temporary storage timestamp is to maintain the order of different disease data packets and avoid the upload order disorder after the signal is restored. In practical implementation, when the reference signal received power is higher than or equal to the preset communication threshold, it indicates that the current link can meet the conditions for immediate upload, and the first transmission strategy is determined at this time; when the reference signal received power is lower than the preset communication threshold, it indicates that the vehicle may be in a tunnel, a remote blind spot, or a severely obstructed area. If upload is still forced at this time, packet loss, long waiting time, or repeated submission may occur. Therefore, the second transmission strategy is determined, and the data is temporarily stored in the local cache queue and the timestamp is recorded. When the received power of the reference signal is subsequently detected to be higher than or equal to the preset communication threshold, the breakpoint resume mechanism is triggered according to the order of the temporary timestamps, and the temporary data is transmitted to the cloud in sequence. This invention combines link status judgment, temporary storage management and time sequence recovery, so that the fault data is not lost in the weak signal network environment and can be sent up in the original order after the signal is restored.

[0034] The system acquires disease edge information and low-resolution thumbnails received from the cloud, analyzes the disease edge information and low-resolution thumbnails based on the preset cloud disease verification model, and generates accurate disease categories and severity levels. Among them, the cloud-based disease verification model refers to a high-precision disease verification model deployed in the data center, which is used to re-evaluate the suspected disease data uploaded by the vehicle terminal. In specific implementation, the cloud-based disease verification model can adopt a deep convolutional network based on residual network or visual Transformer architecture. Its training set contains a large number of historical high-definition road images and corresponding manually accurately annotated disease labels. Feature extraction and background separation refer to removing background factors that interfere with disease identification, such as tree shadows, water stains, traces of repaired grout, and lighting contrast, in order to highlight the content that is truly related to the type and severity of the disease. In practice, the cloud receives disease edge information and low-resolution thumbnails, and inputs both into the cloud-based disease verification model. Disease edge information focuses more on representing the disease edge contour and texture abrupt changes, while the low-resolution thumbnail preserves the overall appearance and structural relationships of the local area. The cloud-based disease verification model first analyzes the disease edge information, filters background interference, and then combines the filtered analysis results with a low-resolution thumbnail to obtain an accurate disease category and severity level. The comprehensive calculation process specifically includes: fusing category identification features in the disease edge information with regional appearance features in the low-resolution thumbnail to improve the stability of the verification judgment results. This invention uses both disease edge information and thumbnails as verification criteria, reducing the impact of background noise and making disease judgment more stable. This invention embodiment uses a cloud-based disease verification model to jointly analyze disease edge information and low-resolution thumbnails, avoiding misjudgments caused by a single basis, thereby overcoming the key technical defect in the prior art that the lack of targeted verification basis makes it difficult to eliminate false alarms in complex scenarios. In road inspection scenarios, common extreme conditions include tunnel passage, weak signal network environment in mountainous areas, concurrent uploads by multiple vehicles, and interference from repair marks on old road surfaces. Traditional solutions typically exhibit two types of problems in these scenarios: one is the indiscriminate forced real-time upload on the network side, which leads to data accumulation or even loss in weak signal network environments; the other is the independent processing on the recognition side, which relies only on the whole image or a single feature, and is prone to mistaking crack sealant or water stain edges as cracks. In this embodiment, the vehicle-mounted terminal adjusts its operating frequency by detecting the frequency and selects whether to upload directly or temporarily store and resume the transmission based on the reference signal receiving power; the cloud uses the edge information of the lesion in conjunction with a low-resolution thumbnail for verification; this controls the power consumption of the vehicle-mounted terminal and reduces the interference of complex backgrounds on the final lesion determination. In this embodiment, the accurate disease category and severity level are compared with the timestamp of the initial local identification result, and the result information including geographical location coordinates and the corresponding accurate disease category and severity level is output, including: Receive the judgment results sent from the cloud, which include the accurate disease type, severity level, and a unique timestamp from the cloud. Extract the initial recognition timestamp recorded in the local initial recognition results; The unique timestamp in the cloud is matched and compared with the initial identification timestamp, and the results of successful timestamp matching are selected as the target verification data for the corresponding local initial identification record. Based on the matched accurate disease category and severity level, update the local road disease status and output the result information including geographical coordinates and the corresponding accurate disease category and severity level.

[0035] After outputting results including geographic coordinates and corresponding accurate disease categories and severity levels, the output also includes: Receive the model parameter update file generated by the cloud after retraining based on periodically collected review and correction samples; The model parameter update file is used to replace and iteratively optimize the parameters of the local preset vehicle defect recognition model, and generate an updated preset vehicle defect recognition model.

[0036] Furthermore, based on the above implementation method, the result correspondence between the vehicle terminal and the cloud and the subsequent model update process are explained; the judgment result issued by the cloud is obtained, and the accurate disease category and severity level are corresponded according to the matching relationship between the unique timestamp of the cloud and the initial identification timestamp of the local location, and the result information including geographical location coordinates is output. Among them, the cloud-based unique timestamp refers to the unique time identifier generated by the cloud for a specific review result; the initial identification timestamp refers to the time identifier recorded by the vehicle-mounted terminal during the local initial identification stage; the actual role of the two in this solution is to accurately match the suspected defect found locally with the precise judgment result returned by the cloud review, avoiding data confusion in the case of continuous collection of the same road segment, multiple frames of similar images, or network latency; in specific implementation, the vehicle-mounted terminal receives the judgment result sent by the cloud, which includes the accurate defect category, severity level, and cloud-based unique timestamp, and extracts the initial identification timestamp recorded in the local initial identification result; The invention compares the unique timestamp in the cloud with the initial identification timestamp. For records that match successfully, the timestamp is used to eliminate the data difference between the local record and the cloud record. Based on the accurate disease category and severity level after matching, the local road disease status is updated. Finally, the result information containing the geographical coordinates and the corresponding accurate disease category and severity level is output. This invention establishes a one-to-one correspondence through timestamp matching, avoiding result mismatch when there is slow transmission, multi-packet continuation, or consecutive occurrence of neighboring diseases in weak signal network environments.

[0037] Obtain the model parameter update file generated by the cloud after retraining based on the reviewed and corrected samples. Based on the model parameter update file, perform parameter replacement and iterative optimization on the local preset vehicle defect recognition model to generate the updated preset vehicle defect recognition model. Among them, the verification and correction samples refer to the difficult samples collected in the cloud during continuous operation and after fine judgment and correction, such as the crack filling glue marks that are easy to misjudge at the edge, complex shadow areas, and low contrast crack areas; the model parameter update file is used to send the parameter changes obtained by retraining in the cloud to the vehicle terminal, so that the local model can gradually adapt to the subsequent road environment. This invention retrains the model based on continuously accumulated verification and correction samples in the cloud and sends the updated files back to the vehicle, enabling the local model to gradually improve its ability to recognize complex road conditions and complex interference.

[0038] The above method is deployed in the vehicle-mounted camera, vehicle-mounted edge computing unit, vehicle-mounted communication module, and cloud data center of the inspection vehicle. When the vehicle is in motion, the vehicle-mounted camera collects road images, and the vehicle-mounted edge computing unit uses the updated preset vehicle-mounted defect recognition model to complete local recognition and initial defect screening. When a suspected defect with a confidence level reaching the threshold is identified, defect edge information and a low-resolution thumbnail are generated, and the vehicle-mounted communication module selects whether to upload directly or temporarily store and continue uploading based on the current reference signal reception power. After receiving the data, the cloud uses the cloud-based disease review model to output the accurate disease category and severity level, and then sends the results, along with the cloud's unique timestamp, back to the vehicle terminal. After the vehicle terminal completes the timestamp matching, it updates the disease status on the display interface and outputs the result information, which includes the geographical coordinates and the corresponding accurate disease category and severity level. In subsequent operation, the cloud can also retrain and issue model parameter update files based on the periodically collected review and correction samples, and the vehicle terminal can use this to complete the local model replacement, thereby adapting to the new road image data. This embodiment enables a continuous processing flow from image acquisition, local initial screening, network upload, cloud verification, result feedback, to model update in a real road inspection system.

[0039] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for extracting and analyzing road defect features based on image recognition, characterized in that, include: The road surface image and the geographical coordinates of the vehicle are obtained. The road surface image is input into the preset vehicle-mounted defect recognition model through the vehicle edge computing unit to extract image texture information and category information, and obtain an initial recognition result containing the initial recognition timestamp, suspected defect category, bounding box coordinates and confidence score. The region corresponding to the bounding box coordinates where the confidence score is not lower than the preset initial screening confidence threshold is taken as the target bounding box region. The image texture information and category information within the target bounding box region are channel-weighted to generate lesion edge information. Using the original pixel region of the target bounding box region as the region of interest, the image compression ratio is calculated based on the corresponding confidence score and the area ratio of the region of interest to the road surface image. Based on the image compression ratio, the region of interest is downsampled to generate a low-resolution thumbnail. The reference signal receiving power of the access wireless network is obtained, and a transmission strategy is determined based on the comparison result between the reference signal receiving power and the preset communication threshold. The lesion edge information and the low-resolution thumbnail are then sent to the cloud. Receive the accurate disease category and severity level returned from the cloud, match and compare the accurate disease category and severity level with the initial identification timestamp, and output result information including the geographical location coordinates and the corresponding accurate disease category and severity level.

2. The method for extracting and analyzing road defect features based on image recognition according to claim 1, characterized in that, The generation process of the preset vehicle defect identification model includes: Obtain the original deep learning model and calculate the scaling factor of the batch normalization layer of each convolutional layer in the original deep learning model. Channels whose absolute value of the scaling factor is lower than the preset pruning threshold are identified as redundant channels and deleted, while channels whose absolute value of the scaling factor is greater than or equal to the preset pruning threshold are retained, thus obtaining the pruned initial model. The model parameters of the pruned initial model are quantized to generate the preset vehicle-mounted defect identification model.

3. The method for extracting and analyzing road defect features based on image recognition according to claim 2, characterized in that, The determination of the preset pruning threshold and the quantization of model parameters of the pruned initial model include: On a preset validation set, the preset initial parameter threshold is gradually increased, and the decrease in the recall rate of the original deep learning model for road defects is calculated. When the recall rate decreases to a preset critical point, the corresponding initial parameter threshold is determined as the preset pruning threshold. The input feature values ​​and floating-point weight values ​​of the pruned initial model are converted from floating-point to fixed-point using a linear quantization mapping formula to obtain quantized model weights. Based on these quantized model weights, the preset vehicle defect recognition model is generated. The linear quantization mapping formula is: In the formula, These are the quantized fixed-point values. For input feature values ​​or floating-point values ​​of model weights, This is the quantization scaling factor. This is the zero-point offset; This is the floor function.

4. The method for extracting and analyzing road defect features based on image recognition according to claim 1, characterized in that, Channel-weighted processing is applied to the image texture and category information within the target bounding box region to generate lesion edge information, including: The activation values ​​of each feature channel in the image texture information are extracted within the target bounding box region, and the spatial variance of the activation values ​​is calculated. Channels with spatial variance greater than a preset variance threshold are identified as the disease edge feature channels, and channels with spatial variance less than or equal to the preset variance threshold are identified as the road surface background feature channels. A first weight value is assigned to the disease edge feature channels, and a second weight value is assigned to the road surface background feature channels, wherein the first weight value is greater than the second weight value. The image texture information and category information within the target bounding box region are weighted and calculated based on the first weight value and the second weight value to generate the lesion edge information.

5. The method for extracting and analyzing road defect features based on image recognition according to claim 1, characterized in that, The image compression ratio is calculated based on the corresponding confidence score and the area ratio of the region of interest to the road surface image, including: Obtain the first pixel area of ​​the region of interest and the second pixel area of ​​the road surface image; Calculate the ratio of the area of ​​the first pixel to the area of ​​the second pixel, and use it as the area ratio; Based on a preset first proportion coefficient and a second proportion coefficient, the confidence score and the area ratio are weighted and fused to obtain the image compression ratio.

6. The method for extracting and analyzing road defect features based on image recognition according to claim 1, characterized in that, The process of acquiring road images captured by vehicle-mounted cameras also includes: Real-time monitoring of the frequency of defects detected in current road surface images; When the detection frequency of the disease is lower than the preset detection frequency threshold, the operating frequency of the vehicle edge computing unit is reduced. When the detection frequency of the disease is higher than or equal to the preset detection frequency threshold, the peak operating frequency of the vehicle edge computing unit is restored.

7. The method for extracting and analyzing road defect features based on image recognition according to claim 1, characterized in that, A transmission strategy is determined based on a comparison between the received power of the reference signal and a preset communication threshold. The disease edge information and the low-resolution thumbnail are then sent to the cloud according to the transmission strategy, including: When the power of the reference signal reception is higher than or equal to the preset communication threshold, a first transmission strategy is determined, and the disease edge information and the low-resolution thumbnail are directly uploaded to the cloud. When the power of the reference signal reception is lower than the preset communication threshold, a second transmission strategy is determined, the lesion edge information and the low-resolution thumbnail are temporarily stored, and the corresponding temporary storage timestamp is recorded. When the received power of the reference signal is detected to be higher than or equal to the preset communication threshold again, the breakpoint resume mechanism is triggered according to the order of the temporary timestamps, and the temporary disease edge information and the low-resolution thumbnail are transmitted to the cloud.

8. The method for extracting and analyzing road defect features based on image recognition according to claim 1, characterized in that, The process of generating the accurate disease category and severity level includes: After receiving the disease edge information and the low-resolution thumbnail, the cloud inputs the disease edge information and the low-resolution thumbnail into a preset cloud disease verification model; The preset cloud-based disease verification model is used to analyze the disease edge information, and to perform feature extraction and background separation. Based on the analysis results after filtering out background interference and the low-resolution thumbnail, the accurate disease category and severity level are calculated.

9. The method for extracting and analyzing road defect features based on image recognition according to claim 1, characterized in that, The accurate disease category and severity level are compared with the initial identification result in the local area using a timestamp comparison. The result information, including the geographical coordinates and the corresponding accurate disease category and severity level, is output, including: Receive the judgment result sent from the cloud, which includes the accurate disease category, severity level, and unique timestamp from the cloud. Extract the initial identification timestamp recorded in the initial identification result locally; The unique timestamp in the cloud is matched and compared with the initial identification timestamp, and the judgment results of successful timestamp matching are selected as the target verification data of the corresponding local initial identification record. Based on the matched accurate disease category and severity level, update the local road disease status and output the result information including the geographical coordinates and the corresponding accurate disease category and severity level.

10. The method for extracting and analyzing road defect features based on image recognition according to claim 1, characterized in that, After outputting the results containing the aforementioned geographic coordinates and the corresponding accurate disease category and severity level, the following is also included: Receive the model parameter update file generated by the cloud after retraining based on periodically collected review and correction samples; The model parameter update file is used to replace and iteratively optimize the parameters of the local preset vehicle defect recognition model, thereby generating an updated preset vehicle defect recognition model.

Images