A machine vision-assisted intelligent road inspection method and system

Abstract

The present invention discloses a machine vision-assisted intelligent road inspection method and system, the method comprising: acquiring road surface condition video image data and vehicle position data; performing road surface condition detection operation and road condition classification operation according to the road surface condition video image data to obtain road condition level data; performing road sign detection operation and lane classification detection operation according to the road surface condition video image data to obtain lane classification data; performing road environment detection operation and traffic diversion classification operation according to the vehicle position data to obtain diversion classification data; performing clustering operation according to the vehicle position data to obtain vehicle clustering data; performing confidentiality identification operation according to the vehicle clustering data to obtain position detection data; performing inspection level judgment operation according to the road condition level data, lane classification data, diversion classification data and position detection data to obtain inspection level data. The method can improve the accuracy of road analysis.

Description

Technical Field

[0001] The present invention relates to the field of machine vision technology, and in particular to a machine vision-assisted intelligent road inspection method and system. Background Art

[0002] At present, with the continuous development of economy and the rapid progress of transportation, the traffic road network is becoming more and more complex, and the maintenance and inspection of roads are becoming increasingly difficult. Behind this phenomenon is the rapid advancement of urbanization and the imbalance of regional development. Urbanization not only promotes the expansion of the transportation network, but also brings about problems such as traffic congestion, three-dimensional traffic and low utilization efficiency of some roads. The study of complex network characteristics and robustness of urban road networks shows that the urban transportation system is a typical large system with openness and complexity composed of road systems, traffic systems and management systems, and the intricate road network is an important component of the urban transportation system. At present, road inspection work is mainly carried out by manual inspection, which relies on the experience and subjective judgment of road inspectors and can no longer meet the needs of modern transportation networks.

[0003] In one existing technology, the image data and timestamps obtained from the inspection are sent to a distributed processing system to realize the continuity of the data flow, and a time-continuous image data stream of vehicles, road surfaces, and signs is obtained. The vehicle's driving speed and mileage are extracted based on the obtained image data stream, and the driving speed is continuously processed to generate a speed change curve, and the speed change curve is compared with a set reference curve. The comparison result is combined with the longitude and latitude coordinates obtained by GPS and displayed in a traffic condition distribution map. An alarm is issued for abnormal conditions, which can accurately and effectively provide highway traffic operation information, and can timely warn of road condition information, providing technical support for maintaining highway traffic conditions to achieve unified command, joint management, and mutual cooperation.

[0004] The existing technology can only rely on the human eye to make judgments, which is subjective and biased, and the labor intensity is too high. At the same time, no specific system architecture and implementation method are proposed, road signs are not detected, and all-round intelligent detection of road conditions cannot be achieved. Summary of the invention

[0005] The present invention provides a machine vision-assisted intelligent road inspection method and system to improve the accuracy of road analysis.

[0006] In a first aspect, in order to solve the above technical problems, the present invention provides a machine vision-assisted intelligent road inspection method, comprising:

[0007] Acquire road condition video image data and vehicle position data;

[0008] Performing a road condition detection operation according to the road condition video image data to obtain road condition data;

[0009] Performing a road condition classification operation according to the road condition data to obtain road condition level data;

[0010] Performing a road sign detection operation according to the road condition video image data to obtain lane sign data;

[0011] Performing a lane classification detection operation according to the lane identification data to obtain lane classification data;

[0012] Performing a road environment detection operation according to the vehicle position data to obtain road environment data;

[0013] Performing traffic diversion classification operations according to the road environment data to obtain diversion classification data;

[0014] Performing a clustering operation according to the vehicle position data to obtain vehicle clustering data;

[0015] Performing a confidentiality identification operation based on the vehicle clustering data to obtain position detection data;

[0016] An inspection level judgment operation is performed based on the road condition level data, the lane classification data, the diversion classification data and the position detection data to obtain inspection level data.

[0017] In an optional implementation, performing a road condition detection operation according to the road condition video image data to obtain road condition data includes:

[0018] Performing a grayscale operation on the road condition video image data to obtain a grayscale image;

[0019] Performing edge extraction operation according to the grayscale image to obtain road surface boundary data;

[0020] Performing a histogram equalization operation according to the road boundary data to obtain equalized image data;

[0021] Performing a fuzzy enhancement operation according to the equalized image data to obtain enhanced image data;

[0022] Performing a numerical normalization operation on the enhanced image data to obtain normalized image data;

[0023] Performing a binarization operation on the normalized image data to obtain binary image data;

[0024] Performing an image deconvolution operation according to the binary image data to obtain restored image data;

[0025] A blur degree calculation operation is performed according to the restored image data to obtain road condition data.

[0026] In an optional implementation, performing a blur degree calculation operation according to the restored image data to obtain road condition data includes:

[0027] The calculation formula of the road condition data is as follows:

[0028]

[0029] in, Indicates the restored image pixels, Indicates the road condition data of the restored image, represents the variance of the restored image, represents the mean of the restored image, is the total number of pixels.

[0030] In an optional implementation, performing a road sign detection operation according to the road condition video image data to obtain lane sign data includes:

[0031] Performing image segmentation operations according to the road condition video image data to obtain lane line image data and lane marking image data;

[0032] Performing a lane line extraction operation according to the lane line image data and the lane marking image data to obtain lane line feature data;

[0033] Performing a lane marking detection operation according to the lane line feature data to obtain lane marking position information;

[0034] A lane marking information recognition operation is performed according to the lane marking line position information to obtain lane marking data.

[0035] In an optional implementation, performing a lane classification detection operation according to the lane identification data to obtain lane classification data includes:

[0036] Performing an area calculation operation according to the lane marking data to obtain damaged area data;

[0037] Performing an automatic completion operation according to the lane identification data to obtain completed identification data;

[0038] Performing an area calculation operation according to the completed identification data to obtain completed area data;

[0039] Performing a damage degree calculation operation according to the damaged area data and the completed area data to obtain lane classification data;

[0040] The calculation formula of the lane classification data is as follows:

[0041]

[0042] in, represents lane classification data, Indicates the damaged area data, Indicates the completed area data.

[0043] In an optional implementation, performing a road environment detection operation according to the vehicle position data to obtain road environment data includes:

[0044] Performing a vehicle flow information detection operation according to the vehicle position data to obtain vehicle flow data;

[0045] performing an exhaust gas information estimation operation according to the vehicle position data to obtain exhaust gas data;

[0046] Performing a vehicle flow density information detection operation according to the vehicle position data to obtain vehicle flow density data;

[0047] According to the vehicle flow data, exhaust gas data and vehicle flow density data, a data packaging operation is performed to obtain road environment data.

[0048] In an optional implementation, performing a clustering operation according to the vehicle position data to obtain vehicle clustering data includes:

[0049] Performing a clustering iterative update operation according to the vehicle position data to obtain vehicle clustering data;

[0050] The clustering iterative update operation is as follows:

[0051] According to the vehicle position data, a cluster center is randomly selected to obtain an initial cluster center;

[0052] Performing a centroid distance calculation operation according to the initial cluster center and the vehicle position data to obtain centroid distance data;

[0053] Performing data partitioning operation according to the centroid distance data to obtain a clustering partitioning result;

[0054] According to the clustering division result, a centroid update operation is performed to obtain an updated cluster center;

[0055] When it is determined that the updated cluster center no longer changes, the iteration is terminated to obtain vehicle clustering data.

[0056] In an optional implementation, performing a confidentiality identification operation based on the vehicle clustering data to obtain position detection data includes:

[0057] According to the vehicle clustering data, an encrypted area distance calculation operation is performed to obtain an encrypted area distance;

[0058] When it is determined that the encrypted area distance data is less than a preset minimum threshold, an encryption operation is performed to obtain position detection data.

[0059] In an optional implementation, performing an inspection level determination operation according to the road condition level data, the lane classification data, the guidance classification data and the position detection data to obtain the inspection level data includes:

[0060] Performing a main cause analysis operation according to the road condition level data to obtain main cause level data;

[0061] Performing a secondary factor analysis operation based on the grooming classification data and the position detection data to obtain secondary factor level data;

[0062] A weighted sum operation is performed according to the primary cause level data and the secondary cause level data to obtain inspection level data.

[0063] In a second aspect, the present invention provides a machine vision-assisted intelligent road inspection system, comprising:

[0064] An input module, used to obtain road condition video image data and vehicle position data;

[0065] A road condition calculation module, used to perform a road condition detection operation according to the road condition video image data to obtain road condition data;

[0066] A road condition level calculation module, used to perform a road condition classification operation according to the road condition data to obtain road condition level data;

[0067] A lane marking recognition module, used to perform a road marking detection operation according to the road condition video image data to obtain lane marking data;

[0068] A lane classification calculation module, used to perform a lane classification detection operation according to the lane identification data to obtain lane classification data;

[0069] A road environment recognition module, used to perform a road environment detection operation according to the vehicle position data to obtain road environment data;

[0070] A traffic diversion classification calculation module is used to perform traffic diversion classification operations according to the road environment data to obtain traffic diversion classification data;

[0071] A vehicle clustering module, used to perform a clustering operation based on the vehicle position data to obtain vehicle clustering data;

[0072] A position encryption module, used to perform a confidentiality identification operation based on the vehicle clustering data to obtain position detection data;

[0073] The inspection level calculation module is used to perform an inspection level judgment operation according to the road condition level data, the lane classification data, the diversion classification data and the position detection data to obtain the inspection level data.

[0074] Compared with the prior art, the present invention has the following beneficial effects:

[0075] The present invention provides a machine vision-assisted intelligent road inspection method, which is characterized by comprising:

[0076] Obtain road condition video image data and vehicle position data; perform road condition detection operation according to the road condition video image data to obtain road condition data; perform road condition classification operation according to the road condition data to obtain road condition level data; perform road sign detection operation according to the road condition video image data to obtain lane sign data; perform lane classification detection operation according to the lane sign data to obtain lane classification data; perform road environment detection operation according to the vehicle position data to obtain road environment data; perform traffic diversion classification operation according to the road environment data to obtain diversion classification data; perform clustering operation according to the vehicle position data to obtain vehicle clustering data; perform confidentiality identification operation according to the vehicle clustering data to obtain position detection data; perform inspection level judgment operation according to the road condition level data, the lane classification data, the diversion classification data and the position detection data to obtain inspection level data. The present invention obtains comprehensive analysis results by machine vision-assisted analysis of multiple data types, which can improve the accuracy of road analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] Figure 1 It is a flowchart of a machine vision-assisted intelligent road inspection method provided by an embodiment of the present invention;

[0078] Figure 2 It is a structural diagram of a machine vision-assisted road intelligent inspection system provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0079] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0080] Reference Figure 1The embodiment of the present invention provides a machine vision-assisted intelligent road inspection method, comprising the following steps:

[0081] S11, acquiring road condition video image data and vehicle position data;

[0082] S12, performing a road condition detection operation according to the road condition video image data to obtain road condition data;

[0083] S13, performing a road condition classification operation according to the road condition data to obtain road condition level data;

[0084] S14, performing a road sign detection operation according to the road condition video image data to obtain lane sign data;

[0085] S15, performing a lane classification detection operation according to the lane identification data to obtain lane classification data;

[0086] S16, performing a road environment detection operation according to the vehicle position data to obtain road environment data;

[0087] S17, performing traffic diversion classification operations according to the road environment data to obtain diversion classification data;

[0088] S18, performing a clustering operation according to the vehicle position data to obtain vehicle clustering data;

[0089] S19, performing a confidentiality identification operation according to the vehicle clustering data to obtain position detection data;

[0090] S110, performing an inspection level determination operation according to the road condition level data, the lane classification data, the traffic diversion classification data and the position detection data to obtain inspection level data.

[0091] In step S11, road condition video image data and vehicle position data are acquired.

[0092] It should be noted that in step S11, the road surface condition video image data and the vehicle position data are two key features. The road surface condition video image data is obtained by cameras installed at the front and rear of the vehicle. These cameras have high shock resistance and can stably capture the road surface conditions, road signs, and surrounding environment images in front of and behind the vehicle during the vehicle's driving process. Exemplarily, the camera model used in the present invention is MV-SUF1200-U3M, which transmits the acquired data and video information to the main control module via a USB cable. Of course, according to user needs and different application scenarios, other models of cameras can also be selected, and the present invention does not limit this. The operation of this process depends on the image capture technology of the camera and the stability of the USB connection to ensure the real-time and accuracy of the data. The vehicle position data involves the longitude, latitude and altitude information of the vehicle. Exemplarily, the present invention uses a 4G module to transmit the information to the main system to achieve accurate positioning of the vehicle position. Of course, according to user needs and different application scenarios, other positioning methods can also be selected, and the present invention does not limit this. The acquisition of this feature depends on the communication module, which is installed on a fixed mounting plate. Exemplarily, the present invention uses an M51 communication module to interact with the processor through an RS485 bus to achieve data communication. Of course, according to different user needs and application scenarios, other types of communication modules can also be selected, and the present invention does not limit this. The processor plays a core role in this process. It is responsible for image analysis of image data obtained by the camera, coordinate analysis of vehicle position data, and classification of road conditions. Exemplarily, the processor model used in the present invention is IMX6UL-1GN2N, which works in conjunction with the memory K9D9G08U1M, and the memory is used to store analysis results and raw data. Of course, according to different user needs and application scenarios, other types of processors can also be selected, and the present invention does not limit this.

[0093] In step S12, a road condition detection operation is performed according to the road condition video image data to obtain road condition data, including:

[0094] Performing a grayscale operation on the road condition video image data to obtain a grayscale image;

[0095] Performing edge extraction operation according to the grayscale image to obtain road surface boundary data;

[0096] Performing a histogram equalization operation according to the road boundary data to obtain equalized image data;

[0097] Performing a fuzzy enhancement operation according to the equalized image data to obtain enhanced image data;

[0098] Performing a numerical normalization operation on the enhanced image data to obtain normalized image data;

[0099] Performing a binarization operation on the normalized image data to obtain binary image data;

[0100] Performing an image deconvolution operation according to the binary image data to obtain restored image data;

[0101] A blur degree calculation operation is performed according to the restored image data to obtain road condition data.

[0102] It is worth noting that, according to the restored image data, a blur degree calculation operation is performed to obtain road condition data, including:

[0103] The calculation formula of the road condition data is as follows:

[0104]

[0105] in, Indicates the restored image pixels, Indicates the road condition data of the restored image, represents the variance of the restored image, represents the mean of the restored image, is the total number of pixels.

[0106] It should be noted that in step S12, the road condition video image data undergoes a series of image processing operations to obtain road condition data. First, the road condition video image data is converted into a grayscale image. This operation converts the color image into a grayscale value through grayscale processing, simplifies the image data and retains important brightness information. Next, the grayscale image undergoes an edge extraction operation. This step obtains road boundary data by identifying areas in the image where the grayscale value changes significantly. These boundary data are crucial for subsequent road condition analysis. The road boundary data then undergoes a histogram equalization operation to obtain balanced image data. Histogram equalization is an image enhancement technology used to improve the contrast of an image, making the brightness distribution in the image more uniform, thereby improving the visibility of road features. The balanced image data then undergoes a fuzzy enhancement operation to obtain enhanced image data. This step is intended to improve the clarity of the image and make the subtle features of the road more obvious. The enhanced image data is then subjected to a numerical normalization operation to obtain normalized image data. Normalization is to scale the pixel values ​​of the image data to a specific range. Exemplarily, the present invention uses [0,1], which helps to eliminate the differences between different images and provide consistent input for the subsequent binarization operation. Of course, other normalization ranges can also be selected according to user needs and application scenarios, and the present invention does not limit this. The normalized image data is subjected to a binarization operation to obtain binary image data. Binarization is the process of converting an image into a process containing only two pixel values, black and white. This step helps to highlight the key features in the image and provide a clear input for the image deconvolution operation. The binary image data is then subjected to an image deconvolution operation to obtain restored image data. Deconvolution is an image processing technique used to restore a clearer original image from a blurred image, which is crucial for accurate detection of road conditions. The restored image data is finally subjected to a blur degree calculation operation to obtain road condition data. This step quantifies the blur degree of the image by applying a specific blur degree calculation formula, thereby evaluating the road condition.

[0107] In the calculation formula of fuzziness, Indicates the restored image pixels, Indicates the road condition data of the restored image, represents the variance of the restored image, Represents the mean of the restored image. This formula evaluates the blur of the image by calculating the deviation of each pixel from its mean and the ratio of these deviations to the variance. The road condition data provides quantitative road condition information for the road inspection system, which can be used for further road condition analysis and decision support.

[0108] In step S13, a road condition classification operation is performed according to the road condition data to obtain road condition level data. It should be noted that in step S13, the road condition data is the basis for the road condition classification operation, and these data are obtained through the previous image processing and analysis steps. The road condition data reflects the degree of damage to the road surface, including but not limited to quantitative indicators of cracks, potholes, wear and tear, etc. The road condition classification operation is an evaluation process based on preset standards, which compares the road condition data with the defined classification standards to determine the current state of the road surface. Exemplarily, the classification standards used in the present invention include levels such as slight damage, moderate damage and severe damage, and each level has its corresponding road condition data range. Exemplarily, slight damage corresponds to road condition data below 0.05, moderate damage corresponds to road condition data between 0.05 and 0.15, and severe damage corresponds to road condition data exceeding 0.15. The present invention is not limited to this. The operation of the road condition classification operation involves the analysis and interpretation of the road condition data. This includes using statistical methods to determine the distribution of road conditions, or applying machine learning algorithms to identify and classify different road conditions. Once the road conditions are graded, these grade data will be used for further decision-making, including the arrangement of road maintenance plans, the implementation of traffic control measures, or the issuance of road use recommendations. The acquisition of road condition grade data provides key information for road management departments to help them understand the current condition of the road and take appropriate maintenance measures accordingly. This data can also be used for long-term road condition monitoring and trend analysis to predict maintenance work that may be needed in the future, optimize resource allocation, and improve the operating efficiency and safety of the road network.

[0109] In step S14, a road sign detection operation is performed according to the road condition video image data to obtain lane sign data, including:

[0110] Performing image segmentation operations according to the road condition video image data to obtain lane line image data and lane marking image data;

[0111] Performing a lane line extraction operation according to the lane line image data and the lane marking image data to obtain lane line feature data;

[0112] Performing a lane marking detection operation according to the lane line feature data to obtain lane marking position information;

[0113] A lane marking information recognition operation is performed according to the lane marking line position information to obtain lane marking data.

[0114] It should be noted that in step S14, the road surface condition video image data is the basis for the road sign detection operation, and these data are obtained by the camera installed on the vehicle. The image segmentation operation is the process of identifying and separating different objects in the image, and specifically refers to separating the lane line and the lane sign from the road surface condition video image data to form the lane line image data and the lane sign image data. This operation involves image processing technology. For example, the present invention adopts the threshold segmentation technology to identify and distinguish the lane line and the sign. Of course, according to the user's needs and different application scenarios, the edge detection or machine learning algorithm can also be selected, and the present invention is not limited to this. The lane line extraction operation is the process of further identifying and extracting the lane line features based on the lane line image data to obtain the lane line feature data. This includes the geometric attributes of the lane line, and the geometric attributes include position, length, width, etc. The lane marking detection operation is to determine the specific position of the lane marking based on the lane line feature data to obtain the lane marking position information. This involves pattern recognition technology to identify the standard pattern or color of the lane marking. The lane sign information recognition operation is to further identify the specific information of the lane sign, including arrow direction, prohibition sign, etc., based on the lane marking position information, to obtain the lane sign data. This involves a more complex image recognition algorithm. For example, the present invention uses a deep learning model to learn and recognize specific signs from images. Of course, other methods can be selected according to user needs and application scenarios, and the present invention does not limit this. These data can be used by traffic management departments to maintain and update road signs to ensure the clarity and accuracy of road signs, thereby improving road safety.

[0115] In step S15, a lane classification detection operation is performed according to the lane identification data to obtain lane classification data, including:

[0116] Performing an area calculation operation according to the lane marking data to obtain damaged area data;

[0117] Performing an automatic completion operation according to the lane identification data to obtain completed identification data;

[0118] Performing an area calculation operation according to the completed identification data to obtain completed area data;

[0119] Performing a damage degree calculation operation according to the damaged area data and the completed area data to obtain lane classification data;

[0120] The calculation formula of the lane classification data is as follows:

[0121]

[0122] in, represents lane classification data, Indicates the damaged area data, Indicates the completed area data.

[0123] It should be noted that in step S15, the lane marking data is the basis for the lane classification detection operation, and these data are derived from the road marking detection operation in the previous step. The area calculation operation is a process of quantifying the area of ​​the damaged part in the lane marking data to obtain damaged area data. This operation involves image processing technology. For example, the present invention uses image segmentation technology to determine the boundary of the damaged area and calculate its area. The present invention is not limited to this. The automatic completion operation is a process of virtually repairing the damaged or missing marking part based on the lane marking data to obtain the completed marking data. For example, the present invention uses area filling technology to simulate its complete form in the image. Of course, according to different user needs or application scenarios, image recognition and image generation technology can also be selected to identify the type and style of the marking, and the present invention is not limited to this. The completed marking data reflects the state if the lane marking is not damaged. The area calculation operation is performed again, this time based on the completed marking data, to obtain the completed area data. This data represents the total area of ​​the lane marking when it is not damaged. The damage degree calculation operation is to compare the damaged area data with the completed area data to calculate the damage degree of the lane and obtain the lane classification data. In the damage degree calculation formula, represents lane classification data, Indicates the damaged area data, Represents the completion area data. This formula quantifies the degree of damage to lane markings by calculating the ratio of the damaged area to the completed area. The acquisition of lane classification data provides road management departments with quantitative information about the degree of damage to lane markings, which can be used to formulate maintenance and repair plans. For example, depending on the degree of damage, it can be determined which lane markings need to be repaired immediately and which can be dealt with later. In addition, lane classification data can also be used to monitor the long-term condition of lane markings, evaluate the effectiveness of maintenance work, and provide data support for urban planning and traffic management.

[0124] In step S16, a road environment detection operation is performed according to the vehicle position data to obtain road environment data, including:

[0125] Performing a vehicle flow information detection operation according to the vehicle position data to obtain vehicle flow data;

[0126] performing an exhaust gas information estimation operation according to the vehicle position data to obtain exhaust gas data;

[0127] Performing a vehicle flow density information detection operation according to the vehicle position data to obtain vehicle flow density data;

[0128] According to the vehicle flow data, exhaust gas data and vehicle flow density data, a data packaging operation is performed to obtain road environment data.

[0129] It should be noted that in step S16, the vehicle position data is the core of the road environment detection operation. These data are obtained through the positioning system. For example, the present invention adopts the GPS positioning system. Of course, according to different user needs or application scenarios, other high-precision positioning technologies can also be selected to obtain, and the present invention does not limit this. The vehicle flow information detection operation is to estimate the number of vehicles passing through a specific road section and time period by analyzing the vehicle position data and combining the time series to obtain the vehicle flow data. This operation involves the identification and statistical analysis of vehicle movement patterns. The exhaust information estimation operation is based on the vehicle position data and the vehicle type or emission standard data to estimate the exhaust gas emitted by the vehicle to obtain the exhaust gas data. The vehicle flow density information detection operation is to determine the density of vehicles in a specific area or road section by performing spatial distribution analysis on the vehicle flow data to obtain the vehicle flow density data. This involves the use of geographic information system (GIS) technology to map vehicle distribution and calculate density. The data packaging operation is to integrate the vehicle flow data, exhaust data and vehicle flow density data into a comprehensive data set to obtain road environment data. This operation ensures that all relevant environmental information is organized and stored for further analysis and application. The acquisition of road environment data provides important information for traffic management and environmental monitoring. These data can be used in real-time traffic monitoring systems to help traffic management departments optimize traffic signal control, reduce congestion, and improve traffic efficiency. At the same time, exhaust data can provide a basis for environmental quality monitoring and pollution control, and vehicle flow density data can help evaluate the traffic carrying capacity of a specific area. In addition, these data can also be used for urban planning, providing data support for road construction and traffic policy formulation.

[0130] In step S17, traffic diversion classification operation is performed according to the road environment data to obtain diversion classification data;

[0131] It should be noted that in step S17, the road environment data is the basis for the traffic diversion classification operation, and these data include vehicle flow data, exhaust data and vehicle flow density data. The traffic diversion classification operation is a comprehensive analysis process, which involves an in-depth understanding and application of road environment data. The vehicle flow data provides the number of vehicles passing through a specific road section within a specific time, which is a key indicator for assessing the degree of traffic congestion. The exhaust data reflects the level of pollutants emitted by vehicles, which is crucial for air quality management and the formulation of traffic environmental protection policies. The vehicle flow density data describes the distribution of vehicles on the road. High density means traffic congestion, while low density means smooth traffic. The traffic diversion classification operation determines the severity of the traffic situation by analyzing these data and classifies them accordingly. This involves setting different diversion levels. For example, the present invention is set as unblocked, light congestion, moderate congestion and severe congestion. Each level has its corresponding vehicle flow, exhaust emission and density threshold. The present invention is not limited to this. Exemplarily, the smooth level corresponds to low traffic volume, low exhaust emissions and low density, while the severe congestion level corresponds to high traffic volume, high exhaust emissions and high density. The present invention is not limited to this. The acquisition of diversion classification data provides real-time traffic status information to the traffic management department, which can be used to implement traffic diversion measures, such as adjusting the timing of traffic lights, issuing traffic warnings, and guiding vehicles to bypass congested sections. In addition, diversion classification data can also be used for traffic planning and infrastructure construction to prevent and reduce the occurrence of traffic congestion.

[0132] In step S18, a clustering operation is performed according to the vehicle position data to obtain vehicle clustering data, including:

[0133] Performing a clustering iterative update operation according to the vehicle position data to obtain vehicle clustering data;

[0134] The clustering iterative update operation is as follows:

[0135] According to the vehicle position data, a cluster center is randomly selected to obtain an initial cluster center;

[0136] Performing a centroid distance calculation operation according to the initial cluster center and the vehicle position data to obtain centroid distance data;

[0137] Performing data partitioning operation according to the centroid distance data to obtain a clustering partitioning result;

[0138] According to the clustering division result, a centroid update operation is performed to obtain an updated cluster center;

[0139] When it is determined that the updated cluster center no longer changes, the iteration is terminated to obtain vehicle clustering data.

[0140] It should be noted that in step S18, the vehicle position data is the basis for clustering operations. These data are obtained through a positioning system. For example, the present invention uses a GPS positioning system. Of course, according to different user needs or application scenarios, other high-precision positioning technologies can also be selected to obtain, and the present invention does not limit this. The clustering iterative update operation is a repeated execution process, which aims to group the vehicle position data by continuously optimizing the cluster center. The cluster center random selection operation is the starting step of the clustering algorithm, which randomly selects several points from the vehicle position data as the initial cluster center. These initial cluster centers are the starting point of the algorithm and are used for subsequent centroid distance calculation and data partitioning. The centroid distance calculation operation involves measuring the distance between each vehicle position data point and each cluster center to obtain centroid distance data. These distances are the key factors in determining which cluster the data point belongs to. For example, the present invention uses Euclidean distance. Of course, according to different user needs or application scenarios, other appropriate distance measurement methods can also be selected for calculation, and the present invention does not limit this. The data partitioning operation is to assign each vehicle position data point to the nearest cluster center according to the centroid distance data to form a cluster partitioning result. This step groups the vehicle location data points for subsequent centroid updates. The centroid update operation is performed after the data is divided. It recalculates the center point of each cluster, that is, the average position of all data points assigned to the cluster, and obtains the updated cluster center. This updated cluster center will be used for the next round of centroid distance calculation. The iterative process will be repeated until the updated cluster center no longer changes, which means that the clustering has stabilized and the algorithm has reached convergence. At this point, the iteration ends and the vehicle clustering data is obtained. The vehicle clustering data provides important information for traffic management and urban planning. These data can be used to identify traffic hotspots, analyze traffic patterns, and predict traffic flow. Vehicle clustering data can be used for the dispatch of emergency services, such as rapid response in traffic accidents or emergencies.

[0141] In step S19, a confidentiality identification operation is performed based on the vehicle clustering data to obtain position detection data, including:

[0142] Performing an encrypted area distance calculation operation according to the vehicle clustering data to obtain an encrypted area distance;

[0143] When it is determined that the encrypted area distance data is less than a preset minimum threshold, an encryption operation is performed to obtain position detection data.

[0144] It should be noted that in step S19, the vehicle clustering data is the basis for the confidentiality identification operation. These data are obtained through the previous clustering iterative update operation and reflect the distribution of vehicles in a specific area. The encrypted area distance calculation operation is to measure the distance from each data point in the vehicle clustering data to the predefined encrypted area boundary to obtain the encrypted area distance. This operation involves the geographic information system (GIS) technology, which determines the proximity of the vehicle position to the encrypted area by calculating the distance between the spatial coordinate points. When the encrypted area distance data is less than the preset minimum threshold, exemplarily, the present invention sets the minimum threshold to 1km, indicating that the vehicle position has entered or is close to the area that needs to be kept confidential. At this time, an encryption operation is performed to protect the location information from unauthorized access or leakage. The encryption operation involves the application of an encryption algorithm. exemplarily, the present invention uses asymmetric encryption to encrypt the location information to obtain location detection data. These encryption algorithms ensure the security of the data during transmission or storage by converting the original location data into an unreadable format. The acquisition of location detection data provides a means for the vehicle management system to protect sensitive area information. These data can be used to monitor the dynamics of the vehicle in the confidential area while ensuring the security of the vehicle location information. Additionally, location detection data can be used for compliance checks to ensure that vehicles comply with regulations in confidential areas.

[0145] In step S110, a patrol level determination operation is performed according to the road condition level data, the lane classification data, the guidance classification data and the position detection data to obtain patrol level data, including:

[0146] Performing a main cause analysis operation according to the road condition level data to obtain main cause level data;

[0147] Performing a secondary factor analysis operation based on the grooming classification data and the position detection data to obtain secondary factor level data;

[0148] A weighted sum operation is performed according to the primary cause level data and the secondary cause level data to obtain inspection level data.

[0149] It should be noted that in step S110, the road condition level data, lane classification data, traffic diversion classification data and location detection data together constitute the basis for the inspection level judgment operation. These data reflect the current condition of the road and the severity of related factors in a quantitative manner. The main cause analysis operation quantifies the road condition level data to determine the main factors affecting the road condition and obtain the main cause level data. Secondary cause analysis combined with guidance and classification data and location detection data , evaluate the impact of traffic diversion and location confidentiality on the inspection level, and obtain secondary factor level data The weighted sum operation is a mathematical process that combines the primary cause level data and the secondary cause level data by weighted summation to calculate the comprehensive inspection level. . This process can be expressed by the following formula:

[0150]

[0151] in, and are the weights of the primary and secondary factors in the inspection level judgment, and these weights reflect the relative importance of each factor in the inspection decision. For example, the weights of the present invention are determined by an optimization algorithm. It is obtained by analyzing quantitative indicators such as road damage index, crack width, pothole depth, etc. Including indicators such as traffic flow, exhaust emissions, and traffic density, and location detection data Involving the distance between the vehicle and the encrypted area or other geographical location related indicators. Inspection level data The data obtained provides decision support for road inspection and maintenance. The data can be used to formulate inspection plans to determine which sections of the road need priority inspection and repair. For example, a high inspection level indicates a section of the road that needs immediate repair, while a low inspection level indicates a section of the road that can be scheduled for treatment during the regular maintenance cycle. In addition, inspection level data can also be used for resource allocation to ensure that key areas receive adequate attention and maintenance. Through these measures, the efficiency of road use can be improved, traffic congestion can be reduced, the urban traffic environment can be improved, and the long-term health and safety of the road network can be ensured.

[0152] In summary, the present invention provides a machine vision-assisted intelligent road inspection method, which is characterized by comprising:

[0153] Obtain road condition video image data and vehicle position data; perform road condition detection operation according to the road condition video image data to obtain road condition data; perform road condition classification operation according to the road condition data to obtain road condition level data; perform road sign detection operation according to the road condition video image data to obtain lane sign data; perform lane classification detection operation according to the lane sign data to obtain lane classification data; perform road environment detection operation according to the vehicle position data to obtain road environment data; perform traffic diversion classification operation according to the road environment data to obtain diversion classification data; perform clustering operation according to the vehicle position data to obtain vehicle clustering data; perform confidentiality identification operation according to the vehicle clustering data to obtain position detection data; perform inspection level judgment operation according to the road condition level data, the lane classification data, the diversion classification data and the position detection data to obtain inspection level data. The present invention obtains comprehensive analysis results by machine vision-assisted analysis of multiple data types, which can improve the accuracy of road analysis.

[0154] Reference Figure 2 The embodiment of the present invention provides a machine vision-assisted intelligent road inspection system, comprising:

[0155] An input module, used to obtain road condition video image data and vehicle position data;

[0156] A road condition calculation module, used to perform a road condition detection operation according to the road condition video image data to obtain road condition data;

[0157] A road condition level calculation module, used to perform a road condition classification operation according to the road condition data to obtain road condition level data;

[0158] A lane marking recognition module, used to perform a road marking detection operation according to the road condition video image data to obtain lane marking data;

[0159] A lane classification calculation module, used to perform a lane classification detection operation according to the lane identification data to obtain lane classification data;

[0160] A road environment recognition module, used to perform a road environment detection operation according to the vehicle position data to obtain road environment data;

[0161] A traffic diversion classification calculation module is used to perform traffic diversion classification operations according to the road environment data to obtain traffic diversion classification data;

[0162] A vehicle clustering module, used to perform a clustering operation based on the vehicle position data to obtain vehicle clustering data;

[0163] A position encryption module, used to perform a confidentiality identification operation based on the vehicle clustering data to obtain position detection data;

[0164] The inspection level calculation module is used to perform an inspection level judgment operation according to the road condition level data, the lane classification data, the diversion classification data and the position detection data to obtain the inspection level data.

[0165] It should be noted that the machine vision-assisted road intelligent inspection system provided in an embodiment of the present invention is used to execute all the process steps of the machine vision-assisted road intelligent inspection method in the above embodiment. The working principles and beneficial effects of the two correspond one to one, and therefore will not be repeated here.

[0166] The embodiment of the present invention further provides an electronic device. The electronic device includes: a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the above-mentioned steps in the method embodiment are implemented, for example Figure 1 Alternatively, the processor implements the functions of each module / unit in the above-mentioned device embodiments when executing the computer program.

[0167] Exemplarily, the computer program may be divided into one or more modules / units, which are stored in the memory and executed by the processor to implement the present invention. The one or more modules / units may be a series of computer program instruction segments capable of implementing specific functions, which are used to describe the execution process of the computer program in the electronic device.

[0168] The electronic device may be a computing device such as a desktop computer, a notebook, a PDA, and a smart tablet. The electronic device may include, but is not limited to, a processor and a memory. Those skilled in the art will appreciate that the above components are merely examples of electronic devices and do not constitute a limitation on the electronic device. The electronic device may include more or fewer components than the above components, or may combine certain components, or different components. For example, the electronic device may also include input and output devices, network access devices, buses, etc.

[0169] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the electronic device, and uses various interfaces and lines to connect various parts of the entire electronic device.

[0170] The memory can be used to store the computer program and / or module, and the processor realizes various functions of the electronic device by running or executing the computer program and / or module stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0171] Wherein, if the module / unit integrated in the electronic device is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Wherein, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0172] It should be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the accompanying drawings of the device embodiments provided by the present invention, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines. A person of ordinary skill in the art may understand and implement it without paying any creative effort.

[0173] The specific embodiments described above further illustrate the purpose, technical solutions and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. It is particularly pointed out that for those skilled in the art, any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of protection of the present invention.

Claims

1. A machine vision-assisted intelligent road inspection method, characterized in that: Executed by a computer, including: Acquire road condition video image data and vehicle position data; Performing a road condition detection operation according to the road condition video image data to obtain road condition data; Performing a road condition classification operation according to the road condition data to obtain road condition level data; Performing a road sign detection operation according to the road condition video image data to obtain lane sign data; Performing a lane classification detection operation according to the lane identification data to obtain lane classification data; Performing a road environment detection operation according to the vehicle position data to obtain road environment data; Performing traffic diversion classification operations according to the road environment data to obtain diversion classification data; Performing a clustering operation according to the vehicle position data to obtain vehicle clustering data; According to the vehicle clustering data, a confidentiality identification operation is performed to obtain position detection data; the position detection data includes an encrypted area distance between the vehicle and the encrypted area, and the encrypted area distance is obtained by measuring the distance from each updated cluster center in the vehicle clustering data to the predefined encrypted area boundary; According to the road condition level data, the lane classification data, the guidance classification data and the position detection data, an inspection level judgment operation is performed to obtain inspection level data, specifically including: Performing a main cause analysis operation according to the road condition level data to obtain main cause level data; Combining the traffic diversion classification data and the location detection data, evaluating the impact of traffic diversion and location confidentiality on the inspection level, and obtaining secondary factor level data; A weighted sum operation is performed according to the primary cause level data and the secondary cause level data to obtain inspection level data.

2. The machine vision-assisted intelligent road inspection method according to claim 1, characterized in that: According to the road condition video image data, a road condition detection operation is performed to obtain road condition data, including: Performing a grayscale operation on the road condition video image data to obtain a grayscale image; Performing edge extraction operation according to the grayscale image to obtain road surface boundary data; Performing a histogram equalization operation according to the road boundary data to obtain equalized image data; Performing a fuzzy enhancement operation according to the equalized image data to obtain enhanced image data; Performing a numerical normalization operation on the enhanced image data to obtain normalized image data; Performing a binarization operation on the normalized image data to obtain binary image data; Performing an image deconvolution operation according to the binary image data to obtain restored image data; A blur degree calculation operation is performed according to the restored image data to obtain road condition data.

3. The machine vision-assisted intelligent road inspection method according to claim 2, characterized in that: According to the restored image data, a blur degree calculation operation is performed to obtain road condition data, including: The calculation formula of the traffic data is as follows: in, Indicates the restored image pixels, Indicates the road condition data of the restored image, represents the variance of the restored image, represents the mean of the restored image, is the total number of pixels.

4. The machine vision-assisted intelligent road inspection method according to claim 1, characterized in that: According to the road condition video image data, a road sign detection operation is performed to obtain lane sign data, including: Performing image segmentation operations according to the road condition video image data to obtain lane line image data and lane marking image data; Performing a lane line extraction operation according to the lane line image data and the lane marking image data to obtain lane line feature data; Performing a lane marking detection operation according to the lane line feature data to obtain lane marking position information; A lane marking information recognition operation is performed according to the lane marking line position information to obtain lane marking data.

5. The machine vision-assisted intelligent road inspection method according to claim 1, characterized in that: According to the lane identification data, a lane classification detection operation is performed to obtain lane classification data, including: Performing an area calculation operation according to the lane marking data to obtain damaged area data; Performing an automatic completion operation according to the lane identification data to obtain completed identification data; Performing an area calculation operation according to the completed identification data to obtain completed area data; Performing a damage degree calculation operation according to the damaged area data and the completed area data to obtain lane classification data; The calculation formula of the lane classification data is as follows: in, represents lane classification data, Indicates the damaged area data, Indicates the completed area data.

6. The machine vision-assisted intelligent road inspection method according to claim 1, characterized in that: According to the vehicle position data, a road environment detection operation is performed to obtain road environment data, including: Performing a vehicle flow information detection operation according to the vehicle position data to obtain vehicle flow data; performing an exhaust gas information estimation operation according to the vehicle position data to obtain exhaust gas data; Performing a vehicle flow density information detection operation according to the vehicle position data to obtain vehicle flow density data; According to the vehicle flow data, exhaust gas data and vehicle flow density data, a data packaging operation is performed to obtain road environment data.

7. The machine vision-assisted intelligent road inspection method according to claim 1, characterized in that: According to the vehicle position data, a clustering operation is performed to obtain vehicle clustering data, including: Performing a clustering iterative update operation according to the vehicle position data to obtain vehicle clustering data; The clustering iterative update operation is as follows: According to the vehicle position data, a cluster center is randomly selected to obtain an initial cluster center; Performing a centroid distance calculation operation according to the initial cluster center and the vehicle position data to obtain centroid distance data; Performing data partitioning operation according to the centroid distance data to obtain a clustering partitioning result; According to the clustering division result, a centroid update operation is performed to obtain an updated cluster center; When it is determined that the updated cluster center no longer changes, the iteration is terminated to obtain vehicle clustering data.

8. A machine vision-assisted intelligent road inspection system, characterized in that: include: An input module, used to obtain road condition video image data and vehicle position data; A road condition calculation module, used to perform a road condition detection operation according to the road condition video image data to obtain road condition data; A road condition level calculation module, used to perform a road condition classification operation according to the road condition data to obtain road condition level data; A lane marking recognition module, used to perform a road marking detection operation according to the road condition video image data to obtain lane marking data; A lane classification calculation module, used to perform a lane classification detection operation according to the lane identification data to obtain lane classification data; A road environment recognition module, used to perform a road environment detection operation according to the vehicle position data to obtain road environment data; A traffic diversion classification calculation module is used to perform traffic diversion classification operations according to the road environment data to obtain traffic diversion classification data; A vehicle clustering module, used to perform a clustering operation based on the vehicle position data to obtain vehicle clustering data; A position encryption module, used to perform a confidentiality identification operation based on the vehicle clustering data to obtain position detection data; the position detection data includes an encrypted area distance between the vehicle and the encrypted area, and the encrypted area distance is obtained by measuring the distance from each updated cluster center in the vehicle clustering data to the predefined encrypted area boundary; An inspection level calculation module is used to perform an inspection level judgment operation according to the road condition level data, the lane classification data, the guidance classification data and the position detection data to obtain inspection level data; wherein, according to the road condition level data, a main cause analysis operation is performed to obtain main cause level data; Combining the traffic diversion classification data and the location detection data, evaluating the impact of traffic diversion and location confidentiality on the inspection level, and obtaining secondary factor level data; A weighted sum operation is performed according to the primary cause level data and the secondary cause level data to obtain inspection level data.

Images