Vehicle-mounted road patrol detection method and related equipment

By dynamically calculating the road loss value and adjusting the patrol frequency, combined with image recognition technology, the problem of inefficient inspections in fixed cycles in the existing technology is solved, and the efficiency and accuracy of on-board road patrol inspections are achieved.

CN119068669APending Publication Date: 2024-12-03BEIJING MUNICIPAL BRIDGE MAINTENANCE MANAGEMENT +2
View PDF 0 Cites 6 Cited by

Patent Information

Application Number
CN202411051674.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-01
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

The existing on-board road patrol inspection and detection methods are based on fixed cycles and cannot dynamically adjust the patrol frequency, resulting in some fast-loss roads not being maintained in time or being inspected too frequently, which is inefficient.

Method used

By obtaining the temperature difference of the road environment, traffic flow and mainstream vehicle types, calculate the road loss value within the preset time period, and compare it with the preset loss threshold, determine the patrol frequency level, dynamically adjust the patrol frequency, combine image recognition and data comparison, Realize differentiated inspections.

Benefits of technology

It improves patrol efficiency, reduces manpower and material consumption, ensures timely maintenance of roads and traffic safety, and improves the accuracy and pertinence of inspections.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119068669A_ABST
    Figure CN119068669A_ABST
Patent Text Reader

Abstract

The invention discloses a vehicle-mounted road patrol detection method and related equipment, and relates to the field of road patrol, and the method comprises the steps: obtaining environment parameters of a current vehicle-mounted road, calculating a road loss value in a preset time according to a road loss calculation formula, comparing and analyzing the road loss value with a loss threshold value, preset in data, of the road in the preset time period, and obtaining a loss threshold value of the road in the preset time period; therefore, the difference between the loss degree of the current road and the expected threshold value can be obtained, the patrol frequency grade of the road can be determined according to the difference, finally, the current road is comprehensively detected according to the determined patrol frequency parameters, differential patrol of different road sections can be achieved, and the patrol efficiency is improved. The problem that road detection is too much or too little due to the fact that only a unified fixed patrol period is determined in the past is effectively avoided, patrol efficiency is improved, and manpower and material resource consumption is reduced.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of road inspection, and in particular to a vehicle-mounted road inspection detection method and related equipment. Background Art

[0002] With the rapid development of urban transportation, road traffic density has increased dramatically, resulting in accelerated road wear and tear, and putting higher demands on road maintenance. Effective road inspection and maintenance are not only related to traffic safety, but also affect the efficiency of urban transportation and public safety.

[0003] In related technologies, vehicle-mounted road patrol detection mainly relies on regular physical and visual inspections, collecting image data of the road surface through on-board cameras. The patrol frequency is usually set based on empirical rules, such as monthly or quarterly inspections, rather than dynamically adjusted according to the actual usage and wear of the road.

[0004] This fixed-cycle inspection method cannot effectively adapt to the actual wear and tear of different roads. It may result in some rapidly worn roads not being maintained in a timely manner, or overly frequent inspections of roads in good condition, resulting in a low conversion rate of vehicle-mounted road inspections. Summary of the invention

[0005] The present application provides a vehicle-mounted road patrol detection method and related equipment for improving the conversion rate of vehicle-mounted road patrol detection.

[0006] In a first aspect, the present application provides a vehicle-mounted road inspection and detection method, which is applied to a vehicle-mounted road inspection and detection integrated device, and the method comprises: Obtaining the road environment temperature difference, traffic volume and mainstream vehicle type of the current vehicle road, where the mainstream vehicle type is the vehicle type with the largest proportion of vehicles traveling on the current vehicle road, obtained by performing image recognition on the monitoring screen of the current vehicle road; inputting the road environment temperature difference, the traffic volume and the mainstream vehicle type into the road loss calculation formula to obtain the road loss value within a preset time period, which is the loss value that will be generated by the current vehicle road within the preset time period; Obtaining a preset loss threshold value of the current vehicle road corresponding to the preset time period, and determining a road inspection frequency level of the current vehicle road according to a difference between the road loss value and the loss threshold value; The current vehicle road is detected according to the road inspection frequency corresponding to the road inspection frequency level.

[0007] In the above embodiment, by obtaining the environmental parameters of the current vehicle road, the road loss value within the preset time is calculated according to the road loss calculation formula, and then compared and analyzed with the loss threshold of the road preset in the data within the time period, the gap between the loss degree of the current road and the expected threshold can be obtained. The patrol frequency level of the road can be determined according to the size of the gap. Finally, the current road is comprehensively inspected according to the determined patrol frequency parameters, which can realize differentiated patrols of different road sections, effectively avoiding the problem of excessive or insufficient road detection that may result from only determining a unified fixed patrol cycle in the past, improving patrol efficiency, and reducing manpower and material resources consumption.

[0008] In conjunction with some embodiments of the first aspect, in some embodiments, the step of detecting the current vehicle road according to the road inspection frequency corresponding to the road inspection frequency level specifically includes: Acquire a real-time road surface image of the current vehicle-mounted road according to the road inspection frequency; Performing image recognition on the real-time road surface image to obtain a road surface disease image of the real-time road surface image, wherein the road surface disease image includes newly added road surface diseases; Compare the pavement damage image with the historical pavement image to obtain the location information, type information and scale parameters of the newly added pavement damage; The point information, the type information and the scale parameter are sent to the client.

[0009] In the above embodiment, when conducting on-board road inspections, real-time high-definition images of the road surface can be obtained through the on-board camera and image recognition algorithm, and compared and deeply analyzed with the historical road surface image information of the road section in the past. Various new types of road surface defects, such as new potholes and cracks, can be accurately identified, and relevant information about the defects, such as location, type, size, etc., can be obtained. In this way, the monitoring of road surface defects and the extraction of defect information can be realized comprehensively, accurately and automatically, thereby improving the accuracy of detection.

[0010] In combination with some embodiments of the first aspect, in some embodiments, after the step of detecting the current vehicle-mounted road according to the road inspection frequency corresponding to the road inspection frequency level, the method further includes: Obtain all road surface images of all vehicle-mounted roads in a preset area; Modeling is performed based on all road surface images to obtain a road surface detection map within the preset area; The vehicle-mounted roads that have been inspected in the preset area are displayed in real time and visualized to obtain real-time inspection map information; The real-time detection map information is sent to the client.

[0011] In the above embodiment, after completing the inspection of a certain road or a certain area, it is possible to further obtain all road surface images for three-dimensional modeling, generate a detailed road surface detection digital map, and perform real-time dynamic visualization. At the same time, the detected road surface information digital map is pushed to the monitoring and maintenance party in real time through the vehicle-mounted equipment, which can achieve accurate visual monitoring of the overall condition of the regional road surface, and pass on the advantage of knowing the road surface conditions at the first time to the daily maintenance party, thereby improving the timeliness and pertinence of road maintenance, and providing very scientific and valuable decision-making support for the planning and implementation of road repair and maintenance.

[0012] In combination with some embodiments of the first aspect, in some embodiments, after the step of detecting the current vehicle-mounted road according to the road inspection frequency corresponding to the road inspection frequency level, the method further includes: Obtaining the traffic sign image on the current vehicle road; Matching the traffic sign image with a preset traffic sign database to determine whether the traffic sign image exists in the preset traffic sign database; If so, calculating the degree of matching between the traffic sign image and a preset sign image, where the preset sign image is an image that matches the traffic sign image in the preset traffic sign database; If the matching degree is lower than a preset matching degree threshold, a first prompt message including the traffic sign image is sent to the client.

[0013] In the above embodiment, when conducting on-board road inspections, the method can also identify traffic sign images on the road and compare them with a preset traffic sign database to calculate the degree of match between the images. If the degree of match is lower than a preset threshold, a prompt message containing the traffic sign is sent to the client, thereby realizing automatic monitoring of traffic signs. When the device detects a problem with a traffic sign, it can quickly prompt maintenance personnel, which is of great significance to ensuring road traffic safety.

[0014] In conjunction with some embodiments of the first aspect, in some embodiments, after the step of matching the traffic sign image with a preset traffic sign database and determining whether the traffic sign image exists in the preset traffic sign database, the method further includes: If not present, the second prompt information including the traffic sign image is sent to the client.

[0015] In the above embodiment, after determining whether the traffic sign image matches the database, if there is no matching result at all, a prompt message of the sign image is sent to the client, which avoids the device from missing the monitoring of illegally added or unknown signs. When illegal additions or unknown signs occur, relevant departments can be quickly notified for timely processing.

[0016] In combination with some embodiments of the first aspect, in some embodiments, after the step of detecting the current vehicle-mounted road according to the road inspection frequency corresponding to the road inspection frequency level, the method further includes: Obtain user feedback data within a preset time range from a preset feedback information database; Performing semantic analysis on the user feedback data to obtain frequency data of all vehicle-mounted roads appearing in the user feedback data, and obtaining a collection of frequency data; The vehicle-mounted road with the highest frequency in the frequency data collection is determined as a first road to be inspected, and the first road to be inspected is inspected.

[0017] In the above embodiment, by analyzing user feedback information, the roads with the highest frequency of problem reports are counted and taken as priority inspection objects. At the same time, the roads that exceed the frequency threshold are included in the inspection plan. It can make full use of user feedback data and reasonably allocate inspection resources to problem roads. It determines the inspection priority based on big data analysis, can continuously optimize and adjust the inspection plan, and improve the inspection efficiency.

[0018] In conjunction with some embodiments of the first aspect, in some embodiments, after the step of performing semantic analysis on the user feedback data to obtain frequency data of all vehicle-mounted roads appearing in the user feedback data and obtaining a collection of frequency data, the method further includes: Add all vehicle-mounted roads in the frequency data collection that are greater than a preset frequency threshold into a preset detection plan; All vehicle-mounted roads are inspected according to the preset inspection plan during a preset future time period.

[0019] In the above embodiment, by performing semantic analysis on user feedback, it is possible to count the problem roads that appear repeatedly in the report and the frequency with which they are mentioned, from which it is possible to find out the common problem roads that users report more frequently, and then set a reasonable frequency threshold. All roads that are greater than this threshold, that is, roads that are reported more frequently, are included in the inspection and detection plan to be rectified, thereby improving the response speed to key problem roads.

[0020] In the second aspect, an embodiment of the present application provides an integrated vehicle-mounted road patrol and detection device, which includes: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the integrated vehicle-mounted road patrol and detection device to execute the method described in the first aspect and any possible implementation method of the first aspect.

[0021] In a third aspect, an embodiment of the present application provides a computer program product comprising instructions. When the above-mentioned computer program product is run on an integrated vehicle-mounted road patrol and detection device, the above-mentioned integrated vehicle-mounted road patrol and detection device executes the method described in the first aspect and any possible implementation method of the first aspect.

[0022] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium comprising instructions. When the above instructions are executed on a vehicle-mounted road patrol and detection integrated device, the above-mentioned vehicle-mounted road patrol and detection integrated device executes the method described in the first aspect and any possible implementation method of the first aspect.

[0023] It is understandable that the vehicle-mounted road inspection and detection integrated device provided in the second aspect, the computer program product provided in the third aspect, and the computer storage medium provided in the fourth aspect are all used to execute the method provided in the embodiment of the present application. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method, and will not be repeated here.

[0024] One or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages: 1. This application obtains the environmental parameters of the current vehicle road, calculates the road loss value within the preset time according to the road loss calculation formula, and then compares and analyzes it with the loss threshold of the road preset in the data within the time period, so as to obtain the gap between the current road loss level and the expected threshold. According to the size of the gap, the patrol frequency level of the road can be determined. Finally, the current road is comprehensively inspected according to the determined patrol frequency parameters, which can realize differentiated patrols of different sections, effectively avoiding the problem of too many or too few road inspections that may be caused by only determining a unified fixed patrol cycle in the past, thereby improving patrol efficiency and reducing manpower and material resources consumption.

[0025] 2. When conducting on-board road inspections, the present application can obtain real-time high-definition images of the road surface through the on-board camera and image recognition algorithm, and compare and deeply analyze the images of the road surface with the historical road surface image information of the road section in the past. It can accurately identify various new road surface defects, such as new potholes and cracks, and obtain relevant information about the defects, such as location, type, size, etc. In this way, the monitoring of road surface defects and the extraction of defect information can be realized comprehensively, accurately and automatically, thereby improving the accuracy of detection.

[0026] 3. When conducting vehicle-mounted road inspections, the method can also identify traffic sign images on the road, compare them with a preset traffic sign database, and calculate the degree of match between the images. If the degree of match is lower than a preset threshold, a prompt message containing the traffic sign is sent to the client, thereby realizing automatic monitoring of traffic signs. When the equipment detects a problem with a traffic sign, it can quickly alert maintenance personnel, which is of great significance to ensuring road traffic safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 It is a flow chart of the vehicle-mounted road inspection detection method in the embodiment of the present application; Figure 2 is another flow chart of the vehicle-mounted road inspection detection method in an embodiment of the present application; Figure 3 This is a schematic diagram of a scenario of the vehicle-mounted road inspection detection method in an embodiment of the present application; Figure 4 is another flow chart of the vehicle-mounted road inspection detection method in an embodiment of the present application; Figure 5 It is a schematic diagram of the structure of a physical device of the vehicle-mounted road inspection and detection integrated equipment in the embodiment of the present application. DETAILED DESCRIPTION

[0028] The terms used in the following embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to be used as limitations to the present application. As used in the specification of the present application, the singular expressions "one", "a kind of", "above", "the" and "this" are intended to also include plural expressions, unless there is a clear indication to the contrary in the context. It should also be understood that the term "and / or" used in the present application refers to any or all possible combinations comprising one or more of the listed items.

[0029] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features, and in the description of the embodiments of the present application, unless otherwise specified, "plurality" means two or more.

[0030] The following is a description of the process of the method provided by this implementation. Figure 1 , which is a flow chart of the vehicle-mounted road inspection and detection method in an embodiment of the present application.

[0031] S101. Obtaining a road environment temperature difference, a traffic volume, and a mainstream vehicle type for a current vehicle road. The mainstream vehicle type is a vehicle type that accounts for the largest proportion of vehicles traveling on the current vehicle road, and is obtained by performing image recognition on a monitoring screen of the current vehicle road.

[0032] Among them, the current vehicle road represents the road section that needs to be inspected, the road environment temperature difference refers to the difference between the road surface temperature and the air temperature, the traffic volume refers to the number of vehicles passing through the current vehicle road in unit time, and the mainstream vehicle type refers to the vehicle type that occupies the largest proportion on the current vehicle road.

[0033] Specifically, before conducting an on-board road inspection, this step requires obtaining the basic parameters of the road section that needs to be inspected as input conditions for calculating road loss. These parameters include: the temperature difference between the road surface and the air temperature, which can reflect the thermodynamic properties of the road; the number of vehicles passing through per unit time, which can indicate the traffic flow of the road; and the type of vehicles that occupies the largest proportion on the road, which can reflect the traffic composition of the road.

[0034] In some embodiments, various parameters of the current vehicle road can be obtained in the following two ways: Optionally, set up an infrared thermal imager to measure the road surface temperature and detect the data of the air temperature sensor to obtain the road environment temperature difference; count the number of vehicles passing per unit time; use image recognition to analyze the proportion of different types of vehicles in the traffic flow and determine the mainstream model.

[0035] Optionally, the real-time road image can be directly obtained by using the on-board camera, and each vehicle type can be detected and identified through image processing algorithms; the number of vehicles per unit time can be counted by traffic statistics software; the road surface temperature can be obtained by analyzing the image using software, and the temperature can be detected to calculate the temperature difference.

[0036] It is understandable that other methods may be used to obtain the current road ambient temperature difference, traffic volume, and mainstream vehicle types, which are not limited here.

[0037] S102, input the road environment temperature difference, the vehicle flow and the mainstream vehicle type into a road loss calculation formula to obtain a road loss value within a preset time period, where the road loss value is the loss value that will be generated by the current vehicle road within the preset time period.

[0038] Among them, the road loss calculation formula is a preset algorithm for calculating the road loss value; the road loss value refers to the loss that is expected to occur on the current road within the preset time; the preset time period refers to a pre-set time range.

[0039] Specifically, this step is to obtain the ambient temperature difference, traffic volume and mainstream vehicle type data of the current road, substitute them into the road loss calculation formula for calculation, and obtain the loss value that is expected to be generated by the road within a preset time period from the current time. The algorithm based on the calculation can comprehensively model the impact of various factors on road loss to obtain a relatively accurate calculation result.

[0040] In some embodiments, this step can be implemented in the following two ways: using a simple linear loss calculation model, directly performing linear weighting according to certain weights on each data to obtain a loss value; establishing a loss calculation model based on multivariate regression analysis, selecting temperature difference, vehicle flow, vehicle model and other factors as independent variables, determining the weight coefficients of each variable through regression analysis, and obtaining a loss prediction model.

[0041] It is understandable that other algorithmic means may also be used to calculate and predict the loss value of the current road within a preset time period, which is not limited here.

[0042] In some embodiments, the road loss calculation formula is: L=k1·T+k2·V+k3·C+k4·V·C k1·T is the effect of temperature difference on road loss, k2·V is the direct effect of vehicle flow on road loss, k3·C is the effect of different vehicle models on road loss, and k4·V·C is the interaction term between vehicle flow and vehicle model, which takes into account the compound effect of different types of vehicle flow on road loss.

[0043] S103, obtaining a preset loss threshold of the current vehicle road corresponding to the preset time period, and determining a road inspection frequency level of the current vehicle road according to a difference between the road loss value and the loss threshold.

[0044] Among them, the preset loss threshold refers to the maximum acceptable loss of a specified road within a preset time; the road loss value is the loss expected to be incurred by the road in this time period obtained by previous calculation; the road inspection frequency level is the classification level of the recommended inspection frequency corresponding to the road loss condition.

[0045] Specifically, in this step, after calculating the predicted loss value of the current road within the preset time period, the device needs to obtain the preset loss threshold of the road within this time period, that is, the maximum loss that the road can withstand as stipulated by the management department. The device then calculates the difference between the predicted loss value and the threshold, and determines the patrol frequency level of the road based on the size of the difference. The larger the difference, the more serious the loss, and the higher the corresponding patrol frequency level, and vice versa.

[0046] In some embodiments, the preset loss threshold can be obtained in two ways: searching the prescribed threshold for the road from the road management database, and calculating the standard recommended threshold by the category, level, and operating time of the road. It can be understood that the rules for determining the loss threshold and patrol level can be set in a variety of ways, which are not limited here.

[0047] In some embodiments, in the process of determining the road inspection frequency level, it is first necessary to obtain the preset loss threshold of the road, that is, the maximum loss limit that can be tolerated within a specific time period specified by the management department. Then, the actual loss value of the road in the corresponding time period is calculated, and the difference with the preset loss threshold is calculated. According to the size of the difference, the corresponding inspection frequency level is found in the preset inspection frequency level table.

[0048] S104: Detect the current vehicle road according to the road inspection frequency corresponding to the road inspection frequency level.

[0049] Among them, the road inspection frequency level is based on the result obtained from the previous calculation, reflecting the inspection frequency classification corresponding to the current road loss condition; the road inspection frequency refers to the time interval frequency for comprehensive road inspections.

[0050] Specifically, this step will obtain the reasonable patrol frequency parameter value corresponding to the road according to the determined road patrol frequency level and the preset level-frequency correspondence rules, and then the equipment will conduct a comprehensive and standardized patrol inspection on the current road according to the requirements of this parameter value.

[0051] This step specifically includes: Acquire a real-time road surface image of the current vehicle-mounted road according to the road inspection frequency; In this step, after determining the reasonable inspection frequency of the current road based on the difference between the road loss value and the loss threshold value calculated in advance, the on-board detection equipment needs to obtain images of the road it is currently responsible for according to the requirements of this frequency parameter. For example, if the inspection frequency is once a week, the equipment needs to drive through the road at the planned time every week, and start the high-definition camera to take all-round and continuous photos of the road surface to obtain real-time images of the road surface. The image acquisition range needs to cover all lanes of the road, taking into account the shoulders and roadbed. The on-board equipment will also accurately record the location coordinate information of the image acquisition through the satellite navigation positioning device. In addition to the road itself, the real-time road surface image also needs to include information about the surrounding area in order to conduct an overall analysis of the road surface conditions. The equipment can take multiple shots at different time periods to make the image information comprehensive and accurate. The acquired images will be saved in the internal memory of the device in real time, providing a data source for subsequent image analysis and processing.

[0052] Performing image recognition on the real-time road surface image to obtain a road surface disease image of the real-time road surface image, wherein the road surface disease image includes newly added road surface diseases; After obtaining the real-time road image of the current vehicle-mounted road, the device will automatically start the built-in intelligent image recognition and analysis algorithm to automatically detect various road problems, mainly including road damage such as cracks, potholes, gravel, and road markings such as lane lines and arrows. The device will automatically compare the newly acquired real-time road image with the historical image data of the road section, determine the newly emerged road problems, that is, new diseases, and intercept the part of the image to obtain the road image of the new disease. New diseases include new cracks, new ruts, and falling road signs. The image recognition algorithm can effectively identify new road problems and achieve early detection and early treatment. The device will save the road image containing the new disease and bind the relevant location coordinates, problem type and other information of the image locally.

[0053] Compare the pavement damage image with the historical pavement image to obtain the location information, type information and scale parameters of the newly added pavement damage; After the on-board detection equipment identifies the new road surface defects contained in the real-time road surface image, the equipment needs to compare and analyze these new defects with the historical road surface images of the road section to further obtain detailed information about the defects. Specifically, the equipment needs to extract the past road surface image information of the road section from the memory, and these historical images do not contain the latest road surface disease conditions. Then, the equipment will use the image processing algorithm to compare the two, determine the precise coordinates of the new defect point, obtain the type of defect, such as cracks or potholes, and the scale parameters of the defect, such as the length and width of the cracks, the diameter and depth of the potholes, etc. By making full use of historical data and conducting a comprehensive quantitative analysis of the attributes of the newly added defect points, the point information, type information and scale parameters can be output.

[0054] The point information, the type information and the scale parameter are sent to the client.

[0055] After obtaining detailed quantitative information about new road surface defects, the on-board detection equipment needs to send these key data to the back-end management and service clients in a timely manner for timely repairs. For example, the equipment can send the precise coordinates of the defect, the result of the defect type identification, and scale parameter information such as crack length and pothole area to the back-end monitoring platform of the road maintenance company in JSON format through the 4G network. The platform can clearly determine the location and scale of the problem based on this information, so that repairs can be dispatched immediately. The equipment can also only temporarily store the data locally and upload it to the maintenance service provider via a wired connection when it is transported back.

[0056] It is understandable that the development trend of existing road diseases and the location, type, scale and other parameters of new road diseases can also be compared and analyzed to provide a reference for road maintenance and management strategies. According to the location information, type information and scale parameters of the road disease, the maintenance strategy with the highest matching degree with the road disease can be found in the historical maintenance strategy database, and the maintenance strategy can be displayed on the visualization screen or sent to the client.

[0057] The following is a more detailed description of the process of the method provided by this implementation. Figure 2 , is another flow chart of the vehicle-mounted road inspection and detection method in an embodiment of the present application.

[0058] S201. Detect the current vehicle road according to the road inspection frequency corresponding to the road inspection frequency level.

[0059] It can be understood that this step is similar to step S104 and will not be described in detail here.

[0060] S202: Acquire all road surface images of all vehicle-mounted roads in a preset area.

[0061] After completing the inspection of the current road, the vehicle-mounted road inspection and detection integrated equipment can also be set to obtain road image information in a larger range. Specifically, the device will obtain images of all vehicle-mounted roads within a preset area, which can be a city area, a county, or a larger area. Through the high-definition camera and satellite navigation positioning equipment installed, the device can continuously scan and obtain images of all vehicle roads in the preset area.

[0062] Whether it is a highway, national highway, provincial highway, county or rural road, or even a city road, the equipment automatically obtains image information of the road section by section, and obtains the road surface, street lights, traffic signs, surrounding environment and other aspects, and stores the image data.

[0063] S203: Modeling is performed based on all road surface images to obtain a road surface detection map within the preset area.

[0064] After obtaining the image data of all vehicle-mounted roads in the preset area, the vehicle-mounted inspection equipment will automatically or cooperate with the management center server to use computer vision and deep learning algorithms to analyze a large number of images, determine the road shape, diseases, landmarks, etc., and integrate these image contents according to the corresponding geographic location information, and finally form a digital three-dimensional map, that is, a digital map of road surface detection in the preset area. Each road section on the map can be associated with the corresponding road surface image information and image recognition results, restoring the specific road conditions of the roads in the area.

[0065] In some embodiments, after obtaining the image data of all vehicle-mounted roads in a preset area, the vehicle-mounted inspection equipment will automatically or in cooperation with the management center server to perform a fine analysis of a large number of images, and use computer vision and deep learning algorithms to identify and process the image content to determine the road surface shape, diseases, markers, etc. Then, the device will model the image according to the optimized patrol line path. For example, an optimized "S"-shaped patrol line path or vehicle-mounted laser SLAM technology can be used, combined with GPS navigation for real-time modeling to avoid repeated routes. During the modeling process, the equipment can accurately locate and mark fault events on the road surface, such as steel plate road surface protrusions, potholes, cracks, etc., for subsequent maintenance, and finally form a digital three-dimensional map, that is, a digital map of road surface detection in the preset area. Each road section on the map can be associated with the corresponding road surface image information and image recognition results to restore the specific road conditions of the roads in the area.

[0066] S204: Display the vehicle-mounted roads that have been inspected in the preset area in real time and visualize them to obtain real-time inspection map information.

[0067] After obtaining the digital map of road surface detection in the preset area, the on-board inspection equipment will mark the road sections or location points that have been inspected in real time on the map, forming a real-time visual detection dynamic map. The display terminal of the equipment can intuitively present the real-time status of the detected sections on the map. For example, different colors are used to represent roads that have been inspected in different time periods, and marks or borders are used to mark the sections that have been newly inspected. The equipment can display the map detection dynamics locally, and can also transmit the map information to the traffic management monitoring center in real time.

[0068] In some embodiments, the real-time data and road surface images of the current vehicle-mounted road, including traffic flow, main vehicle types and current road conditions, can also be visualized. The road environment data (such as temperature difference, traffic flow, vehicle type) and road surface conditions (such as disease images) collected by the device are used, and preprocessed, including data cleaning, formatting and preliminary analysis (such as image recognition and disease detection). The processed data is stored in a central database, and the real-time data and road surface images of the current vehicle-mounted road, including traffic flow, main vehicle types and current road conditions, are displayed. An interface is provided to query historical data, and multi-dimensional filtering by date, section, etc. is supported. The road data comparison function at the same location at different time points is realized, including dynamic changes in the location, type and scale of diseases.

[0069] S205: Send the real-time detection map information to the client.

[0070] After the device generates a real-time road detection visualization map, it can immediately transmit the map information to the back-end road management and monitoring client for relevant personnel to view and analyze. The device can send the map's vector data, image data, and annotation information to the client through 4G network or other wireless communication methods in the form of streaming media or time-based push. The client can be set up in the monitoring center of the Transportation Administration Bureau, or in a road maintenance company or third-party monitoring agency. The map data can be centrally stored and analyzed on the client, or further uploaded to the cloud platform. The real-time transmission of the map enables the supervisory party to grasp the progress of the inspection and detection of regional roads in the first place, so as to arrange subsequent maintenance and planning.

[0071] like Figure 3 FIG. 1 is a schematic diagram of a scenario of a vehicle-mounted road inspection detection method in an embodiment of the present application. Figure 3 The vehicle-mounted road inspection and detection integrated equipment includes an inspection vehicle, image acquisition devices such as a front-view camera and a rear-view camera, a laser point cloud module and a visualization screen. When the inspection vehicle is traveling on the current vehicle-mounted road, it can realize a driver + equipment. The image acquisition device can be used to identify the mainstream vehicle types on the current vehicle-mounted road, and the collected real-time road surface images can be image recognized and analyzed to obtain road surface disease images. The location information, type information and scale parameters of the detected road surface diseases can also be displayed in the inspection vehicle through a visualization screen, and can also be sent to the client.

[0072] The road surface is scanned with high precision by a laser point cloud device installed on the top of the inspection vehicle, creating a detailed three-dimensional model of the road surface, accurately measuring geometric features such as road width and slope, and identifying and locating road defects such as cracks, potholes, and ruts. The laser point cloud data is then integrated with other detection data (such as video, GPS) to construct a complete three-dimensional road model, obtain real-time detection map information, and display the real-time detection map information on a visual interface.

[0073] S206: Acquire the traffic sign image on the current vehicle-mounted road.

[0074] While detecting the road surface, the on-board equipment will also automatically scan and obtain various types of traffic signs on the roadside, such as instruction signs, prohibition signs, warning signs, etc. The equipment uses a high-definition camera to capture images on both sides of the road, uses image recognition technology to identify the location and content of the signs, and then extracts the image of the sign area to obtain a clear and complete picture of the traffic sign. To avoid omissions, the equipment can capture the same sign through multiple shots at different times. The traffic sign images obtained can provide auxiliary judgment basis for subsequent maintenance and management.

[0075] S207: Match the traffic sign image with a preset traffic sign database to determine whether the traffic sign image exists in the preset traffic sign database.

[0076] The device will automatically compare and match the acquired traffic sign image with the sign database in the background management device. The database contains standardized and legal traffic sign sample images. The device uses image feature extraction and matching algorithms to calculate the similarity between the newly acquired sign image and the database sample image. If the similarity exceeds the preset threshold, the traffic sign is considered to match a standard sign in the database, and the sign is legal and valid. Otherwise, it means that the sign does not exist in the database and needs further verification. This technology can help identify illegally added or damaged signs and ensure road traffic safety.

[0077] S208: If so, calculate the matching degree between the traffic sign image and a preset sign image, where the preset sign image is an image that matches the traffic sign image in the preset traffic sign database.

[0078] If the device determines that the acquired traffic sign image has a match in the preset database, the device will call the image processing algorithm to specifically extract the feature vectors of the two images and calculate the similarity between the feature vectors as the match between the two images. The extracted feature vectors can include significant information such as the color, texture, shape, and edges of the image, and a quantitative matching result can be obtained. Generally speaking, the higher the matching result, the more similar the content and details of the two images are, and the fewer the differences. If the matching degree between the newly acquired traffic sign image and the standard sample sign in the database is too low, it often reflects that the new sign may be damaged, deformed, or contaminated to a certain extent, and is no longer clear and complete.

[0079] S209: If the matching degree is lower than a preset matching degree threshold, a first prompt message including the traffic sign image is sent to the client.

[0080] The device will compare the calculated matching degree between the newly acquired traffic sign image and the standard sample image with the preset matching degree threshold. The preset matching degree threshold here needs to be obtained by testing in advance according to different types of signs and database samples. When the comparison finds that the matching degree value of the newly acquired sample sign is obviously low and lower than the predetermined threshold level, it can be considered that the sign is too different from the standard intact state and is likely to be damaged. At this time, the on-board device will immediately send a prompt message to the background management client. The message will include a picture of the problematic sign, clearly indicating that the sign may be damaged and needs to be checked for replacement or repair.

[0081] This prompt message can be instantly transmitted to the monitoring center or the client of the relevant department through 4G, 5G and other wireless networks. After receiving the prompt, the traffic management party can arrange staff to conduct on-site inspection and processing, and replace or repair the damaged signs to avoid traffic accidents and ensure road safety.

[0082] S210: If not, send the second prompt information including the traffic sign image to the client.

[0083] If the device's image recognition algorithm determines that the newly acquired traffic sign sample cannot find any matching standard samples in the preset database, it can be inferred that the sign may have been illegally added to the road, because the database should contain all standard category samples of road traffic signs. At this time, the device will immediately send a warning message to the client. The message will include a picture of the unknown sign, clearly indicating that the sign is of unknown origin and requires further verification. At the same time, relevant location information can be provided. The traffic management department that receives the warning can immediately send relevant personnel to the location of the sign for inspection.

[0084] The following is a more detailed description of the process of the method provided by this implementation. Figure 4 , is another flow chart of the vehicle-mounted road inspection and detection method in an embodiment of the present application.

[0085] S401. Detect the current vehicle road according to the road inspection frequency corresponding to the road inspection frequency level.

[0086] It can be understood that this step is similar to step S101 and will not be described in detail here.

[0087] S402: Obtain user feedback data within a preset time range from a preset feedback information database.

[0088] The vehicle inspection equipment can be connected to the user feedback database of the backend management equipment, which stores user feedback information on roads in various regions within a certain time range. The equipment can extract all user feedback data within the time range from the database according to preset time parameters, such as the last 1 month, 2 months, etc. The feedback data comes from road users using mobile phone applications, and includes the location of the problem section, the type of problem, and pictures describing the problem.

[0089] The source of the user feedback information in the user feedback database may be a QR code mark set up on the roadside, and the user may obtain a link to fill in the feedback information after scanning the QR code.

[0090] S403: Perform semantic analysis on the user feedback data to obtain frequency data of all vehicle-mounted roads appearing in the user feedback data, and obtain a collection of frequency data.

[0091] After obtaining basic feedback data, the device uses natural language processing technology to perform semantic analysis on the feedback content, automatically extracting key information such as the road name and the type of problem that occurred in the user's description, and then can count the total number of times a specific road has been fed back, as well as the most important problem classification.

[0092] For example, the analysis results can show that: XX road section was reported by users 120 times, and the main problems were waterlogging and potholes; YY road section was reported 100 times, and the problem was unclear signs, etc. The feedback frequency of each road is statistically compiled into a frequency data collection for subsequent analysis.

[0093] S404: Determine the vehicle-mounted road with the highest frequency in the frequency data collection as a first road to be inspected, and inspect the first road to be inspected.

[0094] Finally, the device can find the road with the most feedback based on the frequency statistics, that is, the road where users report the most problems. It will be prioritized as the first road to be inspected. For example, if the number of feedbacks on section XX is 120, the highest, the device will be automatically triggered to use sensors such as cameras to re-detect the actual situation of the section in detail to find out the cause of the problem reported by users and provide a basis for repair.

[0095] S405: Add all vehicle-mounted roads in the frequency data collection that are greater than a preset frequency threshold into a preset detection plan.

[0096] After obtaining the user feedback frequency statistics, the vehicle-mounted equipment can independently determine the high-frequency problem road information that exceeds the threshold based on the built-in frequency threshold parameters. The device will store the relevant information of these high-frequency problem roads, including road names, road section coordinates, etc. in the local detection plan. The key problem roads will be included in its own detection plan, and the equipment will focus on these roads in future patrol tasks. This can avoid the road sections that are ignored due to too few feedback times, and realize closed-loop detection of problem roads through autonomous planning.

[0097] S406: Inspect all vehicle-mounted roads according to the preset inspection plan in a preset future time period.

[0098] After the inspection plan is started within the scheduled time, the on-board equipment needs to perform the inspection task according to the inspection plan table stored in itself. The equipment arranges its own operation route reasonably to cover the planned inspection roads as much as possible. For each planned section, the equipment uses various sensors such as high-definition cameras to take full-scale pictures of the road conditions, and compares them with the historical data of the section, and uses image analysis algorithms to identify the cause of the problem. After the inspection of a road is completed, the equipment organizes the inspection results into report format data and stores them in the local storage.

[0099] The following describes the vehicle-mounted road inspection and detection integrated device in the embodiment of the present invention from the perspective of hardware processing. Figure 5 , which is a schematic diagram of the physical device structure of the vehicle-mounted road inspection and detection integrated equipment in the embodiment of the present application.

[0100] It should be noted that Figure 5 The structure of the vehicle-mounted road patrol and detection integrated device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0101] like Figure 5 As shown, the vehicle-mounted road patrol detection integrated device includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 502 or the program loaded from the storage part 508 to the random access memory (RAM) 503, such as executing the method described in the above embodiment. In RAM503, various programs and data required for the operation of the device are also stored. CPU501, ROM502 and RAM503 are connected to each other through a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0102] The following components are connected to the I / O interface 505: an input section 506 including an audio input device, a button switch, etc.; an output section 507 including a liquid crystal display (LCD) and an audio output device, an indicator light, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the I / O interface 505 as needed. A removable medium 511, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 510 as needed so that a computer program read therefrom is installed into the storage section 508 as needed.

[0103] In particular, according to an embodiment of the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present invention includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication section 509, and / or installed from a removable medium 511. When the computer program is executed by a central processing unit (CPU) 501, various functions defined in the present invention are performed.

[0104] It should be noted that specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution device, apparatus, or device.

[0105] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the devices, methods and computer program products according to various embodiments of the present invention. Each box in the flowchart or block diagram may represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the box may also occur in an order different from that marked in the accompanying drawings.

[0106] Specifically, the integrated vehicle-mounted road patrol and detection device of this embodiment includes a processor and a memory. The memory stores a computer program. When the computer program is executed by the processor, the vehicle-mounted road patrol and detection method provided in the above embodiment is implemented.

[0107] As another aspect, the present invention further provides a computer-readable storage medium, which may be included in the vehicle-mounted road patrol detection integrated device described in the above embodiment; or may exist independently without being assembled into the vehicle-mounted road patrol detection integrated device. The above storage medium carries one or more computer programs, and when the above one or more computer programs are executed by a processor of the vehicle-mounted road patrol detection integrated device, the vehicle-mounted road patrol detection integrated device implements the vehicle-mounted road patrol detection method provided in the above embodiment.

[0108] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

[0109] As used in the above embodiments, the term "when..." may be interpreted to mean "if..." or "after..." or "in response to determining..." or "in response to detecting...", depending on the context. Similarly, the phrases "upon determining..." or "if (the stated condition or event) is detected" may be interpreted to mean "if determining..." or "in response to determining..." or "upon detecting (the stated condition or event)" or "in response to detecting (the stated condition or event)", depending on the context.

[0110] Those skilled in the art can understand that to implement all or part of the processes in the above-mentioned embodiments, the processes can be completed by computer programs to instruct related hardware, and the programs can be stored in computer-readable storage media. When the programs are executed, they can include the processes of the above-mentioned method embodiments. The aforementioned storage media include: ROM or random access memory RAM, magnetic disk or optical disk and other media that can store program codes.

Claims

1. A vehicle-mounted road inspection detection method, characterized in that: Applied to an integrated vehicle-mounted road inspection and detection device, the method comprises: Acquire the road environment temperature difference, traffic volume and mainstream vehicle type of the current vehicle road, wherein the mainstream vehicle type is the vehicle type with the largest proportion of vehicles traveling on the current vehicle road, and is obtained by performing image recognition on the monitoring screen of the current vehicle road; Inputting the road environment temperature difference, the traffic volume and the mainstream vehicle type into a road loss calculation formula to obtain a road loss value within a preset time period, wherein the road loss value is the loss value that will be generated by the current vehicle road within the preset time period; Obtaining a preset loss threshold of the current vehicle road corresponding to the preset time period, and determining a road inspection frequency level of the current vehicle road according to a difference between the road loss value and the loss threshold; The current vehicle road is detected according to the road inspection frequency corresponding to the road inspection frequency level.

2. The method according to claim 1, characterized in that: The step of detecting the current vehicle road according to the road inspection frequency corresponding to the road inspection frequency level specifically includes: Acquiring a real-time road surface image of the current vehicle-mounted road according to the road inspection frequency; Performing image recognition on the real-time road surface image to obtain a road surface damage image of the real-time road surface image, wherein the road surface damage image includes newly added road surface damage; Comparing the pavement damage image with the historical pavement image to obtain the location information, type information and scale parameters of the newly added pavement damage; The point information, the type information and the scale parameter are sent to the client.

3. The method according to claim 1 or 2, characterized in that: After the step of detecting the current vehicle-mounted road according to the road inspection frequency corresponding to the road inspection frequency level, the method further includes: Obtain all road surface images of all vehicle-mounted roads in a preset area; Modeling is performed based on all the road surface images to obtain a road surface detection map within the preset area; Displaying the vehicle-mounted roads that have been inspected in the preset area in real time and visually to obtain real-time inspection map information; The real-time detection map information is sent to the client.

4. The method according to claim 1 or 2, characterized in that: After the step of detecting the current vehicle-mounted road according to the road inspection frequency corresponding to the road inspection frequency level, the method further includes: Acquire a traffic sign image on the current vehicle-mounted road; Matching the traffic sign image with a preset traffic sign database to determine whether the traffic sign image exists in the preset traffic sign database; If so, calculating the matching degree between the traffic sign image and a preset sign image, wherein the preset sign image is an image matching the traffic sign image in the preset traffic sign database; If the matching degree is lower than a preset matching degree threshold, a first prompt message including the traffic sign image is sent to the client.

5. The method according to claim 4, characterized in that After the step of matching the traffic sign image with a preset traffic sign database and determining whether the traffic sign image exists in the preset traffic sign database, the method further includes: If not present, the second prompt information including the traffic sign image is sent to the client.

6. The method according to claim 1, characterized in that After the step of detecting the current vehicle-mounted road according to the road inspection frequency corresponding to the road inspection frequency level, the method further includes: Obtain user feedback data within a preset time range from a preset feedback information database; Performing semantic analysis on the user feedback data to obtain frequency data of all vehicle-mounted roads appearing in the user feedback data, and obtaining a collection of frequency data; The vehicle-mounted road with the highest frequency in the frequency data collection is determined as a first road to be inspected, and the first road to be inspected is inspected.

7. The method according to claim 6, characterized in that After the step of performing semantic analysis on the user feedback data to obtain frequency data of all vehicle-mounted roads appearing in the user feedback data and obtaining a collection of frequency data, the method further includes: Adding all vehicle-mounted roads in the frequency data collection that are greater than a preset frequency threshold to a preset detection plan; All the vehicle-mounted roads are inspected according to the preset inspection plan in a preset future time period.

8. An integrated vehicle-mounted road inspection and detection device, characterized in that: The vehicle-mounted road patrol and detection integrated device includes: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the vehicle-mounted road patrol and detection integrated device to execute the method described in any one of claims 1-7.

9. A computer-readable storage medium comprising instructions, characterized in that: When the instruction is executed on the vehicle-mounted road patrol and detection integrated device, the vehicle-mounted road patrol and detection integrated device executes the method as described in any one of claims 1-7.

10. A computer program product, characterized in that When the computer program product runs on the vehicle-mounted road patrol and detection integrated device, the vehicle-mounted road patrol and detection integrated device executes the method as described in any one of claims 1-7.

Citation Information

Cited By

  • Vehicle equipment consumption prediction method and device, model training method and device, computer readable storage medium and electronic equipment

    CN119272807A

  • Road cleaning and maintenance environment assessment method

    CN119314137A

  • A road cleaning and maintenance environment assessment method

    CN119314137B

  • Rail transit protection area automatic patrol method and device based on artificial intelligence

    CN119992679A

  • An automatic inspection method and device for rail transit protection areas based on artificial intelligence

    CN119992679B