A method and system for identifying pavement diseases

Abstract

The present invention relates to a method and system for identifying road surface diseases. The method includes obtaining a first inspection image obtained when an inspection vehicle travels on a first main inspection lane; identifying road surface diseases on the first main inspection lane and the remaining recognizable lanes according to the first inspection image; marking areas with doubtful recognition results in the remaining recognizable lanes; identifying road surface diseases on the second main inspection lane and the remaining lanes that cannot be recognized in the first inspection image according to a second inspection image; wherein, if there are areas with doubtful recognition results in areas other than the second main inspection lane, supplementary road surface disease identification is performed on the areas with doubtful recognition results according to the second inspection image. It can perform supplementary identification on lanes that the inspection vehicle does not travel on, improve the completion degree of disease identification for the entire road, and is beneficial to improving the identification accuracy.

Description

Technical Field

[0001] The present invention relates to the field of pavement disease identification, and particularly to a pavement disease identification method and system. Background Art

[0002] In the work of maintaining urban roads, it is necessary to regularly inspect the roads to promptly detect pavement diseases and repair them. In the past, manual patrols along the roads were usually carried out to find areas with diseases. In order to efficiently identify pavement diseases, a method of identifying diseases by taking pictures along the way with inspection vehicles has been introduced. However, in order to save manpower and material resources, during the daily inspection process, inspection vehicles usually identify diseases in lanes at intervals, for example, alternately inspect lanes 1 and 3, and lanes 2 and 4. Although side cameras can be installed, due to complex environments and long distances, the accuracy of disease identification for lanes that the inspection vehicle does not drive on is low, and the identification difficulty is large, resulting in a low overall disease identification completion rate. Only by conducting higher-density inspections or waiting until the inspection vehicle is arranged to inspect these lanes during the next inspection can more accurate identification results with a higher completion rate be obtained. Summary of the Invention

[0003] The purpose of this application is to propose a pavement disease identification method and system to solve the problem of low accuracy and high difficulty in identifying diseases in lanes that the inspection vehicle does not drive on, resulting in a low overall disease identification completion rate for each inspection.

[0004] The pavement disease identification method of this application includes:

[0005] Obtain a first inspection image obtained when the inspection vehicle is driving on a first main inspection lane;

[0006] Based on the first inspection image, identify pavement diseases on the first main inspection lane and the remaining identifiable lanes;

[0007] Mark the areas with doubtful identification results among the remaining identifiable lanes;

[0008] When obtaining a second inspection image obtained when the inspection vehicle is driving on a second main inspection lane, determine whether there are areas with doubtful identification results in the areas outside the second main inspection lane at the position corresponding to the second inspection image;

[0009] Based on the second inspection image, identify pavement diseases on the second main inspection lane and the remaining lanes that cannot be identified in the first inspection image; wherein, if there are areas with doubtful identification results in the areas outside the second main inspection lane, supplementary pavement disease identification is carried out on the areas with doubtful identification results based on the second inspection image.

[0010] Further, before obtaining the second inspection image obtained when the inspection vehicle travels in the second main inspection lane, the following steps are also included:

[0011] When the inspection vehicle travels in the second main inspection lane, determine whether there is an area where the recognition result is in doubt at the current position of the inspection vehicle;

[0012] At the position where there is an area where the recognition result is in doubt, send an instruction to the image acquisition device on the inspection vehicle to obtain the second inspection image at a higher sampling frequency.

[0013] Further, the road surface disease recognition for supplementing the area where the recognition result is in doubt according to the second inspection image includes:

[0014] Fuse and enhance the areas where the recognition results are in doubt in the first inspection image and the second inspection image;

[0015] Perform road surface disease recognition on the image after the fusion and enhancement process to obtain a supplementary recognition result.

[0016] Further, the road surface disease recognition for supplementing the area where the recognition result is in doubt according to the second inspection image includes:

[0017] Obtain the first recognition result obtained by performing road surface disease recognition based on the first inspection image and the second recognition result obtained by performing road surface disease recognition based on the second inspection image;

[0018] Determine the shooting distances between the shooting positions of the first inspection image and the second inspection image and the area where the recognition result is in doubt;

[0019] Determine the final recognition result according to the change trend of the recognition results in the order of the shooting distances from far to near.

[0020] Further, the step of determining the final recognition result according to the change trend of the recognition results in the order of the shooting distances from far to near includes:

[0021] If it is judged that in the order of the shooting distances from far to near, the possibility of no disease in the recognition results of the road surface disease recognition increases, then modify the recognition result of the area where the recognition result is in doubt to no disease;

[0022] Conversely, modify the recognition result of the area where the recognition result is in doubt to having a disease.

[0023] Further, if it is judged that there is no area where the recognition result is in doubt in the area outside the second main inspection lane at the position corresponding to the second inspection image, then remove the area outside the second main inspection lane in the second inspection image that coincides with the first inspection image.

[0024] Further, before obtaining the second inspection image obtained when the inspection vehicle is traveling in the second main inspection lane, it includes:

[0025] When there are multiple pre-planned main inspection lanes, if there is a lane with a driving direction opposite to that of the first main inspection lane, then select the lane with a driving direction opposite to that of the first main inspection lane as the second main inspection lane.

[0026] Further, before obtaining the second inspection image obtained when the inspection vehicle is traveling in the second main inspection lane, it further includes:

[0027] Identify the lane to which the area with doubtful recognition result belongs;

[0028] Dynamically adjust the pre-planned second main inspection lane according to the number of areas with doubtful recognition results in each lane;

[0029] Send a notice of temporarily adjusting the main inspection lane to a predetermined device;

[0030] Among them, the rule for dynamically adjusting the pre-planned second main inspection lane includes:

[0031] Assign different weights to each area with doubtful recognition results according to the distance relationship between the lane and the inspection vehicle, where the farther the distance, the greater the weight;

[0032] Determine the total weight of the areas with doubtful recognition results when the adjacent lane of the pre-planned second main inspection lane is used as the main inspection lane;

[0033] When there is an adjacent lane whose total weight is smaller than the total weight of the pre-planned second main inspection lane and the difference in total weight exceeds the set threshold, use this adjacent lane as the actual second main inspection lane.

[0034] Further, the set threshold is determined based on the interval since the pre-planned second main inspection lane was last used as the main inspection lane. Among them, the longer the interval since the pre-planned second main inspection lane was last used as the main inspection lane, the larger the set threshold.

[0035] On the other hand, the present application also provides a road surface disease recognition system, including:

[0036] An image acquisition module, configured to acquire a first inspection image obtained when the inspection vehicle is traveling in the first main inspection lane;

[0037] A disease recognition module, configured to recognize road surface diseases on the first main inspection lane and the remaining recognizable lanes according to the first inspection image; and mark the areas with doubtful recognition results in the remaining recognizable lanes.

[0038] A judgment module, configured to, when obtaining a second inspection image obtained when an inspection vehicle is traveling in a second main inspection lane, judge whether there is a region with doubtful recognition results in the region outside the second main inspection lane at the position corresponding to the second inspection image;

[0039] The disease recognition module is further configured to perform road surface disease recognition on the second main inspection lane and the remaining lanes that cannot be recognized in the first inspection image according to the second inspection image; wherein, if there is a region with doubtful recognition results in the region outside the second main inspection lane, supplementary road surface disease recognition is performed on the region with doubtful recognition results according to the second inspection image.

[0040] The road surface disease recognition method provided by this application performs road surface disease recognition on the remaining recognizable lanes around the first main inspection lane, and marks the regions with doubtful recognition results among them; when obtaining a second inspection image obtained when an inspection vehicle is traveling in the second main inspection lane, judge whether there is a region with doubtful recognition results in the region outside the second main inspection lane at the position corresponding to the second inspection image; if so, supplementary road surface disease recognition is performed on the region with doubtful recognition results according to the second inspection image. It realizes the supplementary recognition of the lanes that the inspection vehicle does not travel on, improves the completion degree of disease recognition for the entire road, and is beneficial to improving the recognition accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 is a schematic flowchart of the basic process of the road surface disease recognition method of this embodiment;

[0042] Figure 2 is a schematic diagram of vehicle inspection of this embodiment;

[0043] Figure 3 is a schematic structural diagram of the road surface disease recognition system of this embodiment. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0044] In order to enable those skilled in the art to better understand the solution of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0045] It should be noted that the terms "first", "second", etc. in the description, claims and the above-mentioned drawings of the present invention are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present invention described here can be implemented in an order other than those illustrated or described here. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0046] Embodiment 1:

[0047] This embodiment provides a method for identifying road surface diseases. Refer to Figure 1 , and the method includes the following steps:

[0048] S101. Obtain a first inspection image obtained when the inspection vehicle travels on the first main inspection lane;

[0049] The first inspection image is an image obtained by an image acquisition device mounted on the inspection vehicle. During the process of identifying road surface diseases, arrange the inspection vehicle to drive on the road and take pictures of the image data of the road along the way. By analyzing these image data, the road surface diseases can be identified.

[0050] It should be noted that the main inspection lane is the lane on which the inspection vehicle travels. Since the inspection vehicle travels on the main inspection lane, the main content in the inspection image will relatively completely reflect the road conditions of the main inspection lane. However, even if there is an image acquisition device with a large shooting range on the inspection vehicle, due to the complex environment and distance of the actual road, the information quality of other lanes outside the main inspection lane in the inspection image will necessarily be lower than that of the main inspection lane. Therefore, the identification of road surface diseases in the remaining lanes is often low.

[0051] S102. According to the first inspection image, identify road surface diseases for the first main inspection lane and the remaining identifiable lanes;

[0052] Generally speaking, the identification of road surface diseases can be completed by a pre-trained deep learning model or other models. The specific model for this is not limited in this application. It should be understood that the identifiable lanes refer to other lanes that can be photographed in the first inspection image. For lanes that are not photographed or have a low picture ratio, they will be ignored during the process of identifying road surface diseases this time.

[0053] S103. Mark the areas with doubtful identification results in the remaining identifiable lanes;

[0054] If it is possible to determine the presence or absence of a disease area in the remaining recognizable lanes based on the first inspection image, the disease identification of this lane can be directly completed. Since the shooting position of the first inspection image is relatively far from these lanes, the recognition accuracy is relatively lower. In this embodiment, the areas with doubtful results are marked.

[0055] It should be understood that when determining whether there is a road surface disease in identification methods such as deep learning models, it is usually based on the probability score output by the model. As an example, when the probability score of the model output indicating the presence of a road surface disease is higher than the set value, it is considered that there is a road surface disease in the corresponding area; otherwise, it is considered that there is no road surface disease in the corresponding area.

[0056] Exemplarily, in this embodiment, a probability score interval for doubtful recognition results can be set. When the probability score output by the model is within this probability score interval for doubtful recognition results, it is considered that the recognition result of the road surface disease in this area is doubtful. For example, if this probability score interval is 50%-70%, it means that when the probability score of the model output indicating the presence of a road surface disease is below 50%, the recognition result is that there is no road surface disease; when the probability score is above 70%, the recognition result is that there is a road surface disease; and if the probability score is within the interval of 50%-70%, the corresponding area is marked as having a doubtful recognition result. In practical applications, the type of the disease can also be recognized simultaneously. For example, cracks are 80%, collapses are 15%... Then the recognition result is a disease of the crack type. For the sake of simplicity, the types of diseases are not expanded in the embodiments of this application.

[0057] S104. When the second inspection image obtained when the inspection vehicle is driving in the second main inspection lane is acquired, determine whether there is an area with a doubtful recognition result in the area outside the second main inspection lane at the position corresponding to the second inspection image;

[0058] The second main inspection lane is a lane different from the first main inspection lane. It can be understood that during the process of road surface disease identification, the shooting position of the inspection image is known, which can be obtained through positioning means such as GPS on the inspection vehicle or the equipment carried by the inspection personnel. And when marking the areas with diseases or suspected diseases, the positions of these disease areas will be recorded.

[0059] S105. According to the second inspection image, perform road surface disease identification on the second main inspection lane and the remaining lanes that cannot be recognized in the first inspection image; and perform supplementary road surface disease identification on the areas with doubtful recognition results according to the second inspection image;

[0060] Regions with doubtful recognition results will be recognized again, which can improve the completion and accuracy of disease recognition. The specific method of supplementary road surface disease recognition based on the second inspection image can be the same as that based on the first inspection image; in some implementation processes, different recognition methods can also be adopted compared with the disease recognition based on the first inspection image.

[0061] In practical applications, if there are no regions with doubtful recognition results, then during the process of disease recognition based on the second inspection image, the lanes for which definite recognition results have been obtained when the inspection vehicle is driving in the first main inspection lane can be ignored. During the process of the inspection vehicle driving in the second main inspection lane, road surface disease recognition is mainly carried out on the second main inspection lane and the lanes that cannot be recognized in the first inspection image, which can save computing power and avoid repeated recognition of the same region. That is, if there are no regions with doubtful recognition results, then road surface disease recognition is only carried out on the second main inspection lane and the lanes that cannot be recognized in the first inspection image; when there are regions with doubtful recognition results in the regions outside the second main inspection lane in the second inspection image, supplementary road surface disease recognition is also carried out on the regions with doubtful recognition results.

[0062] For the sake of easy understanding, refer to Figure 2 , and it will be described in combination with a specific lane example. In this example, a two-way 6-lane (3 lanes in one direction) is taken as an example, but the road surface disease recognition method of this embodiment is applicable to any number of lanes and lane directions more than two. For the sake of easy explanation, with reference to the illustrated direction, from left to right are lane 1, lane 2... lane 6 respectively, and the illustrated arrows indicate the lane directions.

[0063] In the actual inspection process, it is often not the case that all lanes are inspected in a single inspection. Exemplarily, the main inspection lanes for a single inspection may be any one of the groups of lane 1 and lane 4, lane 2 and lane 5, lane 3 and lane 6; in the next inspection, the main inspection lanes can be alternately changed so that after a certain number of inspections, all lanes will serve as the main inspection lane at least once. In this example, lane 2 and lane 5 are taken as the main inspection lanes, where lane 2 is the first main inspection lane and lane 5 is the second main inspection lane.

[0064] Continue to refer to Figure 2As shown, the inspection vehicle 1 can usually obtain images within a certain range in the front and / or rear (or plus the sides). For simplicity of explanation, in this example, only the case where the inspection vehicle 1 takes pictures forward is used, and the actual shooting direction and range can be arbitrary. When the inspection vehicle 1 is driving on the first main inspection lane, the obtained first inspection image includes the image data of the first main inspection lane (Lane 2). At the same time, the first inspection image will also include other lanes around the first main inspection lane. For example, Lanes 1, 3, 4, and 5 are also photographed to a certain extent; it is assumed that Lane 6 cannot be recognized due to the long distance (in actual applications, the actually recognizable lanes will be determined based on the actual shooting situation). Figure 2 The shooting range is indicated by a slant line in the figure.

[0065] In this example, when identifying road surface diseases based on the first inspection image, the road surfaces of Lanes 1, 2, 3, 4, and 5 are all identified for road surface diseases. Since in the image information of the first inspection image, the information of the remaining lanes is less than that of the first main inspection lane, and its accuracy is relatively low, the areas with doubtful recognition results in the remaining lanes (Lanes 1, 3, 4, 5) are marked. Figure 2 In the example of the figure, the areas with doubtful recognition results are indicated by circles.

[0066] Obtain the second inspection image obtained when the inspection vehicle is driving on the second main inspection lane. In actual applications, the inspection of the second main inspection lane can be carried out by the same inspection vehicle. For example, after the inspection vehicle has completed the inspection drive on the first main inspection lane, it turns around to the second main inspection lane and drives back in the reverse direction; it can also be carried out by different inspection vehicles for the second main inspection lane. For example, two inspection vehicles are sent out simultaneously for inspection respectively.

[0067] Based on the second inspection image itself or the positioning device (GPS) on the inspection vehicle, etc., the position when the second inspection image is taken can be determined. As Figure 2 shown, when the inspection vehicle is at a certain position in Lane 5, there will be areas with doubtful recognition results in its second inspection image, and these areas with doubtful recognition results are marked after the road surface disease recognition of the first inspection image. Based on the second inspection image, supplementary road surface disease recognition is carried out on the areas with doubtful recognition results in Lanes 3 and 4 to try to obtain a more accurate recognition result. Since Lane 5 is the second main inspection lane, normal road surface disease recognition will be carried out on Lane 5, and the areas with doubtful recognition results in Lane 5 will not be referred to during this process.

[0068] If there are no areas with doubtful recognition results in the second inspection image, only the road surface diseases of Lane 5 and Lane 6 that cannot be effectively photographed by the first inspection image will be recognized.

[0069] Of course, in other embodiments, the driving direction of the second main inspection lane may be the same as that of the first main inspection lane. For example, the first main inspection lane and the second main inspection lane are lane 1 and lane 3 respectively.

[0070] In some embodiments, before obtaining the second inspection image obtained when the inspection vehicle is driving in the second main inspection lane, it further includes:

[0071] S201. When the inspection vehicle is driving in the second main inspection lane, determine whether there is an area where the recognition result is in doubt at the current position of the inspection vehicle;

[0072] S202. At the position of the area where the recognition result is in doubt, send an instruction to the image acquisition device on the inspection vehicle to obtain the second inspection image at a higher sampling frequency;

[0073] The image acquisition device, such as a camera or a camera terminal, is a device used to capture inspection images. It should be understood that the process of identifying road surface diseases based on the second inspection image can be real-time or based on stored image data afterwards. And the above step of sending an instruction to the image acquisition device on the inspection vehicle to obtain the second inspection image at a higher sampling frequency is executed in real time during the inspection process of the inspection vehicle. Among them, the device for sending the instruction can be a terminal on the inspection vehicle for controlling devices such as the image acquisition device, or a server in the cloud; the road surface disease recognition method of this embodiment can be executed by a terminal carried on the inspection vehicle or by a server in the cloud.

[0074] In practical applications, if there are multiple image acquisition devices on the inspection vehicle, it is also possible to only control some of the image acquisition devices to obtain the second inspection image at a higher sampling frequency (the second inspection image here can be understood as a collection of images obtained by multiple image acquisition devices). For example, only control the image acquisition device whose shooting direction is towards the lane where the area with doubtful recognition result is located to obtain the second inspection image at a higher sampling frequency.

[0075] Controlling the image acquisition device to obtain the second inspection image at a higher sampling frequency will be beneficial to the accurate recognition of the area where the recognition result is in doubt, and only increasing the sampling frequency when there is an area where the recognition result is in doubt is beneficial to saving resources such as storage, communication, and computing power.

[0076] In some embodiments, the supplementary road surface disease recognition of the area where the recognition result is in doubt according to the second inspection image includes:

[0077] S301. Perform fusion enhancement processing on the areas where the recognition results are in doubt in the first inspection image and the second inspection image;

[0078] The fusion enhancement process includes, but is not limited to, fusing and splicing the clear parts of the two images, combining the complementary and redundant information in the first inspection image and the second inspection image, and generating a comprehensive image that contains all the key details and has enhanced features. Additionally, the shooting directions of the first inspection image and the second inspection image can be different (i.e., the inspection vehicle shoots in lanes with different driving directions), so there will be a large difference in the shooting angles between the first inspection image and the second inspection image. The three-dimensional information of the area where the recognition result is doubtful can be simulated based on the first inspection image and the second inspection image obtained from two opposite directions, which can also increase the accuracy of road surface disease recognition.

[0079] S302. Perform road surface disease recognition based on the image after fusion enhancement to obtain a supplementary recognition result;

[0080] Since the first inspection image and the second inspection image are different in terms of shooting time, angle, etc., their fusion will be able to reflect more information. By performing road surface disease recognition after integrating the image information of the two, the accuracy of road surface disease recognition can be improved in some implementation processes.

[0081] As another implementation method, the supplementary road surface disease recognition for the area where the recognition result is doubtful based on the second inspection image includes:

[0082] S401. Obtain the first recognition result obtained by performing road surface disease recognition based on the first inspection image and the second recognition result obtained by performing road surface disease recognition based on the second inspection image;

[0083] S402. Determine the shooting distances between the shooting positions of the first inspection image and the second inspection image and the area where the recognition result is doubtful;

[0084] S403. Determine the final recognition result according to the change trend of the recognition results in the order of the shooting distances from far to near;

[0085] Among them, the above step S403 specifically includes:

[0086] If it is judged that in the order of the shooting distances from far to near, the possibility of no disease in the recognition results of road surface disease recognition increases, then modify the recognition result of the area where the recognition result is doubtful to no disease; otherwise, modify the recognition result of the area where the recognition result is doubtful to having disease.

[0087] For ease of understanding, the following will be described with an example. As mentioned before, when identifying road surface diseases, probability scores output by relevant models can be obtained. The above methods respectively obtain probability scores (i.e., the first recognition result and the second recognition result) output based on different inspection images. Assume that the probability score of the existence of diseases in the first recognition result is greater than that in the second recognition result, and the shooting position of the first inspection image is farther from the area with doubtful recognition results. Then it indicates that as the distance gets closer, the change trend of the recognition result approaches the non-existence of diseases. Therefore, the recognition result of the area with doubtful recognition results can be modified to the non-existence of diseases. In this embodiment, the recognition result can be adjusted based on the change trend of the probability score, which has a lower requirement for the accuracy of the inspection image and saves more resources.

[0088] To save resources, if it is determined that there is no area with doubtful recognition results in the area outside the second main inspection lane at the position corresponding to the second inspection image, then the area outside the second main inspection lane in the second inspection image that overlaps with the first inspection image is removed. This embodiment removes based on the existence of areas with doubtful recognition results, without the need to compare and remove duplicates from the images. Therefore, there is no need for a complex image processing process, which can occupy less computing power resources while saving communication and storage resources.

[0089] It should be understood that during the inspection process, there may be more than two main inspection lanes, and the driving directions of some main inspection lanes are the same. When supplementing the recognition results of the areas with doubtful recognition results for road surface disease recognition, any one of the main inspection lanes outside the first main inspection lane can be selected as the second main inspection lane.

[0090] In some implementation processes, in order to further improve the complementary degree of information of the second inspection image for the areas with doubtful recognition results, before obtaining the second inspection image obtained when the inspection vehicle is driving in the second main inspection lane, it includes: when there are multiple pre-planned main inspection lanes, if there is a lane with a driving direction opposite to that of the first main inspection lane, then select the lane with a driving direction opposite to that of the first main inspection lane as the second main inspection lane. Using the lane with the opposite driving direction as the second main inspection lane, that is, selecting the inspection image with the opposite shooting direction as the second inspection image, can supplement the information of the areas with doubtful recognition results from different shooting angles and reflect more three-dimensional information of the areas with doubtful recognition results in some implementation processes.

[0091] In some embodiments, to better ensure the completion degree of road surface disease recognition, before obtaining the second inspection image obtained when the inspection vehicle is driving in the second main inspection lane, it further includes

[0092] S501. Identify the lane to which the area with doubtful recognition results belongs;

[0093] S502. Dynamically adjust the pre-planned second main inspection lane according to the number of regions with doubtful recognition results in each lane;

[0094] For ease of explanation, as an example, assume that the first main inspection lane is lane 3 and the pre-planned second main inspection lane is lane 6. There are relatively many regions with doubtful recognition results in lane 4. In order to provide a better-quality supplementary pavement disease recognition for the regions with doubtful recognition results on lane 4 to improve the overall recognition accuracy, it can be selected to temporarily adjust the second main inspection lane to lane 5, so that the position for shooting the second inspection image is closer to lane 4, in order to improve the recognition effect on the regions with doubtful recognition results on lane 4.

[0095] In practical applications, specific judgment rules for dynamically adjusting the pre-planned second main inspection lane can be formulated according to requirements.

[0096] S503. Send a notice of temporarily adjusting the main inspection lane to a predetermined device;

[0097] This device can be an in-vehicle device or a device such as the mobile phone of the inspection personnel. The purpose is to enable the inspection personnel to know that the main inspection lane has been temporarily adjusted, so that the inspection personnel can conduct inspections according to the adjusted main inspection lane.

[0098] The above steps S501 to S503 can be executed before the inspection vehicle conducts inspections on the second main inspection lane. For example, it can be judged and dynamically adjusted in real time during the inspection process of the first main inspection lane.

[0099] As a specific example, the steps for dynamically adjusting the pre-planned second main inspection lane may include:

[0100] S5021. Assign different weights to each region with doubtful recognition results according to the distance relationship between the lane and the inspection vehicle, where the farther the distance, the greater the weight;

[0101] For ease of understanding, continue to take the first main inspection lane as lane 3 and the pre-planned second main inspection lane as lane 6 as an example. As a simplification, based on the interval distance from lane 6, assume that the weight of the region with doubtful recognition results in lane 5 is 1, the weight of the region with doubtful recognition results in lane 4 is 2, there is no region with doubtful recognition results in lane 3 as the first main inspection lane, the weight of the region with doubtful recognition results in lane 2 is 4, and the weight of the region with doubtful recognition results in lane 1 is 5.

[0102] S5022. Determine the total weight of all regions with doubtful recognition results when the adjacent lane of the pre-planned second main inspection lane is used as the main inspection lane;

[0103] Suppose in a certain section of the road, the numbers of areas with doubtful recognition results in Lane 1, Lane 2, Lane 4, Lane 5, and Lane 6 are 1, 5, 0, 2, and 3 respectively.

[0104] Then when Lane 6 (the pre-planned second main inspection lane) is used as the main inspection lane, the total weight value is: 1*5 + 5*4 + 0*2 + 2*1 + 3*0 = 27.

[0105] Similarly, when Lane 5 (the adjacent lane of the pre-planned second main inspection lane) is used as the main inspection lane, the total weight value is: 1*4 + 5*3 + 0*1 + 2*0 + 3*1 = 22.

[0106] S5023. When there is an adjacent lane whose total weight value is smaller than that of the pre-planned second main inspection lane and the difference in the total weight value exceeds the set threshold, take this adjacent lane as the actual second main inspection lane;

[0107] Suppose the set threshold is 3. In the above example, the difference between the total weight value of Lane 5 and that of Lane 6 is 5, which exceeds the set threshold of 3. Then take Lane 5 as the actual second main inspection lane, that is, send a notice to the predetermined device to temporarily adjust the second main inspection lane to Lane 5. It can be seen that this method can make the main inspection lane move towards the lane close to the areas with more doubtful recognition results temporarily when there are more areas with doubtful recognition results in the lanes far from the second main inspection lane, so as to better obtain the image information of the areas with doubtful recognition results, thereby improving the accuracy of the supplementary road surface disease recognition.

[0108] In practical applications, with the change of the distance relationship between the lane and the inspection vehicle, the weight value change of the areas with doubtful recognition results in the other lanes can be non-linear. The above only takes the linear weight value change as an example for understanding.

[0109] The above example can dynamically determine whether to adjust the main inspection lane to the adjacent lane of the pre-planned second main inspection lane based on the weight value of the areas with doubtful recognition results. It can not only improve the accuracy of supplementary recognition of the areas with doubtful recognition results to a certain extent, but also will not cause the new second main inspection lane to deviate too much from the pre-planned second main inspection lane, and still can ensure relatively accurate road surface disease recognition of the pre-planned second main inspection lane.

[0110] In some embodiments, the above set threshold is determined based on the interval when the pre-planned second main inspection lane was used as the main inspection lane last time. Among them, the longer the interval since the adjacent lane was used as the main inspection lane last time, the larger the set threshold.

[0111] That is to say, the above-set threshold can also be dynamically adjusted in practical applications. If the pre-planned second main inspection lane has not been selected as the main inspection lane for a long time for inspection, the value of the set threshold is increased to avoid the easy change of the pre-planned second main inspection lane. This can ensure that each lane can be selected as the main inspection lane after a certain period of time, and prevent some lanes from being unable to serve as the main inspection lane for a long time, resulting in inaccurate pavement disease identification results. As an assumption of an extreme case, when the interval since the last time the pre-planned second main inspection lane served as the main inspection lane exceeds the set time, the set threshold can be adjusted to a very large value to ensure that the next second main inspection lane will not be adjusted.

[0112] This embodiment also provides a pavement disease identification system 200, as Figure 3 shown, which includes an image acquisition module 201, a disease identification module 202, and a judgment module 203.

[0113] The image acquisition module 201 is used to acquire the first inspection image obtained when the inspection vehicle travels on the first main inspection lane.

[0114] The disease identification module 202 is used to identify pavement diseases on the first main inspection lane and the remaining recognizable lanes according to the first inspection image; and mark the areas with doubtful identification results among the remaining recognizable lanes.

[0115] The judgment module 203 is used to judge whether there are areas with doubtful identification results in the area outside the second main inspection lane at the position corresponding to the second inspection image when the second inspection image obtained when the inspection vehicle travels on the second main inspection lane is acquired.

[0116] Moreover, the disease identification module 202 is also used to perform supplementary pavement disease identification on the areas with doubtful identification results according to the second inspection image when there are areas with doubtful identification results in the area outside the second main inspection lane.

[0117] In addition, the specific steps that the pavement disease identification system 200 of this embodiment can execute can also refer to the steps of the pavement disease identification method provided above in this embodiment, which will not be elaborated in this embodiment.

[0118] In addition, although exemplary embodiments have been described herein, their scope includes any and all embodiments based on the present disclosure that have equivalent, modified, omitted, combined (e.g., solutions that cross various embodiments), adapted, or changed. It is not limited to the examples described in this specification or during the implementation of this application, and the examples will be interpreted as non-exclusive.

[0119] The above description is intended to be illustrative and not restrictive. For example, the above examples (or one or more aspects thereof) can be used in combination with each other. For example, those of ordinary skill in the art can use other embodiments when reading the above description.

[0120] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, various modifications and variations can be made to the embodiments of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for identifying road surface defects, characterized in that: include: Acquire a first inspection image obtained when the inspection vehicle is traveling in the first main inspection lane; According to the first inspection image, road surface disease identification is performed on the first main inspection lane and other identifiable lanes; Mark the areas in the remaining identifiable lanes where the recognition results are questionable; When a second inspection image obtained when the inspection vehicle is traveling in the second main inspection lane is obtained, it is determined whether there is an area outside the second main inspection lane at a position corresponding to the second inspection image where the recognition result is questionable; According to the second inspection image, road surface defects are identified for the second main inspection lane and the remaining lanes that cannot be identified in the first inspection image; wherein, if there is an area outside the second main inspection lane where the identification result is questionable, supplementary road surface defects identification is performed on the area where the identification result is questionable according to the second inspection image; If it is determined that at the position corresponding to the second inspection image, there is no area outside the second main inspection lane where the recognition result is questionable, then the area outside the second main inspection lane in the second inspection image that overlaps with the first inspection image is removed; Before acquiring the second inspection image obtained when the inspection vehicle is traveling in the second main inspection lane, the method further includes: Identifying the lane to which the area in question belongs; According to the number of areas in each lane where the recognition results are questionable, dynamically adjust the pre-planned second main inspection lane; Send a notice to the scheduled equipment to temporarily adjust the main inspection lane; The rules for dynamically adjusting the pre-planned second main inspection lane include: Different weights are assigned to the areas where the recognition results are questionable according to the distance relationship between the lane and the inspection vehicle, wherein the farther the distance, the greater the weight; When determining the adjacent lane of the pre-planned second main inspection lane as the main inspection lane, the sum of weights of the area where the recognition result is in doubt; When there is an adjacent lane whose total weight is smaller than the total weight of the pre-planned second main inspection lane and whose total weight difference exceeds a set threshold, the adjacent lane is used as the actual second main inspection lane.

2. The method for identifying road damage according to claim 1, characterized in that: Before acquiring the second inspection image obtained when the inspection vehicle is traveling in the second main inspection lane, the method further includes: When the inspection vehicle is traveling in the second main inspection lane, determining whether there is an area where the recognition result is questionable at the current position of the inspection vehicle; At the location where the area where the recognition result is questionable exists, an instruction is sent to the image acquisition device on the inspection vehicle to acquire the second inspection image at a higher sampling frequency.

3. The road surface disease identification method according to claim 2, characterized in that: The road surface disease recognition of supplementing the area where the recognition result is questionable according to the second inspection image includes: Performing fusion enhancement processing on the areas in the first inspection image and the second inspection image where the recognition results are questionable; Pavement disease identification is performed based on the fused enhanced image to obtain supplementary identification results.

4. The method for identifying road damage according to claim 2, characterized in that: The road surface disease recognition of supplementing the area where the recognition result is questionable according to the second inspection image includes: Acquire a first recognition result obtained by identifying road surface defects based on the first inspection image, and a second recognition result obtained by identifying road surface defects based on the second inspection image; Determining the shooting distances between the shooting positions of the first inspection image and the second inspection image and the area where the recognition result is in doubt; The final recognition result is determined according to the changing trend of the recognition results in the order of shooting distance from far to near.

5. The road surface disease identification method according to claim 4, characterized in that: The step of determining the final recognition result according to the change trend of the recognition result in the order of shooting distance from far to near includes: If it is determined that the probability of no road disease in the road disease recognition result increases in the order of shooting distance from far to near, the recognition result of the area where the recognition result is questionable is modified to no disease; Otherwise, the recognition result of the area where the recognition result is questionable is modified to indicate that there is a disease.

6. The method for identifying road damage according to claim 1, characterized in that: Before obtaining the second inspection image obtained when the inspection vehicle is traveling in the second main inspection lane, the following steps are included: When there are multiple pre-planned main inspection lanes, if there is a lane with a driving direction opposite to that of the first main inspection lane, the lane with a driving direction opposite to that of the first main inspection lane is selected as the second main inspection lane.

7. The pavement disease identification method according to claim 1, characterized in that: The set threshold is determined based on the interval from the last time the pre-planned second main inspection lane was used as the main inspection lane, wherein the longer the interval from the last time the pre-planned second main inspection lane was used as the main inspection lane, the larger the set threshold.

8. A road surface disease identification system, characterized in that: include: An image acquisition module, used to acquire a first inspection image obtained when the inspection vehicle travels in the first main inspection lane; a disease recognition module, configured to recognize road surface diseases on the first main inspection lane and other identifiable lanes according to the first inspection image; And mark the areas in the remaining identifiable lanes where the recognition results are questionable; a judgment module, for, when acquiring a second inspection image obtained when the inspection vehicle is traveling in the second main inspection lane, judging whether there is an area outside the second main inspection lane at a position corresponding to the second inspection image where the recognition result is questionable; The disease recognition module is further used to perform road disease recognition on the second main inspection lane and the remaining lanes that cannot be recognized in the first inspection image according to the second inspection image; wherein, if there is an area outside the second main inspection lane where the recognition result is questionable, supplementary road disease recognition is performed on the area where the recognition result is questionable according to the second inspection image; If it is determined that at the position corresponding to the second inspection image, there is no area outside the second main inspection lane where the recognition result is questionable, then the area outside the second main inspection lane in the second inspection image that overlaps with the first inspection image is removed; Before acquiring the second inspection image obtained when the inspection vehicle is traveling in the second main inspection lane, the method further includes: Identifying the lane to which the area in question belongs; According to the number of areas in each lane where the recognition results are questionable, dynamically adjust the pre-planned second main inspection lane; Send a notice to the scheduled equipment to temporarily adjust the main inspection lane; The rules for dynamically adjusting the pre-planned second main inspection lane include: Different weights are assigned to the areas where the recognition results are questionable according to the distance relationship between the lane and the inspection vehicle, wherein the farther the distance, the greater the weight; When determining the adjacent lane of the pre-planned second main inspection lane as the main inspection lane, the sum of weights of the area where the recognition result is in doubt; When there is an adjacent lane whose total weight is smaller than the total weight of the pre-planned second main inspection lane and whose total weight difference exceeds a set threshold, the adjacent lane is used as the actual second main inspection lane.

Images