Method and related device for road surface inspection maintenance

Abstract

The application discloses a road pavement inspection and maintenance method and a related device. The method comprises the following steps: acquiring multiple frames of road images of a to-be-detected road; dividing the to-be-detected road into multiple regions according to the multiple frames of road images; selecting a detection point from each region of the multiple regions to obtain a target detection point corresponding to each region; marking each region of the multiple regions according to the target detection point corresponding to each region to obtain multiple pavement type identifiers; and determining a pavement maintenance scheme corresponding to each region according to the pavement type identifiers. The application is beneficial to improving the use performance and service level of the road.

Description

Methods and related equipment for road surface inspection and maintenance Technical Field

[0001] This application relates to the field of road surface inspection and maintenance technology, specifically to a method and related apparatus for road surface inspection and maintenance. Background Technology

[0002] As society develops, the demands on road services are increasing. To maintain roads in good condition and ensure economical operation, regular maintenance and repairs are required. Typically, a comprehensive overhaul is chosen, but this approach involves significant work, causing traffic disruptions such as detours or congestion, and resulting in low service performance.

[0003] Therefore, a method for road surface inspection and maintenance is urgently needed as an important task to improve the performance and service level of roads. Summary of the Invention

[0004] This application provides a method and related apparatus for road surface inspection and maintenance, which divides the road to be inspected into different areas to achieve regional road surface maintenance, thereby improving the road's performance and service level.

[0005] In a first aspect, embodiments of this application provide a method for road surface inspection and maintenance, applied to a road surface inspection and maintenance system, the method comprising:

[0006] Acquire multiple frames of road images of the road to be detected;

[0007] The road to be detected is divided into multiple regions based on the multi-frame road images;

[0008] Detection points are selected from each of the plurality of regions to obtain the target detection points corresponding to each region;

[0009] Based on the target detection points corresponding to each region, each region in the plurality of regions is marked to obtain multiple road surface type identifiers;

[0010] Based on the road surface type identifier, determine the corresponding road surface maintenance plan for each area.

[0011] Secondly, embodiments of this application provide a road surface inspection and maintenance apparatus, applied to a road surface inspection and maintenance system. The apparatus includes: an acquisition unit, a division unit, a selection unit, a marking unit, and a determination unit, wherein...

[0012] The acquisition unit is used to acquire multiple frames of road images of the road to be detected;

[0013] The segmentation unit is used to divide the road to be detected into multiple regions based on the multi-frame road images;

[0014] The selection unit is used to select a detection point from each of the plurality of regions to obtain the target detection point corresponding to each region.

[0015] The marking unit is used to mark each of the multiple regions according to the target detection point corresponding to each region, so as to obtain multiple road surface type identifiers;

[0016] The determining unit is used to determine the road maintenance plan corresponding to each area based on the road surface type identifier.

[0017] Thirdly, embodiments of this application provide an electronic device, including a processor, a memory, a communication interface, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor, and the programs include instructions for performing some or all of the steps described in the first aspect of embodiments of this application.

[0018] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program for electronic data interchange, wherein the computer program causes a computer to perform some or all of the steps described in the first aspect of embodiments of this application.

[0019] Fifthly, embodiments of this application provide a computer program product, wherein the computer program product includes a non-transitory computer-readable storage medium storing a computer program operable to cause a computer to perform some or all of the steps described in the first aspect of embodiments of this application. The computer program product may be a software installation package.

[0020] Implementing the embodiments of this application has the following beneficial effects: acquiring multiple frames of road images of the road to be inspected; dividing the road to be inspected into multiple regions based on the multiple frames of road images; selecting detection points from each of the multiple regions to obtain target detection points corresponding to each region; marking each of the multiple regions based on the target detection points corresponding to each region to obtain multiple road surface type identifiers; and determining the road surface maintenance plan corresponding to each region based on the road surface type identifiers. In this way, the inspection work of the road to be inspected can be realized through target detection points, and the road to be inspected can be divided into different regions to achieve regional road surface maintenance, which is beneficial to improving the road's performance and service level. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 is a flowchart illustrating a method for road surface inspection and maintenance provided in an embodiment of this application;

[0023] Figure 2 is a schematic diagram illustrating the relationship between a camera plane and a focusing plane according to an embodiment of this application;

[0024] Figure 3 is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0025] Figure 4 is a functional unit block diagram of a road surface inspection and maintenance device provided in an embodiment of this application. Detailed Implementation

[0026] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0027] The terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but in one possible example includes steps or units not listed, or in one possible example includes other steps or units inherent to these processes, methods, products, or apparatuses.

[0028] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0029] The electronic devices involved in the embodiments of this application may include various handheld devices, vehicle-mounted devices, wearable devices, computing devices or other processing devices connected to a wireless modem with wireless communication capabilities, as well as various forms of user equipment (UE), mobile station (MS), terminal device, etc. For ease of description, the devices mentioned above are collectively referred to as electronic devices.

[0030] In one possible example, the aforementioned electronic device can be used to manage a road surface inspection and maintenance system, or to acquire relevant data from that system. The road surface inspection and maintenance system may include multiple cameras, each capable of adjusting its focusing plane to adjust the imaging plane. Multiple cameras can be used to acquire or capture multiple frames of road images corresponding to multiple roads within a given area, with each road potentially corresponding to at least one image; alternatively, multiple cameras can be used to capture multiple frames of road images from different sections of the same road. The specific camera layout or shooting method is not limited here.

[0031] It should be noted that in this application, "multiple" can refer to two or more.

[0032] Please refer to Figure 1, which is a flowchart illustrating a method for road surface inspection and maintenance according to an embodiment of this application; applied to electronic devices, the method includes:

[0033] S101. Obtain multiple frames of road images of the road to be detected.

[0034] The road to be detected may include multiple cameras, each of which can be used to capture road images of multiple road segments within the road to be detected.

[0035] The aforementioned roads to be detected can also correspond to multiple roads; for example, multiple frames of road images corresponding to multiple roads within a certain range can be obtained, and each road can correspond to multiple frames of road images captured by at least one camera.

[0036] In one possible example, the method for acquiring multiple frames of road images of the road to be detected may include the following steps: determining a shooting target during each capture of the road image; determining a shooting angle corresponding to each shooting target; determining a plane adjustment parameter corresponding to each road image based on the shooting angle, wherein the plane adjustment parameter is determined relative to the camera plane angle; determining a focal plane corresponding to the road image based on the plane adjustment parameter; and acquiring the road image on the focal plane.

[0037] The aforementioned shooting targets may refer to road shoulders, roadside fences, etc. under normal conditions in road images, or a short section of smooth road, such as a smooth section of road without potholes or puddles; there may be multiple shooting targets.

[0038] The shooting angle can be set by the system default or by the user; the shooting angle can also be the angle corresponding to the last shooting.

[0039] In full-area shooting, since cameras are generally positioned high, it's often difficult to obtain a clear image of the entire target when it's not parallel to the camera's plane. Therefore, it's necessary to determine the plane adjustment parameters for each camera when capturing road images. These parameters represent the angle between the camera's plane and the focusing plane. Adjusting the focusing plane based on these parameters allows for clearer road images. This helps to clearly identify the brightness and darkness of pixels used to mark different road sections, thus improving the accuracy of subsequent road maintenance plans for different areas.

[0040] For example, the angle formed by the camera plane and the focal plane described above can be used to indicate that they are parallel.

[0041] In determining the planar adjustment parameters for each road image based on the shooting angle, a standard image of the target captured by the camera can be obtained. This standard image is taken when the target is not deformed due to the shooting angle during full-area shooting (e.g., when the target is deformed due to the camera being lowered). By comparing the contours of the target in the two standard images and the road image to be shot, the shooting deformation values ​​over time are determined. These deformation values ​​indicate the deformation angle, degree of deformation, etc. Furthermore, the planar adjustment parameters can be determined based on the shooting deformation values, i.e., the angle adjustment parameters between the camera plane and the focusing plane.

[0042] For example, as shown in Figure 2, it is a schematic diagram of the relationship between the camera plane and the focusing plane. The angle difference between the camera plane and the focusing plane can be used as a plane adjustment parameter. The relative position between the camera plane and the focusing plane can be adjusted through this plane adjustment parameter, thereby adjusting the image through the focusing plane.

[0043] As can be seen in this example, the electronic device takes into account the deformation of the road image caused by the angle between the camera plane and the focal plane during the camera shooting process. When acquiring multiple frames of road images of the road to be detected, the focal plane can be adjusted to obtain a more accurate road image, which is beneficial to improving the accuracy of subsequent road maintenance plans for different areas.

[0044] S101. Divide the road to be detected into multiple regions based on the multi-frame road images.

[0045] Each of the aforementioned regions may correspond to different road segments.

[0046] In one possible example, dividing the road to be detected into multiple regions based on the multiple road images includes the following steps: combining the multiple road images according to the focal plane corresponding to each road image to obtain a first image; determining the focal range of each shooting target in the corresponding road image; selecting images in the first image that include shooting targets whose focal range is greater than or equal to a preset value as a second image; dividing the second image into multiple target images according to shooting targets whose focal range is greater than or equal to the preset value, each target image corresponding to one region, and each region including at least one shooting target.

[0047] Because different cameras may have different shooting angles, resulting in overlapping road segments or inconsistent focal planes when shooting the same road, leading to inconsistent road images corresponding to the same target (for example, different color and brightness parameters for the same target may cause the system to identify them as two different targets), multiple road images can be combined based on the focal plane of each road image to merge identical road segments. During the merging process, the inconsistency in the angular relationship between the focal plane and the camera plane can be considered. Based on the relationship between the camera plane and the focal plane for each camera, the pixel values ​​in the road image can be adjusted to alleviate the inconsistency in road images corresponding to the same target caused by inconsistent focal planes between two road images.

[0048] When determining the focal range of each shooting target in the corresponding road image, the electronic device can divide the first image into multiple sub-regions around each shooting target. Each sub-region includes the shooting target. Then, the variance of each sub-region can be evaluated by the variance function to obtain the image gray value transformation range. The image gray value transformation range characterizes the image clarity. All sub-regions with variances greater than the preset variance (large image gray value transformation range) are taken as the focal range corresponding to the shooting target.

[0049] Optionally, when determining the focal range of each captured target in the corresponding road image, the electronic device can also obtain it based on the information entropy evaluation algorithm; the specific implementation method is not limited here.

[0050] The aforementioned focus range can be used to indicate the focus status of the captured road images. For example, if the focus range is smaller than a preset value, it indicates that the image may be out of focus, which could lead to inaccurate detection of surface height change parameters and affect the formulation of road maintenance plans. Therefore, the image corresponding to the captured target with a focus range greater than the preset value can be selected from the first image as the second image. This helps improve the accuracy of the subsequent determination of road maintenance plans.

[0051] S103. Select a detection point from each of the plurality of regions to obtain the target detection point corresponding to each region.

[0052] Each area may include multiple detection points. Electronic equipment can further determine the road conditions through road images corresponding to multiple detection points. For example, for gravel and crushed stone pavements, it can determine whether there are potholes to determine whether a road maintenance plan requires material replenishment; for asphalt concrete pavements, it can determine whether there are cracks or potholes to determine whether a road maintenance plan requires patching or sealing; for roads in cold weather, it can determine whether there is black ice or ice to determine whether a road maintenance plan requires salting or stone chips for anti-skid purposes; for roads with leaves or dead branches obstructing the view, it can determine whether a road maintenance plan requires cleaning, etc., and so on.

[0053] In one possible example, a detection point is selected from each of the plurality of regions to obtain a target detection point corresponding to each region. The above method may include the following steps: for any one of the plurality of regions, arbitrarily select a plurality of candidate detection points, wherein the candidate detection points do not include the shooting target; determine the ground height change parameter corresponding to each candidate detection point; when the ground height change parameter is greater than or equal to a preset threshold, determine the corresponding candidate detection point as the target detection point.

[0054] Among them, the above-mentioned surface height change parameters are used to indicate the range of surface height change relative to the normal ground height caused by factors such as road surface compression, water erosion, potholes, or obstruction by fallen leaves, dead branches, or snow.

[0055] The aforementioned preset threshold can be set by the user or by the system default, and is not limited here. For example, the preset threshold can be different for different situations. For roads to be tested with obvious rain or snow, snow accumulation, or black ice, there can be a corresponding preset threshold. Or a preset threshold can be set for potholes on gravel or gravel roads. The preset threshold can generally refer to the minimum value set according to normal conditions (the normal range of road surface height increase caused by snow accumulation in northern streets in previous years).

[0056] Optionally, the aforementioned preset threshold can also be set dynamically. For example, it can be set according to the actual weather, or multiple preset sub-thresholds can be set. Each preset sub-threshold corresponds to a type of road inspection or road surface (e.g., asphalt concrete road surface, gravel road surface, asphalt road surface, etc.). It can be compared with multiple preset sub-thresholds in turn. If it is less than all the preset sub-thresholds, the detection point is reselected; if it is greater than at least one of the preset sub-thresholds, the above-mentioned arbitrarily or randomly selected candidate detection point is used as the target detection point.

[0057] As can be seen in this example, the electronic device can evaluate whether the candidate detection point can be used as the target detection point by setting a preset threshold. When the specific road conditions of the road to be detected are unknown, it can determine whether it can be used as a detection point for evaluating the road surface type identification. This helps to improve the accuracy of road surface type identification and can overcome the error in judging the road conditions in complex road conditions, such as when water accumulation and potholes coexist. This helps to improve the accuracy of road inspection.

[0058] In one possible example, the above method may further include the following steps: when the surface height range is less than the preset threshold, the corresponding candidate detection point is taken as a critical detection point to obtain at least one critical detection point; when the number of the at least one critical detection point reaches a preset number, a critical region is generated based on the at least one critical detection point; and candidate detection points are reselected from the region that does not include the critical region as the target detection point.

[0059] The preset number can be set by the user or by the system default, and is not limited here. The preset number can be set according to the number of test points to be selected. For example, it can be set as a preset ratio of the number of test points to be selected, which can be set to 30%. If there are 10 test points to be selected, the preset number can be set to 3.

[0060] The aforementioned critical region is used to further subdivide or divide the area corresponding to the candidate detection point. If the candidate detection point does not meet the requirements, the target detection point is reselected in a new area, i.e., an area excluding the critical region, to accurately locate the detection point. This helps to reduce the misjudgment or underjudgment of road conditions in that area.

[0061] S104. Based on the target detection points corresponding to each region, mark each of the multiple regions to obtain multiple road surface type identifiers.

[0062] Among them, road markings can be used to indicate the need for road maintenance, or to characterize the cause of road damage, etc. Electronic equipment can set a road maintenance plan for each road type marking; road type markings can correspond to multiple levels, and each road type marking can correspond to one level. This level can be used to indicate the urgency of road maintenance. The higher the level, the higher the urgency of road maintenance.

[0063] For example, road surface type markings may specifically include at least one of the following: low-grade gravel or crushed stone road surface with potholes; low-grade asphalt or asphalt concrete road surface with cracks or potholes; low-grade road shoulders that need to be cleared in time for fire prevention; medium-grade road surface with dead branches that may affect vehicle collisions and cause tire blowouts or changes in driving direction; medium-grade road surface with snow accumulation that needs to be cleared and is impassable; high-grade road surface affected by cold weather that needs to be salted to prevent the road surface from freezing; high-grade road surface collapse resulting in large potholes, etc., without limitation.

[0064] In one possible example, the step of marking each of the multiple regions according to the target detection points corresponding to each region to obtain multiple road surface type identifiers may include the following steps: for each region, determining the RGB value corresponding to each target detection point in the region; determining the average RGB value in the corresponding region according to the RGB value corresponding to each target detection point; and marking the corresponding region according to the average RGB value to obtain the road surface type identifier corresponding to the region.

[0065] Different road surface types can be distinguished by RGB values ​​to differentiate different road conditions.

[0066] In particular, due to the presence of potholes, ice, snow, branches, etc. on the road surface, the brightness and color of the captured images will vary. For example, whether there is snow, potholes, branches or leaves, etc., the RGB values ​​displayed in the image will be very different. Therefore, the road conditions in the corresponding area can be reflected by identifying the RGB values ​​of the target detection points in the target image corresponding to the color of each area.

[0067] As can be seen, in this example, the electronic device can divide the target image into regions based on the captured multi-frame road images, and combine the divided regions with the road surface type identifier for each region to reflect the road conditions in different regions. Furthermore, by using the RGB values ​​corresponding to the target detection points in the target image obtained in the preceding method to represent the road surface type identifier, there is no need to re-evaluate the road conditions through image comparison or target recognition (pothole image comparison, snow accumulation recognition, etc.), which helps improve the processing efficiency of the electronic device and enables more accurate location of problems in the road. This facilitates the electronic device to select customized road maintenance solutions for each region in the later stages.

[0068] S105. Determine the road maintenance plan corresponding to each area based on the road surface type identifier.

[0069] Among them, electronic devices can set a road maintenance plan for each road surface type.

[0070] For example, for road type markings indicating potholes on low-grade gravel or crushed stone roads, or cracks or potholes on low-grade asphalt or asphalt concrete roads, the corresponding road maintenance plan is to patch or seal the affected area. For road type markings indicating that dead branches could affect vehicle collisions and cause tire blowouts or changes in driving direction, the corresponding road maintenance plan is to send sweepers to the area to clear the dead branches. For road type markings indicating that snow accumulation on medium-grade roads makes them impassable, the corresponding road maintenance plan is to send sweepers to salt the area and remove the snow.

[0071] In one possible example, determining the road maintenance plan corresponding to each area based on the road surface type identifier includes the following steps: converting the RGB average value of the area to YUV value to obtain a Y value, wherein the Y value is used to identify the road inspection type; determining the road maintenance plan corresponding to the area based on the preset mapping relationship between the road inspection type and the road maintenance plan.

[0072] To determine the corresponding color depth, the RGB values ​​in RGB mode are converted to YUV values ​​in YUV mode. The Y value is then used to determine the color depth on the equipment label, which is used to identify the hazard level of the hazardous waste liquid in the IBC container. The larger the Y value, the darker the color, indicating a lower hazard level. This conversion from RGB to YUV values ​​improves the accuracy of color depth determination, and the ability to determine color depth solely through the Y value facilitates more precise identification of any abnormalities in the hazardous waste liquid within the IBC container.

[0073] Since the RGB values ​​are three values, each value needs to be compared when selecting a road maintenance plan, which makes it difficult to select quickly. Also, when comparing RGB values, all three values ​​need to be compared, which may result in multiple road maintenance plans being located. Therefore, the Y value can be used to identify the road inspection type, so that the road maintenance plan can be accurately located from the database corresponding to the electronic device.

[0074] As can be seen in this example, since the RGB or Y values ​​can reflect road surface damage, snow accumulation, and other conditions, the road conditions in each area can be accurately located in a differentiated manner. This is conducive to precise road maintenance for each area and can avoid traffic congestion caused by uniform or large-scale road maintenance on the entire road section or the road to be inspected. Thus, it helps to improve the performance and service level of the road.

[0075] As can be seen, the road surface inspection and maintenance method described in this application involves acquiring multiple frames of road images of the road to be inspected; dividing the road to be inspected into multiple regions based on the multiple frames of road images; selecting detection points from each of the multiple regions to obtain target detection points corresponding to each region; marking each of the multiple regions based on the target detection points corresponding to each region to obtain multiple road surface type identifiers; and determining the road surface maintenance plan corresponding to each region based on the road surface type identifiers. In this way, the inspection of the road to be inspected can be achieved through target detection points, and the road to be inspected can be divided into different regions to achieve regional road surface maintenance, which is beneficial to improving the road's performance and service level.

[0076] Consistent with the above embodiments, please refer to Figure 3, which is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. As shown in the figure, the electronic device includes a processor, a memory, a communication interface, and one or more programs, applied to a road surface inspection and maintenance system. The one or more programs are stored in the memory and configured to be executed by the processor. The programs include instructions for performing the following steps:

[0077] Acquire multiple frames of road images of the road to be detected;

[0078] The road to be detected is divided into multiple regions based on the multi-frame road images;

[0079] Detection points are selected from each of the plurality of regions to obtain the target detection points corresponding to each region;

[0080] Based on the target detection points corresponding to each region, each region in the plurality of regions is marked to obtain multiple road surface type identifiers;

[0081] Based on the road surface type identifier, determine the corresponding road surface maintenance plan for each area.

[0082] In one possible example, regarding the acquisition of multiple frames of road images of the road to be detected, the above procedure includes instructions for performing the following steps:

[0083] The target is determined during each capture of the road image;

[0084] Determine the shooting angle corresponding to each of the aforementioned shooting targets;

[0085] Based on the shooting angle, the plane adjustment parameters corresponding to each road image are determined, wherein the plane adjustment parameters are determined relative to the camera plane angle;

[0086] Based on the plane adjustment parameters, determine the focal plane corresponding to the road image;

[0087] The road image is acquired on the focal plane.

[0088] In one possible example, regarding the division of the road to be detected into multiple regions based on the multi-frame road images, the above procedure includes instructions for performing the following steps:

[0089] Based on the focal plane corresponding to each road image, the multiple road images are combined to obtain a first image;

[0090] Determine the focal range of each of the captured targets in the corresponding road image;

[0091] The first image is selected to include the image corresponding to the target whose focus range is greater than or equal to a preset value, which is then used as the second image.

[0092] Based on the shooting target whose focus range is greater than or equal to a preset value, the second image is divided into multiple target images, each target image corresponding to a region, and each region including at least one shooting target.

[0093] In one possible example, regarding the selection of detection points from each of the plurality of regions to obtain the target detection point corresponding to each region, the above procedure includes instructions for performing the following steps:

[0094] For any one of the multiple regions, arbitrarily select multiple candidate detection points, wherein the candidate detection points do not include the shooting target;

[0095] Determine the surface height change parameters corresponding to each of the candidate detection points;

[0096] When the surface height change parameter is greater than or equal to a preset threshold, the corresponding candidate detection point is determined as the target detection point.

[0097] In one possible example, the above procedure also includes instructions for performing the following steps:

[0098] When the surface height range is less than the preset threshold, the corresponding candidate detection point is taken as the critical detection point, and at least one critical detection point is obtained.

[0099] When the number of at least one critical detection point reaches a preset number, a critical region is generated based on the at least one critical detection point.

[0100] From the region that does not include the critical region, reselect the candidate detection point as the target detection point.

[0101] In one possible example, in the step of marking each of the plurality of regions according to the target detection points corresponding to each region to obtain multiple road surface type identifiers, the above procedure includes instructions for performing the following steps:

[0102] For each region, determine the RGB value corresponding to each target detection point in that region;

[0103] Based on the RGB value corresponding to each target detection point, determine the average RGB value in the corresponding region;

[0104] Based on the average RGB value, the corresponding area is marked to obtain the road surface type identifier corresponding to the area.

[0105] In one possible example, regarding the determination of the road maintenance plan corresponding to each area based on the road surface type identifier, the above procedure includes instructions for performing the following steps:

[0106] The average RGB value corresponding to the region is converted to YUV value to obtain the Y value, wherein the Y value is used to identify the road patrol type;

[0107] Based on the preset mapping relationship between road inspection types and road maintenance plans, the corresponding road maintenance plan for the area is determined.

[0108] As can be seen, the electronic device described in this application embodiment acquires multiple frames of road images of the road to be inspected; divides the road to be inspected into multiple regions based on the multiple frames of road images; selects a detection point from each of the multiple regions to obtain a target detection point corresponding to each region; marks each of the multiple regions based on the target detection point corresponding to each region to obtain multiple road surface type identifiers; and determines a road surface maintenance plan corresponding to each region based on the road surface type identifiers. In this way, the inspection of the road to be inspected can be realized through target detection points, and the road to be inspected can be divided into different regions to achieve regional road surface maintenance, which is beneficial to improving the road's performance and service level.

[0109] Figure 4 is a functional unit block diagram of a road surface inspection and maintenance device 400 according to an embodiment of this application, applied to an electronic device. The road surface inspection and maintenance device 400 includes: an acquisition unit 401, a division unit 402, a selection unit 403, a marking unit 404, and a determination unit 405, wherein...

[0110] The acquisition unit 401 is used to acquire multiple frames of road images of the road to be detected;

[0111] The segmentation unit 402 is used to divide the road to be detected into multiple regions based on the multi-frame road images;

[0112] The selection unit 403 is used to select a detection point from each of the plurality of regions to obtain the target detection point corresponding to each region.

[0113] The marking unit 404 is used to mark each of the multiple regions according to the target detection point corresponding to each region, so as to obtain multiple road surface type identifiers;

[0114] The determining unit 405 is used to determine the road maintenance plan corresponding to each area based on the road surface type identifier.

[0115] In one possible example, regarding the acquisition of multi-frame road images of the road to be detected, the acquisition unit 401 is specifically used for:

[0116] The target is determined during each capture of the road image;

[0117] Determine the shooting angle corresponding to each of the aforementioned shooting targets;

[0118] Based on the shooting angle, the plane adjustment parameters corresponding to each road image are determined, wherein the plane adjustment parameters are determined relative to the camera plane angle;

[0119] Based on the plane adjustment parameters, determine the focal plane corresponding to the road image;

[0120] The road image is acquired on the focal plane.

[0121] In one possible example, regarding the division of the road to be detected into multiple regions based on the multi-frame road images, the division unit 402 is specifically used for:

[0122] Based on the focal plane corresponding to each road image, the multiple road images are combined to obtain a first image;

[0123] Determine the focal range of each of the captured targets in the corresponding road image;

[0124] The first image is selected to include the image corresponding to the target whose focus range is greater than or equal to a preset value, which is then used as the second image.

[0125] Based on the shooting target whose focus range is greater than or equal to a preset value, the second image is divided into multiple target images, each target image corresponding to a region, and each region including at least one shooting target.

[0126] In one possible example, in selecting a detection point from each of the plurality of regions to obtain a target detection point corresponding to each region, the selection unit 403 is specifically used for:

[0127] For any one of the multiple regions, arbitrarily select multiple candidate detection points, wherein the candidate detection points do not include the shooting target;

[0128] Determine the surface height change parameters corresponding to each of the candidate detection points;

[0129] When the surface height change parameter is greater than or equal to a preset threshold, the corresponding candidate detection point is determined as the target detection point.

[0130] In one possible example, the selection unit 403 described above is further used for:

[0131] When the surface height range is less than the preset threshold, the corresponding candidate detection point is taken as the critical detection point, and at least one critical detection point is obtained.

[0132] When the number of at least one critical detection point reaches a preset number, a critical region is generated based on the at least one critical detection point.

[0133] From the region that does not include the critical region, reselect the candidate detection point as the target detection point.

[0134] In one possible example, in the process of marking each of the multiple regions according to the target detection points corresponding to each region to obtain multiple road surface type identifiers, the above-mentioned segmentation unit 402 is specifically used for:

[0135] For each region, determine the RGB value corresponding to each target detection point in that region;

[0136] Based on the RGB value corresponding to each target detection point, determine the average RGB value in the corresponding region;

[0137] Based on the average RGB value, the corresponding area is marked to obtain the road surface type identifier corresponding to the area.

[0138] In one possible example, regarding the determination of the road maintenance plan corresponding to each area based on the road surface type identifier, the determining unit 405 is specifically used for:

[0139] The average RGB value corresponding to the region is converted to YUV value to obtain the Y value, wherein the Y value is used to identify the road patrol type;

[0140] Based on the preset mapping relationship between road inspection types and road maintenance plans, the corresponding road maintenance plan for the area is determined.

[0141] As can be seen, the road surface inspection and maintenance device described in this application embodiment acquires multiple frames of road images of the road to be inspected; divides the road to be inspected into multiple regions based on the multiple frames of road images; selects a detection point from each of the multiple regions to obtain a target detection point corresponding to each region; marks each of the multiple regions based on the target detection point corresponding to each region to obtain multiple road surface type identifiers; and determines the road surface maintenance plan corresponding to each region based on the road surface type identifiers. In this way, the inspection of the road to be inspected can be achieved through target detection points, and the road to be inspected can be divided into different regions to achieve regional road surface maintenance, which is beneficial to improving the road's performance and service level.

[0142] It is understood that the functions of each program module of the road surface inspection and maintenance device in this embodiment can be specifically implemented according to the methods in the above method embodiments. The specific implementation process can be referred to the relevant descriptions in the above method embodiments, which will not be repeated here.

[0143] This application also provides a computer storage medium storing a computer program for electronic data interchange, which causes a computer to perform some or all of the steps of any of the methods described in the above method embodiments, wherein the computer includes an electronic device.

[0144] This application also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program operable to cause a computer to perform some or all of the steps of any of the methods described in the above method embodiments. The computer program product may be a software installation package, and the computer may include an electronic device.

[0145] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0146] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0147] In the several embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical or other forms.

[0148] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0149] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0150] If the aforementioned integrated units are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0151] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage device, which may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0152] The embodiments of this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for road surface inspection and maintenance, characterized in that, An application to a road surface inspection and maintenance system includes the following method: acquiring multiple frames of road images of a road to be inspected; dividing the road to be inspected into multiple regions based on the multiple frames of road images; selecting detection points from each of the multiple regions to obtain target detection points corresponding to each region; marking each of the multiple regions based on the target detection points corresponding to each region to obtain multiple road surface type identifiers; and determining a road surface maintenance plan corresponding to each region based on the road surface type identifiers. The acquisition of multiple frames of road images of the road to be inspected includes: determining a shooting target during each image capture; determining a shooting angle corresponding to each shooting target; determining a plane adjustment parameter corresponding to each road image based on the shooting angle, wherein the plane adjustment parameter is determined relative to the camera plane angle; determining a focal plane corresponding to the road image based on the plane adjustment parameter; and acquiring the road image on the focal plane. The road is divided into multiple regions, including: combining multiple road images according to the focal plane corresponding to each road image to obtain a first image; determining the focal range of each shooting target in the corresponding road image; selecting images in the first image that include the shooting target whose focal range is greater than or equal to a preset value as a second image; dividing the second image into multiple target images according to the shooting target whose focal range is greater than or equal to the preset value, each target image corresponding to one region, and each region including at least one shooting target; wherein, selecting a detection point from each of the multiple regions to obtain a target detection point corresponding to each region includes: arbitrarily selecting multiple candidate detection points for any one of the multiple regions, wherein the candidate detection points do not include the shooting target; determining the ground height change parameter corresponding to each candidate detection point; when the ground height change parameter is greater than or equal to a preset threshold, determining the corresponding candidate detection point as the target detection point.

2. The method according to claim 1, characterized in that, The method further includes: when the surface height range is less than the preset threshold, taking the corresponding candidate detection point as a critical detection point to obtain at least one critical detection point; when the number of the at least one critical detection point reaches a preset number, generating a critical region based on the at least one critical detection point; and reselecting candidate detection points as the target detection point from the region that does not include the critical region.

3. The method according to claim 1, characterized in that, The step of marking each of the multiple regions according to the target detection points corresponding to each region to obtain multiple road surface type identifiers includes: for each region, determining the RGB value corresponding to each target detection point in the region; determining the average RGB value in the corresponding region according to the RGB value corresponding to each target detection point; and marking the corresponding region according to the average RGB value to obtain the road surface type identifier corresponding to the region.

4. The method according to claim 3, characterized in that, The step of determining the road maintenance plan corresponding to each area based on the road type identifier includes: converting the RGB average value of the area to YUV value to obtain a Y value, wherein the Y value is used to identify the road inspection type; and determining the road maintenance plan corresponding to the area based on the preset mapping relationship between the road inspection type and the road maintenance plan.

5. A device for road surface inspection and maintenance, characterized in that, An apparatus for use in road surface inspection and maintenance systems, to implement the method described in any one of claims 1-4, comprises: an acquisition unit, a division unit, a selection unit, a marking unit, and a determination unit, wherein the acquisition unit is used to acquire multiple frames of road images of the road to be inspected; the division unit is used to divide the road to be inspected into multiple regions based on the multiple frames of road images; the selection unit is used to select a detection point from each of the multiple regions to obtain a target detection point corresponding to each region; the marking unit is used to mark each of the multiple regions based on the target detection point corresponding to each region to obtain multiple road surface type identifiers; and the determination unit is used to determine a road surface maintenance plan corresponding to each region based on the road surface type identifiers.

6. An electronic device, characterized in that, It includes a processor and a memory, the memory being used to store one or more programs and configured to be executed by the processor, the programs including instructions for performing the steps of the method as described in any one of claims 1-4.

7. A computer-readable storage medium, characterized in that, A computer program for storing electronic data interchange is provided, wherein the computer program causes a computer to perform the method as described in any one of claims 1-4.

Images