Road pit detection method and device, electronic equipment and storage medium

Abstract

The application discloses a road pit detection method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring first road data, and performing pit detection on the first road data by using a preset pit detection model to obtain a first pit detection result; performing dynamic target detection on the first road data according to the first pit detection result to obtain a corresponding dynamic target detection result; determining a second pit detection result according to the first pit detection result and the corresponding dynamic target detection result; and determining the persistence state of the second pit detection result, and determining a final pit detection result. The application firstly performs preliminary detection on pits in a road by using a pit detection model, further confirms the pit detection result in combination with a dynamic target detection result, finally judges the persistence state of the detected pits, improves the accuracy of road pit detection, reduces the false detection rate, and provides prior information for subsequent vehicle driving planning.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of road detection, and in particular to a road pit detection method and device, an electronic device, and a storage medium. BACKGROUND

[0002] Roads are easily damaged by factors such as vehicle compression, impact, and wear over a long period of time, which may result in pits and cracks. On the one hand, this may affect the service life of the road, and on the other hand, it may also cause damage to vehicle tires and traffic accidents, and even for new energy batteries, there is a certain probability that the battery will be impacted by a deep pit.

[0003] In order to maintain road health and eliminate safety hazards, the road administration department often needs to regularly assign staff to conduct manual road inspections. However, the manual road inspection method has the problems of large workload, high labor cost, time-consuming and labor-intensive, and low efficiency.

[0004] Some existing road pit detection technologies based on visual and laser radar sensors also have the problem of low detection accuracy, which may easily result in false detection and waste of human and material resources. SUMMARY

[0005] The embodiments of the present application provide a road pit detection method, device, electronic device, and storage medium to improve the accuracy of road pit detection.

[0006] The embodiments of the present application adopt the following technical solutions:

[0007] In a first aspect, the embodiments of the present application provide a road pit detection method, and the method comprises:

[0008] Obtaining first road data and performing pit detection on the first road data by using a preset pit detection model to obtain a first pit detection result;

[0009] Performing dynamic target detection on the first road data according to the first pit detection result to obtain a dynamic target detection result corresponding to the first pit detection result;

[0010] Determining a second pit detection result according to the first pit detection result and the dynamic target detection result corresponding to the first pit detection result;

[0011] Determining the persistence state of the second pit detection result, and determining a final pit detection result according to the persistence state of the second pit detection result.

[0012] Optionally, the first road data comprises road image data and laser point cloud data, and the performing pit detection on the first road data by using the preset pit detection model to obtain the first pit detection result comprises:

[0013] perform pit detection on the road image data by using a preset image detection model to obtain a pit detection result of the road image data;

[0014] perform pit detection on the laser point cloud data by using a preset point cloud detection model to obtain a pit detection result of the laser point cloud data;

[0015] determine the first pit detection result according to the pit detection result of the road image data and the pit detection result of the laser point cloud data.

[0016] Optionally, the determining the first pit detection result according to the pit detection result of the road image data and the pit detection result of the laser point cloud data comprises:

[0017] if at least one of the pit detection result of the road image data and the pit detection result of the laser point cloud data is pit, determining that the first pit detection result is pit;

[0018] otherwise, determining that the first pit detection result is no pit.

[0019] Optionally, the first pit detection result comprises a pit position, and the performing dynamic target detection on the first road data according to the first pit detection result to obtain a dynamic target detection result corresponding to the first pit detection result comprises:

[0020] detecting a dynamic target at the pit position;

[0021] if a dynamic target is detected at the pit position, determining whether a detection box of the dynamic target and a pit detection box corresponding to the pit position have an intersection;

[0022] if yes, determining that the dynamic target detection result corresponding to the first pit detection result is dynamic target;

[0023] otherwise, determining that the dynamic target detection result corresponding to the first pit detection result is no dynamic target.

[0024] Optionally, the determining a second pit detection result according to the first pit detection result and the dynamic target detection result corresponding to the first pit detection result comprises:

[0025] if the first pit detection result is pit and the dynamic target detection result is no dynamic target, directly taking the first pit detection result as the second pit detection result, and marking and counting a pit position in the second pit detection result;

[0026] If the first pit detection result is a pit and the dynamic target detection result is a dynamic target, the first pit detection result is discarded.

[0027] Optionally, the second pit detection result comprises a pit position, the determining the persistence state of the second pit detection result, and determining the final pit detection result according to the persistence state of the second pit detection result comprises:

[0028] obtaining a plurality of frames of second road data corresponding to the pit position, and performing pit detection on each frame of the second road data to obtain a third pit detection result;

[0029] determining a cumulative count corresponding to the pit position according to the third pit detection result;

[0030] determining the persistence state of the second pit detection result according to the cumulative count corresponding to the pit position.

[0031] Optionally, the determining the final pit detection result according to the persistence state of the second pit detection result comprises:

[0032] if the cumulative count corresponding to the pit position reaches a preset count threshold, determining the final pit detection result according to the second pit detection result and the third pit detection result, the final pit detection result comprising a pit position and a pit attribute;

[0033] otherwise, determining that the final pit detection result is no pit.

[0034] In a second aspect, the embodiments of the present application further provide a road pit detection device, wherein the device comprises:

[0035] a first detection unit configured to obtain first road data and perform pit detection on the first road data by using a preset pit detection model to obtain a first pit detection result;

[0036] a second detection unit configured to perform dynamic target detection on the first road data according to the first pit detection result to obtain a dynamic target detection result corresponding to the first pit detection result;

[0037] a first determination unit configured to determine a second pit detection result according to the first pit detection result and the dynamic target detection result corresponding to the first pit detection result;

[0038] a second determination unit configured to determine a persistence state of the second pit detection result and determine a final pit detection result according to the persistence state of the second pit detection result.

[0039] In a third aspect, the embodiments of the present application further provide an electronic device, comprising:

[0040] a processor; and

[0041] a memory arranged to store computer-executable instructions that, when executed, cause the processor to perform any of the aforementioned methods.

[0042] In a fourth aspect, the embodiments of the present application further provide a computer-readable storage medium storing one or more programs, which, when executed by an electronic device comprising a plurality of applications, cause the electronic device to perform any of the aforementioned methods.

[0043] The above at least one technical solution adopted by the embodiments of the present application can achieve the following beneficial effects: The road pit detection method of the embodiments of the present application first acquires first road data, and performs pit detection on the first road data by using a preset pit detection model to obtain a first pit detection result. Then, dynamic target detection is performed on a road image according to the first pit detection result to obtain a dynamic target detection result corresponding to the first pit detection result. Subsequently, a second pit detection result is determined according to the first pit detection result and the dynamic target detection result corresponding to the first pit detection result. Finally, a persistent state of the second pit detection result is determined, and a final pit detection result is determined according to the persistent state of the second pit detection result. The road pit detection method of the embodiments of the present application first performs preliminary detection on pits in a road by using a pit detection model, further confirms the pit detection result in combination with a corresponding dynamic target detection result, and finally judges the persistent state of the detected pit, thereby greatly improving the accuracy of road pit detection and reducing the possibility of false detection and false reporting, and providing prior information for subsequent vehicle driving planning. BRIEF DESCRIPTION OF DRAWINGS

[0044] The accompanying drawings, which are included to provide a further understanding of the present application, constitute a part of the present application, illustrate the illustrative embodiments of the present application and their description serve to explain the present application, and do not constitute improper limitations on the present application. In the drawings:

[0045] Figure 1 FIG. 1 is a flowchart of a road pit detection method according to an embodiment of the present application;

[0046] Figure 2 FIG. 2 is a structural diagram of a road pit detection device according to an embodiment of the present application;

[0047] Figure 3 FIG. 3 is a structural diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0048] To make the purpose, technical solutions, and advantages of this application more clear, the technical solutions of this application will be clearly and completely described below in conjunction with the specific embodiments of this application and the corresponding drawings. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0049] The following describes in detail the technical solutions provided by various embodiments of the present application in conjunction with the accompanying drawings.

[0050] The present application embodiment provides a road pothole detection method, such as Figure 1 , a flow chart of a road pothole detection method according to an embodiment of the present application is provided, wherein the method comprises at least the following steps S110 to S140:

[0051] Step S110 , obtaining first road data, and performing pothole detection on the first road data using a preset pothole detection model to obtain a first pothole detection result.

[0052] The road pothole detection method of the embodiment of the present application can be executed by the roadside. The roadside is usually deployed with roadside sensors such as cameras and lidars. Multiple roadside sensors can be deployed in a road section. Each roadside sensor is responsible for perceiving road data within a certain area. The mutual cooperation between multiple roadside sensors can realize road monitoring of the entire road section.

[0053] Based on this, the embodiment of the present application needs to first obtain the first road data collected by the roadside sensor, such as the road image data collected by the roadside camera and the laser point cloud data collected by the lidar, and then use the pre-trained pothole detection model to perform pothole detection on the first road data to obtain a first pothole detection result. The first pothole detection result may be a detection result of potholes or a detection result of no potholes. If it is a detection result of potholes, further subsequent confirmation steps are required.

[0054] The preset pothole detection model can be trained based on an existing deep learning model. For example, an image pothole detection model can be trained based on the YOLO target detection network or based on YOLO combined with the Darknet framework. A laser point cloud pothole detection model can also be trained. Of course, technical personnel in this field can flexibly choose how to train the pothole detection model according to actual needs, and no specific limitation is made here.

[0055] Step S120 , performing dynamic target detection on the first road data according to the first pothole detection result, to obtain a dynamic target detection result corresponding to the first pothole detection result.

[0056] If the first pit detection result is that there is a pit, the first road data can be further dynamically target detected according to the detected pit position. The dynamic target is a target such as a pedestrian, a vehicle, etc. The target will have a certain influence on the detection of the pit. Therefore, the dynamic target near the pit position can be detected to confirm whether the pit detection result is accurate.

[0057] In step S130, a second pit detection result is determined according to the first pit detection result and the dynamic target detection result corresponding to the first pit detection result.

[0058] The dynamic target detection result near the pit position can further guarantee the accuracy of the pit detection result. Therefore, the second pit detection result can be determined by combining the first pit detection result and the dynamic target detection result corresponding to the first pit detection result. That is, different second pit detection results are determined according to different dynamic target detection results. If the determined second pit detection result is still that there is a pit, the subsequent step can be continued.

[0059] In step S140, a persistence state of the second pit detection result is determined, and a final pit detection result is determined according to the persistence state of the second pit detection result.

[0060] In order to further improve the detection accuracy, after the confirmation of the dynamic target detection result, if it is still considered that there is a pit, the persistence state of the pit detection result can be further determined. If the result can persistently and stably exist, it can be finally confirmed that there is a pit at the position, and the related personnel can be fed back in time for processing. In addition, the pit position and other information can be sent to the vehicle that will arrive in the area, so that the vehicle can be noticed and avoided in advance.

[0061] The road pit detection method of the embodiment of the application first detects the pit in the road by using the pit detection model, further confirms the pit detection result by combining the corresponding dynamic target detection result, and finally judges the persistence state of the detected pit. Therefore, the accuracy of the road pit detection is greatly improved, the possibility of false detection and false reporting is reduced, and prior information is provided for the subsequent driving planning of the vehicle.

[0062] In some embodiments of the present application, the first road data includes road image data and laser point cloud data, and the pit detection on the first road data by using the preset pit detection model to obtain the first pit detection result includes: performing pit detection on the road image data by using a preset image detection model to obtain a pit detection result of the road image data; performing pit detection on the laser point cloud data by using a preset point cloud detection model to obtain a pit detection result of the laser point cloud data; and determining the first pit detection result according to the pit detection result of the road image data and the pit detection result of the laser point cloud data.

[0063] The first road data of the embodiments of the present application can include road image data collected by a roadside camera and laser point cloud data collected by a laser radar. For the road image data, a previously trained image detection model such as a YOLO network can be used to detect pits in the road image. The image detection model mainly detects pits from the dimension of image color texture information. For the laser point cloud data, a previously trained point cloud detection model can be used for detection. The point cloud detection model mainly detects the undulating structure of the pit point cloud. Both of them can determine the pit position.

[0064] Since the image detection model and the point cloud detection model detect pits in the current road from different dimensions respectively, in order to improve the accuracy of the detection result, the embodiments of the present application can combine the pit detection results of the two different detection models to determine the first pit detection result.

[0065] In some embodiments of the present application, the determination of the first pit detection result according to the pit detection result of the road image data and the pit detection result of the laser point cloud data includes: if at least one of the pit detection result of the road image data and the pit detection result of the laser point cloud data is a pit, determining that the first pit detection result is a pit; otherwise, determining that the first pit detection result is no pit.

[0066] Since the detection result of the image detection model is easily affected by light, and the detection result of the point cloud detection model is easily affected by the size of the pit, if the pit is small, it may cause the point cloud detection model to misdetect. Therefore, during actual detection, the image detection model may not detect a pit in the current road, or the laser radar may not detect a pit in the current road.

[0067] Based on this, the embodiments of the present application can combine the pit detection results of the two different detection models. As long as at least one detection model detects a pit, it is considered that there may be a pit at the current position, and the subsequent confirmation link can be performed, thereby improving the accuracy of the pit detection result.

[0068] It should be noted here that since there may be more than one pothole on the current road section, if both the image detection model and the point cloud detection model detect potholes, it is possible to further confirm whether the potholes detected by the two are the same pothole. Since the detection result output by the image detection model is a 2D pothole detection frame, and the detection result output by the point cloud detection model is a 3D pothole point cloud, the 3D pothole point cloud can be first projected into the image based on the projection transformation relationship between the pre-calibrated lidar and the camera, thereby obtaining the 2D pothole projection frame of the 3D pothole point cloud in the image, and then the 2D pothole projection frame of the 3D pothole point cloud in the image is matched with the 2D pothole detection frame of the image, so as to determine whether the two correspond to the same pothole target based on the matching results.

[0069] The matching method can be, for example, calculating the intersection-and-union (IoU) of the 2D pothole projection frame and the 2D pothole detection frame. If the IoU is greater than a preset IoU threshold, it can be considered that the two correspond to the same pothole target. Otherwise, they are different pothole targets and need to be processed separately. This is similar to the detection of targets such as vehicles and pedestrians. Different ID identifiers can be used to distinguish different pothole targets.

[0070] In some embodiments of the present application, the first pothole detection result includes a pothole location, and the dynamic target detection is performed on the first road data according to the first pothole detection result to obtain a dynamic target detection result corresponding to the first pothole detection result, including: detecting the dynamic target at the pothole location; if a dynamic target is detected at the pothole location, determining whether there is an intersection between the detection frame of the dynamic target and the pothole detection frame corresponding to the pothole location; if so, determining that the dynamic target detection result corresponding to the first pothole detection result is that there is a dynamic target; otherwise, determining that the dynamic target detection result corresponding to the first pothole detection result is that there is no dynamic target.

[0071] If there are dynamic targets such as vehicles and pedestrians near the pothole, the vehicles and pedestrians will block the pothole, which will affect the accuracy of the pothole detection results. Therefore, the embodiment of the present application can further utilize the existing dynamic target detection model to detect dynamic targets such as vehicles and pedestrians near the pothole. The dynamic target detection model can be implemented using existing networks such as YOLO and RCNN. In addition, the size of the specific detection range around the pothole location can also be set by those skilled in the art according to actual needs. For example, the size of the detected pothole frame can be used as a reference and a certain range can be appropriately expanded outward.

[0072] If a dynamic target is detected near the pit position, the relationship between the detection box of the dynamic target and the detection box corresponding to the pit position can be further determined, that is, it is judged whether the existence of the dynamic target will cause occlusion to the pit. For example, the intersection over union of the detection box of the dynamic target and the detection box corresponding to the pit position can be calculated. If the intersection over union is greater than 0, it indicates that the detection boxes of the two exist intersection, and the existence of the dynamic target will cause a certain degree of occlusion to the detection of the pit. At this time, it can be considered that there is a dynamic target in the pit position in the first pit detection result. Conversely, if no dynamic target is detected near the pit position, or the detected dynamic target does not cause occlusion to the detected pit, it can be considered that there is no dynamic target in the pit position.

[0073] That is, the definition of the dynamic target detection result corresponding to the pit position in the embodiment of the present application is to first determine whether a dynamic target is detected near the pit position. If a dynamic target is detected, it is further determined whether the dynamic target will cause occlusion to the detection of the pit.

[0074] In some embodiments of the present application, the determination of the second pit detection result according to the first pit detection result and the dynamic target detection result corresponding to the first pit detection result includes: if the first pit detection result is a pit and the dynamic target detection result is no dynamic target, directly taking the first pit detection result as the second pit detection result, and marking and counting the pit positions in the second pit detection result; if the first pit detection result is a pit and the dynamic target detection result is a dynamic target, discarding the first pit detection result.

[0075] If the first pit detection result is a pit, and the dynamic target detection result corresponding to the detected pit position is no dynamic target, it indicates that no dynamic target is detected near the pit position or the dynamic target does not cause occlusion to the pit. Therefore, it can be considered that the detected pit in the first pit detection result is probably accurate, and thus the first pit detection result can be directly taken as the second pit detection result, and the detected pit positions are marked and counted. Here, the counting refers to the accumulation of the number of pits detected at the same position, which is the basis for reflecting the continuous state of the pit position in the subsequent process.

[0076] If the first pit detection result is a pit, and the dynamic target detection result corresponding to the detected pit position is a dynamic target, it indicates that there is a dynamic target near the pit position and the existence of the dynamic target causes occlusion to the detection of the pit. Therefore, the first pit detection result obtained at this time is probably inaccurate, and thus the detection result of the frame data can be directly discarded, and the marking and counting accumulation of the pit position are not performed, so as to improve the accuracy and robustness of the pit detection.

[0077] In some embodiments of the present application, the second pit detection result includes a pit position, the determination of the persistence state of the second pit detection result, and the determination of the final pit detection result according to the persistence state of the second pit detection result include: acquiring multiple frames of second road data corresponding to the pit position, and performing pit detection on each frame of the second road data to obtain a third pit detection result; determining an accumulated count corresponding to the pit position according to the third pit detection result; and determining the persistence state of the second pit detection result according to the accumulated count corresponding to the pit position.

[0078] In order to further improve the accuracy of pit detection, the embodiments of the present application further track and determine the persistence state of the second pit detection result, that is, whether the pit position in the second pit detection result can be stably detected multiple times.

[0079] Specifically, the multiple frames of second road data collected for the pit position can be further detected and analyzed. Each frame of data can be detected once using the scheme of the foregoing embodiments. When the detection result meets the count requirement corresponding to the second pit detection result, the count of the pit position is incremented by 1. Finally, the accumulated count of the pit position can be obtained. The size of the accumulated count reflects that the pit position can be accurately and stably detected within a period of time. Therefore, the persistence state of the second pit detection result can be determined according to the accumulated count of the pit position.

[0080] In some embodiments of the present application, the determination of the final pit detection result according to the persistence state of the second pit detection result includes: if the accumulated count corresponding to the pit position reaches a preset count threshold, the final pit detection result is determined according to the second pit detection result and the third pit detection result, and the final pit detection result includes a pit position and a pit attribute; otherwise, the final pit detection result is determined to be no pit.

[0081] If the accumulated count corresponding to the pit position reaches the preset count threshold, it indicates that the detected pit position is accurate and can be stably detected. The final pit detection result can be further determined by combining the second pit detection result and the multiple frames of third pit detection result. The third pit detection result is combined for comprehensive determination because it is desired that the final pit detection result not only includes a pit position and a pit size, but also includes pit depth information, so as to provide more reference for the vehicle end. However, the pit depth information can only be provided by laser point cloud data. If the second pit detection result is detected based on an image detection model, and the point cloud detection model does not detect, then the second pit detection result can only provide pit position and pit size information, and cannot provide pit depth information.

[0082] But the third pit detection result of the subsequent detection is the detection result of multiple frames of data, and the point cloud detection model does not detect the pit in the data frame corresponding to the second pit detection result, which does not mean that it cannot be detected in subsequent data frames. After continuous detection of multiple frames of data, the third pit detection result may contain the pit detection result of the point cloud detection model, and thus the depth information of the pit position can be compensated.

[0083] In addition, it should be noted that in the process of detecting the multiple frames of second road data, some frames may be discarded due to the occlusion of dynamic targets, and therefore the cumulative count of the pit position in the above embodiments does not require continuous counting, that is, as long as the cumulative count within the preset time reaches the threshold requirement. Therefore, if the cumulative count of the pit position within the preset time reaches the threshold requirement, it can be considered that the pit in the second pit detection result is actually present, and the pit position should be at least detected by one detection model without dynamic target occlusion. However, one possible situation is that the pit position is not detected without dynamic target occlusion, and although the final cumulative count meets the requirement, it may not be an accurate result.

[0084] Based on this, after labeling and counting the pit position in the second pit detection result, if the pit is not detected for several consecutive frames without dynamic target occlusion, the labeling of the pit position can be canceled and the previous cumulative count can be cleared, which can to some extent constrain the accuracy of the cumulative count result.

[0085] The embodiments of the present application also provide a road pit detection device 200, as shown in Figure 2 The structure diagram of a road pit detection device in the embodiments of the present application is provided, and the device 200 comprises a first detection unit 210, a second detection unit 220, a first determination unit 230 and a second determination unit 240, wherein:

[0086] The first detection unit 210 is configured to obtain first road data and perform pit detection on the first road data by using a preset pit detection model to obtain a first pit detection result.

[0087] The second detection unit 220 is configured to perform dynamic target detection on the first road data according to the first pit detection result to obtain a dynamic target detection result corresponding to the first pit detection result.

[0088] The first determination unit 230 is configured to determine a second pit detection result according to the first pit detection result and the dynamic target detection result corresponding to the first pit detection result.

[0089] The second determination unit 240 is configured to determine a persistence state of the second pit detection result, and determine a final pit detection result according to the persistence state of the second pit detection result.

[0090] In some embodiments of the present application, the first road data includes road image data and laser point cloud data, and the first detection unit 210 is specifically configured to: perform pit detection on the road image data by using a preset image detection model to obtain a pit detection result of the road image data; perform pit detection on the laser point cloud data by using a preset point cloud detection model to obtain a pit detection result of the laser point cloud data; and determine the first pit detection result according to the pit detection result of the road image data and the pit detection result of the laser point cloud data.

[0091] In some embodiments of the present application, the first detection unit 210 is specifically configured to: if at least one of the pit detection result of the road image data and the pit detection result of the laser point cloud data is a pit, determine that the first pit detection result is a pit; otherwise, determine that the first pit detection result is no pit.

[0092] In some embodiments of the present application, the first pit detection result includes a pit position, and the second detection unit 220 is specifically configured to: detect a dynamic target at the pit position; if a dynamic target is detected at the pit position, determine whether a detection frame of the dynamic target and a pit detection frame corresponding to the pit position exist intersection; if yes, determine that a dynamic target detection result corresponding to the first pit detection result is a dynamic target; otherwise, determine that the dynamic target detection result corresponding to the first pit detection result is no dynamic target.

[0093] In some embodiments of the present application, the first determination unit 230 is specifically configured to: if the first pit detection result is a pit and the dynamic target detection result is no dynamic target, directly take the first pit detection result as the second pit detection result, and mark and count the pit position in the second pit detection result; if the first pit detection result is a pit and the dynamic target detection result is a dynamic target, discard the first pit detection result.

[0094] In some embodiments of the present application, the second pit detection result includes a pit position, and the second determination unit 240 is specifically configured to: obtain multiple frames of second road data corresponding to the pit position, and perform pit detection on each frame of the second road data to obtain a third pit detection result; determine a cumulative count corresponding to the pit position according to the third pit detection result; and determine a persistence state of the second pit detection result according to the cumulative count corresponding to the pit position.

[0095] In some embodiments of the present application, the second determining unit 240 is specifically configured to: if the accumulated count corresponding to the pit position reaches a preset count threshold, determine a final pit detection result according to the second pit detection result and the third pit detection result, the final pit detection result including a pit position and a pit attribute; otherwise, determine that the final pit detection result is no pit.

[0096] It can be understood that the road pit detection device described above can realize each step of the road pit detection method provided in the foregoing embodiments, and the related explanations about the road pit detection method are all applicable to the road pit detection device, which will not be repeated here.

[0097] Figure 3 is a structural schematic diagram of an electronic device of an embodiment of the present application. Please refer to Figure 3 At the hardware level, the electronic device includes a processor, and optionally further includes an internal bus, a network interface, and a memory. The memory can include a memory such as a random-access memory (RAM), and can also include a non-volatile memory such as at least one disk memory. Of course, the electronic device can also include other hardware required by the business.

[0098] The processor, the network interface, and the memory can be connected to each other through the internal bus, which can be an industry standard architecture (ISA) bus, a peripheral component interconnect (PCI) bus, or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, and a control bus. For ease of representation, Figure 3 In the figure, only one bidirectional arrow is used to represent the bus, but it does not mean that there is only one bus or only one type of bus.

[0099] The memory is used to store programs. Specifically, the program can include program code, and the program code includes computer operation instructions. The memory can include a memory and a non-volatile memory, and provides instructions and data to the processor.

[0100] The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs, and forms a road pit detection device at the logical level. The processor executes the program stored in the memory, and is specifically configured to perform the following operations:

[0101] Acquire first road data, and perform pothole detection on the first road data using a preset pothole detection model to obtain a first pothole detection result;

[0102] performing dynamic target detection on the first road data according to the first pothole detection result to obtain a dynamic target detection result corresponding to the first pothole detection result;

[0103] determining a second pothole detection result based on the first pothole detection result and a dynamic target detection result corresponding to the first pothole detection result;

[0104] Determine a persistence status of the second pothole detection result, and determine a final pothole detection result based on the persistence status of the second pothole detection result.

[0105] The above application Figure 1 The methods performed by the pothole detection device disclosed in the illustrated embodiments can be applied to or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be performed by hardware integrated logic circuits within the processor or by software instructions. The processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The methods, steps, and logic block diagrams disclosed in the embodiments of this application can be implemented or executed. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly executed by a hardware decoding processor or by a combination of hardware and software modules within the decoding processor. The software module can be located in a storage medium well-known in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the memory, and the processor reads the information in the memory and, in conjunction with its hardware, completes the steps of the above method.

[0106] The electronic device may also perform Figure 1 The method of the road pothole detection device is implemented in Figure 1The functions of the embodiments shown are not repeated here.

[0107] The embodiments of the present application also provide a computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by an electronic device comprising a plurality of application programs, can enable the electronic device to perform the functions of the embodiments shown. Figure 1 The method performed by the road pit detection device in the embodiments shown, and specifically for performing:

[0108] Obtain first road data, and perform pit detection on the first road data by using a preset pit detection model to obtain a first pit detection result;

[0109] Perform dynamic target detection on the first road data according to the first pit detection result to obtain a dynamic target detection result corresponding to the first pit detection result;

[0110] Determine a second pit detection result according to the first pit detection result and the dynamic target detection result corresponding to the first pit detection result;

[0111] Determine the persistence state of the second pit detection result, and determine a final pit detection result according to the persistence state of the second pit detection result.

[0112] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage media, etc.) containing computer-usable program code.

[0113] The present application is described with reference to flowcharts and / or block diagrams of the method, device (system), and computer program product according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to produce a machine, so that the instructions executed by the computer or other programmable data processing devices produce a device that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 The device that implements the functions specified in one flow or multiple flows and / or blocks. Figure 1 The device that implements the functions specified in one flow or multiple flows and / or blocks.

[0114] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the Figure 1 function specified in the flow or flows and / or blocks Figure 1 of the block or blocks.

[0115] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the Figure 1 function specified in the flow or flows and / or blocks ​ of the block or blocks.

[0116] In one typical configuration, the computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0117] The memory can include non-persistent memory and / or volatile memory, such as random access memory (RAM) about which the computer stores information about an operating system, application software, and / or the like. Memory is an example of computer readable media.

[0118] Computer readable media includes permanent and non-permanent, moveable and non- moveable media that can be implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile discs (DVDs) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that is accessible to a computing device. According to the definition provided herein, a computer readable medium excludes transitory media, such as modulated data signals and carrier waves.

[0119] It is also to be noted that the terms "comprising", "including", and any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises a... " does not, without more constraints, exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the recited element.

[0120] Those skilled in the art will appreciate that embodiments of the present application can be devised for a method, a system, or a computer program product. Accordingly, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) embodying computer-readable program code.

[0121] The embodiments of the present application described above are only used to explain the technical solutions of the present application and not to limit the present application. Although the present application has been described in detail, those skilled in the art will understand that the present application can make various modifications and changes without departing from the spirit and scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the scope of the claims of the present application.

Claims

1. A method for detecting potholes in a road, wherein: The method comprises: Acquire first road data, and perform pothole detection on the first road data using a preset pothole detection model to obtain a first pothole detection result; performing dynamic target detection on the first road data according to the first pothole detection result to obtain a dynamic target detection result corresponding to the first pothole detection result; determining a second pothole detection result based on the first pothole detection result and a dynamic target detection result corresponding to the first pothole detection result; determining a persistence status of the second pothole detection result, and determining a final pothole detection result based on the persistence status of the second pothole detection result; Determining a second pothole detection result based on the first pothole detection result and a dynamic target detection result corresponding to the first pothole detection result includes: If the first pothole detection result indicates a pothole is present and the dynamic target detection result indicates no dynamic target, directly using the first pothole detection result as the second pothole detection result, and marking and counting the pothole positions in the second pothole detection result; If the first pothole detection result indicates that there is a pothole, and the dynamic target detection result indicates that there is a dynamic target, discarding the first pothole detection result; The count refers to the accumulation of the number of times a pothole is detected at the same location, and is used to reflect the persistence of the pothole location.

2. The method according to claim 1, wherein: The first road data includes road image data and laser point cloud data, and the performing pothole detection on the first road data using a preset pothole detection model to obtain a first pothole detection result includes: Performing pothole detection on the road image data using a preset image detection model to obtain a pothole detection result of the road image data; Performing pothole detection on the laser point cloud data using a preset point cloud detection model to obtain a pothole detection result of the laser point cloud data; The first pothole detection result is determined according to the pothole detection result of the road image data and the pothole detection result of the laser point cloud data.

3. The method according to claim 2, wherein: Determining the first pothole detection result according to the pothole detection result of the road image data and the pothole detection result of the laser point cloud data includes: If at least one of the pothole detection result of the road image data and the pothole detection result of the laser point cloud data indicates that a pothole exists, determining that the first pothole detection result is that a pothole exists; Otherwise, it is determined that the first pothole detection result is no pothole.

4. The method according to claim 1, wherein: The first pothole detection result includes a pothole location, and performing dynamic target detection on the first road data according to the first pothole detection result to obtain a dynamic target detection result corresponding to the first pothole detection result includes: detecting a dynamic target at the pothole location; If a dynamic target is detected at the pothole location, determining whether a detection frame of the dynamic target and a pothole detection frame corresponding to the pothole location intersect; If so, determining that the dynamic target detection result corresponding to the first pothole detection result is a dynamic target; Otherwise, it is determined that the dynamic target detection result corresponding to the first pothole detection result is no dynamic target.

5. The method of claim 1, wherein: The second pothole detection result includes a pothole location, and determining a persistence state of the second pothole detection result and determining a final pothole detection result based on the persistence state of the second pothole detection result includes: acquiring multiple frames of second road data corresponding to the pothole location, and performing pothole detection on each frame of the second road data to obtain a third pothole detection result; determining a cumulative count corresponding to the pothole position according to the third pothole detection result; The persistence state of the second pothole detection result is determined according to the accumulated count corresponding to the pothole position.

6. The method of claim 5, wherein: Determining the final pothole detection result according to the persistence state of the second pothole detection result includes: If the accumulated count corresponding to the pothole location reaches a preset count threshold, determining a final pothole detection result based on the second pothole detection result and the third pothole detection result, the final pothole detection result including the pothole location and pothole attributes; Otherwise, the final pothole detection result is determined to be no pothole.

7. A road pothole detection device, wherein: The device comprises: a first detection unit, configured to acquire first road data, and perform pothole detection on the first road data using a preset pothole detection model to obtain a first pothole detection result; a second detection unit, configured to perform dynamic target detection on the first road data according to the first pothole detection result, and obtain a dynamic target detection result corresponding to the first pothole detection result; a first determining unit, configured to determine a second pothole detection result based on the first pothole detection result and a dynamic target detection result corresponding to the first pothole detection result; a second determining unit, configured to determine a persistence state of the second pothole detection result, and determine a final pothole detection result based on the persistence state of the second pothole detection result; The first determining unit is specifically configured to: If the first pothole detection result indicates a pothole is present and the dynamic target detection result indicates no dynamic target, directly using the first pothole detection result as the second pothole detection result, and marking and counting the pothole positions in the second pothole detection result; If the first pothole detection result indicates that there is a pothole, and the dynamic target detection result indicates that there is a dynamic target, discarding the first pothole detection result; The count refers to the accumulation of the number of times a pothole is detected at the same location, and is used to reflect the persistence of the pothole location.

8. An electronic device comprising: processor; as well as A memory arranged to store computer executable instructions, which when executed cause the processor to perform the method of any one of claims 1 to 6.

9. A computer-readable storage medium storing one or more programs, which, when executed by an electronic device including a plurality of application programs, causes the electronic device to execute the method according to any one of claims 1 to 6.

Images