Damage detection method and device, electronic equipment and storage medium

Abstract

The method comprises: obtaining an original image of a preset object by a preset vehicle; obtaining an image of a first target of the preset object based on the original image; obtaining a detection result of the image of the first target based on the image of the first target; detecting a continuous length of a non-edge pixel in the image of the first target row by row or column by column according to the detection result; determining whether the continuous length of the non-edge pixel exceeds a preset damage threshold, wherein the preset damage threshold is determined according to a relative distance between the preset vehicle and the preset object when the original image is obtained; and determining that a row or a column where the non-edge pixel is located is damaged if the continuous length of the non-edge pixel exceeds the preset damage threshold. The disclosure improves the accuracy of guardrail damage detection in a complex environment.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of automatic driving, and in particular to a damage detection method and device, an electronic device and a storage medium. BACKGROUND

[0002] In scenarios such as parks, airports and ports, there is a large demand for damage detection of protective fences. Manual detection of protective fences is time-consuming and inefficient, and it is not suitable for entering harsh environments, making it difficult to achieve 24-hour continuous detection. With the development of automatic driving technology, the application of unmanned patrol vehicles is gradually increasing. Common applications of unmanned patrol vehicles mainly include automatic cruising, video transmission and remote control, and the application of damage detection of protective fences is currently less common.

[0003] Traditional computer vision methods for detecting damage to protective fences usually need to preset a region of interest in an image, perform edge detection on the region of interest, and then detect the damage location based on a preset threshold. Although this method has good interpretability, it requires a large number of preconfigured parameters and lacks flexibility and adaptability. For example, it is difficult to adapt to dynamic changes in the region of interest during driving, and it is difficult to adapt to changes in the distance between the patrol vehicle and the protective fence. Therefore, traditional methods are difficult to adapt to the demand for damage detection of protective fences in complex actual scenarios. SUMMARY

[0004] To solve the above technical problems or at least partially solve the above technical problems, the embodiments of the present disclosure provide a damage detection method, device, electronic device and storage medium, which dynamically determine the relative distance between a preset vehicle and a preset object when the preset vehicle acquires an original image, and realize dynamic damage detection of protective fences in complex environments, thereby improving the damage detection accuracy of protective fences.

[0005] In a first aspect, the embodiments of the present disclosure provide a damage detection method, which comprises:

[0006] acquiring, by a preset vehicle, an original image of a preset object;

[0007] based on the original image, acquiring an image of a first target of the preset object;

[0008] based on the image of the first target, obtaining a detection result of the image of the first target;

[0009] detecting, according to the detection result, the continuous length of non-edge pixels in the image of the first target row by row or column by column;

[0010] judging whether the continuous length of the non-edge pixels exceeds a preset damage threshold, wherein the preset damage threshold is determined according to the relative distance between the preset vehicle and the preset object when the preset vehicle acquires the original image;

[0011] If the continuous length of the non-edge pixel exceeds a preset breakage threshold, it is determined that a row or a column where the non-edge pixel is located has breakage.

[0012] In a second aspect, the embodiments of the present disclosure further provide a breakage detection device, which comprises:

[0013] The acquisition module is configured to acquire an original image of a preset object by a preset vehicle, acquire an image of a first target of the preset object based on the original image, and obtain a detection result of the image of the first target based on the image of the first target.

[0014] The detection module is configured to detect, according to the detection result, a continuous length of a non-edge pixel in the image of the first target row by row or column by column.

[0015] The judgment module is configured to judge whether the continuous length of the non-edge pixel exceeds a preset breakage threshold, wherein the preset breakage threshold is determined according to a relative distance between the preset vehicle and the preset object when the original image is acquired.

[0016] The output module is configured to determine that a row or a column where the non-edge pixel is located has breakage if the continuous length of the non-edge pixel exceeds the preset breakage threshold.

[0017] In a third aspect, the embodiments of the present disclosure further provide an electronic device, which comprises one or more processors, a storage device configured to store one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors implement the breakage detection method as described above.

[0018] In a fourth aspect, the embodiments of the present disclosure further provide a computer-readable storage medium having a computer program stored thereon, and the program is executed by a processor to implement the breakage detection method as described above.

[0019] The breakage detection method provided by the embodiments of the present disclosure solves the problem of insufficient breakage detection accuracy of a protective fence caused by the change of the distance between a vehicle and the protective fence during the driving of an autonomous vehicle by dynamically setting a preset breakage threshold, detecting the continuous length of a non-edge pixel row by row or column by column, judging whether the continuous length of the non-edge pixel exceeds the preset breakage threshold, and determining that a row or a column where the non-edge pixel is located has breakage. BRIEF DESCRIPTION OF DRAWINGS

[0020] The above and other features, advantages, and aspects of the embodiments of the present disclosure will become more apparent as various embodiments of the present disclosure are described in conjunction with the accompanying drawings, in which like reference numbers in the drawings indicate identical or similar elements. It should be understood that the drawings are not to scale, and the elements and features are not necessarily drawn to scale.

[0021] Figure 1 A flow chart of a breakage detection method in an embodiment of the present disclosure;

[0022] Figure 2 A flow chart of a breakage detection method in an embodiment of the present disclosure;

[0023] Figure 3 A flow chart of a cascade edge detector detection method in an embodiment of the present disclosure;

[0024] Figure 4 A structural schematic diagram of a wire mesh in a guardrail in an embodiment of the present disclosure;

[0025] Figure 5 A structural schematic diagram of a guardrail in an embodiment of the present disclosure;

[0026] Figure 6 A flow chart of a breakage detection method in an embodiment of the present disclosure;

[0027] Figure 7 A structural schematic diagram of a breakage detection device in an embodiment of the present disclosure;

[0028] Figure 8 A structural schematic diagram of an electronic device in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0029] Embodiments of the present disclosure will be described in more detail with reference to the drawings. Although some embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein, but rather the embodiments are provided so that the present disclosure can be more thoroughly and completely understood. It should be understood that the drawings and embodiments of the present disclosure are merely for illustrative purposes and are not intended to limit the scope of protection of the present disclosure.

[0030] It should be noted that the terms "first", "second", and the like in the present disclosure are merely used to distinguish different devices, modules or units, and do not imply the order or interdependence of the functions performed by these devices, modules or units.

[0031] The names of the messages or information exchanged between the devices in the embodiments of the present disclosure are merely for illustrative purposes, and are not intended to limit the scope of the messages or information.

[0032] Generally, the traditional computer vision method for guardrail damage detection usually needs to preset the region of interest in the image, perform edge detection on the region of interest, and then detect the damage position according to the preset threshold. Although it has good interpretability, it needs to rely on a large number of pre-configuration parameters, and lacks flexibility and adaptability. For example, it is difficult to adapt to the dynamic changes of the region of interest during driving, and it is difficult to adapt to the changes of the distance between the patrol vehicle and the guardrail. Therefore, the traditional method is difficult to adapt to the demand for guardrail damage detection in complex actual scenes.

[0033] To solve the above problems, the disclosure embodiment provides a damage detection method for dynamic damage detection of guardrails in complex environments. Figure 1 A flowchart of a damage detection method in the disclosure embodiment. The method can be executed by a damage detection device, which can be implemented in software and / or hardware, and can be configured in an electronic device, such as a server. As shown in the figure, the method can specifically include the following steps: Figure 1

[0034] Step 110: Obtain an original image of a preset object through a preset vehicle.

[0035] In some embodiments, the original image of the preset object obtained by the preset vehicle is image data collected by a sensor of an autonomous vehicle, wherein the preset vehicle is an autonomous vehicle, and the preset object is a guardrail. Specifically, the sensor of the autonomous vehicle includes a laser radar, a millimeter wave radar, a camera, etc., and the present application mainly refers to the image data of the guardrail and other objects collected by the camera of the autonomous vehicle. Among them, the camera of the autonomous vehicle can be multiple, installed at different positions of the vehicle body, so as to simultaneously collect images of the guardrail and other objects from different angles. For example, one camera is installed on each side of the vehicle body, and images of the guardrails on both sides are collected simultaneously.

[0036] Step 120: Based on the original image, obtain an image of a first target of the preset object.

[0037] In an embodiment, the original image is preprocessed to obtain a preprocessed image; the preprocessed image is input into a pre-trained deep learning semantic segmentation model to obtain an image of a first target of the preset object. Specifically, the guardrail image collected by the camera of the autonomous vehicle is preprocessed in the form of format adjustment (such as color image to grayscale image), size adjustment, regularization, etc. to generate a preprocessed image.

[0038] The preprocessed image is obtained by designing and training a deep learning semantic segmentation model, and the specific process is as follows:

[0039] The region of interest in the image collected by the sensor is manually labeled (for example, as shown in the figure, the region of interest in the image is manually labeled as a guardrail region of interest), and the labeled image is used as a training set to train the deep learning semantic segmentation model. Figure 5 ​As shown, the guardrail is composed of cement piers, columns and wire mesh, wherein the wire mesh is the region of interest, a deep learning semantic network is designed and trained to realize semantic segmentation of the region of interest by the deep learning network, and a deep learning semantic segmentation model is obtained; the preprocessed image is segmented into an irregular region in the image by the deep learning semantic segmentation model, an outer quadrilateral of the irregular region is obtained, and the outer quadrilateral is taken as a dynamic region of interest. The image of the region of interest is taken as the image of the first target. The dynamic region of interest is the region of interest segmented by the deep learning model according to the environment.

[0040] Step 130, based on the image of the first target, obtaining the detection result of the image of the first target.

[0041] In an embodiment, the image of the first target is input into a cascaded edge detector to obtain the detection result of the image of the first target.

[0042] Specifically, as shown in Figure 2 The image of the first target is input into a cascaded edge detector to obtain the edge density of the first target; wherein the number of cascaded edge detectors is at least two, the edge detection algorithms associated with different edge detectors are the same, and the parameter values of the edge detection algorithms associated with different edge detectors are different; it is judged whether the edge density reaches the edge density threshold; if the edge density reaches the edge density threshold, the cascaded edge detector is exited, and the detection result of the image of the first target is output; wherein the detection result includes edge pixels and non-edge pixels.

[0043] The edge density threshold is an empirical value, which is set according to needs, such as different light intensity numbers collected in different scenes, different time periods and / or different weather, and the specific value is set to 0.3, 0.5, etc.

[0044] The image of the first target is input into a cascaded edge detector to obtain the edge density of the first target; wherein the number of cascaded edge detectors is at least two, the edge detection algorithms associated with different edge detectors are the same, and the parameter values of the edge detection algorithms associated with different edge detectors are different; it is judged whether the edge density reaches the edge density threshold; if the edge density reaches the edge density threshold, the cascaded edge detector is exited, and the detection result of the image of the first target is output; wherein the detection result includes edge pixels and non-edge pixels.

[0045] In an embodiment, as shown in Figure 3 The image of the first target is input into a cascaded edge detector to obtain the edge density of the first target, which includes:

[0046] inputting the image of the first target into a first edge detector in the cascaded edge detectors to obtain a first edge density;

[0047] Correspondingly, the judging whether the edge density reaches the edge density threshold comprises:

[0048] judging whether the first edge density reaches the edge density threshold;

[0049] Correspondingly, the outputting the detection result of the image of the first target by the first edge detector if the first edge density reaches the edge density threshold comprises:

[0050] if the first edge density reaches the edge density threshold, the first edge detector is exited and the detection result of the image of the first target is outputted by the first edge detector.

[0051] In an embodiment, the edge detectors are cascaded according to the density detection thresholds respectively associated with the edge detectors according to a preset rule, to obtain the cascaded edge detectors; wherein the parameters of the edge detection algorithm associated with the edge detectors comprise the density detection thresholds.

[0052] Specifically, the density detection thresholds are determined based on different light intensity data collected in different time periods and / or under different weather conditions, and the edge detectors are cascaded according to the order from high to low of the density detection thresholds, to generate the cascaded edge detectors. The advantage of this setting is that after the image of the first target is obtained, the image of the first target is first inputted into the edge detector with a higher density detection threshold, and if the edge detector cannot detect the image of the target, the image of the first target is inputted into the edge detector with a lower density detection threshold, so as to adaptively find a suitable edge detector for the image of the first target to perform edge detection, thereby ensuring the edge detection accuracy on the premise of ensuring the edge detection rate. It can be understood that the higher the density detection threshold is, the more difficult it is to detect the edge of the preset object.

[0053] The first edge detector is associated with a first density detection threshold and a second density detection threshold, the first density detection threshold is a first high density detection threshold, and the second density detection threshold is a first low density detection threshold. Before the first edge density is obtained, edge pixels are obtained first.

[0054] Specifically, for a target pixel in the image of the first target, if the gradient of the target pixel is greater than the first high density detection threshold, the target pixel is determined as an edge pixel.

[0055] If the gradient of the target pixel is less than the first low density detection threshold, the target pixel is determined as a non-edge pixel;

[0056] If the gradient of the target pixel is less than the first high density detection threshold and greater than the first low density detection threshold, whether the target pixel is an edge pixel is determined based on a set rule. Wherein, whether the target pixel is an edge pixel is determined based on the set rule as follows: if the target pixel is connected with a reference pixel whose gradient is greater than the first high density detection threshold, the target pixel is determined as an edge pixel. After the edge pixel is determined, the ratio of the number of the image edge pixels of the first target to the number of the pixels of the image of the target is calculated to obtain a first edge density. Wherein, the gradient is a first derivative in a required direction (such as x direction or y direction), which can be understood as the position where the adjacent pixels of the image change most in the relevant direction.

[0057] In an embodiment, the reference Figure 3 As shown in FIG. 1, if the first edge density does not reach the edge density threshold, the image of the first target is input to a second edge detector in the cascade edge detector to obtain a second edge density; wherein, the first edge detector comprises a first high density detection threshold and a first low density detection threshold, the second edge detector comprises a second high density detection threshold and a second low density detection threshold; the second high density detection threshold is less than the first high density detection threshold.

[0058] Specifically, the edge detector has an associated edge detection algorithm, such as a Canny algorithm, wherein the Canny algorithm associated with the first edge detector can set a first high-density detection threshold and a first low-density detection threshold, and thus a plurality of Canny algorithms can be cascaded according to the density detection threshold from high to low, thereby cascading different edge detectors to generate a cascaded edge detector. The cascaded edge detector further includes a second edge detector and a third edge detector, the second edge detector corresponding to a second density detection threshold, the second edge detector including a second high-density detection threshold and a second low-density detection threshold, the third edge detector corresponding to a third density detection threshold, the third density detection threshold including a third high-density detection threshold and a third low-density detection threshold, the high-density detection threshold and the low-density detection threshold being monotonically decreasing, but the low threshold of the first edge detector does not necessarily have to be greater than the high threshold of the second edge detector. For example, the first high-density detection threshold and the first low-density detection threshold are 200 and 100 respectively, the second high-density detection threshold and the second low-density detection threshold are 150 and 90 respectively, and the third high-density detection threshold and the third low-density detection threshold are 80 and 50 respectively. The high-density detection threshold and the low-density detection threshold are cascaded in a list according to the order from high to low: high-density detection threshold [200, 150, 80] low-density detection threshold [100, 90, 50]. By such a cascading manner, the edge detectors are sorted according to the density detection threshold from high to low to generate a cascaded detector.

[0059] In another embodiment, continuing to refer to Figure 3 As shown, if the second edge density does not reach the edge density threshold, the image of the first target is input to the next edge detector cascaded with the second edge detector, and until it is determined that the edge density obtained by the last edge detector in the cascaded edge detector still does not reach the edge density threshold, the cascaded edge detector is exited and a notification message of undetectable is output.

[0060] Step 140, according to the detection result, detecting the length of the non-edge pixel in the image of the first target row by row or column by column;

[0061] Referring to Figure 4 As shown, according to the result output by the cascaded detector, it is determined that the edge pixel and the non-edge pixel can be detected, and then the length A of the non-edge pixel (the number of non-edge pixels) is detected row by row or column by column, or the length between adjacent edge pixels is detected, thereby determining the length of the non-edge pixel.

[0062] Step 150, judging whether the continuous length of the non-edge pixel exceeds a preset damage threshold, wherein the preset damage threshold is determined according to the relative distance between the preset vehicle and the preset object when the original image is acquired;

[0063] In an embodiment, a trained key point detection model based on deep learning is acquired; the original image of the preset object is input into the key point detection model to acquire key points; the height of the second target of the preset object and the width between the second targets of the preset object are obtained according to the key points; the relative distance between the vehicle and the preset object is obtained according to the height of the second target of the preset object and the width between the second targets of the preset object; and the preset damage threshold is determined according to the relative distance. Reference Figures 4-5 As shown, the original image of the preset object is a guardrail image, the guardrail includes a cement block, a column and a wire mesh; the first target is the wire mesh, and the second target is the column; according to the height of the column and the width between the columns, the relative distance between the vehicle and the guardrail can be determined, and the preset damage threshold is dynamically determined according to the relative distance; specifically, for example, if the preset column height is 2 m and the column height pixel value is x when the vehicle is 1.5 m away from the guardrail, then the column height pixel value is x / 2 when the vehicle is 3 m away from the guardrail; the farther the actual distance between the vehicle and the guardrail, the larger the space represented by the unit pixel, and the two are inversely proportional; for example Figure 5 In the embodiment, the wire mesh hole exceeding 0.5 m is determined as damaged, the corresponding pixel distance is x when the vehicle is 1 m away from the guardrail, and the corresponding pixel distance is x / 2 when the vehicle is 2 m away from the guardrail; it is not necessary to convert to the actual distance, but only to obtain the pixel distance of the column height or the width between each other; the preset damage threshold is dynamically set in the above manner, which has the advantage of improving the accuracy of damage detection of the autonomous vehicle in different environments and different scenarios, and solving the problem that a single damage threshold cannot meet the actual needs of users.

[0064] Step 160, if the continuous length of the non-edge pixel exceeds the preset damage threshold, it is determined that there is damage in the row or column where the non-edge pixel is located.

[0065] Continuing to refer to Figure 4 As shown, the non-edge pixel is the black part of the wire mesh, the continuous length of the non-edge pixel is A, and it is determined that the continuous length A exceeds the preset damage threshold, so it is determined that there is damage in the row where the non-edge pixel is located; or it is determined that there is damage in the column where the non-edge pixel is located when the continuous length of the non-edge pixel exceeds the preset damage threshold.

[0066] In another example, whether the barbed wire is damaged can also be determined by the edge pixels, which are the white parts of the barbed wire. By the distance between adjacent edge pixels, by determining the continuous length B of the edge pixels and the continuous length of the adjacent edge pixels, the length of the discontinuity in the middle is determined, and the length of the discontinuity is determined as the continuous length of the non-edge pixels, so that the continuous length A exceeding the preset damage threshold can be further determined, and it is determined that the row where the non-edge pixels are located is damaged.

[0067] The damage detection method provided in the embodiment solves the problem of the accuracy of the detection of the dynamic changes of the region of interest in the driving process of the autonomous vehicle in various weather and light conditions and the detection of the damaged region of the protection device in a complex environment.

[0068] Figure 6 The flowchart of another damage detection method in the embodiment of the present disclosure. Based on the above-mentioned embodiment, after determining that the row or column where the non-edge pixels are located is damaged, the step of how to determine the damaged region is further added. The damaged region includes at least two rows or columns where damage exists. As shown in the figure, the method specifically further includes the following steps: Figure 6

[0069] Step 210: obtaining an original image of a preset object by a preset vehicle.

[0070] Step 211: obtaining a first target image of the preset object based on the original image.

[0071] Step 212: obtaining a detection result of the first target image based on the first target image.

[0072] Step 213: detecting the continuous length of non-edge pixels in the first target image row by row or column by column according to the detection result.

[0073] Step 214: determining whether the continuous length of the non-edge pixels exceeds a preset damage threshold, wherein the preset damage threshold is determined according to the relative distance between the preset vehicle and the preset object when the original image is obtained.

[0074] Step 215: if the continuous length of the non-edge pixels exceeds the preset damage threshold, determining that the row or column where the non-edge pixels are located is damaged.

[0075] Step 216: determining whether the continuous length of the non-edge pixels in the continuous row or column starting from the row or column where the non-edge pixels are located in the first target image exceeds the preset damage threshold according to the row or column where the non-edge pixels are located.

[0076] ​Step 217, if yes, determine whether the image of the first target has a damaged area.

[0077] On the basis of the above-mentioned embodiments, with reference to Figure 4 , the continuous length of the non-edge pixels in the continuous row or column is continuously detected, and it is determined whether the continuous length of the non-edge pixels exceeds the preset damage threshold, so as to determine whether the continuous length of the non-edge pixels exceeds the preset damage threshold, and determine whether the wire mesh has a damaged area area.

[0078] In another embodiment, with reference to Figure 6 After the damaged area is determined, the damaged area can be tracked, and the specific steps include:

[0079] Step 218, obtaining the bounding box of the damaged area in the image of the first target.

[0080] Step 219, tracking the bounding box of the damaged area based on a target tracking algorithm to obtain a tracking result, so as to continuously locate the damaged area.

[0081] Specifically, with reference to Figure 4 , the bounding box of the damaged area area in the wire mesh image is determined, and the known target tracking algorithm is used to track the bounding box of the damaged area, so as to continuously locate the damaged area.

[0082] In another embodiment, with reference to Figure 6 , the step further includes:

[0083] Step 220, according to the tracking result, mapping the damaged area to the original image to obtain an original damaged area of the original image, and giving a prompt.

[0084] Specifically, the position coordinates of the bounding box of the damaged area in the image of the first target are obtained, the original position coordinates of the damaged area mapped to the original image are marked at the corresponding position of the original image according to the position coordinates of the bounding box, the original bounding box of the original image is obtained according to the original position coordinates, and the original damaged area of the original image is obtained according to the original bounding box.

[0085] Another damaged area detection method in the embodiments of the present disclosure can continuously locate the damaged area through further tracking of the damaged area, and obtain more accurate damaged area detection accuracy.

[0086] The technical scheme of the embodiment of the present disclosure solves the problems that the existing guardrail damage detection method is difficult to adapt to the dynamic change of the region of interest during vehicle driving, difficult to adapt to the selection of reasonable edge detection parameters under various weather and lighting conditions, and difficult to adapt to the changing distance between the vehicle and the guardrail during the patrol, and achieves the purpose of continuously patrolling and detecting the damage of the guardrail by the vehicle. Specifically, the stable edge detection is realized by the adaptive cascading edge detector; the accurate selection of the region of interest in the dynamic scene is realized by locating the guardrail region in the image by training the semantic segmentation model; the distance detection between the vehicle and the guardrail is realized by detecting the key point information in the guardrail by training the key point detection model, so as to combine the distance between the vehicle and the guardrail for guardrail damage detection, achieve the purpose of improving the detection accuracy, and further realize the gap detection of the guardrail.

[0087] Figure 7 FIG. 1 is a structural schematic diagram of a damage detection device according to an embodiment of the present disclosure. As shown in the figure, the device comprises an acquisition module 410, a detection module 420, a judgment module 430 and an output module 440. The acquisition module 410 is configured to acquire an original image of a preset object by a preset vehicle; based on the original image, acquire an image of a first target of the preset object; based on the image of the first target, obtain a detection result of the image of the first target. The detection module 420 is configured to detect the continuous length of non-edge pixels in the image of the first target row by row or column by column according to the detection result. The judgment module 430 is configured to judge whether the continuous length of the non-edge pixels exceeds a preset damage threshold, wherein the preset damage threshold is determined according to the relative distance between the preset vehicle and the preset object when the original image is acquired. The output module 440 is configured to determine that there is damage in the row or column where the non-edge pixels are located if the continuous length of the non-edge pixels exceeds the preset damage threshold. Figure 7

[0088] Optionally, the device further comprises a first determination module configured to, after determining that there is damage in the row or column where the non-edge pixels are located, judge whether the continuous length of the non-edge pixels in the continuous rows or continuous columns in the image of the first target starting from the row or column where the non-edge pixels are located exceeds the preset damage threshold according to the row or column where the non-edge pixels are located; if yes, determine that there is a damage region in the image of the first target.

[0089] ​Optionally, the apparatus further comprises a second determining module configured to: acquire a trained key point detection model based on deep learning; input the original image of the preset object into the key point detection model to acquire key points; acquire a height of a second target of the preset object and a width between the second target of the preset object according to the key points; acquire the relative distance between the vehicle and the preset object according to the height of the second target of the preset object and the width between the second target of the preset object; and determine the preset damage threshold according to the relative distance.

[0090] Optionally, the apparatus further comprises a tracking module configured to: acquire a bounding box of the damage area in the image of the first target; track the bounding box of the damage area based on a target tracking algorithm to acquire a tracking result, so as to continuously locate the damage area.

[0091] Optionally, the apparatus further comprises a tracking prompting module configured to: map the damage area to the original image according to the tracking result to acquire an original damage area of the original image, and give a prompt.

[0092] Optionally, the tracking prompting module comprises an acquisition unit configured to: acquire position coordinates of a bounding box of a damage area of an image of a first target; mark original position coordinates of the damage area mapped to the original image at a corresponding position of the original image according to the position coordinates of the bounding box; acquire an original bounding box of the original image according to the original position coordinates; and acquire an original damage area of the original image according to the original bounding box.

[0093] Optionally, the acquisition module 410 comprises a first acquisition unit configured to input the image of the first target into a cascaded edge detector to acquire a detection result of the image of the first target.

[0094] Optionally, the first acquisition unit comprises: an input subunit configured to input the image of the first target into a cascaded edge detector to acquire an edge density of the first target; wherein the number of the cascaded edge detectors is at least two, the edge detection algorithms associated with different edge detectors are the same, and the parameter values of the edge detection algorithms associated with different edge detectors are different; a judgment subunit configured to judge whether the edge density reaches an edge density threshold; and an output subunit configured to, if the edge density reaches the edge density threshold, exit the cascaded edge detector and output a detection result of the image of the first target; wherein the detection result comprises edge pixels and non-edge pixels.

[0095] Optionally, the first obtaining unit further comprises a cascading subunit configured to cascade each of the edge detectors according to density detection thresholds respectively associated with each of the edge detectors in a preset rule to obtain the cascaded edge detectors; wherein the parameters of the edge detection algorithm associated with the edge detector include the density detection threshold.

[0096] Optionally, the cascading subunit is specifically configured to cascade each of the edge detectors according to the density detection thresholds from high to low in the preset rule.

[0097] Optionally, the density detection threshold and the edge density threshold are determined based on sample data, and the sample data is different light intensity data collected at different time periods and / or under different weather conditions for the preset object.

[0098] Optionally, the input subunit is specifically configured to input the image of the first target to a first edge detector in the cascaded edge detectors to obtain a first edge density; correspondingly, the judgment subunit is specifically configured to judge whether the first edge density reaches an edge density threshold; correspondingly, the output subunit is specifically configured to, if the first edge density reaches the edge density threshold, exit the first edge detector and output a detection result of the image of the first target by the first edge detector.

[0099] Optionally, the input subunit is further configured to, if the first edge density does not reach the edge density threshold, input the image of the first target to a second edge detector in the cascaded edge detectors to obtain a second edge density; wherein the second edge detector is cascaded with the first edge detector; the first edge detector includes a first high-density detection threshold and a first low-density detection threshold, and the second edge detector includes a second high-density detection threshold and a second low-density detection threshold; and the second high-density detection threshold is less than the first high-density detection threshold.

[0100] Optionally, the input subunit is further configured to, if the second edge density does not reach the edge density threshold, input the image of the first target to a next edge detector cascaded with the second edge detector, until it is determined that the edge density obtained by a last edge detector in the cascaded edge detectors still does not reach the edge density threshold, then exit the cascaded edge detectors and output a notification message that detection is impossible.

[0101] Optionally, the first edge detector is associated with a first density detection threshold and a second density detection threshold; the input subunit is specifically used for target pixels in the image of the first target. If the gradient of the target pixel is greater than the first density detection threshold, the target pixel is determined to be an edge pixel; if the gradient of the target pixel is less than the second density detection threshold, the target pixel is determined to be a non-edge pixel; if the gradient of the target pixel is less than the first density detection threshold and greater than the second density detection threshold, the target pixel is determined to be an edge pixel based on a set rule.

[0102] Optionally, the input sub-unit is specifically used to determine that the target pixel is an edge pixel if the target pixel is connected to a reference pixel whose gradient is greater than the first density detection threshold.

[0103] Optionally, the acquisition module 410 includes a preprocessing unit for preprocessing the original image to obtain a preprocessed image; and an input unit for inputting the preprocessed image into a pre-trained deep learning semantic segmentation model to obtain an image of the first target of the preset object.

[0104] The damage detection device provided in this embodiment can execute the steps in the damage detection method provided in this embodiment, and has the execution steps and beneficial effects, which will not be repeated here.

[0105] Figure 8 This is a schematic diagram of the structure of an electronic device according to an embodiment of this disclosure. See below for details. Figure 8 It shows a schematic diagram of a structure suitable for implementing the electronic device 500 in the embodiments of this disclosure. Figure 8 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0106] like Figure 8 As shown, the electronic device 500 may include a processing device (e.g., a central processing unit, a graphics processor, etc.) 501, which can perform various appropriate actions and processes to implement the methods of the embodiments described herein, based on a program stored in a read-only memory (ROM) 502 or a program loaded from a storage device 508 into a random access memory (RAM) 503. The RAM 503 also stores various programs and data required for the operation of the electronic device 500. The processing device 501, ROM 502, and RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0107] In particular, in accordance with embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for executing the methods illustrated by the flowcharts, thereby implementing the damage detection method as described above. In such embodiments, the computer program can be downloaded and installed from a network by the communication device 509, or installed from the storage device 508, or installed from the ROM 502. When the computer program is executed by the processing device 501, the above-mentioned functions defined in the methods of the embodiments of the present disclosure are performed.

[0108] It should be noted that the computer-readable medium described above in the present disclosure can be a computer-readable signal medium or a computer-readable storage medium or any combination thereof. The computer-readable storage medium, for example, can be, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any suitable combination thereof. More specific examples of the computer-readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, the computer-readable storage medium can be any tangible medium that contains or stores a program used or used in conjunction with an instruction execution system, apparatus, or device. In the present disclosure, the computer-readable signal medium can include a data signal carried in a baseband or as a part of a carrier wave, which carries computer-readable program code. Such a propagated data signal can take on many forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium other than the computer-readable storage medium that can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained in the computer-readable medium can be transmitted by any suitable medium, including but not limited to a wire, a cable, a RF (radio frequency) or the like, or any suitable combination thereof.

[0109] The computer readable medium can be included in the electronic device or can exist separately from the electronic device. The computer readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: acquire an original image of a preset object by a preset vehicle; acquire an image of a first target of the preset object based on the original image; obtain a detection result of the image of the first target based on the image of the first target; detect, according to the detection result, a continuous length of a non-edge pixel in the image of the first target row by row or column by column; determine whether the continuous length of the non-edge pixel exceeds a preset damage threshold, wherein the preset damage threshold is determined according to a relative distance between the preset vehicle and the preset object when the original image is acquired; and determine that a row or a column where the non-edge pixel is located is damaged if the continuous length of the non-edge pixel exceeds the preset damage threshold.

[0110] Optionally, when the one or more programs are executed by the electronic device, the electronic device can further perform other steps described in the above embodiments.

[0111] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the above. More specific examples of the machine-readable storage medium can include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0112] Scheme 1, a damage detection method, the method comprising:

[0113] acquiring an original image of a preset object by a preset vehicle;

[0114] acquiring an image of a first target of the preset object based on the original image;

[0115] obtaining a detection result of the image of the first target based on the image of the first target;

[0116] detecting, according to the detection result, a continuous length of a non-edge pixel in the image of the first target row by row or column by column;

[0117] determine whether the continuous length of the non-edge pixel exceeds a preset damage threshold, wherein the preset damage threshold is determined according to a relative distance between the preset vehicle and the preset object when the original image is acquired;

[0118] If the continuous length of the non-edge pixel exceeds the preset damage threshold, it is determined that there is damage in the row or column where the non-edge pixel is located.

[0119] Scheme 2, according to the method of scheme 1, after determining that the row or column where the non-edge pixel is located has damage, the method further comprises:

[0120] According to the row or column where the non-edge pixel is located, it is determined whether the continuous length of the non-edge pixel in the continuous row or column starting from the row or column where the non-edge pixel is located in the image of the first target exceeds a preset damage threshold;

[0121] If yes, it is determined that there is a damage area in the image of the first target.

[0122] Scheme 3, according to the method of scheme 1, further comprising:

[0123] Acquire a trained key point detection model based on deep learning;

[0124] Input the original image of the preset object into the key point detection model to acquire key points;

[0125] According to the key points, obtain the height of the second target of the preset object and the width between the second target of the preset object;

[0126] According to the height of the second target of the preset object and the width between the second target of the preset object, obtain the relative distance between the vehicle and the preset object;

[0127] According to the relative distance, determine the preset damage threshold.

[0128] Scheme 4, according to the method of scheme 2, the method further comprises:

[0129] Acquire the bounding box of the damage area in the image of the first target;

[0130] Track the bounding box of the damage area based on a target tracking algorithm to obtain a tracking result, so as to continuously locate the damage area.

[0131] Scheme 5, according to the method of scheme 4, further comprising:

[0132] According to the tracking result, map the damage area to the original image to obtain an original damage area of the original image, and give a prompt.

[0133] In scheme 6, the method according to scheme 5, the mapping of the damaged area to the original image according to the tracking result to obtain the original damaged area of the original image comprises:

[0134] Obtaining position coordinates of a bounding box of the damaged area of the image of the first target;

[0135] According to the position coordinates of the bounding box, marking the original position coordinates of the mapping of the damaged area to the original image at the corresponding positions of the original image;

[0136] According to the original position coordinates, obtaining an original bounding box of the original image;

[0137] According to the original bounding box, obtaining an original damaged area of the original image.

[0138] In scheme 7, the method according to scheme 1, the obtaining of the detection result of the image of the first target comprises:

[0139] Inputting the image of the first target into a cascaded edge detector to obtain the detection result of the image of the first target.

[0140] In scheme 8, the method according to scheme 7, the inputting of the image of the first target into a cascaded edge detector to obtain the detection result of the image of the first target comprises:

[0141] Inputting the image of the first target into a cascaded edge detector to obtain an edge density of the first target; wherein the number of the cascaded edge detectors is at least two, the edge detection algorithms associated with different edge detectors are the same, and the parameter values of the edge detection algorithms associated with different edge detectors are different.

[0142] Judging whether the edge density reaches an edge density threshold value;

[0143] If the edge density reaches the edge density threshold value, exiting the cascaded edge detector and outputting the detection result of the image of the first target; wherein the detection result comprises edge pixels and non-edge pixels.

[0144] In scheme 9, the method according to scheme 8 further comprises:

[0145] According to a preset rule, cascading the edge detectors according to the density detection threshold values respectively associated with the edge detectors one by one to obtain the cascaded edge detectors;

[0146] The parameters of the edge detection algorithm associated with the edge detector include the density detection threshold value.

[0147] Solution 10: According to the method described in Solution 9, the step of cascading each edge detector according to a preset rule based on a density detection threshold associated with each edge detector includes:

[0148] The edge detectors are cascaded in descending order of the density detection threshold.

[0149] Option 11: According to the method described in Option 10, the density detection threshold and the edge density threshold are determined based on sample data, which are different light intensity data collected for the preset object at different time periods and / or under different weather conditions.

[0150] Solution 12: According to the method described in Solution 8, the step of inputting the image of the first target into a cascaded edge detector to obtain the edge density of the first target includes:

[0151] The image of the first target is input into the first edge detector in the cascaded edge detector to obtain the first edge density;

[0152] Correspondingly, determining whether the edge density reaches the edge density threshold includes:

[0153] Determine whether the first edge density reaches the edge density threshold;

[0154] Correspondingly, if the edge density reaches the edge density threshold, the cascaded edge detector is exited, and the detection result of the first target image is output, including:

[0155] If the first edge density reaches the edge density threshold, the first edge detector is exited, and the detection result of the first target image is output through the first edge detector.

[0156] Solution 13: According to the method described in Solution 12, if the first edge density does not reach the edge density threshold, the image of the first target is input to the second edge detector in the cascaded edge detector to obtain the second edge density;

[0157] The second edge detector is cascaded with the first edge detector.

[0158] The first edge detector includes a first high-density detection threshold and a first low-density detection threshold, and the second edge detector includes a second high-density detection threshold and a second low-density detection threshold;

[0159] The second high-density detection threshold is less than the first high-density detection threshold.

[0160] The method according to any one of solutions 1-17, wherein the method further comprises:

[0161] If the second edge density does not reach the edge density threshold, inputting the image of the first target to a next edge detector cascaded with the second edge detector until determining that the edge density obtained by a last edge detector in the cascaded edge detectors still does not reach the edge density threshold, then exiting the cascaded edge detectors and outputting a notification message that detection is failed.

[0162] The method according to solution 12, wherein the first edge detector is associated with a first density detection threshold and a second density detection threshold.

[0163] The method according to any one of solutions 1-17, wherein the method further comprises:

[0164] For a target pixel in the image of the first target, if a gradient of the target pixel is greater than the first density detection threshold, determining the target pixel as an edge pixel.

[0165] If the gradient of the target pixel is less than the second density detection threshold, determining the target pixel as a non-edge pixel.

[0166] If the gradient of the target pixel is less than the first density detection threshold and greater than the second density detection threshold, determining whether the target pixel is an edge pixel based on a set rule.

[0167] The method according to solution 15, wherein the determining whether the target pixel is an edge pixel based on the set rule comprises:

[0168] If the target pixel is connected with a reference pixel having a gradient greater than the first density detection threshold, determining the target pixel as an edge pixel.

[0169] The method according to any one of solutions 1-16, wherein the obtaining the image of the first target of the preset object based on the original image comprises:

[0170] Preprocessing the original image to obtain a preprocessed image.

[0171] Inputting the preprocessed image to a pre-trained deep learning semantic segmentation model to obtain the image of the first target of the preset object.

[0172] Solution 18, a damage detection device, comprising:

[0173] The acquisition module is configured to acquire an original image of a preset object by a preset vehicle, acquire an image of a first target of the preset object based on the original image, and obtain a detection result of the image of the first target based on the image of the first target.

[0174] The detection module is configured to detect a continuous length of a non-edge pixel in the image of the first target row by row or column by row according to the detection result.

[0175] The judgment module is configured to judge whether the continuous length of the non-edge pixel exceeds a preset damage threshold, wherein the preset damage threshold is determined according to a relative distance between the preset vehicle and the preset object when the original image is acquired.

[0176] The output module is configured to determine that a row or a column where the non-edge pixel is located is damaged if the continuous length of the non-edge pixel exceeds the preset damage threshold.

[0177] Scheme 19, an electronic device, the electronic device comprising:

[0178] one or more processors;

[0179] a storage device for storing one or more programs;

[0180] When the one or more programs are executed by the one or more processors, the one or more processors implement the method of any one of schemes 1-17.

[0181] Scheme 20, a computer readable storage medium, having stored thereon a computer program, the program being executed by a processor to implement the method of any one of schemes 1-17.

[0182] The above description is merely preferred embodiments of the present disclosure and a description of the principles of the technology used. Those skilled in the art should understand that the disclosed scope of the present disclosure is not limited to the technical solutions formed by the specific combinations of the above technical features, and should also cover other technical solutions formed by any combinations of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features disclosed in the present disclosure (but not limited to) having similar functions to form technical solutions.

Claims

1. A method of breakage detection, characterized in that, The method comprises: obtaining an original image of a preset object by a preset vehicle; based on the original image, obtaining an image of a first target of the preset object; based on the image of the first target, obtaining a detection result of the image of the first target; according to the detection result, detecting the continuous length of non-edge pixels in the image of the first target row by row or column by column; determining whether the continuous length of the non-edge pixels exceeds a preset damage threshold, wherein the preset damage threshold is determined according to the relative distance between the preset vehicle and the preset object when the original image is obtained, and the relative distance is obtained according to the height of the second target of the preset object and the width between the second target of the preset object, and the height of the second target of the preset object and the width between the second target of the preset object are obtained according to the key points; if the continuous length of the non-edge pixels exceeds the preset damage threshold, it is determined that the row or column where the non-edge pixels are located is damaged.

2. The method of claim 1, wherein, After determining that the row or column where the non-edge pixels are located is damaged, the method further comprises: determining whether the continuous length of the non-edge pixels in the continuous row or column starting from the row or column where the non-edge pixels are located in the image of the first target exceeds the preset damage threshold according to the row or column where the non-edge pixels are located; if so, it is determined that the image of the first target has a damaged area.

3. The method of claim 1, wherein, Further comprising: obtaining a trained key point detection model based on deep learning; inputting the original image of the preset object into the key point detection model to obtain key points; obtaining the height of the second target of the preset object and the width between the second target of the preset object according to the key points; obtaining the relative distance between the vehicle and the preset object according to the height of the second target of the preset object and the width between the second target of the preset object; determining the preset damage threshold according to the relative distance.

4. The method of claim 2, wherein, The method further comprises: obtaining a bounding box of the damaged area in the image of the first target; tracking the bounding box of the damaged area based on a target tracking algorithm to obtain a tracking result for continuous positioning of the damaged area.

5. The method of claim 4, wherein, Further comprising: mapping the damaged area to the original image according to the tracking result to obtain an original damaged area of the original image and giving a prompt.

6. The method of claim 5, wherein, The mapping of the damaged area to the original image according to the tracking result to obtain the original damaged area of the original image comprises: obtaining the position coordinates of the bounding box of the damaged area in the image of the first target; labeling the original position coordinates of the mapping of the damaged area to the original image at the corresponding position of the original image according to the position coordinates of the bounding box; obtaining an original bounding box of the original image according to the original position coordinates; obtaining an original damaged area of the original image according to the original bounding box.

7. The method of claim 1, wherein, The obtaining of the detection result of the image of the first target comprises: inputting the image of the first target into a cascaded edge detector to obtain the detection result of the image of the first target.

8. The method of claim 7, wherein, The image of the first target is input into the cascaded edge detector, and a detection result of the image of the first target is obtained, including: The image of the first target is input into the cascaded edge detector, and an edge density of the first target is obtained; wherein the number of the cascaded edge detector is at least two, the edge detection algorithms associated with different edge detectors are the same, and the parameter values of the edge detection algorithms associated with different edge detectors are different; It is judged whether the edge density reaches an edge density threshold value; If the edge density reaches the edge density threshold value, the cascaded edge detector is exited, and a detection result of the image of the first target is output; wherein the detection result includes edge pixels and non-edge pixels.

9. The method of claim 8, wherein, Further comprising: According to a preset rule, the edge detectors are cascaded according to the density detection threshold values respectively associated with each of the edge detectors, and the cascaded edge detector is obtained; The parameters of the edge detection algorithm associated with the edge detector include the density detection threshold value.

10. The method of claim 9, wherein, The cascading of the edge detectors according to the density detection threshold values respectively associated with each of the edge detectors according to the preset rule includes: The edge detectors are cascaded in the order from high to low of the density detection threshold values.

11. The method of claim 10, wherein, The density detection threshold value and the edge density threshold value are determined based on sample data, and the sample data is different light intensity data collected at different time periods and / or under different weather conditions for the preset object.

12. The method of claim 8, wherein, The image of the first target is input into the cascaded edge detector, and an edge density of the first target is obtained, including: The image of the first target is input into the first edge detector in the cascaded edge detector, and a first edge density is obtained; Correspondingly, the judgment of whether the edge density reaches the edge density threshold value includes: It is judged whether the first edge density reaches the edge density threshold value; Correspondingly, if the edge density reaches the edge density threshold value, the cascaded edge detector is exited, and a detection result of the image of the first target is output, including: If the first edge density reaches the edge density threshold value, the first edge detector is exited, and a detection result of the image of the first target is output through the first edge detector.

13. The method of claim 12, wherein, If the first edge density does not reach the edge density threshold value, the image of the first target is input into the second edge detector in the cascaded edge detector, and a second edge density is obtained; The second edge detector is cascaded with the first edge detector; The first edge detector includes a first high-density detection threshold value and a first low-density detection threshold value, and the second edge detector includes a second high-density detection threshold value and a second low-density detection threshold value; The second high-density detection threshold value is less than the first high-density detection threshold value.

14. The method of claim 13, wherein, Further comprising: If the second edge density does not reach the edge density threshold, the image of the first target is input to a next edge detector cascaded with the second edge detector until it is determined that the edge density obtained by the last edge detector in the cascaded edge detectors still does not reach the edge density threshold, then the cascaded edge detectors are exited and a notification message that detection is failed is output.

15. The method of claim 12, wherein, The first edge detector is associated with a first density detection threshold and a second density detection threshold; The inputting of the image of the first target to the first edge detector in the cascaded edge detectors to obtain a first edge density comprises: For a target pixel in the image of the first target, if a gradient of the target pixel is greater than the first density detection threshold, the target pixel is determined as an edge pixel; If the gradient of the target pixel is less than the second density detection threshold, the target pixel is determined as a non-edge pixel; If the gradient of the target pixel is less than the first density detection threshold and greater than the second density detection threshold, whether the target pixel is an edge pixel is determined based on a set rule.

16. The method of claim 15, wherein, The determination of whether the target pixel is an edge pixel based on the set rule comprises: If the target pixel is connected with a reference pixel whose gradient is greater than the first density detection threshold, the target pixel is determined as an edge pixel.

17. The method according to any one of claims 1 to 16, characterized in that, The obtaining of the image of the first target of the preset object based on the original image comprises: The original image is preprocessed to obtain a preprocessed image; The preprocessed image is input to a pre-trained deep learning semantic segmentation model to obtain the image of the first target of the preset object.

18. A breakage detection apparatus characterized by comprising: Comprise: An obtaining module is configured to obtain an original image of a preset object by a preset vehicle, and obtain an image of a first target of the preset object based on the original image; Obtain a detection result of the image of the first target based on the image of the first target; A detecting module is configured to detect a continuous length of a non-edge pixel in the image of the first target row by row or column by column according to the detection result; A judging module is configured to judge whether the continuous length of the non-edge pixel exceeds a preset damage threshold, wherein the preset damage threshold is determined according to a relative distance between the preset vehicle and the preset object when the original image is obtained, the relative distance is obtained according to a height of a second target of the preset object and a width between the second target of the preset object, and the height of the second target of the preset object and the width between the second target of the preset object are obtained according to key points; An output module is configured to determine that a row or a column where the non-edge pixel is located is damaged if the continuous length of the non-edge pixel exceeds the preset damage threshold.

19. An electronic device, comprising: The electronic device comprises: One or more processors; A storage device configured to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method in any one of claims 1-17.

20. A computer readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the method in any one of claims 1-17.

Images