Damage detection method and device, electronic equipment and storage medium

Abstract

The method comprises the following steps: an original image of a preset object is acquired by a preset vehicle; a binary mask image of the preset object is acquired based on the original image; an image of interest of the preset object is acquired according to the binary mask image; whether the image of interest exists a first target is judged according to the image of interest; if the image of interest exists the first target, the image of interest is input into a pre-trained target detection model, and a damage area of the first target is obtained. The present disclosure realizes 24-hour unmanned detection, improves the dynamic damage detection efficiency, makes up for the shortage of manual work, and reduces the production cost.

Description

Technical Field

[0001] This disclosure relates to the field of autonomous driving technology, and in particular to a damage detection method, apparatus, electronic device, and storage medium. Background Technology

[0002] In recent years, security issues at industrial parks and airports have received increasing attention from society. Barbed wire fencing around these areas is a crucial component of security, and both intentional and unintentional damage to it can lead to serious security risks. Therefore, effectively detecting damage to barbed wire fencing has become a key security concern for major airports both domestically and internationally.

[0003] In existing technologies, the common practice is to have park security inspectors conduct regular patrols. However, arranging a large number of inspectors is time-consuming, labor-intensive, and costly. Installing cameras near the fence and having security personnel monitor the footage from the control room can also lead to fatigue, making it impossible to constantly review and analyze the video, which may ultimately result in oversights of damage to the barbed wire. Summary of the Invention

[0004] To solve the above-mentioned technical problems, or at least partially solve them, this disclosure provides a damage detection method, apparatus, electronic device, and storage medium, which enables 24-hour real-time monitoring of guardrails, improves damage detection accuracy, and reduces labor costs.

[0005] In a first aspect, embodiments of this disclosure provide a damage detection method, the method comprising:

[0006] Obtain the original image of the preset object by using the preset vehicle;

[0007] Based on the original image, obtain the binary mask image of the preset object;

[0008] Based on the binary mask image, obtain the image of interest for the preset object;

[0009] Based on the image of interest, determine whether the first target exists in the image of interest;

[0010] If the first target is present in the image of interest, the image of interest is input into a pre-trained target detection model to obtain the damaged area of ​​the first target.

[0011] Secondly, this disclosure also provides a damage detection device, which includes:

[0012] The acquisition module is used to acquire an original image of a preset object through a preset vehicle; acquire a binary mask image of the preset object based on the original image; and acquire an image of interest of the preset object based on the binary mask image.

[0013] The judgment module is used to determine whether the first target exists in the image of interest based on the image of interest.

[0014] The output module is used to determine that the first target exists in the image of interest, input the image of interest into a pre-trained target detection model, and obtain the damaged area of ​​the first target.

[0015] Thirdly, embodiments of this disclosure also provide an electronic device, the electronic device comprising: one or more processors; a storage device for storing one or more programs; and when the one or more programs are executed by the one or more processors, causing the one or more processors to implement the damage detection method as described above.

[0016] Fourthly, embodiments of this disclosure also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the damage detection method described above.

[0017] This disclosure provides a damage detection method that, by determining the presence of a first target in an image of interest, enables the separate detection of different targets. The image of interest of the first target is input into a pre-trained target detection model to obtain the damaged area of ​​the first target. This method enables 24-hour real-time monitoring of guardrails, improves the damage detection accuracy of damaged areas, and reduces labor costs. Attached Figure Description

[0018] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0019] Figure 1 This is a flowchart of a damage detection method according to an embodiment of the present disclosure;

[0020] Figure 2 This is a schematic diagram of the structure of a guardrail according to an embodiment of this disclosure;

[0021] Figure 3 This is a schematic diagram of the structure of a guardrail according to an embodiment of this disclosure;

[0022] Figure 4 This is a flowchart of a damage detection method according to an embodiment of the present disclosure;

[0023] Figure 5 This is a schematic diagram of the structure of a tuple of interest in an embodiment of this disclosure;

[0024] Figure 6 This is a schematic diagram of the structure of a second target image in an embodiment of this disclosure;

[0025] Figure 7 This is a schematic diagram of the structure of a thinned first binary image and a thinned first binary image complement in an embodiment of this disclosure;

[0026] Figure 8 This is a schematic diagram of the structure of a first etched image and a second etched image in an embodiment of this disclosure;

[0027] Figure 9 This is a schematic diagram of the structure of a third erosion image in an embodiment of this disclosure;

[0028] Figure 10 This is a schematic diagram of the structure of a damage detection device according to an embodiment of this disclosure;

[0029] Figure 11 This is a schematic diagram of the structure of an electronic device according to an embodiment of this disclosure. Detailed Implementation

[0030] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0031] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0032] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0033] Security issues at industrial parks and airports are receiving increasing attention from society. Barbed wire fencing around these areas is a crucial component of security, and both intentional and unintentional damage to this fencing can lead to serious security risks. Therefore, effectively detecting damage to barbed wire fencing in real time has become a key security concern for major airports both domestically and internationally.

[0034] Secondly, parks and airports generally use security patrol personnel to conduct regular patrols, but arranging a large number of patrol personnel is time-consuming, labor-intensive, and costly; installing cameras near the fence and having security personnel check the monitoring in the monitoring room can also lead to fatigue periods, making it impossible to check and analyze the video at all times, which will eventually lead to oversights of damage to the barbed wire.

[0035] To address the aforementioned issues, this disclosure provides a damage detection method that solves the problem of 24-hour inspection of guardrails in different scenarios such as parks and airports, improving the accuracy of damage detection and reducing labor costs. Figure 1 This is a flowchart illustrating a damage detection method according to an embodiment of this disclosure. The method can be executed by a damage detection device, which can be implemented in software and / or hardware, and can be configured in an electronic device, such as a server. Figure 1 As shown, the method may specifically include the following steps:

[0036] Step 110: Obtain the original image of the preset object through the preset vehicle.

[0037] In some embodiments, the preset vehicle is an autonomous vehicle, and the preset object is a guardrail. Acquiring the original image of the preset object mainly involves obtaining guardrail image data captured by the autonomous vehicle's cameras. Specifically, the sensors of the autonomous vehicle include LiDAR, millimeter-wave radar, and cameras. This invention primarily refers to the image data of objects such as guardrails captured by the autonomous vehicle's cameras. The autonomous vehicle can have multiple cameras installed at different locations on the vehicle body to simultaneously capture images of the guardrails and other objects from different angles. For example, one camera can be installed on each of the left and right sides of the vehicle body to simultaneously capture images of the guardrails on both sides.

[0038] Step 120: Based on the original image, obtain the binary mask image of the preset object.

[0039] Figure 2 This is a schematic diagram of the structure of a guardrail according to an embodiment of the present disclosure; in one embodiment, as shown... Figure 2 As shown, the original image is input into a pre-trained image semantic segmentation model based on the cloud to obtain a binary mask image of the preset object.

[0040] Specifically, the original images of the guardrail captured by the camera of the autonomous vehicle are transmitted to the cloud via multiple original images through networks such as 4G. The original images of the guardrail are then processed based on the image semantic segmentation model trained in the cloud to obtain a binary mask image of the guardrail. Given the insufficient data processing capabilities of autonomous vehicles, this application utilizes cloud processing to greatly improve the efficiency of data processing.

[0041] Among these, images of interest (such as images collected by vehicle-mounted sensors) can be manually labeled. Figure 2 The guardrail consists of cement blocks, posts, and a ring of wire mesh (the ring of wire mesh being the image of interest). A deep learning image semantic segmentation network, Deeplabv3plus, is then constructed, and the network is trained using images of the ring and wire mesh respectively, resulting in a pre-trained image semantic segmentation model. During the use of this model, the original image is directly input into it to obtain a binary mask image of a preset object. Specifically, the image semantic segmentation model processes and analyzes the original image to obtain the probability distribution of the predicted category for each pixel. The category with the highest probability value is then used as the final category for the corresponding pixel, and the binary mask image of the preset object is obtained according to the above processing method. The binary mask image includes a second binary mask image corresponding to the first target and a first binary mask image corresponding to the second target; that is, the image semantic segmentation model can output the first and second binary mask images respectively.

[0042] Step 130: Obtain the image of interest of the preset object based on the binary mask image.

[0043] In one embodiment, preprocessing the binary mask image to obtain the image of interest for the preset object includes performing connected component analysis on the first and second binary mask images respectively, filtering out regions with smaller areas, thereby obtaining two types of images of interest respectively; such as Figure 2 As shown, the image of interest for the first category is the image of interest for the first target, such as... Figure 3 As shown, the image of interest for the second category is the image of interest for the second target, where the first target is a ring-shaped wire mesh and the second target is a linear wire mesh.

[0044] Step 140: Based on the image of interest, determine whether the image of interest contains a first target.

[0045] Step 150: If the first target exists in the image of interest, input the image of interest into a pre-trained target detection model to obtain the damaged area of ​​the first target.

[0046] In one embodiment, reference Figure 2As shown, based on the acquired image of interest, it is determined whether a ring-shaped wire mesh exists in the image of interest. Once the existence of a ring-shaped wire mesh is confirmed, the image of interest is input into a pre-trained object detection model to obtain the damaged area of ​​the ring-shaped wire mesh. The object detection model is built and trained using the stable shape of the damaged gap in the ring-shaped wire mesh as the target detection problem. Since the damaged area of ​​the ring-shaped wire mesh is mostly a background area without wire in the image, a model containing dilated convolutions is used as the backbone network to extract image features and increase the receptive field in order to better complete the gap detection task. At the same time, to improve the robustness of the model, a multi-scale pyramid feature fusion module is set to jointly enhance the accuracy of the object detection model.

[0047] This embodiment provides a damage detection method that, by determining the first target in the image of interest and using a target detection model to detect the damaged area of ​​the first target, solves the problems of low efficiency and inaccurate accuracy of manual detection, achieves 24-hour unmanned detection, improves the efficiency of dynamic damage detection, makes up for the shortage of manpower, and reduces production costs.

[0048] Figure 4 This is a flowchart of another damage detection method according to an embodiment of this disclosure. This embodiment, based on the above embodiments, performs further damage detection steps after determining that the first target does not exist in the image of interest, such as... Figure 4 As shown, the damage detection method specifically includes the following steps:

[0049] Step 210: Obtain the original image of the preset object through the preset vehicle.

[0050] Step 220: Based on the original image, obtain the binary mask image of the preset object.

[0051] Step 230: Obtain the image of interest of the preset object based on the binary mask image.

[0052] Step 240: Based on the image of interest, determine whether the image of interest contains a first target.

[0053] Step 241: If the first target exists in the image of interest, input the image of interest into a pre-trained target detection model to obtain the damaged area of ​​the first target.

[0054] Step 250: If the first target is not present in the image of interest, determine whether a second target exists in the image of interest.

[0055] Step 251: If the second target does not exist in the image of interest, the damage detection process ends.

[0056] In one embodiment, reference Figure 3 As shown, after determining whether there is a linear wire mesh in the image of interest and confirming that there is no circular wire mesh, step 251 is executed to end the damage detection process.

[0057] Step 252: If the image of interest contains the second target, input the first binary mask image into a preset algorithm to obtain a refined first binary image.

[0058] In one embodiment, reference Figure 3 As shown, the algorithm determines whether a linear wire mesh exists in the image of interest. After confirming the presence of a circular wire mesh, the first binary mask image, i.e., the linear wire mesh binary mask image, is input into the Zhang-Suen thinning algorithm for further thinning. This refines the line width to approximately one pixel, and the thinned first binary image is output. Specifically, the steps of the Zhang-Suen thinning algorithm to refine the first binary mask image include:

[0059] 1) Traverse all pixels of the first binary mask image to obtain the first pixel that meets the preset pixel value; wherein, the preset pixel value is set to 1.

[0060] 2) Determine whether the first pixel satisfies the iteration condition. If the first pixel satisfies the iteration condition, delete the first pixel and record the iteration count.

[0061] The iteration count is 1 for each pixel after filtering it according to the four conditions a, b, c, and d.

[0062] 3) Otherwise, exit the iteration and end the preset algorithm.

[0063] The iteration conditions include:

[0064] The sum of the number of first pixels in the eight neighboring regions of the current pixel is greater than or equal to a first threshold and less than or equal to a second threshold; for example, the first threshold is 2 and the second threshold is 6. The range of threshold values ​​in this application is not limited and can be selected according to actual needs.

[0065] b. In the eight neighborhoods of the current pixel, the number of times that two adjacent pixels change from 0 to the preset pixel value in a clockwise direction is equal to 1.

[0066] c. If the current iteration number is even, the product of the first current pixel (i-1, j), the second current pixel (i, j+1), and the third current pixel (i+1, j) in the eight neighborhood of the current pixel (i, j) is zero, or the product of the third current pixel (i+1, j), the second current pixel (i, j+1), and the fourth current pixel (i, j-1) in the eight neighborhood of the current pixel (i, j) is 0.

[0067] d. If the current iteration number is even, the product of the third current pixel (i+1, j), the second current pixel (i, j+1), and the fourth current pixel (i, j-1) in the eight neighbors of the current pixel (i, j) is zero; or, the product of the third current pixel (i+1, j), the first current pixel (i-1, j), and the fourth current pixel (i, j-1) in the eight neighbors of the current pixel (i, j) is 0. It should be noted that all four iteration conditions (a, b, c, and d) must be satisfied simultaneously.

[0068] Step 253: Construct an interest structure tuple containing preset interest structure elements; perform a hit-or-miss transformation algorithm on the thinned first binary image and the interest structure tuple to obtain the total number of target interest structure elements contained in the thinned first binary image.

[0069] In one embodiment, Figure 5 This is a schematic diagram of a structure of interest tuple containing a preset interest tuple in an embodiment of this disclosure. Figure 6 This is a structural diagram of a wire mesh adhesion and breakage disclosed in this embodiment. First, the type of wire mesh adhesion is preset, and different types of preset structural elements of interest are constructed. Different structural elements of interest are combined to form a structural element group of interest, as shown in the reference. Figure 5 As shown, the tuple of interest is B, and the tuple of interest is b. ； The relationship between the tuple of interest and the tuple of interest is as follows:

[0070] B={b0, b1, b2, b3, b4, b5, b6, b7, b8, b 10 b 11 b 12}

[0071] Will as Figure 6 The different types of thinned first binary images of the linear wire mesh shown are processed using a hit-and-miss transformation algorithm on each target structure element of the structure of interest (SIE) group to obtain the total number of target SIEs contained in the thinned first binary image; the specific steps of the hit-and-miss transformation algorithm are as follows:

[0072] like Figure 7-9 As shown, the first eroded image is obtained by eroding the thinned first binary image using any preset structuring element in the structuring element of interest.

[0073] The complement of the first binary image is eroded using the complement of any preset structuring element of interest to obtain a second eroded image;

[0074] The intersection of the first erosion image and the second erosion image is used to obtain the third erosion image;

[0075] Based on the third erosion image, the total number of target interest structural elements contained in the refined first binary image is obtained.

[0076] Step 260: Determine whether the total number of at least one type of target interest structuring elements is greater than a preset threshold;

[0077] Step 261: If the total number of target interest structural elements of at least one type is greater than a preset threshold, it is determined that the second target has adhesion and damage.

[0078] In one embodiment, the preset threshold can be set to 2, as referenced. Figure 9 As shown, based on the third erosion image, the total number of target interest structural elements contained in the refined first binary image is 3. The total number of target interest structural elements 3 is greater than the preset threshold 2, thus confirming that the linear wire mesh is adhered.

[0079] Step 262: If the total number of the at least one type of target interest structuring elements is less than a preset threshold, proceed to step 251.

[0080] In another embodiment, based on the third erosion image, assuming that the total number of target interest structural elements in the refined first binary image is 1, and the total number of target interest structural elements 1 is less than a preset threshold 2, step 251 is executed to end the damage detection process.

[0081] In one embodiment, the damage detection data is converted into alarm information with timestamps and GPS location, and transmitted to an intelligent alarm platform. The intelligent alarm platform then sends a wire mesh damage alarm to the monitoring screen and mobile client, prompting staff to inspect the damage. The damage detection data includes the damaged area of ​​a first target, or adhesive damage to a second target.

[0082] This disclosure discloses a damage detection method that solves the problems of inefficiency and low accuracy associated with manual inspection in existing technologies. It enables 24 / 7 real-time damage detection of different types of targets, increasing the scope of damage detection, improving accuracy, and reducing labor costs. Specifically, it determines whether the target is a first or second target based on the image of interest, thus expanding the scope of damage detection by detecting different target types. It employs a target detection model for the first target and a hit-or-miss transformation algorithm for the second target, selecting different model algorithms for different targets to improve damage detection accuracy.

[0083] Figure 10This is a schematic diagram of the structure of a damage detection device according to an embodiment of this disclosure. Figure 10 As shown: The device includes: an acquisition module 310, a judgment module 320, and an output module 330;

[0084] The acquisition module 310 is used to acquire an original image of a preset object through a preset vehicle; acquire a binary mask image of the preset object based on the original image; and acquire an image of interest of the preset object based on the binary mask image.

[0085] The judgment module 320 is used to determine whether a first target exists in the image of interest based on the image of interest.

[0086] The output module 330 is used to determine that the first target exists in the image of interest, input the image of interest into a pre-trained target detection model, and obtain the damaged area of ​​the first target.

[0087] Optionally, the device includes a determination output module, used to determine whether the first target does not exist in the image of interest, and to determine whether a second target exists in the image of interest;

[0088] If the second target is not found in the image of interest, the damage detection process ends.

[0089] Optionally, the binary mask image includes a first binary mask, and the device includes a thinning module for determining that the second target exists in the image of interest, and inputting the first binary mask image into a preset algorithm to obtain a thinned first binary image. The specific processing method of the thinning module is as follows:

[0090] Traverse all pixels of the first binary mask image to obtain the first pixel that meets the preset pixel value;

[0091] Determine whether the first pixel satisfies the iteration condition. If the first pixel satisfies the iteration condition, delete the first pixel and record the iteration count.

[0092] Otherwise, exit the iteration and end the preset algorithm.

[0093] The iteration conditions include:

[0094] The sum of the number of first pixels in the eight neighboring regions of the current pixel is greater than or equal to the first threshold and less than or equal to the second threshold;

[0095] And in the eight neighborhoods of the current pixel, the number of times two adjacent pixels change from 0 to the preset pixel value in a clockwise direction is equal to 1;

[0096] And if the current iteration number is even, the product of the first current pixel (i-1,j), the second current pixel (i,j+1), and the third current pixel (i+1,j) in the eight neighborhood of the current pixel (i,j) is zero, or the product of the third current pixel (i+1,j), the second current pixel (i,j+1), and the fourth current pixel (i,j-1) in the eight neighborhood of the current pixel (i,j) is 0;

[0097] And if the current iteration number is even, the product of the third current pixel (i+1,j), the second current pixel (i,j+1), and the fourth current pixel (i,j-1) in the eight neighborhood of the current pixel (i,j) is zero, or the product of the third current pixel (i+1,j), the first current pixel (i-1,j), and the fourth current pixel (i,j-1) in the eight neighborhood of the current pixel (i,j) is 0.

[0098] Optionally, the device further includes a first determining module for constructing a group of interest structs containing preset interest structs;

[0099] A hit-or-miss transformation algorithm is applied to the thinned first binary image and the target structure of interest tuple to obtain the total number of target structure of interest tuples contained in the thinned first binary image;

[0100] Determine whether the total number of at least one type of target interest structuring elements is greater than a preset threshold. If the total number of the at least one type of target interest structuring elements is greater than the preset threshold, it is determined that the second target has adhesion and damage.

[0101] Optionally, the device further includes a second determining module, used to terminate the damage detection process when the total number of the at least one type of target interest structural elements is less than a preset threshold.

[0102] Optionally, the first determining module includes a first determining acquisition unit, used to erode the thinned first binary image using any preset interest structure element in the interest structure tuple, to obtain a first eroded image;

[0103] The complement of the first binary image is eroded using the complement of any preset structuring element of interest to obtain a second eroded image;

[0104] The intersection of the first erosion image and the second erosion image is used to obtain the third erosion image;

[0105] Based on the third erosion image, the total number of target interest structural elements contained in the refined first binary image is obtained.

[0106] Optionally, the acquisition module 310 includes a first acquisition unit, used to input the original image into a cloud-based pre-trained image semantic segmentation model to obtain a binary mask image of the preset object.

[0107] Optionally, the first acquisition unit includes an acquisition subunit, used to acquire the probability distribution of the predicted category of each pixel in the original image, select the category with the highest probability value as the predicted category of the image semantic segmentation model, and output a binary mask image of a preset object, wherein the binary mask image includes a second binary mask image corresponding to the first target and a first binary mask image corresponding to the second target.

[0108] Optionally, the acquisition module 310 includes a second acquisition unit, used to preprocess the binary mask image to acquire the image of interest of the preset object.

[0109] Optionally, the device also includes a data processing module for converting damage detection data into alarm information with timestamps and GPS locations, and transmitting it to an intelligent alarm platform. The damage detection data includes the damaged area of ​​the first target or the presence of adhesive damage on the second target.

[0110] The damage detection device provided in this embodiment can execute the steps in the damage detection method provided in this embodiment, and has the execution steps and beneficial effects, which will not be repeated here.

[0111] Figure 11 This is a schematic diagram of the structure of an electronic device according to an embodiment of this disclosure. See below for details. Figure 11 It shows a schematic diagram of a structure suitable for implementing the electronic device 500 in the embodiments of this disclosure. Figure 11 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0112] like Figure 11 As shown, the electronic device 500 may include a processing device (e.g., a central processing unit, a graphics processor, etc.) 501, which can perform various appropriate actions and processes to implement the methods of the embodiments described herein, based on a program stored in a read-only memory (ROM) 502 or a program loaded from a storage device 508 into a random access memory (RAM) 503. The RAM 503 also stores various programs and data required for the operation of the electronic device 500. The processing device 501, ROM 502, and RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0113] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts, thereby implementing the damage detection method as described above. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 509, or installed from a storage device 508, or installed from a ROM 502. When the computer program is executed by the processing device 501, it performs the functions defined in the methods of embodiments of this disclosure.

[0114] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0115] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: acquire an original image of a preset object using a preset vehicle; acquire a binary mask image of the preset object based on the original image; acquire an image of interest (IOI) of the preset object based on the binary mask image; determine whether a first target exists in the IIO based on the IIO; and if the first target exists in the IIO, input the IIO into a pre-trained target detection model to obtain the damaged area of ​​the first target.

[0116] Optionally, when one or more of the above-described procedures are executed by the electronic device, the electronic device may also perform other steps described in the above embodiments.

[0117] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0118] Option 1: A damage detection method, the method comprising:

[0119] Obtain the original image of the preset object by using the preset vehicle;

[0120] Based on the original image, obtain the binary mask image of the preset object;

[0121] Based on the binary mask image, obtain the image of interest for the preset object;

[0122] Based on the image of interest, determine whether the first target exists in the image of interest;

[0123] If the first target is present in the image of interest, the image of interest is input into a pre-trained target detection model to obtain the damaged area of ​​the first target.

[0124] Option 2: The method described in Option 1, further comprising:

[0125] If the first target is not present in the image of interest, determine whether the second target is present in the image of interest;

[0126] If the second target is not found in the image of interest, the damage detection process ends.

[0127] Option 3: According to the method described in Option 2, the binary mask image includes a first binary mask image, and the method further includes:

[0128] If the second target exists in the image of interest, the first binary mask image is input into a preset algorithm to obtain a refined first binary image.

[0129] Solution 4: According to the method described in Solution 3, the step of inputting the first binary mask image into a preset algorithm to obtain a refined first binary image includes:

[0130] Traverse all pixels of the first binary mask image to obtain the first pixel that meets the preset pixel value;

[0131] Determine whether the first pixel satisfies the iteration condition. If the first pixel satisfies the iteration condition, delete the first pixel and record the iteration count.

[0132] Otherwise, exit the iteration and end the preset algorithm.

[0133] Option 5: According to the method described in Option 4, the iteration conditions include:

[0134] The sum of the number of first pixels in the eight neighboring regions of the current pixel is greater than or equal to the first threshold and less than or equal to the second threshold;

[0135] And in the eight neighborhoods of the current pixel, the number of times two adjacent pixels change from 0 to the preset pixel value in a clockwise direction is equal to 1;

[0136] And if the current iteration number is even, the product of the first current pixel (i-1,j), the second current pixel (i,j+1), and the third current pixel (i+1,j) in the eight neighborhood of the current pixel (i,j) is zero, or the product of the third current pixel (i+1,j), the second current pixel (i,j+1), and the fourth current pixel (i,j-1) in the eight neighborhood of the current pixel (i,j) is 0;

[0137] And if the current iteration number is even, the product of the third current pixel (i+1,j), the second current pixel (i,j+1), and the fourth current pixel (i,j-1) in the eight neighborhood of the current pixel (i,j) is zero, or the product of the third current pixel (i+1,j), the first current pixel (i-1,j), and the fourth current pixel (i,j-1) in the eight neighborhood of the current pixel (i,j) is 0.

[0138] Option 6: The method described in Option 3, further comprising:

[0139] Construct interest structure tuples containing preset interest structure elements;

[0140] A hit-or-miss transformation algorithm is applied to the thinned first binary image and the target structure of interest tuple to obtain the total number of target structure of interest tuples contained in the thinned first binary image;

[0141] Determine whether the total number of at least one type of target interest structuring elements is greater than a preset threshold. If the total number of the at least one type of target interest structuring elements is greater than the preset threshold, it is determined that the second target has adhesion and damage.

[0142] Option 7: The method described in Option 6 further includes:

[0143] If the total number of the at least one type of target interest structuring elements is less than a preset threshold, the damage detection process ends.

[0144] Solution 8: According to the method described in Solution 6, the step of performing a hit-and-miss transformation algorithm on the thinned first binary image and the target structure of interest (SPOI) to obtain the total number of target SPOIs contained in the thinned first binary image includes:

[0145] The first eroded image is obtained by eroding the thinned first binary image using any preset structuring element in the structuring element group of interest.

[0146] The complement of the first binary image is eroded using the complement of any preset structuring element of interest to obtain a second eroded image;

[0147] The intersection of the first erosion image and the second erosion image is used to obtain the third erosion image;

[0148] Based on the third erosion image, the total number of target interest structural elements contained in the refined first binary image is obtained.

[0149] Solution 9: According to the method described in Solution 1, based on the original image, obtain a binary mask image of the preset object, including:

[0150] The original image is input into a cloud-based pre-trained image semantic segmentation model to obtain a binary mask image of the preset object.

[0151] Solution 10: According to the method described in Solution 9, the original image is input into a pre-trained image semantic segmentation model based on the cloud to obtain a binary mask image of the preset object, including:

[0152] Obtain the probability distribution of the predicted category of each pixel in the original image, select the category with the highest probability value as the predicted category of the image semantic segmentation model, and output a binary mask image of a preset object. The binary mask image includes a second binary mask image corresponding to the first target and a first binary mask image corresponding to the second target.

[0153] Solution 11: According to the method described in Solution 1, obtain the image of interest of the preset object based on the binary mask image, including:

[0154] The binary mask image is preprocessed to obtain the image of interest for the preset object.

[0155] Option 12: The method described according to any one of Options 1-11, further comprising:

[0156] The damage detection data is converted into alarm information with timestamps and GPS locations and transmitted to the intelligent alarm platform. The damage detection data includes the damaged area of ​​the first target or the presence of adhesive damage on the second target.

[0157] Option 13: A damage detection device, comprising:

[0158] The acquisition module is used to acquire an original image of a preset object through a preset vehicle; acquire a binary mask image of the preset object based on the original image; and acquire an image of interest of the preset object based on the binary mask image.

[0159] The judgment module is used to determine whether the first target exists in the image of interest based on the image of interest.

[0160] The output module is used to determine that the first target exists in the image of interest, input the image of interest into a pre-trained target detection model, and obtain the damaged area of ​​the first target.

[0161] Option 14: An electronic device, the electronic device comprising:

[0162] One or more processors;

[0163] Storage device for storing one or more programs;

[0164] When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of schemes 1-12.

[0165] Option 15: A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method as described in any one of Options 1-12.

[0166] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

Claims

1. A damage detection method, characterized in that, The method includes: Obtain the original image of the preset object by using the preset vehicle; Based on the original image, obtain the binary mask image of the preset object; Based on the binary mask image, obtain the image of interest for the preset object; Based on the image of interest, determine whether the first target exists in the image of interest; If the first target exists in the image of interest, the image of interest is input into a pre-trained target detection model to obtain the damaged area of ​​the first target; If the first target is not present in the image of interest, determine whether the second target is present in the image of interest; If a second target exists in the image of interest, the image of interest is thinned, and the second target is detected to have adhesion or damage based on the hit-or-miss transformation algorithm; The method for detecting whether the second target has adhesion or damage based on the hit-or-miss transformation algorithm includes: Construct interest structure tuples containing preset interest structure elements; A hit-or-miss transformation algorithm is applied to the thinned first binary image and the target structure of interest tuple to obtain the total number of target structure of interest tuples contained in the thinned first binary image; Determine whether the total number of at least one type of target interest structural elements is greater than a preset threshold. If the total number of the at least one type of target interest structural elements is greater than the preset threshold, it is determined that the second target has adhesion damage. The hit-and-miss transformation algorithm applied to the thinned first binary image and the target structure of interest (SPIC) tuples to obtain the total number of target SPICs contained in the thinned first binary image includes: The thinned first binary image is eroded using any preset structuring element in the structuring element group of interest to obtain a first eroded image; The complement of the thinned first binary image is eroded using the complement of any preset structuring element of interest to obtain a second eroded image; The intersection of the first erosion image and the second erosion image is used to obtain the third erosion image; Based on the third erosion image, the total number of the target interest structuring elements contained in the thinned first binary image is obtained.

2. The method according to claim 1, characterized in that, The method further includes: If the second target is not found in the image of interest, the damage detection process ends.

3. The method according to claim 2, characterized in that, The binary mask image includes a first binary mask image, and the method further includes: If the second target exists in the image of interest, the first binary mask image is input into a preset algorithm to obtain a refined first binary image.

4. The method according to claim 3, characterized in that, The step of inputting the first binary mask image into a preset algorithm to obtain a refined first binary image includes: Traverse all pixels of the first binary mask image to obtain the first pixel that meets the preset pixel value; Determine whether the first pixel satisfies the iteration condition. If the first pixel satisfies the iteration condition, delete the first pixel and record the iteration count. Otherwise, exit the iteration and end the preset algorithm.

5. The method according to claim 4, characterized in that, The iteration conditions include: The sum of the number of first pixels in the eight neighboring regions of the current pixel is greater than or equal to the first threshold and less than or equal to the second threshold; And in the eight neighborhoods of the current pixel, the number of times two adjacent pixels change from 0 to the preset pixel value in a clockwise direction is equal to 1; And if the current iteration number is even, the product of the first current pixel (i-1,j), the second current pixel (i,j+1), and the third current pixel (i+1,j) in the eight neighborhood of the current pixel (i,j) is zero, or the product of the third current pixel (i+1,j), the second current pixel (i,j+1), and the fourth current pixel (i,j-1) in the eight neighborhood of the current pixel (i,j) is 0; And if the current iteration number is even, the product of the third current pixel (i+1,j), the second current pixel (i,j+1), and the fourth current pixel (i,j-1) in the eight neighborhood of the current pixel (i,j) is zero, or the product of the third current pixel (i+1,j), the first current pixel (i-1,j), and the fourth current pixel (i,j-1) in the eight neighborhood of the current pixel (i,j) is 0.

6. The method according to claim 1, characterized in that, The method also includes: If the total number of the at least one type of target interest structuring elements is less than a preset threshold, the damage detection process ends.

7. The method according to claim 1, wherein obtaining the binary mask image of the preset object based on the original image comprises: The original image is input into a cloud-based pre-trained image semantic segmentation model to obtain a binary mask image of the preset object.

8. The method according to claim 7, wherein inputting the original image into a pre-trained image semantic segmentation model based on the cloud to obtain a binary mask image of the preset object comprises: Obtain the probability distribution of the predicted category of each pixel in the original image, select the category with the highest probability value as the predicted category of the image semantic segmentation model, and output the binary mask image of the preset object. The binary mask image includes a second binary mask image corresponding to the first target and a first binary mask image corresponding to the second target.

9. The method according to claim 1, wherein obtaining the image of interest of the preset object based on the binary mask image comprises: The binary mask image is preprocessed to obtain the image of interest for the preset object.

10. The method according to any one of claims 1-9, further comprising: The damage detection data is converted into alarm information with timestamps and GPS location and transmitted to the intelligent alarm platform. The damage detection data includes the damaged area of ​​the first target or the adhesion damage of the second target.

11. A damage detection device, characterized in that, include: The acquisition module is used to acquire the original image of a preset object through a preset vehicle; and to acquire a binary mask image of the preset object based on the original image. Based on the binary mask image, obtain the image of interest for the preset object; The judgment module is used to determine whether the first target exists in the image of interest based on the image of interest. The output module is used to determine that the first target exists in the image of interest, input the image of interest into a pre-trained target detection model, and obtain the damaged area of ​​the first target; The judgment module is further configured to determine whether the first target does not exist in the image of interest, and to determine whether the second target exists in the image of interest; The output module is also used to determine that there is a second target in the image of interest, to refine the image of interest, and to detect whether the second target is adhered or damaged based on the hit-or-miss transformation algorithm; The method for detecting whether the second target has adhesion or damage based on the hit-or-miss transformation algorithm includes: Construct interest structure tuples containing preset interest structure elements; A hit-or-miss transformation algorithm is applied to the thinned first binary image and the target structure of interest tuple to obtain the total number of target structure of interest tuples contained in the thinned first binary image; Determine whether the total number of at least one type of target interest structural elements is greater than a preset threshold. If the total number of the at least one type of target interest structural elements is greater than the preset threshold, it is determined that the second target has adhesion damage. The hit-and-miss transformation algorithm applied to the thinned first binary image and the target structure of interest (SPIC) tuples to obtain the total number of target SPICs contained in the thinned first binary image includes: The thinned first binary image is eroded using any preset structuring element in the structuring element group of interest to obtain a first eroded image; The complement of the thinned first binary image is eroded using the complement of any preset structuring element of interest to obtain a second eroded image; The intersection of the first erosion image and the second erosion image is used to obtain the third erosion image; Based on the third erosion image, the total number of the target interest structuring elements contained in the thinned first binary image is obtained.

12. An electronic device, characterized in that, The electronic device includes: One or more processors; Storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-10.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-10.

Images