A perimeter break detection method, device, apparatus and storage medium

Abstract

The application provides a perimeter damage detection method and device, equipment and a storage medium, relates to the field of airport security technology, and is used for solving the problem of high cost in the airport perimeter damage detection process. The method comprises the following steps: image acquisition is performed on an airport perimeter to obtain a plurality of perimeter collection images; a cross intersection detection is performed on the plurality of perimeter collection images according to a trained target detection model, and a cross intersection coordinate set corresponding to each of the plurality of perimeter collection images is output; for any one of the perimeter collection images, whether there is a perimeter damage area in the any one of the perimeter collection images is determined according to the cross intersection coordinate set corresponding to the any one of the perimeter collection images; and if it is determined that there is a perimeter damage area in the any one of the perimeter collection images, a damage area corresponding to the perimeter damage area is determined.

Description

Technical Field

[0001] This application relates to the field of airport security technology, and provides a method, device, equipment and storage medium for detecting perimeter damage. Background Technology

[0002] As is well known, airport perimeter fencing is the first line of defense for airport security and a crucial component of airport flight area safety. Airport perimeter fencing is primarily composed of fencing netting, typically using a mesh fence material. Therefore, when the fencing is damaged, people and animals outside the airport may enter through the breaches, posing a significant threat to the safety of the airport flight area.

[0003] However, at present, the main way to deal with the problem of damaged airport perimeters in my country is through "manual inspection" by professionally trained personnel on foot, by bicycle, or by car. However, this method has many drawbacks, such as high cost, strong subjectivity, lack of transparency, and lack of standardization.

[0004] Therefore, reducing costs and enhancing objectivity, transparency, and standardization are urgent issues to be addressed in the process of airport perimeter damage detection. Summary of the Invention

[0005] This application provides a method, apparatus, and equipment for detecting damage to airport perimeter fencing, which addresses the problem of high costs in the process of detecting damage to airport perimeter fencing.

[0006] On the one hand, a method for detecting boundary damage is provided, the method comprising:

[0007] Image acquisition was performed on the airport perimeter, resulting in multiple perimeter images.

[0008] Based on the trained target detection model, cross intersections are detected in the multiple boundary acquisition images, and the set of cross intersection coordinates corresponding to each of the multiple boundary acquisition images is output.

[0009] For any given boundary image, determine whether there is a damaged boundary area in the given boundary image based on the set of coordinates of the cross intersection points corresponding to the given boundary image.

[0010] If it is determined that there is a damaged area in any of the captured boundary images, then the damaged area corresponding to the damaged area is determined.

[0011] Optionally, before performing cross-intersection detection on the multiple boundary acquisition images based on the trained target detection model and outputting the set of cross-intersection coordinates corresponding to each of the multiple boundary acquisition images, the method further includes:

[0012] Based on multiple boundary acquisition images to be trained, a pre-extraction set of cross intersections is obtained; wherein, the pre-extraction set of cross intersections contains multiple boundary acquisition images to be trained from which cross intersections have been pre-extracted;

[0013] Based on the pre-extracted set of cross intersections and the preset ratio, construct the training dataset, the validation dataset, and the test dataset;

[0014] The preset basic detection model is trained based on the training dataset, validation dataset, and test dataset to obtain the trained target detection model.

[0015] Optionally, the step of obtaining a pre-extracted set of cross intersections based on multiple boundary acquisition images to be trained includes:

[0016] For each pixel in any bounded acquisition image to be trained, perform the following steps:

[0017] Using any pixel as the center, determine all neighboring pixels within a preset radius of that pixel;

[0018] The number of pixels whose absolute value difference between any given pixel and all neighboring pixels is greater than a preset pixel value difference threshold is determined.

[0019] Determine whether the number is greater than a preset number;

[0020] If it is determined that the number is greater than the preset number, then any pixel is determined as a candidate cross intersection point;

[0021] Determine whether a preset number of candidate cross intersections exist within a preset range of any given pixel;

[0022] If it is determined that there are a preset number of candidate cross intersections within a preset range of any pixel, then the pixel containing the sum of the absolute values ​​of the largest pixel value difference among the preset number of candidate cross intersections is determined as the pre-extracted cross intersection.

[0023] Optionally, before determining the pixel containing the sum of the absolute values ​​of the largest pixel value differences among the preset number of candidate cross intersections as the pre-extracted cross intersection, if a preset number of candidate cross intersections are determined to exist, the method further includes:

[0024] The sum of the absolute values ​​of the pixel value differences corresponding to any given pixel is obtained by taking the absolute value of the pixel value differences between any given pixel and all the neighboring pixels.

[0025] Optionally, for any boundary acquisition image, determining whether there is a damaged boundary area in any boundary acquisition image based on the set of crosshair coordinates corresponding to the any boundary acquisition image includes:

[0026] For any crossroads coordinate of any boundary acquisition image, determine whether the length difference between the crossroads and any adjacent crossroads within a preset range is not less than a preset length threshold.

[0027] If the length difference is determined to be not less than a preset length threshold, then it is determined that there is a boundary damage area between the intersection point and any adjacent intersection point.

[0028] Optionally, for any boundary acquisition image, determining whether there is a damaged boundary area in any boundary acquisition image based on the set of crosshair coordinates corresponding to the any boundary acquisition image includes:

[0029] For any crossroads coordinate of any boundary acquisition image, determine whether the width difference between the crossroads and any adjacent crossroads within a preset range is not less than a preset width threshold.

[0030] If the width difference is determined to be not less than a preset width threshold, then it is determined that there is a damaged area of ​​the perimeter fence between the intersection point and any adjacent intersection point.

[0031] Optionally, if it is determined that a damaged area exists in any of the captured boundary images, determining the damaged area corresponding to the damaged area includes:

[0032] If it is determined that there is a damaged area in any of the captured boundary images, then a bounding rectangle is drawn around the damaged area to determine the damaged area.

[0033] Optionally, if it is determined that a damaged area exists in any of the captured boundary images, after determining the damaged area corresponding to the damaged area, the method further includes:

[0034] Obtain the boundary damage information from the multiple boundary acquisition images; wherein, the boundary damage information includes the multiple boundary acquisition images, the damaged area of ​​the boundary damage region in the multiple boundary acquisition images, and the GPS location information of the boundary damage region in the multiple boundary acquisition images;

[0035] The information on the damage to the perimeter from the multiple perimeter images is transmitted to the front-end interface for display.

[0036] Optionally, the step of acquiring images of the airport perimeter to obtain multiple perimeter images includes:

[0037] The airport perimeter is imaged by an image acquisition device running on a preset track, resulting in multiple perimeter images; wherein the optical axis of the lens of the image acquisition device is perpendicular to the airport perimeter.

[0038] Optionally, after acquiring multiple images of the airport perimeter, the method further includes:

[0039] The multiple boundary acquisition images are subjected to grayscale transformation to obtain multiple boundary grayscale images;

[0040] The multiple boundary grayscale images are subjected to denoising processing to obtain multiple boundary denoised images;

[0041] Then, the step of performing cross-intersection detection on the multiple boundary acquisition images based on the trained target detection model, and outputting the set of cross-intersection coordinates corresponding to each of the multiple boundary acquisition images, includes:

[0042] Based on the trained target detection model, cross intersections are detected on the multiple de-denoised boundary images, and the set of cross intersection coordinates corresponding to each of the multiple de-denoised boundary images is output.

[0043] On the one hand, a boundary damage detection device is provided, the device comprising:

[0044] The image acquisition unit is used to acquire images of the airport perimeter and obtain multiple perimeter images.

[0045] The coordinate set output unit is used to perform cross intersection detection on the multiple boundary acquisition images according to the trained target detection model, and output the coordinate set of the cross intersection corresponding to each of the multiple boundary acquisition images.

[0046] The damaged area determination unit is used to determine whether there is a damaged area in any boundary acquisition image based on the set of cross intersection coordinates corresponding to the any boundary acquisition image.

[0047] The damaged area determination unit is used to determine the damaged area corresponding to the damaged area if it is determined that there is a damaged area in any of the acquired boundary images.

[0048] On one hand, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement any of the methods described above.

[0049] On the one hand, a computer storage medium is provided that stores computer program instructions thereon, which, when executed by a processor, implement any of the methods described above.

[0050] In this embodiment, when detecting damage to the airport perimeter fence, firstly, images of the airport perimeter fence are acquired to obtain multiple fence images; then, based on a trained target detection model, cross-point detection is performed on these multiple fence images, and a set of cross-point coordinates corresponding to each fence image is output; furthermore, for any fence image, the presence of a damaged area in that fence image can be determined based on the set of cross-point coordinates corresponding to that fence image; then, if a damaged area is determined in any fence image, the corresponding damaged area can be determined. Therefore, in this embodiment, since the entire damage detection process is unmanned and automated, compared with the "manual inspection" method, it not only greatly reduces labor costs but also has the advantages of objectivity and transparency; in addition, based on this "unmanned and automated operation" damage detection method, since it can be strictly implemented according to the standards for airport perimeter damage inspection, it also has the advantage of strong standardization. Attached Figure Description

[0051] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0052] Figure 1 This is a schematic diagram of an application scenario provided by an embodiment of this application;

[0053] Figure 2 A schematic flowchart of a boundary damage detection method provided in an embodiment of this application;

[0054] Figure 3 A flowchart illustrating the process of obtaining a trained target detection model provided in an embodiment of this application;

[0055] Figure 4 A schematic flowchart for pre-extracting cross intersections provided in an embodiment of this application;

[0056] Figure 5 A flowchart illustrating the process of determining a damaged boundary area provided in this application embodiment;

[0057] Figure 6 This application provides another flowchart for determining the damaged area of ​​the boundary.

[0058] Figure 7This is a schematic diagram of a boundary damage detection device provided in an embodiment of this application.

[0059] The diagram is labeled as follows: 10-Boundary damage detection equipment, 20-Image acquisition equipment, 101-Processor, 102-Memory, 103-I / O interface, 104-Database, 70-Boundary damage detection device, 701-Image acquisition unit, 702-Coordinate set output unit, 703-Damage area determination unit, 704-Damage area determination unit, 705-Model training unit, 706-Data display unit, 707-Image processing unit. Detailed Implementation

[0060] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application. Unless otherwise specified, the embodiments and features in the embodiments of this application can be arbitrarily combined with each other. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than that shown here.

[0061] As is well known, airport perimeter fencing is the first line of defense for airport security and a crucial component of airport flight area safety. Airport perimeter fencing is primarily composed of fencing netting, typically using a mesh fence material. Therefore, when the fencing is damaged, people and animals outside the airport may enter through the breaches, posing a significant threat to the safety of the airport flight area.

[0062] However, at present, the main way to deal with the problem of damaged airport perimeters in my country is through "manual inspection" by professionally trained personnel on foot, by bicycle, or by car. However, this method has many drawbacks, such as high cost, strong subjectivity, lack of transparency, and lack of standardization.

[0063] Based on this, this application provides a method for detecting perimeter damage. In this method, firstly, images of the airport perimeter are acquired to obtain multiple perimeter images; then, based on a trained target detection model, cross-point detection is performed on these multiple perimeter images, and a set of cross-point coordinates corresponding to each of the multiple perimeter images is output; furthermore, for any given perimeter image, the presence of a perimeter damage area can be determined based on the set of cross-point coordinates corresponding to that given perimeter image; then, if a perimeter damage area is determined to exist in any given perimeter image, the corresponding damage area can be determined. Therefore, in this application embodiment, since the entire damage detection process is unmanned and automated, compared with the "manual inspection" method, it not only significantly reduces labor costs but also has the advantages of objectivity and transparency; furthermore, based on this "unmanned and automated operation" damage detection method, since it can be strictly implemented according to the airport perimeter damage inspection standards, it also has the advantage of strong standardization.

[0064] After introducing the design concept of the embodiments of this application, the following is a brief introduction to the application scenarios to which the technical solutions of the embodiments of this application can be applied. It should be noted that the application scenarios described below are only for illustrating the embodiments of this application and are not intended to limit the scope. In specific implementation, the technical solutions provided by the embodiments of this application can be flexibly applied according to actual needs.

[0065] like Figure 1 The diagram shown illustrates an application scenario provided by an embodiment of this application. This application scenario may include a perimeter damage detection device 10 and an image acquisition device 20.

[0066] The image acquisition device 20 can be used to acquire images of the airport perimeter fence; for example, it can be a camera or mobile phone with image acquisition capabilities. The perimeter damage detection device 10 can be used to detect damage to the airport perimeter fence; for example, it can be a personal computer (PC), server, or laptop. The perimeter damage detection device 10 may include one or more central processing units 101 (CPU), memory 102, I / O interfaces 103, and database 104. Specifically, the processor 101 can be a central processing unit (CPU) or a digital processing unit, etc. Memory 102 can be volatile memory, such as random-access memory (RAM); memory 102 can also be non-volatile memory, such as read-only memory, flash memory, hard disk drive (HDD), or solid-state drive (SSD); or memory 102 can be any other medium capable of carrying or storing desired program code in the form of instructions or data structures that can be accessed by a computer, but is not limited thereto. Memory 102 can be a combination of the above-mentioned memories. Memory 102 can store some program instructions of the perimeter damage detection method provided in the embodiments of this application. When these program instructions are executed by processor 101, they can be used to implement the steps of the perimeter damage detection method provided in the embodiments of this application, so as to solve the problems of high cost, strong subjectivity, lack of transparency, and non-standardization in the airport perimeter damage detection process. Database 104 can be used to store data such as perimeter acquisition images, cross-point coordinate sets, damaged area, and GPS location information of the damaged area involved in the scheme provided in the embodiments of this application.

[0067] In this embodiment, the perimeter fencing damage detection device 10 can acquire multiple perimeter fencing images captured by the image acquisition device 20 through the I / O interface 103. Then, the processor 101 of the perimeter fencing damage detection device 10 will perform airport perimeter fencing damage detection according to the program instructions of the perimeter fencing damage detection method provided in this embodiment, stored in the memory 102, to reduce costs and enhance objectivity, transparency, and standardization during the perimeter fencing damage detection process. Furthermore, the perimeter fencing images, the coordinate set of the intersection points, the damaged area, and the GPS location information of the damaged area can be stored in the database 104.

[0068] Of course, the methods provided in the embodiments of this application are not limited to... Figure 1The application scenarios shown can also be used in other possible scenarios, and this application embodiment does not impose any limitations. Figure 1 The functions that the various devices in the application scenarios shown can achieve will be described in subsequent method embodiments, and will not be elaborated on here. Below, the methods of the embodiments of this application will be described in conjunction with the accompanying drawings.

[0069] like Figure 2 The diagram shown is a flowchart of a boundary damage detection method provided in an embodiment of this application. This method can... Figure 1 The boundary damage detection device 10 and the image acquisition device 20 work together to perform these operations. Of course, in this embodiment, steps 202-204 can be deployed in the boundary damage detection device 10 for storage and computation, or they can be deployed in the image acquisition device 20 itself for local real-time storage and computation. Specifically, the process of this method is described below.

[0070] Step 201: Collect images of the airport perimeter to obtain multiple perimeter images.

[0071] In this embodiment, when acquiring images of the airport perimeter, an image acquisition device 20 running on a preset track can acquire multiple perimeter images. The optical axis of the lens of the image acquisition device 20 is perpendicular to the airport perimeter, and the preset track can be a fixed trajectory S on the airport perimeter patrol road. After acquiring multiple perimeter images, this image data can be transmitted via a 4G / 5G mobile network to a perimeter damage detection device 10 for storage and detection.

[0072] Of course, in this embodiment, the image acquisition device 20 can be a camera or mobile phone, or it can be a mobile robot, an autonomous vehicle, a guide robot, a camera, or other acquisition device. Furthermore, in this embodiment, in addition to acquiring images, it can also acquire video. The acquired video can then be converted into a series of images with a size of M*N by downsampling the video frames.

[0073] Step 202: Based on the trained target detection model, perform cross intersection detection on multiple boundary acquisition images, and output the set of cross intersection coordinates corresponding to each of the multiple boundary acquisition images.

[0074] In this embodiment of the application, multiple boundary acquisition images can be directly input into the trained target detection model to perform cross intersection detection on these multiple boundary acquisition images. Then, based on the output of the trained target detection model, the set of cross intersection coordinates corresponding to each of these multiple boundary acquisition images can be obtained.

[0075] Step 203: For any boundary acquisition image, determine whether there is a damaged area in any boundary acquisition image based on the set of cross intersection coordinates corresponding to any boundary acquisition image.

[0076] Since airport perimeter fencing often uses mesh fencing as the fencing material, in this embodiment, the presence of damaged areas in the airport perimeter fencing can be determined based on the crosshairs on the mesh. Specifically, for any captured perimeter fencing image, the presence of damaged areas in that image can be determined based on the set of crosshair coordinates {(x1,y1),(x2,y2),…} corresponding to that image.

[0077] Step 204: If it is determined that there is a damaged area in any of the boundary acquisition images, then determine the damaged area corresponding to the damaged area.

[0078] In this embodiment, if a damaged area is determined in any captured boundary image, a bounding rectangle can be drawn around the damaged area, and the area of ​​the bounding rectangle can be calculated. The calculated area of ​​the bounding rectangle is then determined as the damaged area corresponding to the damaged boundary region. Of course, in this embodiment, besides determining the damaged area by drawing a bounding rectangle, the damaged area can also be determined by drawing an inscribed rectangle, a circumscribed circle, or an inscribed circle, etc.

[0079] In one possible implementation, to facilitate airport staff in determining the location of the damaged perimeter fence, a GPS positioning module can be installed in the image acquisition device 10, and the GPS location information when acquiring the perimeter fence image can be bound to the acquired perimeter fence image (or video).

[0080] Furthermore, to facilitate immediate on-site handling of damaged perimeter fencing areas by airport staff, in this embodiment, after determining the damaged area, multiple perimeter fencing images can be acquired to obtain perimeter damage information. This information can then be transmitted via a wireless communication module to a front-end interface for display, thus alerting airport staff. The perimeter fencing damage information may include multiple perimeter fencing images, the damaged area of ​​the damaged perimeter fencing in the images, GPS location information of the damaged perimeter fencing in the images, and so on.

[0081] In one possible implementation, since the trained target detection model is directly used to obtain the set of intersection point coordinates during perimeter damage detection, this embodiment uses perimeter images obtained by directly acquiring images of the airport perimeter fence to train the model, thus obtaining a trained target detection model with high detection accuracy. Figure 3 The diagram shown is a flowchart illustrating the process of obtaining a trained target detection model according to an embodiment of this application. The specific process is as follows:

[0082] Step 301: Obtain a pre-extracted set of cross intersections based on multiple boundary acquisition images to be trained.

[0083] In this embodiment, the pre-extraction set of intersection points may include multiple boundary acquisition images for training, from which intersection points have been pre-extracted. These boundary acquisition images for training may be boundary acquisition images obtained by directly acquiring images of the airport boundary network. Furthermore, after obtaining these boundary acquisition images for training, intersection point pre-extraction can be performed on these images, and then the intersection points on these images can be labeled to obtain the pre-extraction set of intersection points.

[0084] Step 302: Construct the training dataset, validation dataset, and test dataset based on the pre-extracted cross-point set and the preset ratio.

[0085] In this embodiment, the preset ratio can be 7:2:1, that is, the ratio of the number of images in the training dataset, validation dataset, and test dataset is 7:2:1. Of course, depending on the actual situation, other ratio values ​​can be flexibly selected, but it is required that "training dataset > validation dataset, training dataset > test dataset".

[0086] Step 303: Train the preset basic detection model based on the training dataset, validation dataset, and test dataset to obtain the trained target detection model.

[0087] In this embodiment, firstly, a pre-defined basic detection model can be trained using a training dataset to obtain a preliminary object detection model. This pre-defined basic detection model can be a You Only Look Once (YOLO) object detection model. Further, this pre-defined basic detection model can be a boundary net minimum rectangular cell detection model based on the YOLO object detection model. It can be assumed that the size of the boundary net minimum rectangular cell is m*n, meaning the distance between the intersection points of two adjacent boundary nets in the same row is m centimeters, and the distance between the intersection points of two adjacent boundary nets in the same column is n centimeters. The label can be the top-left corner coordinate (x, y, y) of the boundary net minimum rectangular cell. l ,y l ) and the coordinates of the lower right corner (x r ,y r Then, the initial object detection model can be continuously adjusted using the validation dataset to select the optimal model and record its hyperparameters. Finally, the optimal model can be evaluated using the test dataset. When the evaluation metric of the initial object detection model no longer increases for K consecutive times (K≥2), the model training ends, and the trained object detection model is obtained.

[0088] In one possible implementation, to obtain a high-precision trained target detection model, in this embodiment, before constructing the training dataset, validation dataset, and test dataset based on the pre-extracted cross-point set and a preset ratio, the cross-points of the acquired airport perimeter fence images can be pre-extracted directly. Furthermore, since the cross-points of the perimeter fence in the acquired images (size M3*N3) exhibit significant abrupt changes and large pixel value variations, while pixel value variations are typically more gradual at other locations, in this embodiment, whether a pixel is a pre-extracted cross-point can be determined by observing the pixel value variations around the pixel.

[0089] Since the pre-extraction process of the cross intersection points of each pixel is the same when traversing and determining the pixels in the bounded acquisition image to be trained, the following detailed explanation uses the pre-extraction process of the cross intersection point of pixel 1 as an example. Figure 4 The diagram shown is a flowchart illustrating a method for pre-extracting cross intersections according to an embodiment of this application. The specific process is as follows:

[0090] Step 401: Using pixel 1 as the center, determine all neighborhood pixels within the preset radius of pixel 1.

[0091] In this embodiment, we assume that the coordinates of pixel 1 are Gray(x, y) and the preset radius is r. Then, taking pixel 1 as the center, a neighborhood circle is drawn on the bounded image with radius r. The series of neighborhood pixels {G1, G2, G3, ...} in this neighborhood circle are all the neighborhood pixels of pixel 1.

[0092] Of course, in this embodiment of the application, in addition to determining all neighborhood pixels of pixel 1 by drawing a circle with pixel 1 as the center, it is also possible to determine all neighborhood pixels of pixel 1 by drawing a square (with size r*r) with pixel 1 as the center point.

[0093] Step 402: Determine the number of pixels where the absolute value of the pixel value difference between pixel 1 and all neighboring pixels is greater than a preset pixel value difference threshold.

[0094] In this embodiment, it is assumed that the preset pixel value difference threshold is Th1. First, the absolute values ​​{diff1, diff2, diff3, ...} of the pixel value differences between pixel 1 and all neighboring pixels {G1, G2, G3, ...} can be calculated. Then, the number m of these absolute values ​​{diff1, diff2, diff3, ...} that are greater than the preset pixel value difference threshold Th1 can be determined.

[0095] Step 403: Determine if the number is greater than the preset number.

[0096] In this embodiment, it is assumed that the preset number is n. Then, it can be determined whether the number m of the absolute values ​​{diff1, diff2, diff3, ...} that are greater than the preset pixel value difference threshold Th1 is greater than the preset number n.

[0097] Step 404: If the number of determinations is greater than the preset number, then pixel 1 is determined as a candidate cross intersection point.

[0098] In this embodiment of the application, if the number m is greater than the preset number n, that is, the pixel values ​​around pixel 1 change significantly, then pixel 1 can be initially determined as a candidate cross intersection point.

[0099] At the same time, based on the absolute values ​​of the pixel value differences between pixel 1 and all neighboring pixels {G1, G2, G3, ...}, we can obtain the sum of the absolute values ​​of the pixel value differences corresponding to pixel 1, sum(x, y) = diff1 + diff2 + diff3 + ... .

[0100] Step 405: Determine whether there is a preset number of candidate cross intersections within the preset range of pixel 1.

[0101] To further determine whether pixel 1 is a pre-extracted cross intersection, in this embodiment of the application, pixel 1 can be compared with the sum of the absolute values ​​of the pixel value differences of other candidate cross intersections within a preset range to determine whether pixel 1 is a pre-extracted cross intersection.

[0102] Since, under normal circumstances (assuming the perimeter fence is damaged), one intersection point will be surrounded by eight intersection points, we can assume here that the preset number is eight. Furthermore, the preset range can also be as described above, such as "a circle with pixel 1 as the center and a preset radius of r".

[0103] Step 406: If it is determined that there is a preset number of candidate cross intersections within the preset range of pixel 1, then the pixel containing the sum of the absolute values ​​of the largest pixel value difference among the preset number of candidate cross intersections is determined as the pre-extracted cross intersection.

[0104] In this embodiment of the application, if it is determined that there are a preset number of candidate cross intersections within a preset range of pixel 1, then the sum of the absolute values ​​of the pixel value differences of pixel 1 (sum(x,y)) and the sum of the absolute values ​​of the pixel value differences of the preset number of candidate cross intersections (sum(x,y)) can be calculated. n ,y n The pixel containing the sum of the absolute values ​​of the pixel value differences is compared with the pixel containing the largest sum of absolute values ​​of the pixel value differences, and the pixel is selected as the pre-extracted cross intersection point. For example, if the preset number is 8, then the sum of the absolute values ​​of the pixel value differences of pixel 1 (sum(x,y)) is compared with the sum of the absolute values ​​of the pixel value differences of these 8 candidate cross intersection points (sum(x,y)). n ,y n The pixels with the largest sum of absolute values ​​among the nine sums are compared and the pixel with the largest sum of absolute values ​​is identified as the pre-extracted cross intersection.

[0105] If pixel 1 contains the sum of the absolute values ​​of the largest pixel value difference, it means that compared with other candidate cross intersections within the preset range, the pixel values ​​around pixel 1 change more, and it has a greater probability of being a pre-extracted cross intersection.

[0106] In this embodiment of the application, in order to improve the accuracy of the pre-extraction of cross intersections, the pre-extracted cross intersections can be checked and confirmed a second time. That is, the pre-extraction process described above is performed again on the pre-extracted cross intersections to eliminate non-boundary net cross intersections.

[0107] In one possible implementation, if the smallest rectangular unit of the perimeter fence has dimensions m*n, meaning the distance between the intersections of two adjacent perimeter fence crosses in the same row is m centimeters and the distance between the intersections of two adjacent perimeter fence crosses in the same column is n centimeters, then the dimensions (length and width) of the smallest rectangular unit in the perimeter fence image acquired by the image acquisition device at any point in time on a preset track (e.g., a fixed track S on an airport perimeter patrol road) are almost constant.

[0108] Therefore, when traversing the set of crossroads coordinates {(x1,y1),(x2,y2),…} to determine whether there are any damaged areas in the boundary acquisition image, since the process of determining the damaged areas at each crossroads is the same, the following detailed explanation will focus on the process of determining the damaged area at crossroads a in boundary acquisition image A. Figure 5 The diagram shown is a flowchart illustrating a method for determining a damaged boundary area according to an embodiment of this application. The specific process is as follows:

[0109] Step 501: For the coordinates of the cross intersection point a of the boundary acquisition image A, determine whether the length difference between the cross intersection point a and any adjacent cross intersection point within the preset range is not less than the preset length threshold.

[0110] In this embodiment, the preset length threshold can be set to 2length, then the coordinates (x, y, y) of the cross intersection point a of the boundary acquisition image A are determined. a ,y a This allows us to determine the length difference dis between intersection point a and any adjacent intersection point within a preset range. l Is it not less than a preset length threshold of 2length? Of course, in this embodiment, the preset length threshold can be set to 2length, or other values ​​as needed, such as 3length, 4length, etc.

[0111] Step 502: If the length difference is determined to be not less than the preset length threshold, then it is determined that there is a boundary damage area between the cross intersection a and any adjacent cross intersection.

[0112] In this embodiment of the application, if the length difference dis is determined l If the length is not less than the preset length threshold 2length, it can be determined that there is at least one missing intersection point between intersection point a and any of its neighboring intersection points. Therefore, it can be determined that there is a boundary damage area between intersection point a and any of its neighboring intersection points.

[0113] In one possible implementation, besides determining the existence of a damaged boundary area based on the length difference between the intersection point and any adjacent intersection point within a preset range, in this embodiment, the existence of a damaged boundary area can also be determined based on the width difference between the intersection point and any adjacent intersection point within a preset range. The following detailed description uses the process of determining the damaged boundary area at intersection point a in boundary acquisition image A as an example. Figure 6 The diagram shown illustrates another process for determining the damaged area of ​​a boundary according to an embodiment of this application. The specific process is as follows:

[0114] Step 601: For the coordinates of the cross intersection point a of the boundary acquisition image A, determine whether the width difference between the cross intersection point a and any adjacent cross intersection point within the preset range is greater than the preset width threshold.

[0115] In this embodiment, the preset width threshold can be set to 2width, then the coordinates (x, y, y) of the cross intersection point a of the boundary-captured image A are determined. a ,y a This allows us to determine the width difference dis between intersection point a and any adjacent intersection point within a preset range. w Is it not less than a preset width threshold of 2width? Of course, in this embodiment, the preset width threshold can be set to 2width, or other values ​​as needed, such as 3width, 4width, etc.

[0116] Step 602: If the width difference is determined to be not less than the preset width threshold, then it is determined that there is a damaged area of ​​the perimeter fence between the intersection point a and any adjacent intersection point.

[0117] In this embodiment of the application, if the width difference dis is determined w If the width is greater than the preset width threshold 2width, it can be determined that there is a missing cross intersection between the cross intersection a and any of its neighboring cross intersections. Therefore, it can be determined that there is a boundary damage area between the cross intersection a and any of its neighboring cross intersections.

[0118] In one possible implementation, to improve the accuracy of damage detection, in the embodiments of the application, firstly, multiple boundary acquisition images can be subjected to grayscale transformation to obtain multiple boundary grayscale images; then, these multiple boundary grayscale images can be denoised to obtain multiple boundary denoised images; thereby, these multiple boundary denoised images can be input into the trained target detection model to perform cross-intersection detection on these multiple boundary denoised images, and output the set of cross-intersection coordinates corresponding to each of these multiple boundary denoised images.

[0119] Specifically, firstly, since the image acquisition devices 20 all operate on a fixed patrol track S, therefore, for any point S on the track... i The distance to the airport perimeter fence is constant. Therefore, based on this condition, the perimeter fence image with a size of M*N is cropped to form a perimeter fence image containing only the perimeter fence and with a size of M1*N1.

[0120] Then, the grayscale transformation of the boundary network image with a size of M1*N1 can be performed to obtain a grayscale boundary image with a size of M2*N2, thereby making the subsequent intelligent image analysis process simpler and faster.

[0121] In this embodiment of the application, the grayscale transformation model of the following formula (1) can be used to perform grayscale transformation:

[0122] Gray(x,y)=[R(x,y)+G(x,y)+B(x,y)] / 3 (1)

[0123] Where Gray(x, y) represents the pixel coordinates after grayscale transformation, R(x, y) represents the pixel coordinates of the R channel before grayscale transformation, G(x, y) represents the pixel coordinates of the G channel before grayscale transformation, and B(x, y) represents the pixel coordinates of the B channel before grayscale transformation. In other words, after traversing all pixels in the boundary net image, the grayscale transformed pixel value Gray(x, y) is the average of the R, G, and B channel pixel values ​​of the boundary net image before transformation.

[0124] In practical applications, the grayscale image of the boundary may contain objects such as weeds, trees, and rain or fog, which can cause noise interference. Therefore, in the embodiments of this application, Gaussian filtering can be used to denoise the grayscale image of the boundary to obtain a denoised boundary image with a size of M3*N3.

[0125] Specifically, the filter function expression of the following formula (2) can be used for noise reduction:

[0126]

[0127] Where g(x,y) are the pixel coordinates after denoising.

[0128] In summary, in this embodiment of the application, since the entire damage detection process is unmanned and automated, it not only greatly reduces labor costs compared to the "manual inspection" method, but also has the advantages of objectivity and transparency. In addition, based on this "unmanned and automated operation" damage detection method, since it can be strictly implemented in accordance with the standards for airport perimeter damage inspection, it also has the advantage of strong standardization.

[0129] Based on the same inventive concept, embodiments of this application provide a boundary damage detection device 70, such as... Figure 7 As shown, the device includes:

[0130] Image acquisition unit 701 is used to acquire images of the airport perimeter and obtain multiple perimeter images;

[0131] The coordinate set output unit 702 is used to perform cross intersection detection on multiple boundary acquisition images based on the trained target detection model, and output the coordinate set of the cross intersection corresponding to each of the multiple boundary acquisition images.

[0132] The damaged area determination unit 703 is used to determine whether there is a damaged area in any boundary acquisition image based on the set of cross intersection coordinates corresponding to any boundary acquisition image.

[0133] The damaged area determination unit 704 is used to determine the damaged area corresponding to the damaged area if it is determined that there is a damaged area in any of the boundary acquisition images.

[0134] Optionally, the device 70 further includes a model training unit 705, which is used for:

[0135] Based on multiple boundary acquisition images to be trained, a pre-extraction set of cross intersections is obtained; wherein, the pre-extraction set of cross intersections contains multiple boundary acquisition images to be trained from which cross intersections have been pre-extracted.

[0136] Based on the pre-extracted set of cross intersections and the preset ratio, construct the training dataset, the validation dataset, and the test dataset;

[0137] The pre-defined basic detection model is trained using the training dataset, validation dataset, and test dataset to obtain the trained object detection model.

[0138] Optionally, model training unit 705 is also used for:

[0139] For each pixel in any bounded acquisition image to be trained, perform the following steps:

[0140] Using any pixel as the center, determine all neighboring pixels within a preset radius of any given pixel;

[0141] Determine the number of pixels whose absolute value difference between any given pixel and all neighboring pixels is greater than a preset pixel value difference threshold;

[0142] Determine if the number is greater than the preset number;

[0143] If the number of points is greater than the preset number, then any pixel will be selected as a candidate cross intersection point.

[0144] Determine whether a preset number of candidate cross intersections exist within a preset range of any pixel;

[0145] If it is determined that there are a preset number of candidate cross intersections within a preset range of any pixel, then the pixel containing the sum of the absolute values ​​of the largest pixel value difference among the preset number of candidate cross intersections is determined as the pre-extracted cross intersection.

[0146] Optionally, model training unit 705 is also used for:

[0147] The sum of the absolute values ​​of the pixel value differences corresponding to any pixel is obtained by taking the absolute value of the pixel value differences between any pixel and all neighboring pixels.

[0148] Optionally, the damaged area determination unit 703 is also used for:

[0149] For any crossroads coordinate in any boundary acquisition image, determine whether the length difference between the crossroads and any neighboring crossroads within a preset range is not less than a preset length threshold.

[0150] If the length difference is determined to be no less than the preset length threshold, then a boundary damage area is determined to exist between the intersection point and any adjacent intersection point.

[0151] Optionally, the damaged area determination unit 703 is also used for:

[0152] For any crossroads coordinate in any boundary acquisition image, determine whether the width difference between the crossroads and any adjacent crossroads within a preset range is not less than a preset width threshold.

[0153] If the width difference is determined to be no less than the preset width threshold, then a damaged area of ​​the perimeter fence is determined to exist between the intersection point and any adjacent intersection point.

[0154] Optionally, the damaged area determination unit 704 is also used for:

[0155] If it is determined that there is a damaged area in any of the boundary acquisition images, then draw a bounding rectangle for the damaged area to determine the corresponding damaged area.

[0156] Optionally, the device 70 further includes a data display unit 706, which is used for:

[0157] Acquire boundary damage information from multiple boundary images; the boundary damage information includes multiple boundary images, the damaged area of ​​the boundary damage region in the multiple boundary images, and the GPS location information of the boundary damage region in the multiple boundary images;

[0158] Information on perimeter damage from multiple perimeter images is transmitted to the front-end interface for display.

[0159] Optionally, the image acquisition unit 701 is also used for:

[0160] The airport perimeter is captured by an image acquisition device that runs on a preset track, resulting in multiple perimeter images; the optical axis of the lens of the image acquisition device is perpendicular to the airport perimeter.

[0161] Optionally, the device 70 further includes an image processing unit 707, which is used to: perform grayscale transformation on multiple boundary acquisition images to obtain multiple boundary grayscale images;

[0162] Denoising is performed on multiple boundary grayscale images to obtain multiple denoised boundary images;

[0163] Then, based on the trained object detection model, cross intersections are detected in multiple boundary acquisition images, and the set of cross intersection coordinates corresponding to each of the multiple boundary acquisition images is output, including:

[0164] Based on the trained target detection model, cross intersections are detected on multiple de-denoised boundary images, and the coordinate sets of the cross intersections corresponding to each of the multiple de-denoised boundary images are output.

[0165] This device can be used to perform Figures 2-6 The method performed by the boundary damage detection device in the illustrated embodiment is described above. Therefore, the functions that each functional module of the device can achieve can be referred to... Figures 2-6 The embodiments shown are described in detail below.

[0166] In some possible implementations, various aspects of the methods provided in this application can also be implemented as a program product comprising program code that, when run on a computer device, causes the computer device to perform the steps of the methods according to the various exemplary embodiments of this application described above. For example, the computer device may perform actions such as... Figures 2-6 The method performed by the boundary damage detection device in the illustrated embodiment.

[0167] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks. Alternatively, if the integrated units of the present invention are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

[0168] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0169] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for detecting boundary damage, characterized in that, The method includes: Image acquisition was performed on the airport perimeter, resulting in multiple perimeter images. Based on the trained target detection model, cross intersection detection is performed on the multiple boundary acquisition images, and the set of cross intersection coordinates corresponding to each of the multiple boundary acquisition images is output. The trained target detection model is obtained by training a YOLO model using a pre-extracted cross intersection set. The pre-extracted cross intersection set contains multiple boundary acquisition images for training with pre-extracted cross intersections. The step of obtaining the pre-extracted cross intersections includes: for each pixel in any boundary acquisition image for training, performing the following steps respectively: using any pixel as the center, determining all... The process involves identifying neighborhood pixels; determining the number of pixels where the absolute value of the pixel value difference between any given pixel and all neighborhood pixels is greater than a preset pixel value difference threshold; determining whether this number is greater than a preset number; if the number is greater than the preset number, then identifying any given pixel as a candidate cross intersection; determining whether a preset number of candidate cross intersections exist within a preset range of any given pixel; if a preset number of candidate cross intersections exist within a preset range of any given pixel, then identifying the pixel among the preset number of candidate cross intersections that contains the sum of the absolute values ​​of the largest pixel value differences as a pre-extracted cross intersection. For any given boundary image, based on the set of crosshair coordinates corresponding to the given boundary image, determine whether there is a damaged boundary area in the given boundary image; wherein, determining whether there is a damaged boundary area in any given boundary image based on the set of crosshair coordinates corresponding to the given boundary image includes: For any intersection point coordinate of any boundary acquisition image, determine whether the length difference between the intersection point and any adjacent intersection point within a preset range is not less than a preset length threshold; if the length difference is determined to be not less than the preset length threshold, then it is determined that there is a boundary damage area between the intersection point and any adjacent intersection point. Alternatively, for any intersection point coordinate of any of the boundary acquisition images, determine whether the width difference between the intersection point and any adjacent intersection point within a preset range is not less than a preset width threshold; if the width difference is determined to be not less than the preset width threshold, then determine that there is a boundary damage area between the intersection point and any adjacent intersection point. If it is determined that there is a damaged area in any of the captured boundary images, then the damaged area corresponding to the damaged area is determined.

2. The method as described in claim 1, characterized in that, Before performing cross-intersection detection on the multiple boundary acquisition images based on the trained target detection model and outputting the set of cross-intersection coordinates corresponding to each of the multiple boundary acquisition images, the method further includes: Based on multiple boundary acquisition images to be trained, a pre-extracted set of cross intersection points is obtained; Based on the pre-extracted set of cross intersections and the preset ratio, construct the training dataset, the validation dataset, and the test dataset; The preset basic detection model is trained based on the training dataset, validation dataset, and test dataset to obtain the trained target detection model.

3. The method as described in claim 1, characterized in that, Before determining the pixel containing the sum of the absolute values ​​of the largest pixel value differences among the preset number of candidate cross intersections as the pre-extracted cross intersection, if a preset number of candidate cross intersections are determined to exist, the method further includes: The sum of the absolute values ​​of the pixel value differences corresponding to any given pixel is obtained by taking the absolute value of the pixel value differences between any given pixel and all the neighboring pixels.

4. The method as described in claim 1, characterized in that, If it is determined that a damaged area exists in any of the captured boundary images, then determining the damaged area corresponding to the damaged area includes: If it is determined that there is a damaged area in any of the captured boundary images, then a bounding rectangle is drawn around the damaged area to determine the damaged area.

5. The method as described in claim 1, characterized in that, If it is determined that a damaged area exists in any of the acquired boundary images, and after determining the damaged area corresponding to the damaged area, the method further includes: Obtain boundary damage information from the multiple boundary acquisition images; wherein, the boundary damage information includes the multiple boundary acquisition images, the damaged area of ​​the boundary damage region in the multiple boundary acquisition images, and the GPS location information of the boundary damage region in the multiple boundary acquisition images; The information on the damage to the perimeter from the multiple perimeter images is transmitted to the front-end interface for display.

6. The method as described in claim 1, characterized in that, The process of acquiring images of the airport perimeter results in multiple perimeter images, including: The airport perimeter is imaged by an image acquisition device running on a preset track, resulting in multiple perimeter images; wherein the optical axis of the lens of the image acquisition device is perpendicular to the airport perimeter.

7. The method as described in claim 1, characterized in that, After acquiring multiple images of the airport perimeter, the method further includes: The multiple boundary acquisition images are subjected to grayscale transformation to obtain multiple boundary grayscale images; The multiple boundary grayscale images are subjected to denoising processing to obtain multiple boundary denoised images; Then, the step of performing cross-intersection detection on the multiple boundary acquisition images based on the trained target detection model, and outputting the set of cross-intersection coordinates corresponding to each of the multiple boundary acquisition images, includes: Based on the trained target detection model, cross intersections are detected on the multiple de-denoised boundary images, and the set of cross intersection coordinates corresponding to each of the multiple de-denoised boundary images is output.

8. A boundary damage detection device, characterized in that, The device includes: The image acquisition unit is used to acquire images of the airport perimeter and obtain multiple perimeter images. The coordinate set output unit is used to perform cross-intersection point detection on the multiple boundary acquisition images based on the trained target detection model, and output the coordinate set of the cross-intersection points corresponding to each of the multiple boundary acquisition images; wherein, the trained target detection model is obtained by training a YOLO model using a cross-intersection point pre-extraction set; the cross-intersection point pre-extraction set contains multiple boundary acquisition images to be trained from which cross-intersection points have been pre-extracted; and the step of obtaining the pre-extracted cross-intersection points includes: for each pixel in any boundary acquisition image to be trained, performing the following steps respectively: taking any pixel as the center, determining the preset radius at any pixel. Within the range of all neighborhood pixels; determine the number of pixels where the absolute value of the pixel value difference between any pixel and all neighborhood pixels is greater than a preset pixel value difference threshold; determine whether the number is greater than a preset number; if the number is greater than the preset number, then determine any pixel as a candidate cross intersection; determine whether a preset number of candidate cross intersections exist within a preset range of any pixel; if a preset number of candidate cross intersections exist within a preset range of any pixel, then determine the pixel containing the sum of the absolute values ​​of the largest pixel value difference among any pixel and the preset number of candidate cross intersections as a pre-extracted cross intersection; A damaged area determination unit is used to determine, for any captured boundary image, whether a damaged boundary area exists in the captured boundary image based on the set of crossroads coordinates corresponding to the captured boundary image. Specifically, determining whether a damaged boundary area exists in the captured boundary image based on the set of crossroads coordinates corresponding to the captured boundary image includes: for any crossroads coordinate in the captured boundary image, determining whether the length difference between the crossroads and any adjacent crossroads within a preset range is not less than a preset length threshold; if the length difference is not less than the preset length threshold, then a damaged boundary area exists between the crossroads and any adjacent crossroads; or, for any crossroads coordinate in the captured boundary image, determining whether the width difference between the crossroads and any adjacent crossroads within a preset range is not less than a preset width threshold; if the width difference is not less than the preset width threshold, then a damaged boundary area exists between the crossroads and any adjacent crossroads. The damaged area determination unit is used to determine the damaged area corresponding to the damaged area if it is determined that there is a damaged area in any of the acquired boundary images.

9. An electronic device, characterized in that, The device includes: Memory, used to store program instructions; A processor is configured to invoke program instructions stored in the memory and execute the method described in any one of claims 1-7 according to the obtained program instructions.

10. A storage medium, characterized in that, The storage medium stores computer-executable instructions for causing a computer to perform the method described in any one of claims 1-7.

Images