A fence recognition method based on ENET and a graphics algorithm

Abstract

The application discloses a fence recognition method based on ENET and a graphics algorithm, relates to the fields of computer vision and image processing, and obtains pictures containing fence facilities and pre-processes the pictures; fence areas in the pictures are manually segmented and corresponding masks are generated; original fence data and the corresponding masks are randomly divided to obtain a training set, a verification set and a test set; the divided training set and the verification set are input into an ENET neural network for training; test pictures containing the fence facilities are input into the trained network to obtain a preliminary prediction graph; computer graphics algorithms are used to process the preliminary prediction graph to obtain a complete fence surface; and a final fence recognition effect graph is output; the method to some extent overcomes the problem that traditional target detection has poor recognition effect on fence facilities with large scale variation range and hollow structure, and has better performance in fence boundary processing by using computer graphics algorithms for processing.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and image processing technology, and specifically to a fence recognition method based on ENET and graphics algorithms. Background Technology

[0002] Fences are a common safety protection facility at construction sites, especially in electrical work scenarios, used to separate dangerous live electrical components from workers.

[0003] In recent years, due to continuous improvements in equipment and policies, dedicated safety supervisors have been assigned to some hazardous work scenarios, effectively curbing the occurrence of safety accidents. However, accidents still occur due to complacency stemming from long-term safe construction practices and negligence on the part of safety supervisors. This is especially true in power grid work scenarios, where numerous ultra-high voltage and hazardous equipment exist. A safety officer's inadvertent glance or a worker's momentary lapse in attention can often lead to unbearable consequences. By combining cameras deployed at construction sites with intelligent alerts in scenarios where accidents may occur, safety officers can be assisted in their supervision, which can reduce the occurrence of accidents to some extent.

[0004] Object detection is the most commonly used method in deep learning for object recognition. However, detecting objects with large scale variations has always been a challenge. Due to their unique hollow structure and the significant size variations in fence images captured by cameras at different distances, fence detection performance on this type of data does not meet expectations. Using the end-to-end segmentation network ENET to identify fences can mitigate these problems to some extent. However, in power grid construction scenarios, high segmentation accuracy is often required, necessitating a large amount of manually labeled data as the training set. Given the high density of fences, traditional pixel-by-pixel annotation would be extremely time-consuming. Furthermore, while segmentation networks can better identify the contour information of objects, simply recognizing the fence structure is insufficient for hollow targets like fences, as the gaps between fences are also a crucial factor for safe construction. Therefore, this paper proposes a fence recognition method based on ENET and computer graphics algorithms to address these issues. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a fence recognition method based on ENET and computer graphics algorithms. Without consuming a large amount of storage space and computation time, it utilizes partial fence segmentation results to identify all fenced areas contained in an image, thus solving the problems existing in the prior art.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A fence recognition method based on ENET and computer graphics algorithms includes the following steps:

[0008] Obtain an image containing fence facilities, and use a pixel-by-pixel annotation method to annotate the fence in the image to generate a corresponding mask;

[0009] The original fence data and its corresponding mask are divided proportionally to obtain training set, validation set and test set; the divided training set and validation set are input into ENET neural network for training, and then the test set images are input into the trained network to obtain prediction map;

[0010] Computer graphics algorithms are used to process the initial prediction image to obtain the complete fence surface. The prediction image is divided into several rectangular structures by setting a small rectangular kernel, and after traversing them, dilation and erosion operations are performed. Then, a large rectangular kernel is set to repeat the dilation and erosion operations on the segmented result after the operation. After traversing the image partitions after the erosion operation, polygon dilation is performed on each individual fence partition in turn. The segmented image after polygon dilation is traversed column by column again to form the complete fence surface.

[0011] Output the final fence recognition result image.

[0012] Furthermore, the obtained fence image is converted into a uniform 1280*720 format.

[0013] Furthermore, the original fence data and its corresponding mask are divided into a training set, a validation set, and a test set in a ratio of 7:2:1.

[0014] Furthermore, during the training of the ENET neural network, PRelu is used as the activation function, CrossEntropy is used as the loss function, Adam is used as the optimizer, and Crossover Union Ratio (CUP) is used as the evaluation metric.

[0015] Furthermore, the size of the small rectangular core is set to 3*3, and the size of the large rectangular core is set to 50*10.

[0016] Furthermore, the expansion calculation formula is as follows:

[0017]

[0018] Where P is the dilated image, M represents the segmentation result, and N is a number of structuring elements after segmentation using a rectangular kernel. This represents the expansion operation.

[0019] Furthermore, the corrosion calculation formula is as follows:

[0020] S=P⊙N={x,y|N x,y∈P}

[0021] Where S is the eroded image, and ⊙ represents the erosion operation.

[0022] This invention provides a fence recognition method based on ENET and computer graphics algorithms, which has the following advantages:

[0023] (1) This invention trains the ENET neural network by annotating the red part of the fence, and obtains the entire fence area by processing the partially segmented fence through algorithms. The partial annotation method reduces manpower and time resources. The segmentation network can solve the problem of poor target detection effect for this type of data. The ENET segmentation network has the feature of real-time segmentation. The computer graphics algorithm is used to further process the segmentation results. Overall, it has the advantages of low cost, high accuracy and low computational complexity.

[0024] (2) The present invention performs two operations of expansion followed by erosion, and by setting rectangular kernels of different sizes to traverse the segmentation results, it can make up for the defects such as concave points and small cracks in the segmentation results, so that some fence images with poor segmentation results due to uneven lighting can be segmented more smoothly; at the same time, the computational complexity of the expansion and erosion operation is low, and the processing time is within a reasonable range.

[0025] (3) Before using the polygon expansion method, the present invention divides the fence into sections based on the distribution characteristics of the fence. By performing polygon expansion on the fences in different sections, the blank spaces between the fences are connected, avoiding irregular and erroneous connections after polygon expansion, thus forming a complete fence surface. The entire fence area is successfully recovered from the partial fence segmentation results, greatly reducing the time and economic cost of the fully labeled training set. Attached Figure Description

[0026] Figure 1 This is a complete flowchart of the present invention;

[0027] Figure 2 This is a schematic diagram illustrating the creation of the training set for this invention;

[0028] Figure 3 This is a schematic diagram of the segmentation results in an embodiment of the present invention;

[0029] Figure 4 This is a schematic diagram showing the results of mathematical operations and polygon filling on a partially segmented structure according to the present invention. Detailed Implementation

[0030] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0031] A fence recognition method based on ENET and computer graphics algorithms includes the following steps:

[0032] 1. Read the fence image and convert it into a uniform format with a length of 1280 and a width of 720;

[0033] 2. By marking the clearly colored parts in the fence image, corresponding masks are generated;

[0034] 3. Divide the fence images and their corresponding masks obtained in step 2 into training, validation, and test sets in a ratio of 7:2:1;

[0035] 4. Input the training and validation sets into the ENET network for training;

[0036] 5. Input the test image containing the fence into the trained neural network to obtain the prediction image;

[0037] 6. Perform morphological algorithm processing on the preliminary prediction map from step 5 to recover the entire fenced area;

[0038] 7. Output the final fence recognition image.

[0039] In step 1, the fence data was collected by cameras at the power grid construction site, containing a total of 4270 fence images, including both vertical and intersecting types. The fence images of different sizes were converted into a unified 1280*720 format for neural network training.

[0040] In step 2, the red fences in the dataset are labeled to reduce the workload of full labeling to some extent and generate the corresponding mask.

[0041] In step 3, the 4270 data points are divided into a training set, a validation set, and a test set in a ratio of 7:2:1, with 2989 data points in the training set, 854 data points in the validation set, and 427 data points in the test set.

[0042] In step 4, 2989 training images and 854 validation images are input into the ENET neural network for training. During training, cross-entropy is used as the loss function, Adam is used as the optimizer, and crossover ratio is used as the evaluation metric for the model. Then, the test set images are input into the trained network to obtain the prediction image.

[0043] In step 6, the predicted image is processed using computer graphics algorithms to identify the complete fence surface based on the partial segmentation results. This includes the following steps:

[0044] First, a small rectangular kernel of size 3*3 is set to divide the prediction image into several rectangular structures. By traversing these rectangular structures, dilation and erosion operations are performed to eliminate small particle noise and some misidentifications contained in the target region, making the segmentation result smoother.

[0045] Next, set a large rectangular kernel of size 50*10, and repeat the dilation and erosion operation on the above segmentation result to connect the closely spaced fences laterally, initially forming a fence surface. The dilation and erosion formulas are as follows, where... ⊙ represents the dilation operation, and ⊙ represents the erosion operation.

[0046]

[0047] Where P is the dilated image, M represents the segmentation result, and N is a number of structuring elements after segmentation by a rectangular kernel. The above formula means that the segmentation result is traversed through N, and if there is pixel intersection, the entire structuring element is retained.

[0048] S=P⊙N={x,y|N x,y ∈P}

[0049] Where S is the eroded image, and geometrically it is represented by traversing the dilated image P using the structuring element N, retaining only the pixels where the two have intersection.

[0050] The image S processed in step 2 is partitioned to provide a better framework for the next step. The partitioning steps are as follows: Two empty arrays are set up. Image S is traversed column by column. The coordinates of the first 255 pixels appearing in each partition are stored in the first array, and the coordinates of the last 255 pixels appearing in the second array, until the last partition. Two empty arrays are set up to store the pixel positions of the start and end boundaries, respectively. Then, the binary image S processed in the first two steps is traversed column by column. Among the continuously appearing foreground pixels, the point with the smallest column coordinate value is stored in the first array, and the point with the largest column coordinate value is stored in the second array, thus forming a local fence partition. Simultaneously, a pixel length threshold is set in the column direction to eliminate erroneous connections formed in the previous operations. The main purpose of the fence partition is to prevent subsequent polygon dilation processing from forming erroneous connections, which could lead to incorrect recognition results.

[0051] By traversing the pre-divided fence blocks, polygon dilation is performed on each block sequentially. The actual operation of polygon dilation involves connecting each contour point and saving the information of each edge, then using a structure to fill the contour edges. Finally, the polygon-dilated image is traversed column-by-column to fill the blank points between pixels, forming a complete fence surface. The segmented image after polygon dilation is then traversed column-by-column again. By reading the coordinates of foreground pixels in each column and filling the pixels between their coordinates with 255, a threshold of 10 is set. Columns with a foreground pixel coordinate difference less than 10 are considered incorrectly connected, and their pixels are set to 0.

[0052] The positions corresponding to the foreground pixels in the input image and the segmented image processed by the graphics algorithm are retained, while the pixels at other positions are set to 0, thus retaining only the area where the fence is located in the input image, and finally outputting the fence recognition result.

[0053] This invention trains an ENET neural network by annotating the red portions of fences. By processing the partially segmented fences using algorithms, the entire fence area can be obtained. Partial annotation reduces manpower and time resources compared to full annotation. Using a segmentation network to identify fences solves the problem of poor performance in object detection for fence-type data. Combining the real-time segmentation capabilities of the ENET segmentation network with computer graphics algorithms for further processing of the segmentation results, the overall approach offers advantages such as low cost, high accuracy, and low computational complexity.

[0054] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A fence recognition method based on ENET and computer graphics algorithms, characterized in that, Includes the following steps: Obtain the original fence image, and use a pixel-by-pixel annotation method to annotate the fence in the image to generate the corresponding mask; The original fence image and its corresponding mask are divided proportionally to obtain the training set, validation set and test set; The predefined training and validation sets are input into the ENET neural network for training, and then the test set images are input into the trained ENET neural network to obtain the prediction image; The prediction image is divided into several rectangular structures by setting a small rectangular kernel. After traversing these rectangular structures, dilation and erosion operations are performed sequentially. Then, the prediction image is divided into several rectangular structures by setting a large rectangular kernel again. After traversing these rectangular structures sequentially, dilation and erosion operations are performed again to obtain the fence binary image. The size of the small rectangular kernel is 3*3, and the size of the large rectangular kernel is 50*10. The fence binary image is partitioned, and polygon dilation is performed on the partitioned fence images sequentially. The partitioning steps include: setting two empty arrays, traversing the fence binary image column by column, storing the coordinates of the first 255-pixel point in each partition into the first array, and the coordinates of the last 255-pixel point into the second array, until the last partition; setting two empty arrays to store the pixel positions of the start and end boundaries respectively; then traversing the processed fence binary image column by column, storing the point with the smallest column coordinate value among the continuously occurring foreground pixels into the first array, and the point with the largest column coordinate value into the second array, thus forming a local fence partition. The fence image after polygon expansion is traversed column by column again to form a complete fence area, and the final fence recognition result is output.

2. The fence recognition method based on ENET and graphics algorithms according to claim 1, characterized in that, The obtained fence images are converted into a uniform 1280*720 format.

3. The fence recognition method based on ENET and computer graphics algorithms according to claim 1, characterized in that, The original fence data and its corresponding mask are divided into training set, validation set and test set in a ratio of 7:2:

1.

4. The fence recognition method based on ENET and computer graphics algorithms according to claim 1, characterized in that, During the training of the ENET neural network, PRelu was used as the activation function, CrossEntropy as the loss function, Adam as the optimizer, and Crossover Union Ratio (CUP) as the evaluation metric.

5. The fence recognition method based on ENET and computer graphics algorithms according to claim 1, characterized in that, The expansion calculation formula is as follows: Where P is the dilated image, M represents the segmentation result, and N is a number of structuring elements after segmentation using a rectangular kernel. This indicates the expansion operation.

6. The fence recognition method based on ENET and computer graphics algorithms according to claim 1, characterized in that, The corrosion calculation formula is as follows: Where S is the image after erosion. This represents the erosion operation.

Images