A Method and System for Locating Fence Areas at Construction Sites Based on Deep Learning

Abstract

The present invention discloses a method and system for positioning a fence area at a construction site based on deep learning. The method includes the steps of: 1) extracting the red fence in the construction site picture according to the fence color feature, and then converting it into a binary image to train the ENet network to obtain a trained ENet network model; 2) using the trained ENet network model to segment the fence in the construction site picture to obtain the red fence in the fence; 3) according to the segmented red fence, performing rectangular filling and polygon dilation on the red fence pixels to generate a surface area covering all the fences, and at the same time marking the positions corresponding to the surface area position in the construction site picture to realize the positioning of the fence area at the construction site. The present invention can quickly and accurately segment the position where the construction site fence is located, and has the advantages of low cost, high accuracy and low computational complexity.

Description

Technical Field

[0001] The present invention mainly relates to the technical field of image processing, and particularly relates to a method and system for positioning a construction site fence area based on deep learning. Background Art

[0002] In some construction environments, there are some dangerous items that need to be avoided. For example, in a power construction site, there are usually many live conductors, which are extremely easy for construction workers to touch, and the risk factor is relatively high. It is necessary to separate the operating equipment and the safe area by a fence, and remind the construction workers when they are too close to the fence to improve the safety of the construction site.

[0003] One or more cameras are often arranged at the construction site, and the management personnel will remind the construction workers to stay away from the live conductors through the images uploaded by the cameras. However, there are problems of untimely and human oversight in manual reminders, and arranging monitoring management personnel alone cannot greatly improve the safety.

[0004] Currently, the commonly used methods for target object positioning are image segmentation based on deep learning and object detection based on deep learning. For the object detection algorithm based on deep learning, it is necessary to mark the position of the fence with a detection frame. If the entire area where the fence is located is marked with a detection frame, the proportion of the target pixels in the detection frame to the original construction site picture is too large, and it is impossible to well distinguish the feature information of the target area and the background area, and the training effect is not very ideal. If a detection frame is drawn and marked for each small part of the fence separately, since most of the construction site fences are composed of relatively narrow vertical fences arranged at a certain distance interval, there will be many detection frames in a picture at this time, and the marking process will be relatively cumbersome, requiring a large amount of time resources. At the same time, since the network has learned a large number of features of vertical fences, it is easy to cause misidentification of fences, affecting the accuracy of fence area positioning.

[0005] For the image segmentation algorithm based on deep learning, the above problems can be avoided to a certain extent. Through training with some manually marked segmentation data, real-time and accurate segmentation of foreground pixels and background can be achieved; however, under the premise of relatively high segmentation accuracy requirements, a large amount of data is often required as the training set, and the density of construction site fences is relatively high. Adopting the traditional segmentation dataset marking method requires a large amount of manpower and time. Summary of the Invention

[0006] The technical problem to be solved by the present invention is: aiming at the problems existing in the prior art, the present invention provides a method and system for positioning a construction site fence area based on deep learning with low cost, high accuracy, and low computational complexity.

[0007] To solve the above technical problems, the technical solution proposed by the present invention is:

[0008] A method for positioning the fence area at the construction site based on deep learning, comprising the steps of:

[0009] 1) Extract the red fences in the construction site pictures according to the fence color characteristics, and then convert them into binary images to train the ENet network, obtaining a trained ENet network model;

[0010] 2) Use the trained ENet network model to segment the fences in the construction site pictures, obtaining the red fences in the fences;

[0011] 3) According to the segmented red fences, perform rectangular filling and polygon dilation on the red fence pixels to generate a surface area covering all the fences, and at the same time mark the positions corresponding to the surface area positions in the construction site pictures, realizing the positioning of the fence area at the construction site.

[0012] Preferably, in step 2), a rectangular kernel with a specified size is set to divide the red fences in the fences into several rectangular structural elements, and then a closing operation is performed on the rectangular structural elements to fill the concave corners.

[0013] Preferably, the closing operation is performed in the manner of first dilation operation and then erosion operation, where the dilation operation is used to fill the disconnections in the image or the holes in the back of the target area, and the erosion operation is used to eliminate some noises outside the target contour; the mathematical formula of the closing operation is as follows:

[0014]

[0015] Where A represents the segmented binary image, B is several rectangular structural elements divided by the rectangular kernel, represents the dilation operation, and ⊙ represents the erosion operation.

[0016] Preferably, the specific mathematical formula of the dilation operation is as follows:

[0017]

[0018] Where Bx,y represents moving the origin of the rectangular structural element to the point (x,y), and its geometric meaning represents traversing the binary image A through the structural element B. If there is a pixel intersection between the structural element B and the binary image A, the entire structural element B is retained.

[0019] Preferably, the specific mathematical formula for performing the erosion operation on the image P after the dilation operation is as follows:

[0020] A·B = P⊙B = {x,y|B x,y ∈P}

[0021] Its geometric meaning is that the structural element B traverses the binary image P after dilation operation, retains the pixel points where the structural element B intersects with the binary image P, and removes other non-intersecting pixel points.

[0022] Preferably, in step 3), before performing rectangular filling and polygon dilation, the red fence after closing operation is partitioned in combination with the distribution area of foreground pixels, and at the same time, the vertical pixel threshold is set to n to judge the columns with less than n pixel points in the column direction to eliminate the wrong horizontal connections generated by the closing operation.

[0023] Preferably, in step 3), after performing polygon dilation, the columns with foreground pixels after polygon dilation are traversed, and all pixels between the topmost pixel point and the bottommost pixel point of this column are set to 255 to generate a surface area covering the entire fence.

[0024] Preferably, the specific process of step 1) is: obtaining a preliminary red fence probability map from the construction site picture through a color recognition algorithm; then, by setting a pixel threshold, the pixels greater than the threshold in the construction site picture are set to 255, and the pixels less than the threshold are set to 0, preprocessing the obtained red fence, and finally converting it into a binary map as the training set of the ENet network.

[0025] The present invention also discloses a construction site fence area positioning system based on deep learning, including:

[0026] The first program module is used to extract the red fence in the construction site picture according to the fence color feature, and then convert it into a binary image to train the ENet network to obtain a trained ENet network model;

[0027] The second program module is used to segment the fence in the construction site picture by using the trained model to obtain the red fence in the fence;

[0028] The third program module is used to perform rectangular filling and polygon dilation on the red fence pixels according to the segmented red fence, generate a surface area covering the entire fence, and at the same time mark the positions corresponding to the surface area position in the construction site picture to realize the positioning of the construction site fence area.

[0029] The present invention further discloses a computer device, including a memory and a processor, where a computer program is stored on the memory, and the computer program executes the steps of the above-mentioned construction site fence area positioning method based on deep learning when being run by the processor.

[0030] Compared with the prior art, the advantages of the present invention are:

[0031] 1. The present invention uses a fence with obvious color features to train a neural network, and processes the segmentation result through computer graphics algorithms, which can quickly and accurately segment the position where the construction site fence is located, and has the advantages of low cost, high accuracy and low computational complexity; by processing some of the segmented red fences algorithmically, the entire area where the fences are located is obtained, and the fences are marked in the construction site. The characteristics of the red and white alternating fences are combined in the early stage of data processing, and the traditional segmentation data annotation method is not adopted. The training set is obtained through color features, which greatly reduces the human and time resources compared with the traditional method.

[0032] 2. The present invention connects the blank spaces between fences by means of polygon dilation to form a fence surface, so as to fit the entire area where the fences are located from the segmentation results of some fences. Without consuming a large amount of storage space and computing time, the positioning of the construction site fence area from part to whole and from piece to surface is realized, saving the time cost and economic cost of training with a large number of fully annotated training sets, and at the same time, the fence area can be accurately and quickly positioned, thus providing guarantee for the personal safety of the personnel at the electric power construction site.

[0033] 3. The present invention divides the segmentation result into several rectangular structural elements by setting a rectangular kernel of a specified size, and then fills the concave corners and bridges the small cracks of these small structural elements through closing operation, so that some fences with poor segmentation effect due to uneven illumination in the segmentation result can be segmented more smoothly. At the same time, the computational complexity of using the closing operation is relatively low, and the processing time is also within a reasonable range. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a flowchart of the method of the present invention in an embodiment.

[0035] Figure 2 It is a schematic diagram of the neural network training set in the present invention.

[0036] Figure 3 It is a schematic diagram of segmenting part of the red fence in the present invention.

[0037] Figure 4 It is a schematic diagram of the result of mathematical operations and polygon filling on part of the segmentation structure in the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0038] The present invention will be further described below in conjunction with the accompanying drawings of the specification and specific embodiments.

[0039] As Figure 1 shown, the method for positioning the construction site fence area based on deep learning in the embodiment of the present invention includes the steps:

[0040] 1) Extract the red fence portion from the construction site image based on the fence color feature (e.g., the fence is red and white), then convert it into a binary image to train the ENet network, obtaining a trained ENet network model;

[0041] 2) Use the trained ENet network model to detect whether there is a fence in the construction site image. If there is no fence, obtain the next construction site image. If there is a fence, segment the fence in the construction site image to obtain the red fence in the fence;

[0042] 3) Based on the segmented red fence, the foreground pixels containing the red fence are filled with rectangles and expanded with polygons to generate a surface area covering the entire fence. At the same time, the positions corresponding to the surface areas in the construction site image are marked to achieve the positioning of the construction site fence area.

[0043] In specific applications, if the human body is too close to the fence area, an alarm will be issued.

[0044] The present invention utilizes fences with obvious color features to train a neural network and processes the segmentation results through a computer graphics algorithm. It can quickly and accurately segment the location of the construction site fences and has the advantages of low cost, high accuracy and low computational complexity.

[0045] This method algorithmically processes the partially segmented red fences to identify the entire fence area and identify the fences at the construction site. By incorporating the red and white color signature of the fences in the early stages of data processing, rather than using traditional segmented data annotation methods, the training set is generated based on color features, significantly reducing labor and time resources compared to traditional methods.

[0046] The present invention connects the blank spaces between fences through polygon expansion to form a fence surface, thereby fitting the area where the entire fence is located from the segmentation results of the partial fence. Without consuming a lot of storage space and computing time, the present invention realizes the positioning of the construction site fence area from part to whole, from piece to surface, saving the time and economic cost of training with a large number of fully labeled training sets, and can accurately and quickly realize the positioning of the fence area, thereby providing protection for the safety of personnel at the power construction site.

[0047] In a specific embodiment, in step 1), the initial construction site picture is passed through a color recognition algorithm, and a pixel threshold is set so that pixels greater than the threshold are 255 and pixels less than the threshold are 0 in the construction site picture, thereby obtaining a preliminary red probability map. Pixels in the preliminary red probability map are then deleted and supplemented, and data with strong features are selected and converted into a 0-1 binary map and used as an ENet network training set, as shown in FIG. Figure 2 shown.

[0048] In a specific embodiment, in step 2), a rectangular kernel with a specified size (such as [50, 10]) is set to divide the red fence in the fence into several rectangular structural elements, and then a closing operation is performed on the rectangular structural elements to fill the concave corners. The image after the closing operation can better bridge the small cracks in the prediction map directly output by the model, and at the same time connect the horizontal fences as much as possible, thereby making up for the influence of uneven illumination, picture shooting angle, and possible plastic film coverage of some fences on the segmentation result (not smooth). In addition, the computational complexity of the closing operation is relatively low, and the processing time is also within a reasonable range.

[0049] Specifically, the closing operation is carried out in the way of first performing a dilation operation and then an erosion operation. The dilation operation can fill in some small disconnections or holes in the back of the target area in the image, and then the erosion operation can eliminate some noises outside the target contour. The corresponding mathematical formula for the closing operation is as follows:

[0050]

[0051] In the above formula, A represents the segmented binary image, and B is several structural elements divided by the rectangular kernel. represents the dilation operation, and ⊙ represents the erosion operation.

[0052] Let the image after the dilation operation be P, and the specific mathematical formula for the dilation operation is as follows:

[0053]

[0054] Bx,y in the above formula means moving the origin of the structural element to the point (x, y). Its geometric meaning is to traverse the binary image A through the structural element B. As long as there is a pixel intersection between the structural element B and the binary image A, the entire structural element B is retained.

[0055] The specific mathematical formula for performing an erosion operation on the image P after the dilation operation is as follows:

[0056] A·B = P⊙B = {x,y|B x,y ∈ P}

[0057] Its geometric meaning is to traverse the binary image P after dilation through the structural element B, retain the pixel points where there is an intersection between the structural element B and the image P after the dilation operation, and remove other non-intersecting pixel points.

[0058] Further, in step 3), for the red fence after closing operation, the fence is partitioned in combination with the distribution area of foreground pixels, and at the same time, the vertical pixel threshold is set to 10 to determine the columns with less than 10 pixel points in the column direction, so as to eliminate the incorrect horizontal connections generated by the closing operation, and then the fence pixels are subjected to polygon dilation processing by area.

[0059] Furthermore, after polygon dilation processing, the columns with foreground pixels are traversed, and all pixel values between the topmost pixel point and the bottommost pixel point of this column are set to 255, so as to eliminate the influence that the fence cannot be fully restored in some scenarios of polygon dilation. Finally, the construction site picture is colored at the position corresponding to the processed fence area to mark the fence area.

[0060] The present invention also discloses a construction site fence area positioning system based on deep learning, including:

[0061] The first program module is used to extract the red fence in the construction site picture according to the fence color feature, and then convert it into a binary image to train the ENet network, and obtain a trained ENet network model;

[0062] The second program module is used to segment the fence in the construction site picture by using the trained model to obtain the red fence in the fence;

[0063] The third program module is used to perform rectangular filling and polygon dilation on the red fence pixels according to the segmented red fence, generate a surface area covering the entire fence, and at the same time mark the position corresponding to the surface area position in the construction site picture to realize the positioning of the construction site fence area.

[0064] The construction site fence area positioning system based on deep learning of the present invention corresponds to the above method and has the same advantages as those described in the above method.

[0065] The present invention also discloses a computer device, including a memory and a processor, wherein a computer program is stored on the memory, and the computer program executes the steps of the above-mentioned construction site fence area positioning method based on deep learning when being run by the processor.

[0066] The following further elaborates on the method of the present invention in conjunction with the accompanying drawings:

[0067] (1) Obtain the training set of the deep learning network ENet, set the pixel threshold, obtain the preliminary red fence probability map, perform pixel deletion and supplementation on the preliminary probability map through an algorithm, and select the data with stronger features as the training set of the ENet network to train the ENet network model, and obtain a trained ENet network model, as Figure 2 shown;

[0068] (2) Obtain any construction site picture and put it into the ENet network model trained in step (1). If the construction site picture does not contain fencing facilities, automatically obtain the next construction site picture. If it contains fencing, generate a preliminary predicted segmentation map, as Figure 3 shown.

[0069] (3) Perform a closing operation (close_operation) and polygon dilation (fill) on the predicted segmentation map obtained through the ENet network model, and mark the complete fencing area in the original construction site picture. The overall process is as Figure 4 shown.

[0070] The above is only the preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions falling within the idea of the present invention belong to the protection scope of the present invention. It should be noted that for those of ordinary skill in the art, several improvements and refinements made without departing from the principle of the present invention should be regarded as within the protection scope of the present invention.

Claims

1. A method for locating the fence area at a construction site based on deep learning, wherein the fence includes a red fence, characterized in that, Including the steps: 1) Extract the red fences in the construction site pictures according to the fence color characteristics, then convert the extracted red fences into binary images as the training set of the ENet network to train the ENet network, and obtain the trained ENet network model; 2) Use the trained ENet network model to segment the fences in the construction site pictures to obtain the red fences in the fences; 3) According to the segmented red fences, perform rectangular filling and polygon dilation on the red fence pixels to generate a surface area covering all the fences, and at the same time mark the positions corresponding to the surface area position in the construction site pictures to achieve the positioning of the construction site fence area; In step 3), set a rectangular kernel of a specified size to divide the red fences in the fences into several rectangular structural elements, and then perform a closing operation on the rectangular structural elements to fill the concave corners; In step 3), after performing polygon dilation, traverse the columns with foreground pixels after polygon dilation, so that all pixels between the topmost pixel point and the bottommost pixel point of this column are 255, generating a surface area covering all the fences.

2. The method for positioning a construction site fence area based on deep learning according to claim 1, wherein The closing operation is performed in the way of first dilation operation and then erosion operation. Among them, the dilation operation is used to fill the disconnections in the image or the holes in the back of the target area, and the erosion operation is used to eliminate some noises outside the target contour; the mathematical formula of the closing operation is as follows: Among them, A represents the binarized image after segmentation, and B is several rectangular structural elements obtained by dividing the rectangular kernel. represents the dilation operation. represents the erosion operation.

3. The method for positioning the fence area at the construction site based on deep learning according to claim 2, characterized in that, Among them, the specific mathematical formula of the dilation operation is as follows: Where Bx,y represents moving the origin of the rectangular structural element to the point (x,y), and its geometric meaning represents traversing the binary image A through the structural element B. If there is a pixel intersection between the structural element B and the binary image A, then the entire structural element B is retained.

4. The method for positioning the fence area at the construction site based on deep learning according to claim 3, characterized in that, The specific mathematical formula for performing the erosion operation on the image P after the dilation operation is as follows: Its geometric meaning represents that the structural element B traverses the binary image P after the dilation operation, retains the pixel points where there is an intersection between the structural element B and the binary image P, and removes other non-intersecting pixel points.

5. The method for positioning a construction site fence area based on deep learning according to any one of claims 1 to 4, characterized in that, In step 3), before performing rectangular filling and polygon dilation, partition the red fences after the closing operation in combination with the distribution area of the foreground pixels, and at the same time set the vertical pixel point threshold to n to judge the columns with less than n pixel points in the column direction to eliminate the incorrect horizontal connections generated by the closing operation.

6. The method for positioning the fence area of the construction site based on deep learning according to any one of claims 1 to 4, characterized in that, The specific process of step 1) is: obtain the preliminary red fence probability map of the construction site pictures through the color recognition algorithm; then set the pixel threshold so that the pixels greater than the threshold in the construction site pictures are 255, and the pixels less than the threshold are 0, preprocess the obtained red fences, and finally convert them into binary images as the training set of the ENet network.

7. A construction site fence area positioning system based on deep learning, which is used to execute the steps of the construction site fence area positioning method based on deep learning according to any one of claims 1-6, characterized in that, Including: The first program module is used to extract the red fences in the construction site pictures according to the fence color characteristics, and then convert them into binary images to train the ENet network, and obtain the trained ENet network model; The second program module is used to segment the fences in the construction site pictures by using the trained model to obtain the red fences in the fences; The third program module is used to perform rectangular filling and polygon dilation on the red fence pixels according to the segmented red fence, generate a surface area covering all the fences, and at the same time identify the positions corresponding to the surface area position in the construction site picture, so as to realize the positioning of the fence area in the construction site.

8. A computer device, comprising a memory and a processor, wherein a computer program is stored on the memory, characterized in that, When the computer program is run by a processor, it executes the steps of the method for positioning the fence area in the construction site based on deep learning according to any one of claims 1 to 6.

Images