Drainage well lid blockage detection system and method based on machine vision

Abstract

The invention provides a drainage well lid blockage detection system and method based on machine vision, and the system comprises a data collection module, a preprocessing module, an edge detection module, a rectangular calibration module, a blockage detection module and an early warning judgment module. Whether the point is a blockage point or not is judged at the corresponding position in the grey-scale map according to the pixel value, the blockage point is summed, the ratio of the blockage point to the total pixel value serves as the standard for judging the blockage rate, and a corresponding early warning and solution scheme is provided according to the blockage degree grade, so that the blockage rate can be accurately judged even if the blockage object is cleaned. And water accumulation in rainy days is avoided to a certain extent.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent detection, and in particular relates to a drainage manhole cover blockage detection system and method based on machine vision. Background Art

[0002] The problem of manhole cover blockage is an issue that cannot be ignored. It is not only related to the cleanliness and beauty of the city, but also directly related to the travel safety and quality of life of citizens. This problem is particularly prominent in the rainy season. Rainwater cannot be smoothly discharged into the underground pipe network, resulting in serious waterlogging on the road. It not only affects the passage of vehicles and increases the risk of traffic accidents, but may also flood low-lying areas due to excessive water accumulation, posing a direct threat to pedestrians and vehicles, especially small vehicles such as electric bicycles and motorcycles, which are more likely to lose control in the water and cause traffic accidents.

[0003] Regarding the research on manhole cover issues, in 2017, in the invention of "An intelligent manhole cover early warning device and early warning control system", sensor equipment was used to detect the status of the manhole cover and the water level, but the sensor device cannot be installed on rainy days, which will damage the parts. In 2021, in the invention of "A blockage detection system and blockage detection method for drainage manhole covers", hardware was installed in the holes of the manhole cover grate to detect the water level and transmit pressure signals to see if there is water accumulation. In 2022, in the invention of "An intelligent monitoring and drainage system for urban road flooding", by collecting information on the depth of water accumulation on the road, the sensor device detects whether the manhole cover is missing and determines whether a blockage occurs by calculating the water flow velocity. The above research on manhole covers is all carried out after water accumulation, and additional equipment needs to be installed, which is less feasible and has high economic costs.

[0004] Nowadays, there are many manhole covers and surveillance cameras in the city. These cameras capture video images 24 hours a day. In addition to monitoring urban safety and saving urban dynamics, they can also be used to detect drainage manhole covers. Whether it is sunny or rainy, they can capture photo information without the need for additional hardware facilities. Summary of the invention

[0005] The purpose of the present invention is to solve the defects of the above-mentioned prior art and to provide a drainage manhole cover blockage detection system and method based on machine vision, so as to design a detection system with low cost, high feasibility and early warning, clear the blockage in time and avoid the occurrence of water accumulation on rainy days to a certain extent.

[0006] The present invention adopts the following technical solution:

[0007] The machine vision-based drainage manhole cover blockage detection system includes:

[0008] Data acquisition module,The data acquisition module is used to capture and obtain manhole cover images and provide raw data for subsequent processing and analysis.

[0009] The preprocessing module is used to preprocess the collected manhole cover images, including converting them into grayscale images.

[0010] The edge detection module is used to detect the polarity edges of the grayscale image using an edge detection algorithm, extract the edge information of the manhole cover and the manhole cover holes, and form an edge detection image.

[0011] Rectangle calibration module,The rectangle calibration module is used to calibrate the rectangle of the manhole cover hole in the edge detection image and determine the position and size of the calibrated rectangle.

[0012] The blockage detection module is used to traverse the grayscale value of each pixel in the grayscale image at the calibrated rectangular position using an algorithm to determine whether there is a blockage point and calculate the blockage rate.

[0013] The early warning judgment module is used to make early warning judgments based on the congestion rate. If the congestion rate reaches a preset value, an early warning will be issued to remind relevant personnel to handle it.

[0014] Beneficial effects: (1) Since the risk of manhole cover blockage is very high, timely warning and treatment can improve social stability and safety and eliminate hidden dangers. The present invention has great feasibility and good engineering application.

[0015] (2) Through rectangular calibration, the hole positions in the edge detection image of the manhole cover map are marked, and then the corresponding position in the grayscale image is judged whether the point is a blockage point according to the size of the pixel value. After summing the blockage points, the ratio of the sum to the total pixel value is used as the standard for judging the blockage rate, and corresponding warnings and solutions are proposed according to the degree of blockage, so as to facilitate the timely cleaning of blockages and avoid the occurrence of waterlogging on rainy days to a certain extent.

[0016] (3) The system can automatically complete the entire process from data collection to congestion detection and early warning judgment, reducing the frequency and intensity of manual inspections and improving work efficiency.

[0017] (4) Through advanced image processing and pattern recognition algorithms, the system can accurately identify the edges of manhole covers and holes, precisely calibrate rectangular positions, and effectively determine blockage points, reducing human errors.

[0018] (5) Since the system can work automatically and continuously for a long time, the cost of manual inspection and maintenance is reduced, while the utilization rate and life of the equipment are improved. At the same time, the manhole cover image can be obtained through the camera within the range of the manhole cover, which reduces the cost of data collection.

[0019] Furthermore, when the preprocessing module is converted into a grayscale image, it is processed using a frequency tuning algorithm, the manhole cover image is Gaussian filtered to smooth the image, the filtered image is converted from the RGB color space to the LAB color space, the images of the three channels of L, A, and B are averaged, the saliency value is calculated, and the maximum value is used to normalize the saliency map.

[0020] Beneficial effects: Gaussian filtering as a preprocessing step can effectively smooth the image and remove high-frequency noise, which helps reduce interference and errors in subsequent processing steps. Converting images from RGB color space to LAB color space can better simulate people's visual perception. LAB space has a wider color range and is closer to the human eye's separation of brightness and color, which makes the grayscale processing more refined and more in line with the visual habits of the human eye.

[0021] By taking the mean of the images of the three channels L, A, and B, the overall hue and brightness information of the image can be extracted. This information is retained during the grayscale process, which helps to maintain the important features of the original image. The saliency value is obtained by calculating the Euclidean distance between the mean image and the filtered image. This method can accurately measure the difference between each pixel and the overall feature. The generation of a saliency map helps to highlight the key areas in the image, so that these areas can receive more attention during the grayscale process.

[0022] Normalizing the saliency map using the maximum value can map the saliency value to a uniform range, which is convenient for subsequent image processing and analysis. The normalized saliency map not only retains the key information in the image, but also enhances the readability and usability of the grayscale image.

[0023] Furthermore, when the edge detection module performs edge detection, the Sobel operator is used for edge detection, wherein two 3*3 templates are used, and the template convolution method is adopted during the operation to move the template on the image, and at each position, each coefficient on the template is multiplied and summed with the grayscale value of each corresponding pixel under the template, and the gradient value of the corresponding center pixel is calculated to extract the horizontal and vertical edges of the lower drainage manhole cover hole in the image.

[0024] Beneficial effects: The Sobel operator uses template convolution, which is a fast and effective method. By using two 3*3 templates (one for detecting horizontal edges and one for detecting vertical edges), it can quickly move across the entire image and calculate the gradient value, thereby achieving efficient edge detection.

[0025] The Sobel operator combines Gaussian smoothing and differential derivation, so it can resist the influence of noise to a certain extent and locate edges more accurately. The implementation of the Sobel operator is relatively simple and does not require complex mathematical operations or a large amount of computing resources. The Sobel operator works better for images with grayscale gradients and more noise. It can detect edges based on the changes in brightness next to the edge of the image, which helps to extract the outline of the target object in a complex background.

[0026] Furthermore, when calibrating the rectangle of the manhole cover hole, the rectangle calibration module draws the minimum circumscribed rectangle according to the algorithm and performs red line calibration on it.

[0027] Beneficial effects: Red line calibration makes the position and size of the manhole cover hole clear in the image, which is very intuitive. This is extremely beneficial for subsequent blockage detection and manual review. By drawing the minimum circumscribed rectangle, it can ensure that the calibration frame fits the edge of the manhole cover hole tightly, thereby reducing errors and improving the accuracy of position and size information.

[0028] Furthermore, the blockage detection module uses an algorithm to traverse the grayscale value of each pixel in the grayscale image, calibrate the position of the manhole in the rectangular calibration image, and after finding the empty position, perform detection calculations on the grayscale image, marking from left to right, from top to bottom, from 1 to n, and the entire manhole cover matrix A is represented; traverse the rectangular box in the image, from A 11 First, traverse the pixels in each rectangular box in turn;

[0029] The gray value range of the pixel is 0 to 255, 0 represents pure black, 255 represents white, the larger the value, the brighter the pixel, in the processed image, white represents no blockage, black represents complete blockage, and in the process of pixel value increasing from 0 to 255, the larger the value, the smaller the blockage, and 128 is specified as the dividing line;

[0030] Compare the grayscale value of each pixel with 128. When the grayscale value is less than 128, the counter accumulates 1, indicating that the pixel position is blocked. After the traversal is completed, the ratio of the accumulator value to the total pixel value of the image represents the proportion of the blocked part. After traversing all the rectangular boxes, add up the proportion values ​​to obtain the total proportion of the blocked coverage in the entire manhole cover map, and then divide it by 15 to get the unit ratio value.

[0031] Beneficial effects: By calibrating the position of the manhole in the rectangular calibration image and traversing the pixel points of the entire manhole cover, it is possible to ensure that the blockage of each part of the manhole cover is fully detected without omission. The grayscale value is used to determine the degree of blockage, which can be accurate to the pixel level. By setting a suitable threshold (such as 128), the blocked and unblocked areas can be accurately distinguished, thereby obtaining an accurate blockage ratio.

[0032] By calculating the proportion of the blocked part and the total proportion of blocked coverage in the entire manhole cover map, the blockage situation can be quantified into specific values, which is convenient for evaluating and comparing the blockage situation of different manhole covers or at different times. Using a grayscale image to represent the degree of blockage is simple and intuitive, easy to understand and explain. White represents no blockage, black represents complete blockage, and the intermediate grayscale values ​​represent different degrees of blockage. This representation method is in line with people's intuitive perception.

[0033] Furthermore, when the early warning judgment module makes an early warning judgment based on the congestion rate, if the congestion rate is greater than or equal to 0 and less than 40%, the congestion level is recorded as 1, and the corresponding plan is: observe, no alarm; if the congestion rate is greater than or equal to 40% and less than 70%, the congestion level is recorded as 2, and the corresponding plan is: issue an early warning and find an opportunity to clear; if the congestion rate is greater than or equal to 70% and less than 100%, the congestion level is recorded as 3, and the corresponding plan is: serious early warning and immediate clearing.

[0034] Beneficial effects: By setting different congestion rate thresholds, the warning is divided into multiple levels. This hierarchical warning mechanism can more accurately reflect the severity of the congestion and provide more targeted guidance for subsequent processing. When the congestion rate reaches the preset threshold, the system will immediately issue a warning to remind relevant personnel to handle it. This timeliness helps reduce potential harm and losses caused by congestion.

[0035] By setting a lower congestion rate threshold (such as 40%), the system can alert observers before the congestion problem becomes serious, so that preventive measures can be taken to prevent the problem from worsening. This preventive management helps reduce maintenance costs and risks. The formulation of the plan takes into account the conditions of different congestion levels, so the treatment measures can be flexibly adjusted according to the actual congestion situation. This flexibility enables the system to adapt to a variety of complex and changing scenarios. Each congestion level corresponds to a clear plan, which provides clear operational guidance for the handling personnel. Clarity helps reduce confusion and errors in the processing process and improve processing efficiency.

[0036] Furthermore, it also includes an image enhancement module, which is used to perform image enhancement on the manhole cover image after shooting and acquiring the manhole cover image. The image enhancement includes but is not limited to contrast enhancement, noise reduction and sharpening.

[0037] Beneficial effects: Through processing technologies such as contrast enhancement, noise reduction and sharpening, the visual quality of manhole cover images can be significantly improved, the accuracy of feature recognition can be improved, the stability of the algorithm can be enhanced, and a higher quality data foundation can be provided for image processing and analysis.

[0038] Furthermore, it also includes a multi-perspective fusion module, which is used to combine the image information of different angles of the manhole cover pictures obtained by the data acquisition module from different manhole cover shots to generate a manhole cover status view.

[0039] Beneficial effects: The multi-view fusion module can provide more comprehensive and accurate manhole cover status information by combining manhole cover images taken from different angles by multiple cameras, which helps to improve the accuracy and efficiency of manhole cover status detection.

[0040] The drainage manhole cover blockage detection method based on machine vision is completed by using the drainage manhole cover blockage detection system based on machine vision, including:

[0041] Step 1: Data collection and multi-perspective fusion: Use multiple cameras to simultaneously shoot the manhole cover from different angles through the data collection module to obtain multi-perspective images, and use the multi-perspective fusion module to combine the image information from these different angles to generate a manhole cover status view.

[0042] Step 2: Image enhancement: The fused manhole cover image is processed using the image enhancement module; the processing includes contrast enhancement, noise reduction and sharpening.

[0043] Step 3, preprocessing: Use the preprocessing module to convert the manhole cover image into a grayscale image, use the frequency tuning algorithm to smooth the image, and remove noise through Gaussian filtering. Convert the filtered image from RGB color space to LAB color space, and calculate the average value of the three channels L, A, and B, and calculate the significance value; and perform normalization to obtain the grayscale processed image.

[0044] Step 4: Edge detection: Use the Sobel operator in the edge detection module to perform edge detection on the grayscale image. The Sobel operator performs convolution operations with the image through two 3*3 templates to extract the edge information of the manhole cover and the manhole hole.

[0045] Step 5: Rectangle calibration: In the edge detection image, automatically simulate and draw the minimum circumscribed rectangle of the manhole cover hole, and calibrate the rectangle to determine its position and size.

[0046] Step 6, blockage detection: Use the blockage detection module to traverse each pixel point in the calibrated rectangular area of ​​the grayscale image, and judge the grayscale value of each pixel point. If the grayscale value is less than 128, it is considered that there is a blockage at the harness point position; record the number of blockage points through the counter and calculate the blockage rate.

[0047] Step 7, early warning judgment: The early warning judgment module makes corresponding early warning judgments according to the congestion rate; the judgment criteria are as follows: the congestion rate is less than 40%: it is recorded as congestion level 1, indicating slight congestion or no congestion, no alarm is required, only observation is required; the congestion rate is between 40% and 70%: it is recorded as congestion level 2, a warning is issued and plans are made to clean up at the appropriate time; the congestion rate is greater than 70%: it is recorded as congestion level 3, indicating serious congestion, and a serious warning needs to be issued immediately and cleaned up as soon as possible.

[0048] Step 8, response and record: Based on the early warning judgment results, the system automatically triggers the corresponding response measures, and all detection and processing results should be recorded and saved by the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 Schematic diagram of a drainage manhole cover blockage detection system based on machine vision according to an embodiment of the present invention;

[0050] Figure 2 This is a flow chart of a drainage manhole cover detection system based on machine vision according to an embodiment of the present invention.

[0051] Figure 3 1 is a grayscale image of manhole cover a, manhole cover b, manhole cover c and manhole cover d according to an embodiment of the present invention. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention is described clearly and completely below. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0053] like Figure 1-Figure 3 As shown, the drainage manhole cover blockage detection system based on machine vision of the present invention includes:

[0054] Data acquisition module,The data acquisition module is used to capture and obtain manhole cover images and provide raw data for subsequent processing and analysis.

[0055] The multi-view fusion module is used to combine the image information from different angles of the manhole cover pictures obtained by the data acquisition module from different manhole cover photos to generate a manhole cover status view.

[0056] Image enhancement module: The image enhancement module is used to process the manhole cover image after multi-view fusion and perform image enhancement on the manhole cover image. Image enhancement includes but is not limited to contrast enhancement, noise reduction and sharpening.

[0057] Preprocessing module, the preprocessing module is used to preprocess the manhole cover image after image enhancement, including but not limited to converting it into a grayscale image; wherein, using a frequency tuning algorithm, the manhole cover image is Gaussian filtered to smooth the image, the filtered image is converted from the RGB color space to the LAB color space, and the images of the three channels L, A, and B are averaged respectively; then the saliency value is calculated, that is, the Euclidean distance between the mean image of the three channels and the filtered image is taken and summed, and finally the saliency map is normalized using the maximum value to achieve grayscale processing.

[0058] Edge detection module, the edge detection module is used to use the edge detection algorithm to detect the polar edges of the grayscale image, extract the edge information of the manhole cover and the manhole cover hole, and form an edge detection image; wherein, the Sobel operator is used for edge detection, wherein two 3*3 templates are used, and the template convolution method is adopted during the operation, the template is moved on the image, and at each position, each coefficient on the template is multiplied and summed with the grayscale value of each corresponding pixel under the template, and the gradient value of the corresponding central pixel is calculated to extract the horizontal and vertical edges of the lower drainage manhole cover hole in the image.

[0059] The rectangle calibration module is used to calibrate the rectangle of the manhole cover hole in the edge detection image and determine the position and size of the calibrated rectangle; wherein, the minimum circumscribed rectangle is drawn according to the algorithm and the red line is calibrated.

[0060] The blockage detection module is used to traverse the grayscale value of each pixel in the grayscale image at the calibrated rectangular position using an algorithm to determine whether there is a blockage point and calculate the blockage rate. When the algorithm is used to traverse the grayscale value of each pixel in the grayscale image, the position of the manhole in the rectangular calibration image is calibrated. After finding the empty position, the detection calculation is performed on the grayscale image, marking from left to right, from top to bottom, and from 1 to n. The entire manhole cover is represented by matrix A. The rectangular frame in the image is traversed from A. 11 Start by traversing the pixels in each rectangular box; for example, the hole positions are represented as follows:

[0061] A 11 A 12 A 13 A 14 A 15

[0062] A 21 A 22 A 23 A 24 A 25

[0063] A 31 A 32 A 33 A 34A 35

[0064] Among them, the grayscale value range of the pixel is 0 to 255, 0 represents pure black, 255 represents white, the larger the value, the brighter the pixel, in the processed image, white represents no blockage, black represents complete blockage, and as the pixel value increases from 0 to 255, the larger the value, the smaller the degree of blockage, and 128 is set as the dividing line.

[0065] Compare the grayscale value of each pixel with 128. When the grayscale value is less than 128, the counter accumulates 1, indicating that the pixel position is blocked. After the traversal is completed, the ratio of the accumulator value to the total pixel value of the image indicates the proportion of the blocked part. After traversing all the rectangular boxes, add up the proportion values ​​to get the total proportion of the blocked coverage in the entire manhole cover map, and then divide it by 15 to get the unit ratio value.

[0066] The early warning judgment module is used to make early warning judgments based on the congestion rate. If the congestion rate is greater than or equal to 0% and less than 40%, the congestion level is recorded as 1, and the corresponding plan is: just observe, no alarm; if the congestion rate is greater than or equal to 40% and less than 70%, the congestion level is recorded as 2, and the corresponding plan is: issue a warning and find an opportunity to clear; if the congestion rate is greater than or equal to 70% and less than 100%, the congestion level is recorded as 3, and the corresponding plan is: serious warning and immediate clearing.

[0067] Another example, such as Figure 2 As shown, a drainage manhole cover blockage detection method based on machine vision is provided, which relies on a drainage manhole cover blockage detection system based on machine vision, including:

[0068] Step 1. Data collection and multi-view fusion: Use multiple cameras to simultaneously capture the manhole cover from different angles through the data collection module to obtain multi-view images. These images will serve as the basis for subsequent processing and analysis. Use the multi-view fusion module to combine the image information from these different angles to generate a more comprehensive and accurate view of the manhole cover status. This helps reduce misjudgments that may be caused by viewing angle limitations.

[0069] Step 2. Image enhancement: Use the image enhancement module to process the fused manhole cover image to improve the image quality. The processing includes contrast enhancement, noise reduction and sharpening, etc., in order to make the manhole cover and obstruction in the image more clearly visible.

[0070] Step 3. Preprocessing: The preprocessing module converts the manhole cover image into a grayscale image to reduce the amount of data and complexity of subsequent processing. The frequency tuning algorithm is used for image smoothing, and Gaussian filtering is used to remove noise. The filtered image is converted from RGB color space to LAB color space, and the average values ​​of the three channels L, A, and B are calculated. The grayscale processed image is obtained by calculating the saliency value (sum of Euclidean distance) and normalizing it.

[0071] Step 4. Edge detection: Use the Sobel operator in the edge detection module to perform edge detection on the grayscale image. The Sobel operator performs convolution operations with the image through two 3*3 templates to extract the edge information of the manhole cover and the manhole hole. The resulting edge detection image will clearly show the outline of the manhole cover and the location of the hole.

[0072] Step 5. Rectangle calibration: In the edge detection map, automatically simulate and draw the minimum circumscribed rectangle of the manhole cover hole. Calibrate these rectangles (such as marking them with red lines) to determine their position and size. The calibration information will be used for subsequent blockage detection.

[0073] Step 6. Blockage detection: The blockage detection module traverses each pixel point in the calibrated rectangular area of ​​the grayscale image and judges the grayscale value of each pixel point. If the grayscale value is less than 128 (the set dividing line), it is considered that there is a blockage at the harness point position.

[0074] The number of blocked points is recorded by a counter, and the blocking rate (ratio of the number of blocked points to the total number of pixels) is calculated. If a more detailed blockage analysis is required, the grayscale value can be further graded.

[0075] Step 7. Early warning judgment: The early warning judgment module makes corresponding early warning judgments based on the congestion rate. The judgment criteria are as follows: If the congestion rate is less than 40%, it is recorded as congestion level 1, indicating slight congestion or no congestion, no alarm is required, only observation is required. If the congestion rate is between 40% and 70%, it is recorded as congestion level 2, and an early warning is issued and plans are made to clear the road at an appropriate time. If the congestion rate is greater than 70%, it is recorded as congestion level 3, indicating severe congestion, and a severe early warning needs to be issued immediately and cleared as soon as possible.

[0076] Step 8. Respond to the record: Based on the warning judgment result, the system automatically triggers the corresponding response measures, such as sending an alarm notification to relevant personnel or automatically recording the blockage situation and handling suggestions.

[0077] All detection and processing results should be systematically recorded and saved for subsequent analysis and optimization.

[0078] Example:

[0079] The implementation site of the machine vision-based drainage manhole cover blockage detection system was selected in several main streets in the city center. These streets have a large flow of people and vehicles, and the drainage manhole covers are frequently used. They are easily affected by various garbage and debris, making them ideal places to test the actual effect of the detection system. Several representative drainage manhole covers were selected on each street for monitoring.

[0080] Four high-definition cameras were installed around each selected manhole cover, located in the east, south, west and north directions of the manhole cover to ensure that the image of the manhole cover and its surrounding environment can be fully captured. Figure 3 As shown in the figure, images of manhole covers a, b, c, and d were collected under rainy conditions. Since the actual content of the garbage does not affect the grayscale value analysis and blockage rate calculation, leaves are used as simulated garbage, covering 0% (manhole cover a), 12% (manhole cover b), 17% (manhole cover c), and 37% (manhole cover d) of the manhole cover area respectively. Through the multi-view fusion module, the images taken by the four cameras are fused in real time to generate a view that can fully reflect the status of the manhole cover, providing a high-quality data source for subsequent processing.

[0081] After image enhancement, preprocessing and grayscale, edge detection and rectangle calibration in the above scheme, the blockage detection module traverses each pixel point in the calibrated rectangular area, and determines whether there is blockage by comparing the grayscale value of the pixel point with the set boundary line (128). The blockage rate is calculated according to the number of blockage points. The blockage rates of manhole covers a, b, c, and d are 0%, 13.5%, 19.2%, and 36.8% respectively.

[0082] According to the blockage rate, manhole cover a is blockage level 1 (no alarm required), manhole covers b and c are blockage level 2 (warning issued and planned cleaning), and manhole cover d is blockage level 3 (serious warning and immediate cleaning). The system automatically records the blockage situation and sends a warning notification to relevant personnel, suggesting that manhole covers b, c, and d be cleaned in a timely manner.

[0083] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. The drainage manhole cover blockage detection system based on machine vision is characterized by ,include: Data acquisition module: The data acquisition module is used to capture and obtain manhole cover images and provide raw data for subsequent processing and analysis; A preprocessing module is used to preprocess the collected manhole cover images, including converting them into grayscale images; The edge detection module is used to detect the polarity edge of the grayscale image using an edge detection algorithm, extract edge information of the manhole cover and the manhole cover hole, and form an edge detection image; Rectangle calibration module: The rectangle calibration module is used to calibrate the rectangle of the manhole cover hole in the edge detection image and determine the position and size of the calibrated rectangle; The congestion detection module is used to traverse the grayscale value of each pixel in the grayscale image at the calibrated rectangular position using an algorithm to determine whether there is a congestion point and calculate the congestion rate; The early warning judgment module is used to make early warning judgments based on the congestion rate. If the congestion rate reaches a preset value, an early warning will be issued to remind relevant personnel to handle it.

2. The system according to claim 1, characterized in that When the preprocessing module is converted into a grayscale image, the frequency tuning algorithm is used for processing. The manhole cover image is Gaussian filtered to smooth the image. The filtered image is converted from the RGB color space to the LAB color space. The images of the three channels of L, A, and B are averaged respectively, the saliency value is calculated, and the maximum value is used to normalize the saliency map.

3. The system according to claim 1, characterized in that When the edge detection module performs edge detection, the Sobel operator is used for edge detection. Two 3*3 templates are used, and the template convolution method is adopted during the operation. The template is moved on the image, and each coefficient on the template is multiplied and summed with the grayscale value of each corresponding pixel under the template at each position, and the gradient value of the corresponding center pixel is calculated to extract the horizontal and vertical edges of the lower drainage manhole cover hole in the image.

4. The system according to claim 1, characterized in that When calibrating the rectangle of the manhole cover hole, the rectangle calibration module draws the minimum circumscribed rectangle according to the algorithm and performs red line calibration on it.

5. The system according to claim 1, characterized in that The blockage detection module uses an algorithm to traverse the grayscale value of each pixel in the grayscale image, calibrate the position of the manhole in the rectangular calibration image, and after finding the empty position, perform detection calculations on the grayscale image, marking from left to right, from top to bottom, from 1 to n, and the entire manhole cover matrix A is represented; traverse the rectangular frame in the image, from A 11 First, traverse the pixels in each rectangular box in turn; The gray value range of the pixel is 0 to 255, 0 represents pure black, 255 represents white, the larger the value, the brighter the pixel, in the processed image, white represents no blockage, black represents complete blockage, and in the process of pixel value increasing from 0 to 255, the larger the value, the smaller the blockage, and 128 is specified as the dividing line; Compare the grayscale value of each pixel with 128. When the grayscale value is less than 128, the counter accumulates 1, indicating that the pixel position is blocked. After the traversal is completed, the ratio of the accumulator value to the total pixel value of the image represents the proportion of the blocked part. After traversing all the rectangular boxes, add up the proportion values ​​to obtain the total proportion of the blocked coverage in the entire manhole cover map, and then divide it by 15 to get the unit ratio value.

6. The system according to claim 1, characterized in that When the early warning judgment module makes an early warning judgment based on the congestion rate, if the congestion rate is greater than or equal to 0 and less than 40%, the congestion level is recorded as 1, and the corresponding plan is: observation, no alarm; if the congestion rate is greater than or equal to 40% and less than 70%, the congestion level is recorded as 2, and the corresponding plan is: issue an early warning and find an opportunity to clear; if the congestion rate is greater than or equal to 70% and less than 100%, the congestion level is recorded as 3, and the corresponding plan is: serious early warning and immediate clearing.

7. The system according to claim 1, characterized in that It also includes an image enhancement module, which is used to perform image enhancement on the manhole cover image after shooting and acquiring the manhole cover image. The image enhancement includes but is not limited to contrast enhancement, noise reduction and sharpening.

8. The system according to claim 1, characterized in that It also includes a multi-perspective fusion module, which is used to combine the image information from different angles of the manhole cover pictures obtained by the data acquisition module from different shootings of the manhole cover to generate a manhole cover status view.

9. The detection method using the machine vision-based drainage manhole cover blockage detection system as claimed in claim 1 is characterized in that: include: Step 1. Data collection and multi-view fusion: Use multiple cameras to simultaneously shoot the manhole cover from different angles through the data collection module to obtain multi-view images, and use the multi-view fusion module to combine the image information from different angles to generate a manhole cover status view; Step 2. Image enhancement: The image enhancement module is used to process the fused manhole cover image; the processing includes contrast enhancement, noise reduction and sharpening; Step 3. Preprocessing: Use the preprocessing module to convert the manhole cover image into a grayscale image, use the frequency tuning algorithm to smooth the image, and remove noise through Gaussian filtering. Convert the filtered image from RGB color space to LAB color space, and calculate the average value of the three channels L, A, and B. Calculate the significant value; and perform normalization to obtain the grayscale processed image; Step 4. Edge detection: Use the Sobel operator in the edge detection module to perform edge detection on the grayscale image. The Sobel operator performs convolution operations with the image through two 3*3 templates to extract the edge information of the manhole cover and the manhole hole; Step 5. Rectangle calibration: In the edge detection image, automatically simulate and draw the minimum circumscribed rectangle of the manhole cover hole, and calibrate the rectangle to determine its position and size; Step 6. Blockage detection: Use the blockage detection module to traverse each pixel point in the calibrated rectangular area of ​​the grayscale image, and judge the grayscale value of each pixel point. If the grayscale value is less than 128, it is considered that there is a blockage at the position of the harness point; record the number of blockage points through the counter, and calculate the blockage rate; Step 7. Early warning judgment: The early warning judgment module makes corresponding early warning judgments according to the congestion rate; the judgment criteria are as follows: if the congestion rate is less than 40%, it is recorded as congestion level 1, indicating slight congestion or no congestion, no alarm is required, only observation is required; if the congestion rate is between 40% and 70%, it is recorded as congestion level 2, and an early warning is issued and plans are made to clear the road at an appropriate time; if the congestion rate is greater than 70%, it is recorded as congestion level 3, indicating severe congestion, and a severe early warning needs to be issued immediately and the road needs to be cleared as soon as possible; Step 8. Response and Recording: Based on the warning judgment results, the system automatically triggers the corresponding response measures, and all detection and processing results should be recorded and saved by the system.

Images