A method for detecting the state of a manhole cover

Abstract

The application discloses a kind of inspection methods of inspection well lid state, comprising:A: setting mark point on the back of inspection well lid, collecting the clear and complete back image of inspection well lid as calibration image and storage, extracting the feature of mark point in calibration image as calibration feature point;B: the detection image of inspection well lid is collected and carries out gamma operation;C: the standard detection image is obtained by binaryzation processing to detection image;D: the feature of mark point in standard detection image is extracted as detection feature point, and detection feature point is matched in calibration image;E: detection feature point is compared with calibration feature point, and the distance difference of each detection feature point and corresponding each calibration feature point is calculated, then according to distance difference, the various states of inspection well lid are discriminated.The method can real-time and intuitively identify the damage, subsidence, inclination and movement state of inspection well lid using one image sensor, and solves the problems of existing technology, such as unable to detect comprehensively and in real time, high labor intensity, waste of resources and the like.

Description

Technical Field

[0001] This invention relates to the field of manhole cover inspection technology, and in particular to a method for inspecting the condition of manhole covers. Background Technology

[0002] In recent years, with the continuous acceleration of urbanization and the sustained high-speed development of urban infrastructure construction in my country, the number of underground pipelines for various municipal utilities such as water supply and drainage, gas, heating, electricity, and communications has been increasing. Consequently, the number of manholes on urban roads has also been increasing, making the management of urban road manhole covers increasingly important. Due to poor management of urban manhole covers, various incidents of injury and vehicle damage have occurred frequently throughout the country. Inadequate management of manhole covers leads to the mixing of sewage and rainwater pipes, causing environmental pollution; blockage of sewer pipes, causing urban flooding; and the accumulation of flammable and explosive gases, potentially causing explosions and disasters, seriously affecting the personal safety of citizens and causing adverse social impacts. Therefore, how to improve and strengthen the management of urban manhole covers and underground pipelines has become a difficult and pressing issue for municipal infrastructure management departments across the country.

[0003] With the rapid advancement of IoT technology, the intelligent transformation of urban manhole covers, based on the combination of sensors, satellite positioning, and mobile communication technologies, has begun to become widespread in the construction of smart cities in China, further improving the level of urban management. Currently, accelerometers and gyroscopes are commonly used to monitor the status of manhole covers in real time. However, these sensors have limited functionality; multiple sensors are needed to detect subsidence, damage, and missing parts of a single manhole cover, and the detection results are not intuitive, sometimes requiring manual on-site assessment of the extent of damage.

[0004] Furthermore, patent document CN114399735A discloses a method and system for inspecting problematic manhole covers on urban roads. The method includes acquiring road surface images at a preset image acquisition frequency, performing neural network image recognition on the road surface images to determine if a manhole cover is detected; when a manhole cover is detected, recording the position data of the manhole cover's border and the position data of any defects; copying the corresponding content from the road surface image based on the manhole cover's border, and inputting the copied content into a trained deep learning classification neural network model to determine if the manhole cover and road surface have defects. This method can quickly screen manhole covers at high frequency, promptly identifying and addressing problematic manhole covers. However, in practice, this method uses a vehicle-mounted inspection method to check the condition of manhole covers, which has a detection interval, making real-time monitoring difficult and unable to monitor underground pipe networks. Moreover, the inspection requires personnel and vehicles to work on the road, resulting in high labor intensity, wasted resources, and potential traffic congestion. Additionally, this inspection method has blind spots, such as sidewalks where vehicles cannot reach, which can easily create safety hazards. Summary of the Invention

[0005] The purpose of this invention is to provide a method for detecting the status of manhole covers. This method combines artificial intelligence technology, video image technology, and broadband-IoT communication technology. It uses an image sensor to identify the damage, subsidence, tilting, and movement of manhole covers in real time and intuitively. It can also immediately retrieve images of manhole covers for identification and confirmation, enabling precise management without leaving home. This effectively solves the technical problems of existing technologies, such as the inability to detect comprehensively in real time, high manual labor intensity, and waste of resources.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] A method for detecting the condition of manhole covers, characterized by comprising the following steps:

[0008] Step A: Set multiple marker points on the back of the manhole cover, and set an image sensor, main control MCU and supplementary light inside the manhole. The main control MCU controls the image sensor to acquire a clear and complete image of the back of the manhole cover under supplementary light. Store the back image as a calibration image, and extract the features of the marker points from the calibration image as calibration feature points.

[0009] Step B: The main control MCU controls the image sensor to periodically acquire detection images of the manhole cover, and performs gamma calculations on the acquired detection images to improve the contrast of the detection images;

[0010] Step C: Use Otsu's method to obtain the optimal binarization threshold for the detection image, and use the optimal binarization threshold to perform binarization processing on the detection image to obtain the standard detection image;

[0011] Step D: Extract the features of the marked points in the standard detection image as detection feature points, and use Hamming distance to match the detection feature points to the calibration image;

[0012] Step E: Compare the detected feature points with the calibrated feature points, and calculate the distance difference between each detected feature point and its corresponding calibrated feature point. If the distance difference between each detected feature point and its corresponding calibrated feature point is zero, it indicates that the manhole cover is in normal condition. If there is a sharp change in the distance difference, it indicates that the manhole cover is damaged. If all distance differences change linearly, it indicates that the manhole cover is tilted. If all distance differences increase, it indicates that the manhole cover is sinking.

[0013] In step A, the number of marking points is at least three, and they are evenly sprayed on the back of the manhole cover;

[0014] In step A, the color of the marker is including but not limited to one or more of white, yellow, red, and mixed colors; the shape of the marker is including but not limited to one or more of circle, triangle, square, and trapezoid.

[0015] In step B, the image sensor is only powered on when acquiring images, and the fill light and image sensor work synchronously.

[0016] In step D, if no detection feature points are extracted in the detection image, it indicates that the manhole cover is either open or missing.

[0017] In the detection method described above, the ORB algorithm is used to extract the features of the marker points.

[0018] The image sensor and supplementary light are both installed within a dustproof and waterproof structural component. This component includes a base, a fixed spherical cover, a rotating spherical cover, a rotating shaft, and a micro motor. The fixed spherical cover is fixed to one side of the base surface and has a transparent camera window. The micro motor, image sensor, and supplementary light are all mounted within the fixed spherical cover via the base, with the image sensor and supplementary light facing the transparent camera window. The rotating shaft extends through both ends of the fixed spherical cover, and the rotating spherical cover is fixed to the rotating shaft. The micro motor is connected to the rotating shaft. When the micro motor controls the rotating spherical cover to rotate to the other side of the base surface via the rotating shaft, the transparent camera window is hidden. When the micro motor controls the rotating spherical cover to rotate onto the fixed spherical cover via the rotating shaft, the transparent camera window is opened, allowing the image sensor to acquire images.

[0019] The base has an inner cavity and an openable and closable bottom cover. An adapter is provided in the middle of the bottom cover. One end of the adapter is connected to a micro motor, an image sensor and a fill light, respectively, and the other end is connected to the main control MCU through a waterproof adapter cable.

[0020] The advantages of using this invention are:

[0021] 1. In step A of this invention, directly storing the calibration image improves the subsequent response speed of the calibration image, thereby shortening the image processing time. In step B, performing gamma calculations on the acquired detection image improves its contrast. In step C, using the optimal binarization threshold to binarize the detection image further enhances the features of the marked points. Overall, this invention combines artificial intelligence, video imaging, and broadband-IoT communication technologies. Using a single image sensor, it can identify the damage, subsidence, tilt, and movement of manhole covers in real time and intuitively, and can immediately retrieve images of the manhole covers for identification and confirmation. This enables precise management without leaving the site, effectively solving the technical problems of existing technologies such as inability to perform comprehensive real-time detection, high manual labor intensity, and resource waste.

[0022] 2. This invention, through supplementary lighting design, ensures that the image sensor can acquire better quality images even in the absence of light underground.

[0023] 3. The use of multiple marker points in this invention helps to improve the accuracy of manhole cover status detection.

[0024] 4. This invention installs both the image sensor and the supplementary light inside a dustproof and waterproof structural component, which is beneficial for the normal operation of each electronic component in complex underground working conditions.

[0025] 5. This invention can also transmit images of manhole covers, allowing users to view their actual condition remotely without the need for on-site inspection. Attached Figure Description

[0026] Figure 1 This is a flowchart of the present invention;

[0027] Figure 2 This is a diagram showing the effect of binarization processing in this invention;

[0028] Figure 3 This is an exploded structural diagram of the dustproof and waterproof structural component in this invention;

[0029] Figure 4 This is a schematic diagram of the dustproof and waterproof structural component of the present invention when it is opened;

[0030] Figure 5 This is a schematic diagram of the dustproof and waterproof structural component in this invention when it is closed.

[0031] The following are marked in the diagram: 1. Base, 2. Bottom cover, 3. Fixed spherical cover, 4. Rotating spherical cover, 5. Rotating shaft, 6. Miniature motor, 7. Transparent camera window, 8. Image sensor, 9. Adapter, 10. Waterproof adapter cable. Detailed Implementation

[0032] This invention provides a method for detecting the condition of manhole covers. This method is used to detect various conditions of manhole covers in underground pipe networks. It mainly involves installing (or spraying) marking points on the back of the manhole cover, capturing images of the back of the manhole cover using a camera, identifying the marking points on the manhole cover, and then using an image processing algorithm to extract the marking point parameters in the image. These marking point parameters are compared with pre-stored parameters in a standard template. By comparing the differences between the standard template and the data values ​​of the collected marking points, it is determined whether the manhole cover is tilted, sunken, damaged, or opened in actual use.

[0033] like Figure 1 As shown, it includes the following steps:

[0034] Step A: Set multiple marker points on the back of an intact manhole cover, and place an image sensor, a main control MCU, and supplementary lights at appropriate locations inside the manhole. The number and brightness of the supplementary lights are adjustable. After setup, the main control MCU controls the image sensor to acquire a clear and complete image of the back of the manhole cover under supplementary lighting conditions, and stores this image as a calibration image. Then, the ORB algorithm is used to extract the features of the marker points from the calibration image as calibration feature points.

[0035] It should be noted that each manhole cover should have at least three marking points, and preferably the marking points should be evenly sprayed on the back of the manhole cover. The color and shape of the marking points should be easy for image sensors to recognize. Specifically, the color should include, but is not limited to, one or more of white, yellow, red, and mixed colors, and the shape should include, but is not limited to, one or more of circles, triangles, squares, and trapezoids.

[0036] Preferably, the main control MCU serves as the control core, and a domestically produced ESP32-S module with a CPU frequency of up to 240MHz is preferred. The image sensor used is the OV2640, a 1 / 4-inch CMOS UXGA (1632*1232) image sensor manufactured by OV (OmniVision). This camera is small in size, operates at low voltage, and provides all the functions of a single UXGA camera and image processor. Controlled via the SCCB bus, it can output various resolutions of 8 / 10-bit image data in various modes, including full frame, subsampling, scaling, and windowing. Furthermore, all hardware devices need to be connected to the smart pipeline network terminal acquisition equipment; therefore, each hardware device must conform to the sensor interface specifications of the smart pipeline network terminal acquisition equipment. To ensure compatibility with other sensor module interfaces, each hardware device provides a standard 5-pin connector interface for direct connection to the sensor interface of the smart pipeline network monitoring equipment.

[0037] Step B: The main control MCU controls the image sensor to periodically acquire detection images of the manhole cover. These images are essentially images of the back of the manhole cover. The acquisition period can be customized according to actual needs, such as acquiring images at different time intervals like 30 minutes, 1 hour, or 2 hours. After acquiring the images, gamma calculations are performed to improve their contrast. Furthermore, to address the difficulties of outdoor power access and power shortages, it is preferable that the image sensor and supplementary light are only powered on during operation, and that the image sensor and supplementary light work synchronously to reduce the time the supplementary light is on.

[0038] Step C: Use Otsu's method to obtain the optimal binarization threshold for the detection image obtained in Step B, and use the optimal binarization threshold to binarize the detection image to find pure white marker points. The specific effect after binarization is as follows: Figure 2 As shown, a standard detection image is obtained after processing.

[0039] Step D: The ORB algorithm is used to extract the features of the marked points in the standard detection image as detection feature points, and the Hamming distance is used to match the detection feature points to the calibration image. If no detection feature points are extracted in the detection image, it indicates that the manhole cover is in the open or missing state.

[0040] Step E: If detection feature points are extracted in step D, compare the detection feature points with the calibration feature points and calculate the distance difference between each detection feature point and its corresponding calibration feature point. If the distance difference between each detection feature point and its corresponding calibration feature point is zero, it indicates that the manhole cover is in normal condition. If the distance difference shows a sharp drop, it indicates that the manhole cover is damaged. If all distance differences show a linear change, it indicates that the manhole cover is tilted. If all distance differences increase, it indicates that the manhole cover is sinking. This achieves real-time detection of the manhole cover's condition. It should also be noted that when the damage to the manhole cover is located exactly on a marked point, or when it is deformed or simply has a hole, it will cause a sharp drop in the distance difference, in which case the manhole cover is determined to be damaged.

[0041] It should also be noted that after all hardware devices are installed, this invention requires installation and debugging, as follows:

[0042] 1) Image sensor focusing

[0043] Image sensors can only acquire clear images within a certain range; different acquisition ranges require focusing the image sensor. After installation, the focusing can be adjusted on-site using the installation software app to ensure image clarity.

[0044] 2) Installation status adjustment

[0045] Once focusing is complete, it only guarantees image clarity, but cannot guarantee that the image sensor can completely cover the entire manhole cover. Therefore, the installer needs to adjust the angle of the image sensor so that the image it acquires can completely cover the entire manhole cover. This requires structural support for adjusting the image sensor's acquisition direction.

[0046] 3) Marker point

[0047] Since all algorithms are based on marker points, if the marker points are not reliable, it will cause false alarms from the device. Therefore, it is necessary to verify that the marker points are working properly after the device is installed.

[0048] 4) Equipment debugging

[0049] After the equipment is installed, the image sensor uses WiFi to transmit the images it collects to the debugging terminal equipment. The debugging personnel then observe whether the collected images are normal on the terminal equipment.

[0050] Equipment commissioning includes the following steps:

[0051] Equipment Image Recognition: During debugging, a debugging terminal device is connected to WiFi. Debugging personnel view real-time images through the debugging app on the terminal device, adjusting the device's position and angle to ensure the image sensor captures a clear image covering the entire manhole cover area.

[0052] Fill light adjustment: Adjust the number and / or brightness of the fill lights via the App to make the captured image markers stand out most prominently and be clearly distinguishable from the surrounding images.

[0053] Marker point calibration: After the equipment is installed and debugged, a calibration is required. The calibration is performed by the installer clicking the button in the APP to generate the relevant commands.

[0054] Algorithm verification: After the device is debugged, test run the device on the APP to observe whether the recognition algorithm is running normally.

[0055] Device activation: After the device is debugged and working properly, activate the device and the device's WiFi function will be automatically turned off.

[0056] In this invention, since the main control MCU, image sensor, and supplementary light are all installed in a very dark and humid underground pipe network, in order to avoid water, air, impurities, etc. affecting the equipment's operating status and service life, this invention also designs a dustproof and waterproof structural component and a sealed box. The image sensor and supplementary light are installed in the dustproof and waterproof structural component, and the main control MCU is installed in the sealed box to achieve the purpose of waterproofing and dustproofing the equipment.

[0057] Furthermore, such as Figure 3-5As shown, the dustproof and waterproof structural component includes a base 1, a fixed spherical cover 3, a rotating spherical cover 4, a rotating shaft 5, and a micro motor 6. The base 1 can be circular and has an inner cavity and an openable / closable bottom cover 2. Both the fixed spherical cover 3 and the rotating spherical cover 4 are approximately a quarter-sphere of the overall sphere, but the area of ​​the rotating spherical cover 4 is larger than that of the fixed spherical cover 3 to facilitate the rotating spherical cover 4 being rotatably mounted on the fixed spherical cover 3. Specifically, a fixed spherical cover 3 is fixed to one side of the base 1. A transparent camera window 7 is provided on the fixed spherical cover 3. A micro motor 6, an image sensor 8, and a supplementary light (not shown) are all mounted inside the fixed spherical cover 3 via the base 1. The image sensor 8 and the supplementary light both face the transparent camera window 7. A rotating shaft 5 movably passes through both ends of the fixed spherical cover 3. A rotating spherical cover 4 is fixed to the rotating shaft 5. The micro motor 6 is connected to the rotating shaft 5. The main control MCU can control the opening and closing of the image sensor 8, the supplementary light, and the micro motor 6 respectively. Furthermore, to improve the drying effect, sufficient desiccant can be placed inside the fixed spherical cover 3. An adapter 9 is provided in the middle of the bottom cover 2. One end of the adapter 9 is connected to the micro motor 6, the image sensor 8, and the supplementary light respectively, and the other end is connected to the main control MCU via a waterproof adapter cable 10. When the micro motor 6 controls the rotating spherical housing 4 to rotate to the other side of the base 1 via the rotating shaft 5, the transparent camera window 7 is hidden. At this time, the image sensor 8, the supplementary light, etc., can be hidden and protected to prevent water droplets and dirt from getting on the lens. When the micro motor 6 controls the rotating spherical housing 4 to rotate onto the fixed spherical housing 3 via the rotating shaft 5, the transparent camera window 7 is opened, the image sensor 8 can acquire images, and the supplementary light can provide corresponding illumination.

[0058] In addition, the present invention can also mount detection sensors on the base 1 as needed, such as liquid sensors, gas sensors, etc., to facilitate the detection of liquid and gas conditions in the underground pipe network.

[0059] Overall, this invention combines artificial intelligence, video imaging, and broadband-IoT communication technologies. Using a single image sensor 8, it can identify the damage, subsidence, tilting, and movement of manhole covers in real time and intuitively. It can also immediately retrieve images of the manhole covers for identification and confirmation, enabling precise management without leaving home. This effectively solves the technical problems of existing technologies, such as the inability to detect comprehensively in real time, high manual labor intensity, and waste of resources.

[0060] The above description is merely a specific embodiment of the present invention. Any feature disclosed in this specification may be replaced by other equivalent or similar features unless otherwise specified. All features or steps in the disclosed methods or processes may be combined in any way, except for mutually exclusive features and / or steps.

Claims

1. A method of detecting the state of a manhole cover, characterized by The method comprises the following steps: Step A: a plurality of mark points are arranged on the back of the manhole cover, an image sensor, a main control MCU and a light supplement lamp are arranged in the well, the image sensor is controlled by the main control MCU to collect a clear and complete back image of the manhole cover in a light supplement state, the back image is stored as a calibration image, and the features of the mark points in the calibration image are extracted as calibration feature points; Step B: the image sensor is controlled by the main control MCU to collect detection images of the manhole cover at a period, and the collected detection images are subjected to gamma operation to improve the contrast of the detection images; Step C: the best binary threshold of the detection images is obtained by using the Otsu method, and the detection images are subjected to binary processing by using the best binary threshold to obtain standard detection images; Step D: the features of the mark points in the standard detection images are extracted as detection feature points, and the detection feature points are matched to the calibration feature points by using Hamming distance; Step E: the detection feature points are compared with the calibration feature points, and the distance differences between each detection feature point and each corresponding calibration feature point are calculated; if the distance differences between each detection feature point and each corresponding calibration feature point are all zero, it indicates that the manhole cover is in a normal state; if the distance differences present a cliff-like change, it indicates that the manhole cover is in a damaged state; if all the distance differences present a linear change, it indicates that the manhole cover is in an inclined state; and if all the distance differences increase, it indicates that the manhole cover is in a subsidence state; The image sensor and the light supplement lamp are both installed in a dustproof and waterproof structure, which comprises a base, a fixed spherical cover shell, a rotating spherical cover shell, a rotating shaft and a micro motor, the fixed spherical cover shell is fixed on one side of the surface of the base, the fixed spherical cover shell is provided with a transparent camera window, the micro motor, the image sensor and the light supplement lamp are all installed in the fixed spherical cover shell through the base, and the image sensor and the light supplement lamp both face the transparent camera window, the rotating shaft is movably penetrated through both ends of the fixed spherical cover shell, the rotating spherical cover shell is fixed on the rotating shaft, and the micro motor is connected with the rotating shaft; when the micro motor controls the rotating spherical cover shell to rotate to the other side of the surface of the base through the rotating shaft, the transparent camera window is hidden; when the micro motor controls the rotating spherical cover shell to rotate to the fixed spherical cover shell through the rotating shaft, the transparent camera window is opened, and the image sensor can collect images.

2. The method of claim 1, wherein: In step A, the number of mark points is at least three, and the mark points are uniformly sprayed on the back of the manhole cover.

3. The method according to claim 1 or 2, characterized in that: In step A, the color of the mark points includes but is not limited to one or more of white, yellow, red and mixed color; and the shape of the mark points includes but is not limited to one or more of circle, triangle, square and trapezoid.

4. The method of claim 1, wherein: In step B, the image sensor is only connected to the power supply when collecting images, and the light supplement lamp and the image sensor work synchronously.

5. The method of claim 1, wherein: In step D, if no detection feature point is extracted in the detection image, it indicates that the manhole cover is in an open state or a missing state.

6. The method of claim 1, wherein: In the detection method, the features of the mark points are extracted by using the ORB algorithm.

7. The method of claim 1, wherein: The base has an inner cavity and a bottom cover which can be opened and closed, the bottom cover is provided with an adapter in the middle, one end of the adapter is connected with the micro motor, the image sensor and the light supplement lamp respectively, and the other end is connected with the main control MCU through a waterproof adapter line.

Images